Currying in hardware - scala

I'm trying to implement a simple Address Decoder with a curried function inside. The code below won't compile, could anybody help me with this?
class AddrDecoder[T<:UInt] (dType:T, n:Int) extends Module {
val io = IO (new Bundle {
//val range = (Vec(Seq.fill(n){(dType,dType)})) // This won't compile, how to fix ?
val range = (List.fill(n){(dType,dType)})
val addr = Input (dType)
val en = Input (Bool())
val sel = Output(Bool())
})
def inside (range:(T,T))(addr:T):Bool = {
addr >= range._1 && addr < range._1 + range._2
}
when (io.en) {
io.sel := io.range map (inside(_)(io.addr))
}
}
[error] found : List[chisel3.Bool]
[error] (which expands to) List[chisel3.core.Bool]
[error] required: chisel3.core.Data
[error] io.sel := io.range map (inside(_)(io.addr))

#jkoenig provided an excellent solution. Posting it here for other's benefit
class AddrDecoder[T<:Data with Num[T]] (dType:T, n:Int) extends Module {
val io = IO (new Bundle {
val range0 = Input (Vec(n,dType))
val range1 = Input (Vec(n,dType))
val addr = Input (dType)
val en = Input (Bool())
val sel = Output(Vec(n,Bool()))
})
// Curried function which accepts a tuple and an input addr
// Use map to apply it to inputs
def inside (range:(T,T))(addr:T):Bool = {
addr >= range._1 && addr < range._1 + range._2
}
// MUX output
for (i <- 0 until n) {
io.sel(i) := false.B
}
when (io.en) {
io.sel := io.range0 zip io.range1 map (inside(_)(io.addr))
}
// $onehot0 output encoding check
assert (PopCount(io.sel) <= 1.U, "Invalid addr decoding")
}

Related

How to write a Muller C Element in Chisel Scala Language for async circuit?

When I wrote this :
class MullerC(val WIDTH: Int = 2) extends Module {
val io = IO(new Bundle {
val in = Input(Vec(WIDTH, Bool()))
val out = Output(Bool())
})
io.out := false.B
when (io.in.reduce(_ & _)) {
io.out := true.B
}.elsewhen (io.in.map(!_).reduce(_ & _)) {
io.out := false.B
}
}
I got Verilog like this :
module MullerC(
input clock,
input reset,
input io_in_0,
input io_in_1,
output io_out
);
assign io_out = io_in_0 & io_in_1;
endmodule
That is a simple and gate instead of a C gate.
But when I tried to add otherwise like this :
class MullerC(val WIDTH: Int = 2) extends Module {
val io = IO(new Bundle {
val in = Input(Vec(WIDTH, Bool()))
val out = Output(Bool())
})
io.out := false.B
when (io.in.reduce(_ & _)) {
io.out := true.B
}.elsewhen (io.in.map(!_).reduce(_ & _)) {
io.out := false.B
}.otherwise {
io.out := io.out
}
}
It could not be compiled any more:
Exception in thread "main" firrtl.transforms.CheckCombLoops$CombLoopException: : [module MullerC] Combinational loop detected:
MullerC.io_out
MullerC._GEN_0 #[----.scala 14:38 ----.scala 15:12 ----.scala 17:12]
MullerC._GEN_1 #[----.scala 12:30 ----.scala 13:12]
MullerC.io_out
How should I implement the Muller C in Chisel? Many thanks.
I found the answer via this link: Disable FIRRTL pass that checks for combinational loops
I should use otherwise and then add --no-check-comb-loops as a parameter to emit verilog code. Thanks.
By the way, I also tried this and it works as well.
class MullerC(val WIDTH: Int = 2) extends Module {
val io = IO(new Bundle {
val in = Input(Vec(WIDTH, Bool()))
val out = Output(Bool())
})
io.out := false.B
val allTrue = Wire(Bool())
val allFalse = Wire(Bool())
allTrue := io.in.reduce(_ & _);
allFalse := io.in.map(!_).reduce(_ & _)
io.out := Mux(allTrue | allFalse, Mux(allTrue, true.B, false.B), io.out)
}
This will generate more beautiful verilog code althought it does not matter.

Carry Select Adder Scala Chisel

I'm trying to implement a Carry Select Adder using Chisel in Scala. However after various efforts i keep getting errors. The ripple carry adder and multiplexers are tested and working. This is my current Version. Any Input is greatly appreciated!
class CarrySelectAdder(val n:Int, val m:Int) extends Module {
val io = IO(new Bundle {
val a = Input(UInt(n.W))
val b = Input(UInt(n.W))
val cin = Input(UInt(1.W))
val sum = Output(UInt((n+1).W))
})
object carryselecthelper{ //not used atm
def n_sum_idx(stage:UInt) : UInt = {
return stage * (stage + 1.U) / 2.U;
}
}
//val test = math.ceil(n/m)
val rcas = Array.fill(2 * (n/m)){Module(new RcaAdder(m)).io} //2*math.ceil(n/m)
val muxs = Array.fill(n/m){Module(new Multiplexer(m+1)).io} //math.ceil(n/m)
val Sum = Wire(Vec(n, Bool()))
for (i <- 0 until n) {
rcas(i*2).a := io.a.apply(i*m+m,i*m)
rcas(i*2).b := io.b.apply(i*m+m,i*m)
rcas(i*2).cin := 0.asUInt
rcas(2*i+1).a := io.a.apply(i*m+m,i*m)
rcas(2*i+1).b := io.b.apply(i*m+m,i*m)
rcas(2*i+1).cin := 1.asUInt
muxs(i).a := rcas(i*2).sum
muxs(i).b := rcas(2*i+1).sum
if(i > 0){
muxs(i).sel := io.cin
}
if(i > 0){
if(muxs(i).sel == 0){
muxs(i).sel := rcas(2*i).cout
}else{
muxs(i).sel := rcas(2*i+1).cout
}
}
Sum(i) := muxs(i).out
}
io.sum := Sum.asUInt
}
Currently I'm getting an Exception in "ChiselGeneratorAnnotation":
l3.internal.ChiselException: Exception thrown when elaborating ChiselGeneratorAnnotation
at chisel3.stage.ChiselGeneratorAnnotation.elaborate(ChiselAnnotations.scala:55)
ChiselAnnotations.scala:55
at chisel3.stage.phases.Elaborate.$anonfun$transform$1(Elaborate.scala:19)
Elaborate.scala:19
at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:245)
TraversableLike.scala:245
at scala.collection.immutable.List.foreach(List.scala:392)
List.scala:392
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245)
TraversableLike.scala:245
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242)
TraversableLike.scala:242
at scala.collection.immutable.List.flatMap(List.scala:355)
List.scala:355
at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:18)
Elaborate.scala:18
at chisel3.iotesters.setupTreadleBackend$.apply(TreadleBackend.scala:143)
TreadleBackend.scala:143
at chisel3.iotesters.Driver$.$anonfun$execute$2(Driver.scala:53)
Driver.scala:53
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at logger.Logger$.$anonfun$makeScope$2(Logger.scala:168)
Logger.scala:168
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
DynamicVariable.scala:62
at logger.Logger$.makeScope(Logger.scala:166)
Logger.scala:166
at logger.Logger$.makeScope(Logger.scala:127)
Logger.scala:127
at chisel3.iotesters.Driver$.$anonfun$execute$1(Driver.scala:38)
Driver.scala:38
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
DynamicVariable.scala:62
at chisel3.iotesters.Driver$.execute(Driver.scala:38)
Driver.scala:38
Upon request I added the Multiplexer and RCA class, they should work fine, since they ran all tests without issues. Thanks!
class Multiplexer(val n:Int) extends Module {
val io = IO(new Bundle {
val a = Input(UInt(n.W))
val b = Input(UInt(n.W))
val sel = Input(UInt(n.W))
val out = Output(UInt(n.W))
})
io.out := (io.sel & io.a) | (~io.sel & io.b)
}
class RcaAdder(val n:Int) extends Module {
val io = IO(new Bundle {
val a = Input(UInt(n.W))
val b = Input(UInt(n.W))
val cin = Input(UInt(1.W))
val sum = Output(UInt(n.W))
val cout = Output(UInt(1.W))
})
val FAs = Array.fill(n){Module(new FullAdder()).io}
val carry = Wire(Vec(n+1, UInt(1.W)))
val sum = Wire(Vec(n, Bool()))
carry(0) := io.cin
for (i <- 0 until n) {
FAs(i).a := io.a(i)
FAs(i).b := io.b(i)
FAs(i).cin := carry(i)
carry(i+1) := FAs(i).cout
sum(i) := FAs(i).sum.asBool
}
io.sum := sum.asUInt
io.cout := carry(n)
}
(This is what is used to test the code.)
class CarrySelectAdderTester(dut: CarrySelectAdder, n: Int) extends PeekPokeTester(dut) {
val max=scala.math.pow(2,n).toInt-1
for(i <- 0 to max){
for (j <- 0 to max){
for (k <- 0 to 1){
poke(dut.io.a,i)
poke(dut.io.b,j)
poke(dut.io.cin,k)
step(100)
expect(dut.io.sum,(i+j+k))
}
}
}
}
object CarrySelectAdderTester extends App{
val bitWidth=6
val bitsperblock=3
println("Testing the Carry Select Adder")
iotesters.Driver.execute(Array[String](), () => new CarrySelectAdder(bitWidth, bitsperblock)){
c => new CarrySelectAdderTester(c,bitWidth)
}
}

Chisel Passing Enum type as IO

This is a related topic on chisel enum I have already looked at chisel "Enum(UInt(), 5)" failed
I am building a RISC-V in chisel and am running into a roadblock. I would like to abstract the ALU opcode from a combination of opcode, funct3, and funct7 to the actual ALU operation. Below I have a SystemVerilog module that shows the type of behavior I would like to emulate
// Defined in a types file
typedef enum bit [2:0] {
alu_add = 3'b000,
alu_sll = 3'b001,
alu_sra = 3'b010,
alu_sub = 3'b011,
alu_xor = 3'b100,
alu_srl = 3'b101,
alu_or = 3'b110,
alu_and = 3'b111
} alu_ops;
module alu
(
input alu_ops aluop,
input [31:0] a, b,
output logic [31:0] f
);
always_comb
begin
unique case (aluop)
alu_add: f = a + b;
alu_sll: f = a << b[4:0];
alu_sra: f = $signed(a) >>> b[4:0];
alu_sub: f = a - b;
alu_xor: f = a ^ b;
alu_srl: f = a >> b[4:0];
alu_or: f = a | b;
alu_and: f = a & b;
endcase
end
endmodule : alu
This is the current chisel file that I am using
AluType.scala
import chisel3._
import chisel3.Driver
import chisel3.experimental.ChiselEnum
package ALUType {
object AluOP extends ChiselEnum {
// type AluOP = Value
val ADDI, SLTI, SLTIU, XORI, ORI, ANDI, SLLI, SLRI, SRAI, ADD, SUB, SLL, SLT, SLTU, XOR, SRL, SRA, OR, AND = Value
}
}
AluFile.scala
import chisel3._
import chisel3.Driver
import chisel3.experimental.ChiselEnum
import ALUType.AluOP._
class ALUFile(val dl_size: Int, val op_size: Int, val funct3_size: Int, val funct7_size: Int) extends Module {
val io = IO(new Bundle {
val val_a = Input(UInt(dl_size.W))
val val_b = Input(UInt(dl_size.W))
val aluop = Input(ALUType.AluOP.Type)
//val opcode = Input(UInt(op_size.W))
//val funct3 = Input(UInt(funct3_size.W))
//val funct7 = Input(UInt(funct7_size.W))
val val_out = Output(UInt(dl_size.W))
})
// Actual function
}
This is the result of the sbt run
$ sbt run
[info] welcome to sbt 1.4.7 (Oracle Corporation Java 1.8.0_281)
[info] loading project definition from C:\Chisel\Test1\RegFile\project
[info] loading settings for project regfile from build.sbt ...
[info] set current project to regfile (in build file:/C:/Chisel/Test1/RegFile/)
[info] compiling 1 Scala source to C:\Chisel\Test1\RegFile\target\scala-2.12\classes ...
[error] C:\Chisel\Test1\RegFile\src\main\scala\ALUFile.scala:43:30: inferred type arguments [ALUType.AluOP.Type.type] do not conform to method apply's type parameter bounds [T <: chisel3.Data]
[error] val aluop = Input(Wire(ALUType.AluOP.Type))
[error] ^
[error] C:\Chisel\Test1\RegFile\src\main\scala\ALUFile.scala:43:49: type mismatch;
[error] found : ALUType.AluOP.Type.type
[error] required: T
[error] val aluop = Input(Wire(ALUType.AluOP.Type))
[error] ^
[error] two errors found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 4 s, completed Feb 11, 2021 8:35:18 PM
Do I just need to assign it to a UInt higher up, then decode it again? Seems silly to have to encode, then decode just to pass it from one module to the next. Is there a way to get AluOP.Type to conform to T? I would have thought that it would simply because it is a ChiselEnum.
EDIT :: 1
I tried enumerating with UInt but it says that is a non-literal type
[info] running ALUFile
[info] [0.000] Elaborating design...
[error] chisel3.internal.ChiselException: AluOp defined with a non-literal type
[error] ...
[error] at AluOp$.<init>(ALUFile.scala:41)
[error] at AluOp$.<clinit>(ALUFile.scala)
[error] at ALUFile$$anon$1.<init>(ALUFile.scala:49)
[error] at ALUFile.<init>(ALUFile.scala:46)
[error] at ALUFile$.$anonfun$new$6(ALUFile.scala:80)
object AluOp extends ChiselEnum {
val addi, slti, sltiu, xori, ori, andi, slli, slri, srai, add, sub, sll, slt, sltu, xor, srl, sra, or, and = Value(UInt()) // Line where error occurs
}
import AluOp._
class ALUFile(val dl_size: Int, val op_size: Int, val funct3_size: Int, val funct7_size: Int) extends Module {
val io = IO(new Bundle {
val val_a = Input(UInt(dl_size.W))
val val_b = Input(UInt(dl_size.W))
val aluop = Input( AluOp() )
val val_out = Output(UInt(dl_size.W))
})
switch (io.aluop) {
is (addi) {
io.val_out := 1.U
}
is (slti) {
io.val_out := 2.U
}
}
// Output result
io.val_out := 0.U
}
From https://github.com/chipsalliance/chisel3/blob/dd6871b8b3f2619178c2a333d9d6083805d99e16/src/test/scala/chiselTests/StrongEnum.scala
The only example they have for using enum in a switch statement, but they manually map values to the type WHICH IS NOT AN ENUM!!!
object StrongEnumFSM {
object State extends ChiselEnum {
val sNone, sOne1, sTwo1s = Value
val correct_annotation_map = Map[String, BigInt]("sNone" -> 0, "sOne1" -> 1, "sTwo1s" -> 2)
}
}
class StrongEnumFSM extends Module {
import StrongEnumFSM.State
import StrongEnumFSM.State._
// This FSM detects two 1's one after the other
val io = IO(new Bundle {
val in = Input(Bool())
val out = Output(Bool())
val state = Output(State())
})
val state = RegInit(sNone)
io.out := (state === sTwo1s)
io.state := state
switch (state) {
is (sNone) {
when (io.in) {
state := sOne1
}
}
is (sOne1) {
when (io.in) {
state := sTwo1s
} .otherwise {
state := sNone
}
}
is (sTwo1s) {
when (!io.in) {
state := sNone
}
}
}
}
One solution that might exist can be found here https://github.com/chipsalliance/chisel3/issues/885 where he defined his own Scala object and allowed for the call to return a UInt.
Additionally if I just use Enum, I can get it to compile which may just be the best solution for now. I would like to see a ChiselEnum be able to easily define states or operations at UInts and be able to pass them as IO so that I can get away from using numbers to define states and make them much more readable.
object AluOp {
val addi :: slti :: sltiu :: xori :: ori :: andi :: slli :: slri :: srai :: add :: sub :: sll :: slt :: sltu :: xor :: srl :: sra :: or :: and :: Nil = Enum(19)
}
import AluOp._
class ALUFile(val dl_size: Int, val op_size: Int, val funct3_size: Int, val funct7_size: Int) extends Module {
val io = IO(new Bundle {
val val_a = Input(UInt(dl_size.W))
val val_b = Input(UInt(dl_size.W))
// val aluop = Input( UInt(AluOp.getWidth.W) )
val aluop = Input(UInt(5.W))
// val opcode = Input(UInt(op_size.W))
// val funct3 = Input(UInt(funct3_size.W))
// val funct7 = Input(UInt(funct7_size.W))
val val_out = Output(UInt(dl_size.W))
})
// val reg_last = RegNext()
switch (io.aluop) {
is (addi) {
io.val_out := 1.U
}
is (slti) {
io.val_out := 2.U
}
}
I think all you need to do is use
val aluop = Input(AluOP())
Some simple example code can be found in the chisel3 unit tests
OP Edit ::
By adding a mapping:
import chisel3._
import chisel3.Driver
import chisel3.util._
import chisel3.experimental.ChiselEnum
package ALUType {
// object AluOp {
// val addi :: slti :: sltiu :: xori :: ori :: andi :: slli :: slri :: srai :: add :: sub :: sll :: slt :: sltu :: xor :: srl :: sra :: or :: and :: Nil = Enum(19)
// }
object AluOp extends ChiselEnum {
val add, sub, sll, slt, sltu, xor, srl, sra, or, and = Value
val correct_annotation_map = Map[String, UInt](
"add" -> 0.U,
"sub" -> 1.U,
"sll" -> 2.U,
"slt" -> 3.U,
"sltu" -> 4.U,
"xor" -> 5.U,
"srl" -> 6.U,
"sra" -> 7.U,
"or" -> 8.U,
"and" -> 9.U
)
}
}
The value can be passed as an input:
import ALUType.AluOp._
class ALUFile(val dl_size: Int, val op_size: Int, val funct3_size: Int, val funct7_size: Int) extends Module {
import ALUType._
import ALUType.AluOp._
val io = IO(new Bundle {
val val_a = Input(UInt(dl_size.W))
val val_b = Input(UInt(dl_size.W))
val aluop = Input( AluOp() )
// val aluop = Input(UInt(5.W))
// val opcode = Input(UInt(op_size.W))
// val funct3 = Input(UInt(funct3_size.W))
// val funct7 = Input(UInt(funct7_size.W))
val val_out = Output(UInt(dl_size.W))
})
// switch (io.aluop) {
// is (add) {
// io.val_out := io.val_a + io.val_b
// }
// is (slt) {
// io.val_out := 2.U
// }
// }
switch (io.aluop) {
is (add) {
io.val_out := io.val_a + io.val_b
}
is (slt) {
io.val_out := 2.U
}
}
// Output result
io.val_out := 0.U
}
This is still not ideal as I would like to not have to manually map the strings to UInt values, but it is what it is. Maybe a Scala foreach loop could take the tedious assign out, who knows.

Chisel: Access to Module Parameters from Tester

How does one access the parameters used to construct a Module from inside the Tester that is testing it?
In the test below I am passing the parameters explicitly both to the Module and to the Tester. I would prefer not to have to pass them to the Tester but instead extract them from the module that was also passed in.
Also I am new to scala/chisel so any tips on bad techniques I'm using would be appreciated :).
import Chisel._
import math.pow
class TestA(dataWidth: Int, arrayLength: Int) extends Module {
val dataType = Bits(INPUT, width = dataWidth)
val arrayType = Vec(gen = dataType, n = arrayLength)
val io = new Bundle {
val i_valid = Bool(INPUT)
val i_data = dataType
val i_array = arrayType
val o_valid = Bool(OUTPUT)
val o_data = dataType.flip
val o_array = arrayType.flip
}
io.o_valid := io.i_valid
io.o_data := io.i_data
io.o_array := io.i_array
}
class TestATests(c: TestA, dataWidth: Int, arrayLength: Int) extends Tester(c) {
val maxData = pow(2, dataWidth).toInt
for (t <- 0 until 16) {
val i_valid = rnd.nextInt(2)
val i_data = rnd.nextInt(maxData)
val i_array = List.fill(arrayLength)(rnd.nextInt(maxData))
poke(c.io.i_valid, i_valid)
poke(c.io.i_data, i_data)
(c.io.i_array, i_array).zipped foreach {
(element,value) => poke(element, value)
}
expect(c.io.o_valid, i_valid)
expect(c.io.o_data, i_data)
(c.io.o_array, i_array).zipped foreach {
(element,value) => poke(element, value)
}
step(1)
}
}
object TestAObject {
def main(args: Array[String]): Unit = {
val tutArgs = args.slice(0, args.length)
val dataWidth = 5
val arrayLength = 6
chiselMainTest(tutArgs, () => Module(
new TestA(dataWidth=dataWidth, arrayLength=arrayLength))){
c => new TestATests(c, dataWidth=dataWidth, arrayLength=arrayLength)
}
}
}
If you make the arguments dataWidth and arrayLength members of TestA you can just reference them. In Scala this can be accomplished by inserting val into the argument list:
class TestA(val dataWidth: Int, val arrayLength: Int) extends Module ...
Then you can reference them from the test as members with c.dataWidth or c.arrayLength

scala priority queue not ordering properly?

I'm seeing some strange behavior with Scala's collection.mutable.PriorityQueue. I'm performing an external sort and testing it with 1M records. Each time I run the test and verify the results between 10-20 records are not sorted properly. I replace the scala PriorityQueue implementation with a java.util.PriorityQueue and it works 100% of the time. Any ideas?
Here's the code (sorry it's a bit long...). I test it using the tools gensort -a 1000000 and valsort from http://sortbenchmark.org/
def externalSort(inFileName: String, outFileName: String)
(implicit ord: Ordering[String]): Int = {
val MaxTempFiles = 1024
val TempBufferSize = 4096
val inFile = new java.io.File(inFileName)
/** Partitions input file and sorts each partition */
def partitionAndSort()(implicit ord: Ordering[String]):
List[java.io.File] = {
/** Gets block size to use */
def getBlockSize: Long = {
var blockSize = inFile.length / MaxTempFiles
val freeMem = Runtime.getRuntime().freeMemory()
if (blockSize < freeMem / 2)
blockSize = freeMem / 2
else if (blockSize >= freeMem)
System.err.println("Not enough free memory to use external sort.")
blockSize
}
/** Sorts and writes data to temp files */
def writeSorted(buf: List[String]): java.io.File = {
// Create new temp buffer
val tmp = java.io.File.createTempFile("external", "sort")
tmp.deleteOnExit()
// Sort buffer and write it out to tmp file
val out = new java.io.PrintWriter(tmp)
try {
for (l <- buf.sorted) {
out.println(l)
}
} finally {
out.close()
}
tmp
}
val blockSize = getBlockSize
var tmpFiles = List[java.io.File]()
var buf = List[String]()
var currentSize = 0
// Read input and divide into blocks
for (line <- io.Source.fromFile(inFile).getLines()) {
if (currentSize > blockSize) {
tmpFiles ::= writeSorted(buf)
buf = List[String]()
currentSize = 0
}
buf ::= line
currentSize += line.length() * 2 // 2 bytes per char
}
if (currentSize > 0) tmpFiles ::= writeSorted(buf)
tmpFiles
}
/** Merges results of sorted partitions into one output file */
def mergeSortedFiles(fs: List[java.io.File])
(implicit ord: Ordering[String]): Int = {
/** Temp file buffer for reading lines */
class TempFileBuffer(val file: java.io.File) {
private val in = new java.io.BufferedReader(
new java.io.FileReader(file), TempBufferSize)
private var curLine: String = ""
readNextLine() // prep first value
def currentLine = curLine
def isEmpty = curLine == null
def readNextLine() {
if (curLine == null) return
try {
curLine = in.readLine()
} catch {
case _: java.io.EOFException => curLine = null
}
if (curLine == null) in.close()
}
override protected def finalize() {
try {
in.close()
} finally {
super.finalize()
}
}
}
val wrappedOrd = new Ordering[TempFileBuffer] {
def compare(o1: TempFileBuffer, o2: TempFileBuffer): Int = {
ord.compare(o1.currentLine, o2.currentLine)
}
}
val pq = new collection.mutable.PriorityQueue[TempFileBuffer](
)(wrappedOrd)
// Init queue with item from each file
for (tmp <- fs) {
val buf = new TempFileBuffer(tmp)
if (!buf.isEmpty) pq += buf
}
var count = 0
val out = new java.io.PrintWriter(new java.io.File(outFileName))
try {
// Read each value off of queue
while (pq.size > 0) {
val buf = pq.dequeue()
out.println(buf.currentLine)
count += 1
buf.readNextLine()
if (buf.isEmpty) {
buf.file.delete() // don't need anymore
} else {
// re-add to priority queue so we can process next line
pq += buf
}
}
} finally {
out.close()
}
count
}
mergeSortedFiles(partitionAndSort())
}
My tests don't show any bugs in PriorityQueue.
import org.scalacheck._
import Prop._
object PriorityQueueProperties extends Properties("PriorityQueue") {
def listToPQ(l: List[String]): PriorityQueue[String] = {
val pq = new PriorityQueue[String]
l foreach (pq +=)
pq
}
def pqToList(pq: PriorityQueue[String]): List[String] =
if (pq.isEmpty) Nil
else { val h = pq.dequeue; h :: pqToList(pq) }
property("Enqueued elements are dequeued in reverse order") =
forAll { (l: List[String]) => l.sorted == pqToList(listToPQ(l)).reverse }
property("Adding/removing elements doesn't break sorting") =
forAll { (l: List[String], s: String) =>
(l.size > 0) ==>
((s :: l.sorted.init).sorted == {
val pq = listToPQ(l)
pq.dequeue
pq += s
pqToList(pq).reverse
})
}
}
scala> PriorityQueueProperties.check
+ PriorityQueue.Enqueued elements are dequeued in reverse order: OK, passed
100 tests.
+ PriorityQueue.Adding/removing elements doesn't break sorting: OK, passed
100 tests.
If you could somehow reduce the input enough to make a test case, it would help.
I ran it with five million inputs several times, output matched expected always. My guess from looking at your code is that your Ordering is the problem (i.e. it's giving inconsistent answers.)