I am trying to come up with a better title.
I am new in Chisel and Scala. Below there is a Chisel code defining and testing an module.
import chisel3._
import chiseltest._
import org.scalatest.flatspec.AnyFlatSpec
class DeviceUnderTest extends Module {
val io = IO(new Bundle {
val a = Input(UInt(2.W))
val b = Input(UInt(2.W))
val out = Output(UInt(2.W))
})
io.out := io.a & io.b
}
class WaveformTestWithIteration extends AnyFlatSpec with ChiselScalatestTester {
"WaveformIteration" should "pass" in {
test(new DeviceUnderTest)
.withAnnotations(Seq(WriteVcdAnnotation)) ( dut => // ???
{
for (a <- 0 until 4; b <- 0 until 4) {
dut.io.a.poke(a.U)
dut.io.b.poke(b.U)
dut.clock.step()
}
}
)
}
}
The code line with comments ??? is where I am quite puzzled. Where does the variable dut is defined? It seems an reference to instance gained by new DeviceUnderTest.
test(new DeviceUnderTest).withAnnotations(Seq(WriteVcdAnnotation)) return an TestBuilder[T] with apply method:
class TestBuilder[T <: Module](...) {
...
def apply(testFn: T => Unit): TestResult = {
runTest(defaults.createDefaultTester(dutGen, finalAnnos))(testFn)
}
...
}
So, dut => {...} is a function (T) => Unit? But it does not look like a standard lambda ((x:T) => {...})? Or it is something else?
What is this syntax in scala exactly?
Consider the following boiled-down version with a similar structure:
def test[A](testedThing: A)(testBody: A => Unit): Unit = testBody(testedThing)
What it's essentially doing is taking a value x: A and a function f: A => Unit, and applying f to x to obtain f(x).
Here is how you could use it:
test("foo"){ x =>
println(if x == "foo" then "Success" else "Failure")
} // Success
test("bar"){ x =>
println(if x == "baz" then "Success" else "Failure")
} // Failure
In both cases, the "string under test" is simply passed to the body of the "test".
Now, you could introduce a few more steps between the creation of the value under test and the specification of the body. For example, you could create a TestBuilder[A], which is essentially just a value a with some bells and whistles (in this case, list of "annotations" - plain strings in the following example):
type Annotation = String
case class TestOutcome(annotations: List[Annotation], successful: Boolean)
trait Test:
def run: TestOutcome
// This simply captures the value under test of type `A`
case class TestBuilder[A](
theThingUnderTest: A,
annotations: List[Annotation]
):
// Bells and whistles: adding some metadata
def withAnnotations(moreAnnotations: List[Annotation]): TestBuilder[A] =
TestBuilder(theThingUnderTest, annotations ++ moreAnnotations)
// Combining the value under test with the body of the test produces the
// actual test
def apply(testBody: A => Unit): Test = new Test:
def run =
try {
testBody(theThingUnderTest)
TestOutcome(annotations, true)
} catch {
case t: Throwable => TestOutcome(annotations, false)
}
// This constructs the thing that's being tested, and creates a TestBuilder around it
def test[A](thingUnderTest: A) = TestBuilder(thingUnderTest, Nil)
println(
test("hello")
.withAnnotations(List("size of hello should be 5")){ h =>
assert(h.size == 5)
}
.run
)
println(
test("hello")
.withAnnotations(List("size of hello should be 42")){ h =>
assert(h.size == 42)
}
.run
)
The principle remains the same: test(a) saves the tested value a, then TestBuilder adds some configuration, and once you add a body { thingUnderTest => /* assertStuff */ } to it, you get a full Test, which you can then run to obtain some results (TestOutcomes, in this case). Thus, the above snippet produces
TestOutcome(List(size of hello should be 5),true)
TestOutcome(List(size of hello should be 42),false)
I think I did not notice that sometimes we can omit type declaration in lambda.
class tester{
def apply( fn: (Int) => Int):Int = fn(5)
}
We can write (new tester)(x => {x+1}) instead of (new tester)((x:Int) => {x+1}).
I wrote simple callback(handler) function which i pass to async api and i want to wait for result:
object Handlers {
val logger: Logger = Logger("Handlers")
implicit val cs: ContextShift[IO] =
IO.contextShift(ExecutionContext.Implicits.global)
class DefaultHandler[A] {
val response: IO[MVar[IO, A]] = MVar.empty[IO, A]
def onResult(obj: Any): Unit = {
obj match {
case obj: A =>
println(response.flatMap(_.tryPut(obj)).unsafeRunSync())
println(response.flatMap(_.isEmpty).unsafeRunSync())
case _ => logger.error("Wrong expected type")
}
}
def getResponse: A = {
response.flatMap(_.take).unsafeRunSync()
}
}
But for some reason both tryPut and isEmpty(when i'd manually call onResult method) returns true, therefore when i calling getResponse it sleeps forever.
This is the my test:
class HandlersTest extends FunSuite {
test("DefaultHandler.test") {
val handler = new DefaultHandler[Int]
handler.onResult(3)
val response = handler.getResponse
assert(response != 0)
}
}
Can somebody explain why tryPut returns true, but nothing puts. And what is the right way to use Mvar/channels in scala?
IO[X] means that you have the recipe to create some X. So on your example, yuo are putting in one MVar and then asking in another.
Here is how I would do it.
object Handlers {
trait DefaultHandler[A] {
def onResult(obj: Any): IO[Unit]
def getResponse: IO[A]
}
object DefaultHandler {
def apply[A : ClassTag]: IO[DefaultHandler[A]] =
MVar.empty[IO, A].map { response =>
new DefaultHandler[A] {
override def onResult(obj: Any): IO[Unit] = obj match {
case obj: A =>
for {
r1 <- response.tryPut(obj)
_ <- IO(println(r1))
r2 <- response.isEmpty
_ <- IO(println(r2))
} yield ()
case _ =>
IO(logger.error("Wrong expected type"))
}
override def getResponse: IO[A] =
response.take
}
}
}
}
The "unsafe" is sort of a hint, but every time you call unsafeRunSync, you should basically think of it as an entire new universe. Before you make the call, you can only describe instructions for what will happen, you can't actually change anything. During the call is when all the changes occur. Once the call completes, that universe is destroyed, and you can read the result but no longer change anything. What happens in one unsafeRunSync universe doesn't affect another.
You need to call it exactly once in your test code. That means your test code needs to look something like:
val test = for {
handler <- TestHandler.DefaultHandler[Int]
_ <- handler.onResult(3)
response <- handler.getResponse
} yield response
assert test.unsafeRunSync() == 3
Note this doesn't really buy you much over just using the MVar directly. I think you're trying to mix side effects inside IO and outside it, but that doesn't work. All the side effects need to be inside.
I am trying to write a function which can add a context to those functions given in parameters.
The idea is here
object example {
def withOne(f : => T) = {
val a = 1 //some context
f
}
def foo() = withOne {
println(a)
}
}
I think the context could be passed in implicit.
The idea is to not have the content of f constraint by the surrounding function f should be able to use the context or not.
For now the only way i seen to do that is like that
object example {
def withOne(f : => Int => T) = {
val a = 1 //some context
f(a)
}
def foo() = withOne { a =>
println(a)
}
}
But this forces to declare a 'a' witch is not obvious for others devs :x
I'm afraid you cannot work around this, since you cannot inject an implicit into a function.
There's a proposal to add this feature in the typelevel/scala fork, but it seems hard to achieve as of today.
My suggestion here is to use proper naming, so that you won't surprise your users. For instance if you provide a method like:
def withConnection[A](f: Connection => A): A = {
try {
val conn = ???
f(conn)
} finally {
conn.close()
}
}
it won't surprise me to do:
withConnection { implicit c =>
// db stuff
}
Let's say I want to make a little wrapper along the lines of:
def wrapper(f: (Any) => Any): Any = {
println("Executing now")
val res = f
println("Execution finished")
res
}
wrapper {
println("2")
}
Does this make sense? My wrapper method is obviously wrong, but I think the spirit of what I want to do is possible. Am I right in thinking so? If so, what's the solution? Thanks!
If you want your wrapper method to execute the wrapped method inside itself, you should change the parameter to be 'by name'. This uses the syntax => ResultType.
def wrapper(f: => Any): Any = {
println("Executing now")
val res = f
println("Execution finished")
res
}
You can now do this,
wrapper {
println("2")
}
and it will print
Executing now
2
Execution finished
If you want to be able to use the return type of the wrapped function, you can make your method generic:
def wrapper[T](f: => T): T = {
println("Executing now")
val res: T = f
println("Execution finished")
res
}
In your case you are already executing the function println and then pass the result to your wrapper while it is expecting a function with one arguments (Any) and that return Any.
Not sure if this answer to your question but you can use a generic type parameter and accept a function with no arguments that return that type:
def wrapper[T](f: () => T) = {
println("Executing now")
val res = f() // call the function
println("Execution finished")
res
}
wrapper {
()=>println("2") // create an anonymous function that will be called
}
I've defined 'using' function as following:
def using[A, B <: {def close(): Unit}] (closeable: B) (f: B => A): A =
try { f(closeable) } finally { closeable.close() }
I can use it like that:
using(new PrintWriter("sample.txt")){ out =>
out.println("hellow world!")
}
now I'm curious how to define 'using' function to take any number of parameters, and be able to access them separately:
using(new BufferedReader(new FileReader("in.txt")), new PrintWriter("out.txt")){ (in, out) =>
out.println(in.readLIne)
}
Starting Scala 2.13, the standard library provides a dedicated resource management utility: Using.
More specifically, the Using#Manager can be used when dealing with several resources.
In our case, we can manage different resources such as your PrintWriter or BufferedReader as they both implement AutoCloseable, in order to read and write from a file to another and, no matter what, close both the input and the output resource afterwards:
import scala.util.Using
import java.io.{PrintWriter, BufferedReader, FileReader}
Using.Manager { use =>
val in = use(new BufferedReader(new FileReader("input.txt")))
val out = use(new PrintWriter("output.txt"))
out.println(in.readLine)
}
// scala.util.Try[Unit] = Success(())
Someone has already done this—it's called Scala ARM.
From the readme:
import resource._
for(input <- managed(new FileInputStream("test.txt")) {
// Code that uses the input as a FileInputStream
}
I've been thinking about this and I thought maybe there was an other way to address this. Here is my take on supporting "any number" of parameters (limited by what tuples provide):
object UsingTest {
type Closeable = {def close():Unit }
final class CloseAfter[A<:Product](val x: A) {
def closeAfter[B](block: A=>B): B = {
try {
block(x);
} finally {
for (i <- 0 until x.productArity) {
x.productElement(i) match {
case c:Closeable => println("closing " + c); c.close()
case _ =>
}
}
}
}
}
implicit def any2CloseAfter[A<:Product](x: A): CloseAfter[A] =
new CloseAfter(x)
def main(args:Array[String]): Unit = {
import java.io._
(new BufferedReader(new FileReader("in.txt")),
new PrintWriter("out.txt"),
new PrintWriter("sample.txt")) closeAfter {case (in, out, other) =>
out.println(in.readLine)
other.println("hello world!")
}
}
}
I think I'm reusing the fact that 22 tuple/product classes have been written in the library... I don't think this syntax is clearer than using nested using (no pun intended), but it was an interesting puzzle.
using structural typing seems like a little overkill since java.lang.AutoCloseable is predestined for usage:
def using[A <: AutoCloseable, B](resource: A)(block: A => B): B =
try block(resource) finally resource.close()
or, if you prefer extension methods:
implicit class UsingExtension[A <: AutoCloseable](val resource: A) extends AnyVal {
def using[B](block: A => B): B = try block(resource) finally resource.close()
}
using2 is possible:
def using2[R1 <: AutoCloseable, R2 <: AutoCloseable, B](resource1: R1, resource2: R2)(block: (R1, R2) => B): B =
using(resource1) { _ =>
using(resource2) { _ =>
block(resource1, resource2)
}
}
but imho quite ugly - I would prefer to simply nest these using statements in the client code.
Unfortunately, there isn't support for arbitrary-length parameter lists with arbitrary types in standard Scala.
You might be able to do something like this with a couple of language changes (to allow variable parameter lists to be passed as HLists; see here for about 1/3 of what's needed).
Right now, the best thing to do is just do what Tuple and Function do: implement usingN for as many N as you need.
Two is easy enough, of course:
def using2[A, B <: {def close(): Unit}, C <: { def close(): Unit}](closeB: B, closeC: C)(f: (B,C) => A): A = {
try { f(closeB,closeC) } finally { closeB.close(); closeC.close() }
}
If you need more, it's probably worth writing something that'll generate the source code.
Here is an example that allows you to use the scala for comprehension as an automatic resource management block for any item that is a java.io.Closeable, but it could easily be expanded to work for any object with a close method.
This usage seems pretty close to the using statement and allows you to easily have as many resources defined in one block as you want.
object ResourceTest{
import CloseableResource._
import java.io._
def test(){
for( input <- new BufferedReader(new FileReader("/tmp/input.txt")); output <- new FileWriter("/tmp/output.txt") ){
output.write(input.readLine)
}
}
}
class CloseableResource[T](resource: =>T,onClose: T=>Unit){
def foreach(f: T=>Unit){
val r = resource
try{
f(r)
}
finally{
try{
onClose(r)
}
catch{
case e =>
println("error closing resource")
e.printStackTrace
}
}
}
}
object CloseableResource{
implicit def javaCloseableToCloseableResource[T <: java.io.Closeable](resource:T):CloseableResource[T] = new CloseableResource[T](resource,{_.close})
}
It is a good idea to detatch the cleanup algorithm from the program path.
This solution lets you accumulate closeables in a scope.
The scope cleanup will happen on after the block is executed, or the scope can be detached. The cleaning of the scope can then be done later.
This way we get the same convenience whitout being limited to single thread programming.
The utility class:
import java.io.Closeable
object ManagedScope {
val scope=new ThreadLocal[Scope]();
def managedScope[T](inner: =>T):T={
val previous=scope.get();
val thisScope=new Scope();
scope.set(thisScope);
try{
inner
} finally {
scope.set(previous);
if(!thisScope.detatched) thisScope.close();
}
}
def closeLater[T <: Closeable](what:T): T = {
val theScope=scope.get();
if(!(theScope eq null)){
theScope.closeables=theScope.closeables.:+(what);
}
what;
}
def detatchScope(): Scope={
val theScope=scope.get();
if(theScope eq null) null;
else {
theScope.detatched=true;
theScope;
}
}
}
class Scope{
var detatched=false;
var closeables:List[Closeable]=List();
def close():Unit={
for(c<-closeables){
try{
if(!(c eq null))c.close();
} catch{
case e:Throwable=>{};
}
}
}
}
The usage:
def checkSocketConnect(host:String, portNumber:Int):Unit = managedScope {
// The close later function tags the closeable to be closed later
val socket = closeLater( new Socket(host, portNumber) );
doWork(socket);
}
def checkFutureConnect(host:String, portNumber:Int):Unit = managedScope {
// The close later function tags the closeable to be closed later
val socket = closeLater( new Socket(host, portNumber) );
val future:Future[Boolean]=doAsyncWork(socket);
// Detatch the scope and use it in the future.
val scope=detatchScope();
future.onComplete(v=>scope.close());
}
This solution doesn't quite have the syntax you desire, but I think it's close enough :)
def using[A <: {def close(): Unit}, B](resources: List[A])(f: List[A] => B): B =
try f(resources) finally resources.foreach(_.close())
using(List(new BufferedReader(new FileReader("in.txt")), new PrintWriter("out.txt"))) {
case List(in: BufferedReader, out: PrintWriter) => out.println(in.readLine())
}
Of course the down side is you have to type out the types BufferedReader and PrintWrter in the using block. You might be able to add some magic so that you just need List(in, out) by using multiple ORed type bounds for type A in using.
By defining some pretty hacky and dangerous implicit conversions you can get around having to type List (and another way to get around specifying types for the resources), but I haven't documented the detail as it's too dangerous IMO.
here is my solution to the resource management in Scala:
def withResources[T <: AutoCloseable, V](r: => T)(f: T => V): V = {
val resource: T = r
require(resource != null, "resource is null")
var exception: Throwable = null
try {
f(resource)
} catch {
case NonFatal(e) =>
exception = e
throw e
} finally {
closeAndAddSuppressed(exception, resource)
}
}
private def closeAndAddSuppressed(e: Throwable,
resource: AutoCloseable): Unit = {
if (e != null) {
try {
resource.close()
} catch {
case NonFatal(suppressed) =>
e.addSuppressed(suppressed)
}
} else {
resource.close()
}
}
I used this in multiple Scala apps including managing resources in Spark executors. and one should be aware that we are other even better ways to manage resource like in CatsIO: https://typelevel.org/cats-effect/datatypes/resource.html. if you are ok with pure FP in Scala.
to answer your last question, you can definitely nest the resource like this:
withResource(r: File)(
r => {
withResource(a: File)(
anotherR => {
withResource(...)(...)
}
)
}
)
this way, not just that those resources are protected from leaking, they will also be released in the correct order(like stack). same behaviour like the Resource Monad from CatsIO.