Question
Would like to get assistance to understand the cause of the error. The original is from Coursera Scala Design Functional Random Generators.
Task
With the factories for random int and random boolean, trying to implement a random tree factory.
trait Factory[+T] {
self => // alias of 'this'
def generate: T
def map[S](f: T => S): Factory[S] = new Factory[S] {
def generate = f(self.generate)
}
def flatMap[S](f: T => Factory[S]): Factory[S] = new Factory[S] {
def generate = f(self.generate).generate
}
}
val intFactory = new Factory[Int] {
val rand = new java.util.Random
def generate = rand.nextInt()
}
val boolFactory = intFactory.map(i => i > 0)
Problem
The implementation in the 1st block causes the error but if it changed into the 2nd block, it does not. I believe Factory[+T] meant that Factory[Inner] and Factory[Leaf] could be both treated as Factory[Tree].
I have no idea why the same if expression in for block is OK but it is not OK in yield block. I appreciate explanations.
trait Tree
case class Inner(left: Tree, right: Tree) extends Tree
case class Leaf(x: Int) extends Tree
def leafFactory: Factory[Leaf] = intFactory.map(i => new Leaf(i))
def innerFactory: Factory[Inner] = new Factory[Inner] {
def generate = new Inner(treeFactory.generate, treeFactory.generate)
}
def treeFactory: Factory[Tree] = for {
isLeaf <- boolFactory
} yield if (isLeaf) leafFactory else innerFactory
^^^^^^^^^^^ ^^^^^^^^^^^^
type mismatch; found : Factory[Inner] required: Tree
type mismatch; found : Factory[Leaf] required: Tree
However, below works.
def treeFactory: Factory[Tree] = for {
isLeaf <- boolFactory
tree <- if (isLeaf) leafFactory else innerFactory
} yield tree
I have no idea why the same if expression in for block is OK but it is
not OK in yield block
Because they are translated differently by the compiler. The former example is translated into:
boolFactory.flatMap((isLeaf: Boolean) => if (isLeaf) leafFactory else innerFactor)
Which yields the expected Factory[Tree], while the latter is being translated to:
boolFactory.map((isLeaf: Boolean) => if (isLeaf) leafFactory else innerFactory)
Which yields a Factory[Factory[Tree]], not a Factory[Tree], thus not conforming to your method signature. This isn't about covariance, but rather how for comprehension translates these statements differently.
Related
I am trying to come up with a better title.
I am new in Chisel and Scala. Below there is a Chisel code defining and testing an module.
import chisel3._
import chiseltest._
import org.scalatest.flatspec.AnyFlatSpec
class DeviceUnderTest extends Module {
val io = IO(new Bundle {
val a = Input(UInt(2.W))
val b = Input(UInt(2.W))
val out = Output(UInt(2.W))
})
io.out := io.a & io.b
}
class WaveformTestWithIteration extends AnyFlatSpec with ChiselScalatestTester {
"WaveformIteration" should "pass" in {
test(new DeviceUnderTest)
.withAnnotations(Seq(WriteVcdAnnotation)) ( dut => // ???
{
for (a <- 0 until 4; b <- 0 until 4) {
dut.io.a.poke(a.U)
dut.io.b.poke(b.U)
dut.clock.step()
}
}
)
}
}
The code line with comments ??? is where I am quite puzzled. Where does the variable dut is defined? It seems an reference to instance gained by new DeviceUnderTest.
test(new DeviceUnderTest).withAnnotations(Seq(WriteVcdAnnotation)) return an TestBuilder[T] with apply method:
class TestBuilder[T <: Module](...) {
...
def apply(testFn: T => Unit): TestResult = {
runTest(defaults.createDefaultTester(dutGen, finalAnnos))(testFn)
}
...
}
So, dut => {...} is a function (T) => Unit? But it does not look like a standard lambda ((x:T) => {...})? Or it is something else?
What is this syntax in scala exactly?
Consider the following boiled-down version with a similar structure:
def test[A](testedThing: A)(testBody: A => Unit): Unit = testBody(testedThing)
What it's essentially doing is taking a value x: A and a function f: A => Unit, and applying f to x to obtain f(x).
Here is how you could use it:
test("foo"){ x =>
println(if x == "foo" then "Success" else "Failure")
} // Success
test("bar"){ x =>
println(if x == "baz" then "Success" else "Failure")
} // Failure
In both cases, the "string under test" is simply passed to the body of the "test".
Now, you could introduce a few more steps between the creation of the value under test and the specification of the body. For example, you could create a TestBuilder[A], which is essentially just a value a with some bells and whistles (in this case, list of "annotations" - plain strings in the following example):
type Annotation = String
case class TestOutcome(annotations: List[Annotation], successful: Boolean)
trait Test:
def run: TestOutcome
// This simply captures the value under test of type `A`
case class TestBuilder[A](
theThingUnderTest: A,
annotations: List[Annotation]
):
// Bells and whistles: adding some metadata
def withAnnotations(moreAnnotations: List[Annotation]): TestBuilder[A] =
TestBuilder(theThingUnderTest, annotations ++ moreAnnotations)
// Combining the value under test with the body of the test produces the
// actual test
def apply(testBody: A => Unit): Test = new Test:
def run =
try {
testBody(theThingUnderTest)
TestOutcome(annotations, true)
} catch {
case t: Throwable => TestOutcome(annotations, false)
}
// This constructs the thing that's being tested, and creates a TestBuilder around it
def test[A](thingUnderTest: A) = TestBuilder(thingUnderTest, Nil)
println(
test("hello")
.withAnnotations(List("size of hello should be 5")){ h =>
assert(h.size == 5)
}
.run
)
println(
test("hello")
.withAnnotations(List("size of hello should be 42")){ h =>
assert(h.size == 42)
}
.run
)
The principle remains the same: test(a) saves the tested value a, then TestBuilder adds some configuration, and once you add a body { thingUnderTest => /* assertStuff */ } to it, you get a full Test, which you can then run to obtain some results (TestOutcomes, in this case). Thus, the above snippet produces
TestOutcome(List(size of hello should be 5),true)
TestOutcome(List(size of hello should be 42),false)
I think I did not notice that sometimes we can omit type declaration in lambda.
class tester{
def apply( fn: (Int) => Int):Int = fn(5)
}
We can write (new tester)(x => {x+1}) instead of (new tester)((x:Int) => {x+1}).
I have a case where I wish to apply modifications to an object based on the presence of (a few, say, 5 to 10) optionals. So basically, if I were to do it imperatively, what I'm aiming for is :
var myObject = ...
if (option.isDefined) {
myObject = myObject.modify(option.get)
}
if (option2.isDefined) {
myObject = myObject.someOtherModification(option2.get)
}
(Please note : maybe my object is mutable, maybe not, that is not the point here.)
I thought it'd look nicer if I tried to implement a fluent way of writing this, such as (pseudo code...) :
myObject.optionally(option, _.modify(_))
.optionally(option2, _.someOtherModification(_))
So I started with a sample code, which intelliJ does not highlight as an error, but that actually does not build.
class MyObject(content: String) {
/** Apply a transformation if the optional is present */
def optionally[A](optional: Option[A], operation: (A, MyObject) => MyObject): MyObject =
optional.map(operation(_, this)).getOrElse(this)
/** Some possible transformation */
def resized(length : Int): MyObject = new MyObject(content.substring(0, length))
}
object Test {
val my = new MyObject("test")
val option = Option(2)
my.optionally(option, (size, value) => value.resized(size))
}
Now, in my case, the MyObject type is of some external API, so I created an implicit conversion to help, so what it really does look like :
// Out of my control
class MyObject(content: String) {
def resized(length : Int): MyObject = new MyObject(content.substring(0, length))
}
// What I did : create a rich type over MyObject
class MyRichObject(myObject: MyObject) {
def optionally[A](optional: Option[A], operation: (A, MyObject) => MyObject): MyObject = optional.map(operation(_, myObject)).getOrElse(myObject)
}
// And an implicit conversion
object MyRichObject {
implicit def apply(myObject: MyObject): MyRichObject = new MyRichObject(myObject)
}
And then, I use it this way :
object Test {
val my = new MyObject("test")
val option = Option(2)
import MyRichObject._
my.optionally(option, (size, value) => value.resized(size))
}
And this time, it fails in IntelliJ and while compiling because the type of the Option is unknown :
Error:(8, 26) missing parameter type
my.optionally(option, (size, value) => value.resized(size))
To make it work, I can :
Actively specify a type of the size argument : my.optionally(option, (size: Int, value) => value.resized(size))
Rewrite the optionally to a curried-version
None of them is really bad, but if I may ask :
Is there a reason that a curried version works, but a multi argument version seems to fail to infer the parametrized type,
Could it be written in a way that works without specifying the actual types
and as a bonus (although this might be opinion based), how would you write it (some sort of foldLeft on a sequence of optionals come to my mind...) ?
One option for your consideration:
// Out of my control
class MyObject(content: String) {
def resized(length : Int): MyObject = new MyObject(content.substring(0, length))
}
object MyObjectImplicits {
implicit class OptionalUpdate[A](val optional: Option[A]) extends AnyVal {
def update(operation: (A, MyObject) => MyObject): MyObject => MyObject =
(obj: MyObject) => optional.map(a => operation(a, obj)).getOrElse(obj)
}
}
object Test {
val my = new MyObject("test")
val option = Option(2)
import MyObjectImplicits._
Seq(
option.update((size, value) => value.resized(size)),
// more options...
).foldLeft(my)(_)
}
Might as well just use a curried-version of your optionally, like you said.
A nicer way to think about the need to add the type there is write it this way:
object Test {
val my = new MyObject("test")
val option = Some(2)
my.optionally[Int](option, (size, value) => value.resized(size))
}
Another way, if you only will manage one type since the object creation, is to move the generic to the class creation, but be careful, with this option you only can have one type per instance:
class MyObject[A](content: String) {
def optionally(optional: Option[A], operation: (A, MyObject[A]) => MyObject[A]): MyObject[A] =
optional.map(operation(_, this)).getOrElse(this)
def resized(length : Int): MyObject[A] = new MyObject[A](content.substring(0, length))
}
object Test {
val my = new MyObject[Int]("test")
val option = Some(2)
my.optionally(option, (size, value) => value.resized(size))
}
As you can see, now all the places where the generics was is taken by the Int type, because that is what you wanted in the first place, here is a pretty answer telling why:
(just the part that I think applies here:)
4)When the inferred return type would be more general than you intended, e.g., Any.
Source: In Scala, why does a type annotation must follow for the function parameters ? Why does the compiler not infer the function parameter types?
I've problems understanding where to put type informations in scala, and how to put it. Here I create several sequences of Actors and I don't type them. Even if I had to, I wouldn't know which type of sequence map produces to give them the proper type.
Then later when the compiler yells at me because I'm trying to sum Anys, I've no idea where to begin filling in the gaps.
Here is my code, I tried to minimize it while still letting the necessary info available.
object Actors {
def main(args: Array[String]) {
val array = randomArray(5)
val master = new Master(array, 5)
master.start
}
def randomArray(length: Int): Array[Int] = {
val generator = new Random
new Array[Int](length) map((_:Int) => generator nextInt)
}
}
class Master(array: Array[Int], slavesNumber: Int) extends Actor {
def act () {
val slaves = (1 to slavesNumber).map(_ => new Slave)
slaves.foreach(s => s.start)
val futures = slaves.map(s => s !! Work(array))
val results = awaitAll(3000, futures:_*)
val res2 = results.flatMap(x => x)
println((0 /: res2)(_+_))
}
}
class Slave() extends Actor {
def act () {
Actor.loop {
receive {
case Work(slice) =>
reply((slice :\ 0)(_+_))
}
}
}
}
I'd appreciate too some good pointers towards comprehensive doc on the matter.
The object that are passed between actors are not typed, actors have to filter the object themselves -- as you already do in the Slave actor. As you can see, !! is defined as
def !!(msg: Any): Future[Any]
so there is no type information in the returned Future. Probably the easiest solution is to replace the line var res2 .. with
val res2 = results collect {case Some(y:Int) => y}
this filters out just those Some results that are of type Int.
Is it possible to remove some types from the following code:
import util.continuations._
object TrackingTest extends App {
implicit def trackable(x: Int) = new {
def tracked[R] = shift { cf: (Int => (R, Set[Int])) =>
cf(x) match {
case (r, ints) => (r, ints + x)
}
}
}
def track[R](body: => R #cpsParam[(R, Set[Int]), (R, Set[Int])]) = reset {
(body, Set[Int]())
}
val result = track(7.tracked[Int] + 35.tracked[Int])
assert(result == (42, Set(7, 35)))
val differentTypes = track(9.tracked[String].toString)
assert(differentTypes == ("9", Set(9)))
}
track function tracks calls of tracked on Int instances (e.g. 7.tracked).
Is it possible to infer type parameter on tracked implicit, so the following would compile:
track(7.tracked + 35.tracked)
Your question made me think of how continuations can track state. So I adapted that to your case and came up with this:
import util.continuations._
object TrackingTest extends App {
type State = Set[Int]
type ST = State => State
implicit class Tracked(val i: Int) extends AnyVal {
def tracked = shift{ (k: Int=>ST) => (state:State) => k(i)(state + i) }
}
def track[A](thunk: => A#cps[ST]): (A, State) = {
var result: A = null.asInstanceOf[A]
val finalSate = (reset {
result = thunk
(state:State) => state
}).apply(Set[Int]())
(result, finalSate)
}
val result = track(7.tracked + 35.tracked)
assert(result == (42, Set(7, 35)))
val differentTypes = track(9.tracked.toString)
assert(differentTypes == ("9", Set(9)))
}
This is using 2.10.1 but it works fine with 2.9.1 as well provided you replace the 2.10.x implicit value class with:
implicit def tracked(i: Int) = new {
def tracked = shift{ (k: Int=>ST) => (state:State) => k(i)(state + i) }
}
The key change I made is to have tracked not use any type inference, fixing to Int#cps[ST]. The CPS plugin then maps the computation to the right type (like String#cps[ST]) as appropriate. The state is threaded by the continuation returning a State=>State function that takes the current state (the set of ints) and returns the next state. The return type of reset is a function from state to state (of type ST) that will take the initial state and will return the final state.
The final trick is to use a var to capture the result while still keeping the expected type for reset.
While the exact answer to this question can be given only by the authors of the compiler, we can guess it is not possible by giving a look to the continuation plugin source code.
If you look to the source of the continuations you can see this:
val anfPhase = new SelectiveANFTransform() {
val global = SelectiveCPSPlugin.this.global
val runsAfter = List("pickler")
}
val cpsPhase = new SelectiveCPSTransform() {
val global = SelectiveCPSPlugin.this.global
val runsAfter = List("selectiveanf")
}
The anfPhase phase is executed after the pickler phase, and the cpsPhase after selectiveAnf. If you look to SelectiveANFTransform.scala
abstract class SelectiveANFTransform extends PluginComponent with Transform with
TypingTransformers with CPSUtils {
// inherits abstract value `global' and class `Phase' from Transform
import global._ // the global environment
import definitions._ // standard classes and methods
import typer.atOwner // methods to type trees
/** the following two members override abstract members in Transform */
val phaseName: String = "selectiveanf"
If we use scalac -Xshow-phases, we can see the phases during the compilation process:
parser
namer
packageobjects
typer
superaccessors
pickler
refchecks
selectiveanf
liftcode
selectivecps
uncurry
......
As you can see the typer phase is applied before the selectiveAnf and selectiveCps phases. It should be confirmed that type inference occurs in the typer phase, but if this is really the case and it would make sense, it should be now clear why you can't omit the Int type on 7.tracked and 35.tracked.
Now if you are not satisfied yet, you should know that the compiler works by performing a set of transformations on "trees", which you might look, using the following options:
-Xprint: shows your scala code after a certain phase have been executed
-Xprint: -Yshow-trees shows your scala code and the trees after the phase have been executed
-YBrowse: opens a GUI to surf both.
Why is a return statement required to allow this while statement to be
evaluated properly? The following statement allows
import java.io.File
import java.io.FileInputStream
import java.io.InputStream
import java.io.BufferedReader
import java.io.InputStreamReader
trait Closeable {
def close ()
}
trait ManagedCloseable extends Closeable {
def use (code: () => Unit) {
try {
code()
}
finally {
this.close()
}
}
}
class CloseableInputStream (stream: InputStream)
extends InputStream with ManagedCloseable {
def read = stream.read
}
object autoclose extends App {
implicit def inputStreamToClosable (stream: InputStream):
CloseableInputStream = new CloseableInputStream(stream)
override
def main (args: Array[String]) {
val test = new FileInputStream(new File("test.txt"))
test use {
val reader = new BufferedReader(new InputStreamReader(test))
var input: String = reader.readLine
while (input != null) {
println(input)
input = reader.readLine
}
}
}
}
This produces the following error from scalac:
autoclose.scala:40: error: type mismatch;
found : Unit
required: () => Unit
while (input != null) {
^
one error found
It appears that it's attempting to treat the block following the use as an
inline statement rather than a lambda, but I'm not exactly sure why. Adding
return after the while alleviates the error:
test use {
val reader = new BufferedReader(new InputStreamReader(test))
var input: String = reader.readLine
while (input != null) {
println(input)
input = reader.readLine
}
return
}
And the application runs as expected. Can anyone describe to me what is going
on there exactly? This seems as though it should be a bug. It's been
persistent across many versions of Scala though (tested 2.8.0, 2.9.0, 2.9.1)
That's because it's use is declared as () => Unit, so the compiler expects the block you are giving use to return something that satisfies this signature.
It seems that what you want is to turn the entire block into a by-name parameter, to do so change def use (code : () => Unit) to def use (code : => Unit).
() => Unit is the type of a Function0 object, and you've required the use expression to be of that type, which it obviously isn't. => Unit is a by name parameter, which you should use instead.
You might find my answer to this question useful.
To go the heart of the matter, blocks are not lambdas. A block in Scala is a scope delimiter, nothing more.
If you had written
test use { () =>
val reader = new BufferedReader(new InputStreamReader(test))
var input: String = reader.readLine
while (input != null) {
println(input)
input = reader.readLine
}
}
Then you'd have a function (indicated by () =>) which is delimited by the block.
If use had been declared as
def use (code: => Unit) {
Then the syntax you used would work, but not because of any lambda thingy. That syntax indicates the parameter is passed by name, which, roughly speaking, means you'd take the whole expression passed as parameter (ie, the whole block), and substitute it for code inside the body of use. The type of code would be Unit, not a function, but the parameter would not be passed by value.
return or return expr has the type Nothing. You can substitute this for any type, as it never yields a value to the surrounding expression, instead it returns control to the caller.
In your program, it masquerades as the required type () => Unit.
Here's an occasionally convenient use for that (although you might be tarnished as unidiomatic if you use it too often, don't tell anyone you heard this from me!)
def foo(a: Option[Int]): Int = {
val aa: Int = a.getOrElse(return 0)
aa * 2
}
For the record, you should probably write:
def foo(a: Option[Int]): Int =
a.map(_ * 2).getOrElse(0)
You can get an insight into the mind of the compiler by checking the output of scala -Xprint:typer -e <one-liner>. Add -Ytyper-debug if you like sifting through the reams of output!
scala210 -Ytyper-debug -Xprint:typer -e 'def foo: Any = {val x: () => Any = { return }}'
... elided ...
typed return (): Nothing
adapted return (): Nothing to () => Any,