Choice of implicit conversion location with a block - scala

In the following code, the implicit conversion is applied around the println(2) line; I had foolishly expected it to apply around the entire block { println(1); println(2) }. How should I reason about where the compiler places the implicit?
object Executor {
private var runnable: Runnable = _
def setRunnable(runnable: Runnable) {
this.runnable = runnable
}
def execute() { runnable.run() }
}
object Run extends App {
implicit def blockToRunnable(p: ⇒ Any): Runnable =
new Runnable { def run() = p }
Executor.setRunnable {
println(1)
println(2)
}
println("Before execute")
Executor.execute()
}

I rationalize this behavior like this: according to the spec, the type of a block {s1; s2; ...; sn; e } is the type of the last expression e.
So the compiler takes e and type checks it against Runnable. That fails, so it searches for an implicit conversion that will convert e to Runnable. So it would like this:
{ s1; s2; ... sn; convert(e) }
This is confirmed with scala -Xprint:typer on this small example:
class A
implicit def convert(a: A): String = a.toString
def f(s: String) { println(s) }
f{ println(1); new A }
prints:
private[this] val res0: Unit = $line3.$read.$iw.$iw.f({
scala.this.Predef.println(1);
$line2.$read.$iw.$iw.convert(new $line1.$read.$iw.$iw.A())
});

According to the spec, an implicit conversion is applied when the type of an expression does not match the expected type. The key observation is how the expected type is threaded when typing blocks.
if an expression e is of type T, and T does not conform to the expression’s expected type pt. In this case an implicit v is searched which is applicable to e and whose result type conforms to pt.
In Section 6.11 Blocks, the expected type of a block's last expression is defined as
The expected type of the final expression e is the expected type of the block.
Given this spec, it seems the compiler has to behave this way. The expected type of the block is Runnable, and the expected type of println(2) becomes Runnable as well.
A suggestion: if you want to know what implicits are applied, you can use a nightly build for 2.1 of the Scala IDE for Eclipse. It can 'highlight implicits'.
Edited: I admit it is surprising when there's a call-by-name implicit in scope.

The problem is that you are thinking of blocks as if they were thunks, as if they were pieces of code. They aren't. { a; b; c } is not a piece of code that can be passed around.
So, how should you reason about implicits? Actually, how should you reason about views, which are implicit conversions. Views are applied to the value that needs to be changed. In your example, the value of
{
println(1)
println(2)
}
is being passed to setRunnable. The value of a block is the value of its last expression, so it is passing the result of println(2) to setRunnable. Since that is Unit and setRunnable requires a Runnable, then an implicit is searched for and found, so println(2) is passed to the grossly misnamed blockToRunnable.
Bottom line is, and this is an advice I have given many times already on Stack Overflow (lots of people try to do the same thing) is to get the following in your head:
THERE ARE NO BLOCKS IN SCALA.
There are functions, but not blocks.
Technically, that statement is incorrect -- there are blocks in Scala, but they are not what you think they are, so just completely remove them from your mind. You can learn what blocks in Scala are latter, from a clean slate. Otherwise, you're bound to try to get them to work in ways they don't, or infer that things work in a certain way when they work in a different way.

I liked a lot the explanation given in the first scala puzzle.
In other words, what will be the output of:
List(1, 2).map { i => println("Hi"); i + 1 }
List(1, 2).map { println("Hi"); _ + 1 }

Related

Implicit conversions and by-name parameters of target type

consider this code:
class DelayedInt(val value: Int) {
var locked = true
}
object DelayedInt {
implicit def delayedIntToInt(del: DelayedInt) = {
if (del.locked) throw new RuntimeException("not yet!")
del.value
}
}
object Main {
var queue: Seq[() => Int] = Seq.empty
def queueInt(int: => Int): Unit = {
queue :+= int _
}
def printQueue(): Unit =
for (f <- queue)
println("int is: " + f.apply())
def main(args: Array[String]): Unit = {
val di = new DelayedInt(42)
// println(5 + di) // throws exception
queueInt(5 + di) // OK
queueInt(di) // OK
di.locked = false
printQueue()
}
}
The DelayedInt class throws an exception upon conversion to int, unless it is explicitly unlocked.
The println function must fail, because it invokes the implicit conversion.
queueInt function, works fine, seemingly because it uses by-name parameters and the implicit conversion is not invoked until printQueue.
What is unclear to me is how does Scala decide when to run the implicit conversion? Especially in case of queueInt(di), it must somehow figure out that the result will be of the right type, without using the implicit conversion.
What are the steps to evaluate this?
What is unclear to me is how does Scala decide when to run the implicit conversion?
It doesn't, directly. It just inserts it where required by the expected type, so after that compiler stage your code becomes
val di = new DelayedInt(42)
// println(5 + DelayedInt.delayedIntToInt(di)) // throws exception
queueInt(5 + DelayedInt.delayedIntToInt(di)) // OK
queueInt(DelayedInt.delayedIntToInt(di)) // OK
di.locked = false
printQueue()
and then the queueInt cases behave normally for by-name parameters.
it must somehow figure out that the result will be of the right type, without using the implicit conversion.
This is of course based on types and doesn't require running anything.
My understanding is in both cases, queueInt(5 + di) and queueInt(di), implicit conversion should happen before any evaluation takes place. queueInt accepts a by-name argument, and then due to int _ syntax it adds an unapplied function to queue: Seq[() => Int], so passed in arguments do not get evaluated after queueInt completes. Only when printQueue() is called should element functions of queue evaluate due to f.apply() call.
In the case of println(5 + di) the argument is not passed by-name thus after implicit conversion it evaluates before it is passed to println, hence the throw, but the implicit conversion still happens first, just like for queueInt cases.
There does not seem to be anything strange happening here.

Implicit conversions weirdness

I am trying to understand why exactly an implicit conversion is working in one case, but not in the other.
Here is an example:
case class Wrapper[T](wrapped: T)
trait Wrapping { implicit def wrapIt[T](x: Option[T]) = x.map(Wrapper(_))
class NotWorking extends Wrapping { def foo: Option[Wrapper[String]] = Some("foo") }
class Working extends Wrapping {
def foo: Option[Wrapper[String]] = {
val why = Some("foo")
why
}
}
Basically, I have an implicit conversion from Option[T] to Option[Wrapper[T]], and am trying to define a function, that returns an optional string, that gets implicitly wrapped.
The question is why, when I try to return Option[String] directly (NotWorking above), I get an error (found : String("foo") required: Wrapper[String]), that goes away if I assign the result to a val before returning it.
What gives?
I don't know if this is intended or would be considered a bug, but here is what I think is happening.
In def foo: Option[Wrapper[String]] = Some("foo") the compiler will set the expected type of the argument provided to Some( ) as Wrapper[String]. Then it sees that you provided a String which it is not what is expected, so it looks for an implicit conversion String => Wrapper[String], can't find one, and fails.
Why does it need that expected type stuff, and doesn't just type Some("foo") as Some[String] and afterwards try to find a conversion?
Because scalac wants to be able to typecheck the following code:
case class Invariant[T](t: T)
val a: Invariant[Any] = Invariant("s")
In order for this code to work, the compiler can't just type Invariant("s") as Invariant[String] because then compilation will fail as Invariant[String] is not a subtype of Invariant[Any]. The compiler needs to set the expected type of "s" to Any so that it can see that "s" is an instance of Any before it's too late.
In order for both this code and your code to work out correctly, I think the compiler would need some kind of backtracking logic which it doesn't seem to have, perhaps for good reasons.
The reason that your Working code does work, is that this kind of type inference does not span multiple lines. Analogously val a: Invariant[Any] = {val why = Invariant("s"); why} does not compile.

F-Bounded Polymorphic return types in Scala?

I'm going crazy trying to make F-Bounded Polymorphism work as I want in Scala.
The following code will not compile:
object TestTypeBounds {
trait Upper[T <: Upper[T]] {
def map() : T
}
class Impl extends Upper[Impl] {
def map() : Impl = this
}
type Arr = Upper[T] forSome {type T <: Upper[T]}
def testBounds() {
// Though any is specified as the type parameter, the definition of Upper specifies
// an upper bound of Upper
val upper: Upper[_] = new Impl()
// This must 'logically' be an Upper, but the compiler thinks it's an Any
val mapped = upper.map()
// This line will fail!
mapped.map().map().map()
}
def main(args: Array[String]): Unit = {
testBounds()
}
}
The problem here is that the compiler complains that the type of mapped is Any, and therefore it has no method map. It's not clear to me why the compiler does not assign mapped the type Upper since this is in fact the upper type bound of the parameter type of Upper, even if any was specified in this instance.
Note that replacing the type of "val upper...:" with the alias Arr would work, because now Scala can see that the type is recursive and will always be an Upper. Unfortunately, this approach does also not work for me because I am implementing a Java library which passes Upper[_] arguments to functions, and these then run into the above problem. The compiler also does not accept code where such functions are overridden as having "Arr" arguments, i.e. the alias does not work in that scenario.
Edit: The final paragraph is not entirely correct, see my answer below
As #Rado Buransky pointed out, you cannot just omit the type constructor parameter by using an underscore. The following works for example:
def testBounds[T <: Upper[T]](make: => T): Unit = {
val upper: T = make
val mapped = upper.map()
mapped.map().map().map()
}
testBounds(new Impl)
Also this, using an existential type:
def testBounds: Unit = {
val upper: Upper[T] forSome { type T <: Upper[T] } = new Impl
val mapped = upper.map()
mapped.map().map().map()
}
My view on this is that you should not use underscore "_". It tells the compiler that you don't care about the type parameter. But you do. I know that there is the upper bound, but probably there is an optimization which makes the compiler really don't care.
Just a hint, sometimes, for me, if nothing works, there is always the asInstanceOf[T] method. Maybe this helps you:
def giveMeUpper[T <: Upper[T]] = (new Impl).asInstanceOf[Upper[T]]
...
val upper = giveMeUpper[Impl]
In terms of the 'pure' Scala part of the question, 0__ is correct and I have accepted his answer.
Regarding the Java Part: It turns out that if a Java function returns Upper and the Upper interface is defined in Java equivalently to the Scala implementation above, then the compiler does in fact correctly assign it the type Upper[_$2] forSome {type $2 <: Upper[$2]} - i.e. it interoperates correctly. The final problem I had was in fact caused by implicit functions defined in Scala that still returned Upper[_]. Mea Culpa.

What's the difference between using and no using a "=" in Scala defs ?

What the difference between the two defs below
def someFun(x:String) { x.length }
AND
def someFun(x:String) = { x.length }
As others already pointed out, the former is a syntactic shortcut for
def someFun(x:String): Unit = { x.length }
Meaning that the value of x.length is discarded and the function returns Unit (or () if you prefer) instead.
I'd like to stress out that this is deprecated since Oct 29, 2013 (https://github.com/scala/scala/pull/3076/), but the warning only shows up if you compile with the -Xfuture flag.
scala -Xfuture -deprecation
scala> def foo {}
<console>:1: warning: Procedure syntax is deprecated. Convert procedure `foo` to method by adding `: Unit =`.
def foo {}
foo: Unit
So you should never use the so-called procedure syntax.
Martin Odersky itself pointed this out in his Scala Day 2013 Keynote and it has been discussed in the scala mailing list.
The syntax is very inconsistent and it's very common for a beginner to hit this issue when learning the language. For this reasons it's very like that it will be removed from the language at some point.
Without the equals it is implicitly typed to return Unit (or "void"): the result of the body is fixed - not inferred - and any would-be return value is discarded.
That is, def someFun(x:String) { x.length } is equivalent to def someFun(x:String): Unit = { x.length }, neither of which are very useful here because the function causes no side-effects and returns no value.
The "equals form" without the explicit Unit (or other type) has the return type inferred; in this case that would be def someFun(x:String): Int = { x.length } which is more useful, albeit not very exciting.
I prefer to specify the return type for all exposed members, which helps to ensure API/contract stability and arguably adds clarity. For "void" methods this is trivial done by using the Procedure form, without the equals, although it is a stylistic debate of which is better - opponents might argue that having the different forms leads to needless questions about such ;-)
The former is
def someFun(x: String): Unit = {
x.length
() // return unit
}
And the latter is
def someFun(x: String): Int = {
x.length // returned
}
Note that the Scala Style guide always recommend using '=', both in
method declaration
Methods should be declared according to the following pattern:
def foo(bar: Baz): Bin = expr
function declaration
Function types should be declared with a space between the parameter type, the arrow and the return type:
def foo(f: Int => String) = ...
def bar(f: (Boolean, Double) => List[String]) = ...
As per Scala-2.10, using equals sign is preferred. Infact you must use equals sign in call declarations except the definitions returning Unit.
As such there is no difference but the first one is not recommended anymore and it should not be used.

Two seemingly identical semantics: one binds implicitly, the other does not

Hello: I've been learning Scala recently (my related background is mostly in C++ templates), and I've run into something I currently don't understand about Scala, and it is driving me insane. :(
(Also, this is my first post to StackOverflow, where I've noticed most of the really awesome Scala people seem to hang out, so I'm really sorry if I do something horrendously stupid with the mechanism.)
My specific confusion relates to implicit argument binding: I have come up with a specific case where the implicit argument refuses to bind, but a function with seemingly identical semantics does.
Now, it of course could be a compiler bug, but given that I just started working with Scala, the probability of me having already run into some kind of serious bug are sufficiently small that I'm expecting someone to explain what I did wrong. ;P
I have gone through the code and whittled it quite a bit in order to come up with the single example that doesn't work. Unfortunately, that example is still reasonably complex, as the problem seems to only occur in the generalization. :(
1) simplified code that does not work in the way I expected
import HList.::
trait HApplyOps {
implicit def runNil
(input :HNil)
(context :Object)
:HNil
= {
HNil()
}
implicit def runAll[Input <:HList, Output <:HList]
(input :Int::Input)
(context :Object)
(implicit run :Input=>Object=>Output)
:Int::Output
= {
HCons(0, run(input.tail)(context))
}
def runAny[Input <:HList, Output <:HList]
(input :Input)
(context :Object)
(implicit run :Input=>Object=>Output)
:Output
= {
run(input)(context)
}
}
sealed trait HList
final case class HCons[Head, Tail <:HList]
(head :Head, tail :Tail)
extends HList
{
def ::[Value](value :Value) = HCons(value, this)
}
final case class HNil()
extends HList
{
def ::[Value](value :Value) = HCons(value, this)
}
object HList extends HApplyOps {
type ::[Head, Tail <:HList] = HCons[Head, Tail]
}
class Test {
def main(args :Array[String]) {
HList.runAny( HNil())(null) // yay! ;P
HList.runAny(0::HNil())(null) // fail :(
}
}
This code, compiled with Scala 2.9.0.1, returns the following error:
broken1.scala:53: error: No implicit view available from HCons[Int,HNil] => (java.lang.Object) => Output.
HList.runAny(0::HNil())(null)
My expectation in this case is that runAll would be bound to the implicit run argument to runAny.
Now, if I modify runAll so that, instead of taking its two arguments directly, it instead returns a function that in turn takes those two arguments (a trick I thought to try as I saw it in someone else's code), it works:
2) modified code that has the same runtime behavior and actually works
implicit def runAll[Input <:HList, Output <:HList]
(implicit run :Input=>Object=>Output)
:Int::Input=>Object=>Int::Output
= {
input =>
context =>
HCons(0, run(input.tail)(context))
}
In essence, my question is: why does this work? ;( I would expect that these two functions have the same overall type signature:
1: [Input <:HList, Output <:HList] (Int::Input)(Object):Int::Output
2: [Input <:Hlist, Output <:HList] :Int::Input=>Object=>Int::Output
If it helps understand the problem, some other changes also "work" (although these change the semantics of the function, and therefore are not usable solutions):
3) hard-coding runAll for only a second level by replacing Output with HNil
implicit def runAll[Input <:HList, Output <:HList]
(input :Int::Input)
(context :Object)
(implicit run :Input=>Object=>HNil)
:Int::HNil
= {
HCons(0, run(input.tail)(context))
}
4) removing the context argument from the implicit functions
trait HApplyOps {
implicit def runNil
(input :HNil)
:HNil
= {
HNil()
}
implicit def runAll[Input <:HList, Output <:HList]
(input :Int::Input)
(implicit run :Input=>Output)
:Int::Output
= {
HCons(0, run(input.tail))
}
def runAny[Input <:HList, Output <:HList]
(input :Input)
(context :Object)
(implicit run :Input=>Output)
:Output
= {
run(input)
}
}
Any explanation anyone may have for this would be much appreciated. :(
(Currently, my best guess is that the order of the implicit argument with respect to the other arguments is the key factor that I'm missing, but one that I'm confused by: runAny has an implicit argument at the end as well, so the obvious "implicit def doesn't work well with trailing implicit" doesn't make sense to me.)
When you declare an implicit def like this:
implicit def makeStr(i: Int): String = i.toString
then the compiler can automatically create an implicit Function object from this definition for you, and will insert it where an implicit of type Int => String is expected. This is what happens in your line HList.runAny(HNil())(null).
But when you define implicit defs which themselves accept implicit parameters (like your runAll method), it doesn't work any more, as the compiler cannot create a Function object whose apply method would require an implicit — much less guarantee that such an implicit would be available at the call site.
The solution to this is to define something like this instead of runAll:
implicit def runAllFct[Input <: HList, Output <: HList]
(implicit run: Input => Object => Output):
Int :: Input => Object => Int :: Output =
{ input: Int :: Input =>
context: Object =>
HCons(0, run(input.tail)(context))
}
This definition is a bit more explicit, as the compiler now won't need to try to create a Function object from your def, but will instead call your def directly to get the needed function object. And, while calling it, will automatically insert the needed implicit parameter, which it can resolve right away.
In my opinion, whenever you expect implicit functions of this type, you should provide an implicit def that does indeed return a Function object. (Other users may disagree… anyone?) The fact that the compiler is able to create Function wrappers around an implicit def is there mainly, I suppose, to support implicit conversions with a more natural syntax, e.g. using view bounds together with simple implicit defs like my first Int to String conversion.
(Note: this is a summary of the discussion that took place in possibly more detail in the comments section of another answer on this question.)
It turns out that the problem here is that the implicit parameter is not first in runAny, but not because the implicit binding mechanism is ignoring it: instead, the issue is that the type parameter Output is not bound to anything, and needs to be indirectly inferred from the type of the run implicit parameter, which is happening "too late".
In essence, the code for "undetermined type parameters" (which is what Output is in this circumstance) only gets used in situations where the method in question is considered to be "implicit", which is determined by its direct parameter list: in this case, runAny's parameter list is actually just (input :Input), and isn't "implicit".
So, the type parameter for Input manages to work (getting set to Int::HNil), but Output is simply set to Nothing, which "sticks" and causes the type of the run argument to be Int::HNil=>Object=>Nothing, which is not satisfiable by runNil, causing runAny's type inferencing to fail, disqualifying it for usage as an implicit argument to runAll.
By reorganizing the parameters as done in my modified code sample #2, we make runAny itself be "implicit", allowing it to first get its type parameters fully determined before applying its remaining arguments: this happens because its implicit argument will first get bound to runNil (or runAny again for more than two levels), whose return type will get taken/bound.
To tie up the loose ends: the reason that the code sample #3 worked in this situation is that the Output parameter wasn't even required: the fact that it was bound to Nothing didn't affect any subsequent attempts to bind it to anything or use it for anything, and runNil was easily chosen to bind to its version of the run implicit parameter.
Finally, code sample #4 was actually degenerate, and shouldn't even have been considered to "work" (I had only verified that it compiled, not that it generated the appropriate output): the data types of its implicit parameters were so simplistic (Input=>Output, where Input and Output were actually intended to be the same type) that it would simply get bound to conforms:<:<[Input,Output]: a function that in turn acted as the identity in this circumstance.
(For more information on the #4 case, see this apparently dead-on related question: problem with implicit ambiguity between my method and conforms in Predef.)