I was thinking about one of the questions asked here (Why does Scala require a return type for recursive functions?) and how to improve the code.
Anyway, I was thinking something like this:
def simpledb_update(name: String, metadata: Map[String, String]) = {
def inner_update(attempt: int): Unit = {
try {
db(config("simpledb_db")) += (name, metadata)
return
} catch {
case e =>
if (attempt >= 6) {
AUlog(name + ": SimpleDB Failed")
return
}
}
inner_update(attempt+1)
}
inner_update(0)
}
Or
def simpledb_update(name: String, metadata: Map[String, String]) {
def inner_update(attempt: int): Unit = {
try {
db(config("simpledb_db")) += (name, metadata)
} catch {
//Do I need the pattern match, since I don't
// care what exception is thrown???
if (attempt >= 6) {
AUlog(name + ": SimpleDB Failed")
} else {
inner_update(attempt+1)
}
}
}
inner_update(0)
}
Is the second implementation still tail recursive (is the first???). I'm still a bit hazy on when a function is tail recursive, and when it's not.
Yes, both examples are still tail-recursive, you're doing the checking/processing first, with the recursive call last.
The Jargon File put it succinctly: Tail Recursion (n): If you aren't sick of it already, see tail recursion.
In Scala, you can add tailrecursive annotation to the function, and then the compiler will check whether or not it can do tail call optimisation on the function.
Tail recursion is simple to understand - if the last expression, from all code paths of a function F, is simply a function call then it is tail recursive. But is it tail-recursive for Scala compiler to optimize - depends. The fact that JVM doesn't do TCO means scala compiler does the magic.
And the reason scalac might fail to optimize has to do with polymorphism. If the Function can be overridden in a subclass, scalac cannot optimize it out.
#Timo Rantalaiho, using the tailrec annotation is the safest way to verify if TCO can be done.
Related
Why won't the Scala compiler apply tail call optimization unless a method is final?
For example, this:
class C {
#tailrec def fact(n: Int, result: Int): Int =
if(n == 0)
result
else
fact(n - 1, n * result)
}
results in
error: could not optimize #tailrec annotated method: it is neither private nor final so can be overridden
What exactly would go wrong if the compiler applied TCO in a case such as this?
Consider the following interaction with the REPL. First we define a class with a factorial method:
scala> class C {
def fact(n: Int, result: Int): Int =
if(n == 0) result
else fact(n - 1, n * result)
}
defined class C
scala> (new C).fact(5, 1)
res11: Int = 120
Now let's override it in a subclass to double the superclass's answer:
scala> class C2 extends C {
override def fact(n: Int, result: Int): Int = 2 * super.fact(n, result)
}
defined class C2
scala> (new C).fact(5, 1)
res12: Int = 120
scala> (new C2).fact(5, 1)
What result do you expect for this last call? You might be expecting 240. But no:
scala> (new C2).fact(5, 1)
res13: Int = 7680
That's because when the superclass's method makes a recursive call, the recursive call goes through the subclass.
If overriding worked such that 240 was the right answer, then it would be safe for tail-call optimization to be performed in the superclass here. But that isn't how Scala (or Java) works.
Unless a method is marked final, it might not be calling itself when it makes a recursive call.
And that's why #tailrec doesn't work unless a method is final (or private).
UPDATE: I recommend reading the other two answers (John's and Rex's) as well.
Recursive calls might be to a subclass instead of to a superclass; final will prevent that. But why might you want that behavior? The Fibonacci series doesn't provide any clues. But this does:
class Pretty {
def recursivePrinter(a: Any): String = { a match {
case xs: List[_] => xs.map(recursivePrinter).mkString("L[",",","]")
case xs: Array[_] => xs.map(recursivePrinter).mkString("A[",",","]")
case _ => a.toString
}}
}
class Prettier extends Pretty {
override def recursivePrinter(a: Any): String = { a match {
case s: Set[_] => s.map(recursivePrinter).mkString("{",",","}")
case _ => super.recursivePrinter(a)
}}
}
scala> (new Prettier).recursivePrinter(Set(Set(0,1),1))
res8: String = {{0,1},1}
If the Pretty call was tail-recursive, we'd print out {Set(0, 1),1} instead since the extension wouldn't apply.
Since this sort of recursion is plausibly useful, and would be destroyed if tail calls on non-final methods were allowed, the compiler inserts a real call instead.
Let foo::fact(n, res) denote your routine. Let baz::fact(n, res) denote someone else's override of your routine.
The compiler is telling you that the semantics allow baz::fact() to be a wrapper, that MAY upcall (?) foo::fact() if it wants to. Under such a scenario, the rule is that foo::fact(), when it recurs, must activate baz::fact() rather than foo::fact(), and, while foo::fact() is tail-recursive, baz::fact() may not be. At that point, rather than looping on the tail-recursive call, foo::fact() must return to baz::fact(), so it can unwind itself.
What exactly would go wrong if the compiler applied TCO in a case such as this?
Nothing would go wrong. Any language with proper tail call elimination will do this (SML, OCaml, F#, Haskell etc.). The only reason Scala does not is that the JVM does not support tail recursion and Scala's usual hack of replacing self-recursive calls in tail position with goto does not work in this case. Scala on the CLR could do this as F# does.
The popular and accepted answer to this question is actually misleading, because the question itself is confusing. The OP does not make the distinction between tailrec and TCO, and the answer does not address this.
The key point is that the requirements for tailrec are more strict than the requirements for TCO.
The tailrec annotation requires that tail calls are made to the same function, whereas TCO can be used on tail calls to any function.
The compiler could use TCO on fact because there is a call in the tail position. Specifically, it could turn the call to fact into a jump to fact by adjusting the stack appropriately. It does not matter that this version of fact is not the same as the function making the call.
So the accepted answer correctly explains why a non-final function cannot be tailrec because you cannot guarantee that the tail calls are to the same function and not to an overloaded version of the function. But it incorrectly implies that it is not safe to use TCO on this method, when in fact this would be perfectly safe and a good optimisation.
[ Note that, as explained by Jon Harrop, you cannot implement TCO on the JVM, but that is a restriction of the compiler, not the language, and is unrelated to tailrec ]
And for reference, here is how you can avoid the problem without making the method final:
class C {
def fact(n: Int): Int = {
#tailrec
def loop(n: Int, result: Int): Int =
if (n == 0) {
result
} else {
loop(n - 1, n * result)
}
loop(n, 1)
}
}
This works because loop is a concrete function rather than a method and cannot be overridden. This version also has the advantage of removing the spurious result parameter to fact.
This is the pattern I use for all recursive algorithms.
I am trying to create a generic wrapper function that can be wrapped around any method that returns an object. Very similar to the answer in this SO question. I tried the following:
def wrapper_function[T](f: => T): T = {
println("Executing now");
val ret: T = f;
println("Execution finished");
ret
}
def multiply2( x: Int ): Int = wrapper_function {
println("inside multiply2");
return x*2
}
However, I am observing that nothing is getting executed after the function call inside the wrapper function. Specifically, "Execution finished" is not getting printed.
scala> val x = multiply2(4)
Executing now
inside multiply2
x: Int = 8
I am using scala 2.11.8
Am I doing something wrong here? This is puzzling and I would appreciate some help.
I believe your problem is the "return" statement.
Return in scala doesn't work the same as in java. You can take a look in this answer but basically it is something of a "stack unwinding" which would cause you to return from the wrapper function.
Consider that when you do f: => T you are actually taking the block and running it. This block has a return which simply breaks from the wrapper and returns the value (as opposed to not using return in which case its result would be used for the assignment).
In general, if you are using return in scala at the end of a function or block, you are almost always doing something wrong...
Assaf Mendelson's answer is correct for most situations. However, it does not work in scenarios where you don't own the code of the inner function that you are wrapping, or when there is a legitimate case for using return in the inner function (see here)
For those cases, it will work by executing the inner function in a try-finally block:
def wrapper_function[T](f: => T): T = {
println("Executing now");
val ret: T = try f finally {
println("Execution finished");
}
ret
}
I want do a Log annotation use scala meta.Usage simple as:
#Log
def f1(a: Int) {
Future(a + 1) //result is a Future
}
// after parsed ====>
def f1(a: Int) {
Future(a + 1).map{x =>
println(x)
x
}
}
How can I check if f1 is a Future[ _ ] type? Thanks
Besides, I don't want use reflection(performance cost) in Log annotation and I think method result type can be decided at compile time.
This is not possible with scalameta macro annotations. The macro annotations are syntactic so they don't have access to semantic information such as types of trees.
PS. I am not sure if a macro annotation is suitable for this use-case, you could write out the log statement explicitly with something like this
def log[T](x: T): T = { println(x); x }
Future(1).map(log)
or even more concisely with an extension method
implicit class XtensionFuture[T](future: Future[T]) {
def logged: Future[T] = future.map(log)
}
Future(1).logged
Macros should ideally be used as a last resort, not for convenience only.
I don't know if I am doing something wrong or it is just a property of the Scala compiler - I get the mentioned compilation error when I try to compile this code:
#tailrec
def shiftDown2(x: Int, bound: Int) {
val childOfX = chooseChild(x, bound)
for (child <- childOfX) {
swap(x, child)
shiftDown2(child, bound)
}
}
whilst the following code compiles without problems:
#tailrec
def shiftDown(x: Int, bound: Int) {
val childOfX = chooseChild(x, bound)
if (childOfX.isDefined) {
swap(x, childOfX.get)
shiftDown(childOfX.get, bound)
}
}
I believe that above code fragments are semantically the same and both should work with tail recursion.
Tail recursion optimization will not work with recursive invocations inside for loop, because for loop here is just a syntactic sugar for calling foreach higher-order method. So, your code is equivalent to:
#tailrec
def shiftDown2(x: Int, bound: Int) {
val childOfX = chooseChild(x, bound)
childOfX.foreach { child =>
swap(x, child)
shiftDown2(child, bound)
}
}
scalac can optimize tail calls only if recursive method is tail-calling itself directly - by translating it into something similar to while loop in bytecode.
Unfortunately, this is not the case here - shiftDown2 calls childOfX.foreach passing it an anonymous function. Then, foreach (potentially) calls apply on that anonymous function and that anonymous function finally calls shiftDown2 again. So this is an indirect recursion and cannot be optimized by scalac. This limitation has its roots in the JVM, which doesn't have native tail call support.
Why won't the Scala compiler apply tail call optimization unless a method is final?
For example, this:
class C {
#tailrec def fact(n: Int, result: Int): Int =
if(n == 0)
result
else
fact(n - 1, n * result)
}
results in
error: could not optimize #tailrec annotated method: it is neither private nor final so can be overridden
What exactly would go wrong if the compiler applied TCO in a case such as this?
Consider the following interaction with the REPL. First we define a class with a factorial method:
scala> class C {
def fact(n: Int, result: Int): Int =
if(n == 0) result
else fact(n - 1, n * result)
}
defined class C
scala> (new C).fact(5, 1)
res11: Int = 120
Now let's override it in a subclass to double the superclass's answer:
scala> class C2 extends C {
override def fact(n: Int, result: Int): Int = 2 * super.fact(n, result)
}
defined class C2
scala> (new C).fact(5, 1)
res12: Int = 120
scala> (new C2).fact(5, 1)
What result do you expect for this last call? You might be expecting 240. But no:
scala> (new C2).fact(5, 1)
res13: Int = 7680
That's because when the superclass's method makes a recursive call, the recursive call goes through the subclass.
If overriding worked such that 240 was the right answer, then it would be safe for tail-call optimization to be performed in the superclass here. But that isn't how Scala (or Java) works.
Unless a method is marked final, it might not be calling itself when it makes a recursive call.
And that's why #tailrec doesn't work unless a method is final (or private).
UPDATE: I recommend reading the other two answers (John's and Rex's) as well.
Recursive calls might be to a subclass instead of to a superclass; final will prevent that. But why might you want that behavior? The Fibonacci series doesn't provide any clues. But this does:
class Pretty {
def recursivePrinter(a: Any): String = { a match {
case xs: List[_] => xs.map(recursivePrinter).mkString("L[",",","]")
case xs: Array[_] => xs.map(recursivePrinter).mkString("A[",",","]")
case _ => a.toString
}}
}
class Prettier extends Pretty {
override def recursivePrinter(a: Any): String = { a match {
case s: Set[_] => s.map(recursivePrinter).mkString("{",",","}")
case _ => super.recursivePrinter(a)
}}
}
scala> (new Prettier).recursivePrinter(Set(Set(0,1),1))
res8: String = {{0,1},1}
If the Pretty call was tail-recursive, we'd print out {Set(0, 1),1} instead since the extension wouldn't apply.
Since this sort of recursion is plausibly useful, and would be destroyed if tail calls on non-final methods were allowed, the compiler inserts a real call instead.
Let foo::fact(n, res) denote your routine. Let baz::fact(n, res) denote someone else's override of your routine.
The compiler is telling you that the semantics allow baz::fact() to be a wrapper, that MAY upcall (?) foo::fact() if it wants to. Under such a scenario, the rule is that foo::fact(), when it recurs, must activate baz::fact() rather than foo::fact(), and, while foo::fact() is tail-recursive, baz::fact() may not be. At that point, rather than looping on the tail-recursive call, foo::fact() must return to baz::fact(), so it can unwind itself.
What exactly would go wrong if the compiler applied TCO in a case such as this?
Nothing would go wrong. Any language with proper tail call elimination will do this (SML, OCaml, F#, Haskell etc.). The only reason Scala does not is that the JVM does not support tail recursion and Scala's usual hack of replacing self-recursive calls in tail position with goto does not work in this case. Scala on the CLR could do this as F# does.
The popular and accepted answer to this question is actually misleading, because the question itself is confusing. The OP does not make the distinction between tailrec and TCO, and the answer does not address this.
The key point is that the requirements for tailrec are more strict than the requirements for TCO.
The tailrec annotation requires that tail calls are made to the same function, whereas TCO can be used on tail calls to any function.
The compiler could use TCO on fact because there is a call in the tail position. Specifically, it could turn the call to fact into a jump to fact by adjusting the stack appropriately. It does not matter that this version of fact is not the same as the function making the call.
So the accepted answer correctly explains why a non-final function cannot be tailrec because you cannot guarantee that the tail calls are to the same function and not to an overloaded version of the function. But it incorrectly implies that it is not safe to use TCO on this method, when in fact this would be perfectly safe and a good optimisation.
[ Note that, as explained by Jon Harrop, you cannot implement TCO on the JVM, but that is a restriction of the compiler, not the language, and is unrelated to tailrec ]
And for reference, here is how you can avoid the problem without making the method final:
class C {
def fact(n: Int): Int = {
#tailrec
def loop(n: Int, result: Int): Int =
if (n == 0) {
result
} else {
loop(n - 1, n * result)
}
loop(n, 1)
}
}
This works because loop is a concrete function rather than a method and cannot be overridden. This version also has the advantage of removing the spurious result parameter to fact.
This is the pattern I use for all recursive algorithms.