Scala Tail recursion future returns - scala

Question

how to implement a tail recursion functions in scala with futures as return values:
Example code
def getInfo(lists: List[Int]): Future[List[Int]] = {
def getStudentIDs(lists: List[Int]): Future[List[Int]] = {
//here a web service call that returns a future WS response
val response=ws.call(someURL)
response.map(r => {
r.status match {
case 200 =>
var newList = lists + (r.json \ ids)
.as[List[Int]] //here add the ws call response json..
getStudentIDs(newList)
case 429 =>Future.sucessful(lists)
case _ => getStudentIDs(lists)
}
})
}
getStudentIDs(List.empty[Int])
}

Answer 1

I think it's an XY-problem. You probably don't want it "tail-recursive" in the sense of "having a #tailrec"-annotation. What you want is stack safety, so that this method does not blow the stack after a few hundred retries to connect to your webservice.
For this, there are libraries, for example Cats.
In Cats, there is a typeclass called Monad, and this typeclass provides a special method for what seems to be exactly what you want:
tailRecM[A, B](a: A)(f: (A) => F[Either[A, B]]): F[B]
Quote from the docs:
Keeps calling f until a scala.util.Right[B] is returned.
Implementations of this method should use constant stack space [...]
There is an implementation of this for Future in FutureInstances available. The implementation seems trivial though, because it mixes in StackSafeMonad.
You could of course look into implementation of StackSafeMonad, and then try to understand why it is sufficient in the case of Future, but you could also just use the library implementation instead and not worry about whether your recursive method can fail with a StackOverflowError.

Answer 2

Here is the problem (code simplified to make it runnable):
import scala.concurrent._
import scala.concurrent.duration._
import annotation.tailrec
import scala.concurrent.ExecutionContext.Implicits.global
def getInfo(lists: List[Int]): Future[List[Int]] = {
#tailrec
def getStudentIDs(lists: List[Int]): Future[List[Int]] = {
Future(List(1, 2, 3)).flatMap(x => getStudentIDs(x ::: lists))
}
getStudentIDs(List.empty[Int])
}
Gives an error:
error: could not optimize #tailrec annotated method getStudentIDs:
it contains a recursive call not in tail position
Future(1).flatMap(x => getStudentIDs(x :: lists))
^
The problem is not only with a Future. The actual problem is that getStudents is not in terminal/tail position - it's called from map. This would be a problem if you use no Futures and use regular map from collections or any other function for that matter. For example:
def getInfo(lists: List[Int]): List[Int] = {
#tailrec
def getStudentIDs(lists: List[Int]): List[Int] = {
List(1).flatMap(x => getStudentIDs(x :: lists))
}
getStudentIDs(List.empty[Int])
}
Gives the same error:
error: could not optimize #tailrec annotated method getStudentIDs:
it contains a recursive call not in tail position
List(1).flatMap(x => getStudentIDs(x :: lists))
^
What makes it difficult here is that you can't just get a result from future directly to use it in getStudents because you don't know if it's completed and it's not a good practice to block on Future and await for result. So you sort of forced to use map. Here is a very bad example of how to make it tail recursive (just for science :)). Don't do it in production code:
def getInfo(lists: List[Int]): Future[List[Int]] = {
#tailrec
def getStudentIDs(lists: List[Int]): Future[List[Int]] = {
val r = Await.ready(Future(List(1, 2, 3)), Duration.Inf).value.get.getOrElse(lists)
getStudentIDs(r ::: lists)
}
getStudentIDs(List.empty[Int])
}
The compiler is happy, but this might lead to many problems - read about Await, blocking and thread pools for more on that.
I think it's probably not a big issue that your function is not tail recursive because you probably don't want to create lots of futures this way anyway. There are other concurrency frameworks you could try if that's really a problem like actors (Akka), etc.