waiting for "recursive" futures in scala - scala

a simple code sample that describes my problem:
import scala.util._
import scala.concurrent._
import scala.concurrent.duration._
import ExecutionContext.Implicits.global
class LoserException(msg: String, dice: Int) extends Exception(msg) { def diceRoll: Int = dice }
def aPlayThatMayFail: Future[Int] = {
Thread.sleep(1000) //throwing a dice takes some time...
//throw a dice:
(1 + Random.nextInt(6)) match {
case 6 => Future.successful(6) //I win!
case i: Int => Future.failed(new LoserException("I did not get 6...", i))
}
}
def win(prefix: String): String = {
val futureGameLog = aPlayThatMayFail
futureGameLog.onComplete(t => t match {
case Success(diceRoll) => "%s, and finally, I won! I rolled %d !!!".format(prefix, diceRoll)
case Failure(e) => e match {
case ex: LoserException => win("%s, and then i got %d".format(prefix, ex.diceRoll))
case _: Throwable => "%s, and then somebody cheated!!!".format(prefix)
}
})
"I want to do something like futureGameLog.waitForRecursiveResult, using Await.result or something like that..."
}
win("I started playing the dice")
this simple example illustrates what i want to do. basically, if to put it in words, i want to wait for a result for some computation, when i compose different actions on previous success or failed attampts.
so how would you implement the win method?
my "real world" problem, if it makes any difference, is using dispatch for asynchronous http calls, where i want to keep making http calls whenever the previous one ends, but actions differ on wether the previous http call succeeded or not.

You can recover your failed future with a recursive call:
def foo(x: Int) = x match {
case 10 => Future.successful(x)
case _ => Future.failed[Int](new Exception)
}
def bar(x: Int): Future[Int] = {
foo(x) recoverWith { case _ => bar(x+1) }
}
scala> bar(0)
res0: scala.concurrent.Future[Int] = scala.concurrent.impl.Promise$DefaultPromise#64d6601
scala> res0.value
res1: Option[scala.util.Try[Int]] = Some(Success(10))
recoverWith takes a PartialFunction[Throwable,scala.concurrent.Future[A]] and returns a Future[A]. You should be careful though, because it will use quite some memory when it does lots of recursive calls here.

As drexin answered the part about exception handling and recovering, let me try and answer the part about a recursive function involving futures. I believe using a Promise will help you achieve your goal. The restructured code would look like this:
def win(prefix: String): String = {
val prom = Promise[String]()
def doWin(p:String) {
val futureGameLog = aPlayThatMayFail
futureGameLog.onComplete(t => t match {
case Success(diceRoll) => prom.success("%s, and finally, I won! I rolled %d !!!".format(prefix, diceRoll))
case Failure(e) => e match {
case ex: LoserException => doWin("%s, and then i got %d".format(prefix, ex.diceRoll))
case other => prom.failure(new Exception("%s, and then somebody cheated!!!".format(prefix)))
}
})
}
doWin(prefix)
Await.result(prom.future, someTimeout)
}
Now this won't be true recursion in the sense that it will be building up one long stack due to the fact that the futures are async, but it is similar to recursion in spirit. Using the promise here gives you something to block against while the recursion does it's thing, blocking the caller from what's happening behind the scene.
Now, if I was doing this, I would probable redefine things like so:
def win(prefix: String): Future[String] = {
val prom = Promise[String]()
def doWin(p:String) {
val futureGameLog = aPlayThatMayFail
futureGameLog.onComplete(t => t match {
case Success(diceRoll) => prom.success("%s, and finally, I won! I rolled %d !!!".format(prefix, diceRoll))
case Failure(e) => e match {
case ex: LoserException => doWin("%s, and then i got %d".format(prefix, ex.diceRoll))
case other => prom.failure(new Exception("%s, and then somebody cheated!!!".format(prefix)))
}
})
}
doWin(prefix)
prom.future
}
This way you can defer the decision on whether to block or use async callbacks to the caller of this function. This is more flexible, but it also exposes the caller to the fact that you are doing async computations and I'm not sure that is going to be acceptable for your scenario. I'll leave that decision up to you.

This works for me:
def retryWithFuture[T](f: => Future[T],retries:Int, delay:FiniteDuration) (implicit ec: ExecutionContext, s: Scheduler): Future[T] ={
f.recoverWith { case _ if retries > 0 => after[T](delay,s)(retryWithFuture[T]( f , retries - 1 , delay)) }
}

Related

Propagate errors through a chain of Scala futures

Considering a sequence of futures each returning Either[Status, Resp].
How would you propagate error status codes through a for comprehension which is using Future and not Either?
The code bellow does not work, since the parsing exception is not caught by .recover of the last future
The use case is Scala Play ActionRefiners which returns Future[Either[Status, TRequest[A]]].
def parseId(id: String):Future[Int] = {
Future.successful(Integer.parseInt(id))
}
def getItem(id: Int)(implicit ec: ExecutionContext): Future[Either[Status, String]] =
Future(Some("dummy res from db " + id)).transformWith {
case Success(opt) => opt match {
case Some(item) => Future.successful(Right(item))
case _ => Future.successful(Left(NotFound))
}
case Failure(_) => Future.successful(Left(InternalServerError))
}
(for {
id <- parseId("bad request")
resp <- getItem(id)
} yield resp).recover {
case _:NumberFormatException => Left(BadRequest)
}
I could move the .recover to parseId, but this makes the for comprehension very ugly - having to treat the Either[Status, id] in the middle
def parseId(id: String):Future[Either[Status, Int]] = {
Future.successful(Right(Integer.parseInt(id))).recover {
case _:NumberFormatException => Left(BadRequest)
}
}
Your exception is not caught because you are not throwing it inside the Future: Future.successful is immediately satisfied with the result of the expression you give it, if it throws an exception, it is executed on the current thread.
Try removing the .successful: Future(id.toInt) will do what you want.
Also, I would recommend to get rid of all the Eithers: these are highly overrated/overused, especially in the context of Future (that already wrap their result into Try anyhow), and just make the code more complicated and less readable without offering much benefit.
case class FailureReason(status: Status)
extends Exception(status.toString)
def notFound() = throw FailureReason(NotFound)
def internalError() = throw FailureReason(InternalError)
def badRequest() = throw FailureReason(BadRequest)
def parseId(id: String):Future[Int] = Future(id.toInt)
def getItem(id: Int): Future[String] = Future(Some("dummy"))
.map { _.getOrElse(notFound) }
.recover { _ => internalError }
// this is the same as your for-comprehension, just looking less ugly imo :)
parseId("foo").flatMap(getItem).recover {
case _: NumberFormatException => badRequest()
}
// if you still want `Either` in the end for some reason:
.map(Right.apply[Status, String])
.recover {
case _: NumberFormatException => Left(BadRequest) // no need for the first recover above if you do this
case FailureReason(status) => Left(status)
}

How to write an asynchronous code that Awaited concisely?

I'm a beginner in Scala.
Please let me know if there is a more concise part in the code below.
To supplement, I'd like to call each Future method synchronously.
◆getUser method:
def getUser: Option[User] = {
Await.ready(
twitterService.getUser(configService.getString(TWITTER_USERNAME_CONF)),
Duration.Inf)
.value
.flatMap(x => Option(x.getOrElse(null)))
}
◆ process method:
def process : Unit =
for {
user <- getUser
} yield {
Await.ready(
twitterService.delete(user.id, configService.getString(TWITTER_SEARCH_KEYWORD)),
Duration.Inf)
.value
.foreach {
case Success(tweets) => tweets.foreach(tweet => println(s"Delete Successfully!!. $tweet"))
case Failure(exception) => println(s"Failed Delete.... Exception:[$exception]")
}
}
I made some assumptions on user and tweet data types but I would rewrite that to:
def maybeDeleteUser(userName: String, maybeUser: Option[User]): Future[String] =
maybeUser match {
case Some(user) =>
twitterService.delete(user.id, configService.getString(TWITTER_SEARCH_KEYWORD)).map {
case Failure(exception) => s"Failed Delete.... Exception:[${exception.getMessage}]"
case Success(tweets) => tweets.map(tweet => s"Delete Successfully!!. $tweet").mkString(System.lineSeparator())
}
case _ => Future.successful(s"Failed to find user $userName")
}
def getStatusLogMessage: Future[String] = {
val userName = configService.getString(TWITTER_USERNAME_CONF)
for {
maybeUser <- twitterService.getUser(configService.getString(TWITTER_USERNAME_CONF))
statusLogMessage <- maybeDeleteUser(userName, maybeUser)
} yield statusLogMessage
}
def process: Unit = {
val message = Await.result(getStatusLogMessage, Duration.Inf)
println(message)
}
That way your side effect, i.e. println is isolated and other methods can be unit tested. If you need to block the execution, do it only at the end and use map and flatMap to chain Futures if you need to order the execution of those. Also be careful with Duration.Inf, if you really need to block, then you'd want to have some defined timeout.

Scala Futures for-comprehension with a list of values

I need to execute a Future method on some elements I have in a list simultaneously. My current implementation works sequentially, which is not optimal for saving time. I did this by mapping my list and calling the method on each element and processing the data this way.
My manager shared a link with me showing how to execute Futures simultaneously using for-comprehension but I cannot see/understand how I can implement this with my List.
The link he shared with me is https://alvinalexander.com/scala/how-use-multiple-scala-futures-in-for-comprehension-loop/
Here is my current code:
private def method1(id: String): Tuple2[Boolean, List[MyObject]] = {
val workers = List.concat(idleWorkers, activeWorkers.keys.toList)
var ready = true;
val workerStatus = workers.map{ worker =>
val option = Await.result(method2(worker), 1 seconds)
var status = if (option.isDefined) {
if (option.get._2 == id) {
option.get._1.toString
} else {
"INVALID"
}
} else "FAILED"
val status = s"$worker: $status"
if (option.get._1) {
ready = false
}
MyObject(worker.toString, status)
}.toList.filterNot(s => s. status.contains("INVALID"))
(ready, workerStatus)
}
private def method2(worker: ActorRef): Future[Option[(Boolean, String)]] = Future{
implicit val timeout: Timeout = 1 seconds;
Try(Await.result(worker ? GetStatus, 1 seconds)) match {
case Success(extractedVal) => extractedVal match {
case res: (Boolean, String) => Some(res)
case _ => None
}
case Failure(_) => { None }
case _ => { None }
}
}
If someone could suggest how to implement for-comprehension in this scenario, I would be grateful. Thanks
For method2 there is no need for the Future/Await mix. Just map the Future:
def method2(worker: ActorRef): Future[Option[(Boolean, String)]] =
(worker ? GetStatus).map{
case res: (Boolean, String) => Some(res)
case _ => None
}
For method1 you likewise need to map the result of method2 and do the processing inside the map. This will make workerStatus a List[Future[MyObject]] and means that everything runs in parallel.
Then use Future.sequence(workerStatus) to turn the List[Future[MyObject]] into a Future[List[MyObject]]. You can then use map again to do the filtering/ checking on that List[MyObject]. This will happen when all the individual Futures have completed.
Ideally you would then return a Future from method1 to keep everything asynchronous. You could, if absolutely necessary, use Await.result at this point which would wait for all the asynchronous operations to complete (or fail).

Scala Future error handling with Either

I am writing a wrapper for an API and I want to do error handling for applications problems. Each request returns a Future so in order to do this I see 2 options: using a Future[Either] or using exceptions to fail the future immediately.
Here is a snippet with both situations, response is a future with the return of the HTTP request:
def handleRequestEither: Future[Either[String, String]] = {
response.map {
case "good_string" => Right("Success")
case _ => Left("Failed")
}
}
def handleRequest: Future[String] = {
response.map {
case "good_string" => "Success"
case _ => throw new Exception("Failed")
}
}
And here is the snippet to get the result in both cases:
handleRequestEither.onComplete {
case Success(res) =>
res match {
case Right(rightRes) => println(s"Success $res")
case Left(leftRes) => println(s"Failure $res")
}
case Failure(ex) =>
println(s"Failure $ex")
}
handleRequest.onComplete {
case Success(res) => println(s"Success $res")
case Failure(ex) => println(s"Failure $ex")
}
I don't like to use exceptions, but using Future[Either] makes it much more verbose to get the response afterwards, and if I want to map the result into another object it gets even more complicated. Is this the way to go, or are there better alternatives?
Let me paraphrase Erik Meijer and consider the following table:
Consider then this two features of a language construct: arity (does it aggregate one or many items?) and mode (synchronous when blocking read operations until ready or asynchronous when not).
All of this imply that Try constructs and blocks manage the success or failure of the block generating the result synchronously. You'll control whether your resources provides the right answer without encountering problems (those described by exceptions).
On the other hand a Future is a kind of asynchronous Try. That means that it successfully completes when no problems (exceptions) has been found then notifying its subscribers. Hence, I don't think you should have a future of Either in this case, that is your second handleRequest implementation is the right way of using futures.
Finally, if what disturbs you is throwing an exception, you could follow the approach of Promises:
def handleRequest: Future[String] = {
val p = Promise[String]
response.map {
case "good_string" => p.success("Success")
case _ => p.failure(new Exception("Failed"))
}
p.future
}
Or:
case class Reason(msg: String) extends Exception
def handleRequest: Future[String] = {
val p = Promise[String]
response.map {
case "good_string" => p.success("Success")
case _ => p.failure(Reason("Invalid response"))
}
p.future
}
I'd rather use your second approach.
You could use special type for that: EitherT from the scalaz library.
It works with scalaz enhanced version of Either : \/
It could transform combination of any monad and \/ into a single monad. So using scalaz instances for scala.concurent.Future you could achieve the desired mix. And you could go further with monad transformers if you wish. Read this beautiful blog if you're interested.
Here not prettified but working with scalaz 7.1 example for you:
import scala.concurrent.duration.Duration
import scala.concurrent.{Await, Future}
import scalaz._
import scalaz.std.scalaFuture._
import EitherT._
import scala.concurrent.ExecutionContext.Implicits.global
object EitherFuture {
type ETFS[X] = EitherT[Future, String, X]
val IntResponse = "result (\\d+)".r
def parse(response: Future[String]) =
eitherT(response map {
case IntResponse(num) ⇒ \/-(num.toInt)
case _ ⇒ -\/("bad response")
})
def divideBy2(x: Validation[String, Int]) =
x.ensure("non divisible by 2")(_ % 2 == 0).map(_ / 2)
def handleResponse(response: Future[String]) = for {
num ← parse(response).validationed(divideBy2)
} yield s"half is $num"
def main(args: Array[String]) {
Map(
'good → "result 10",
'proper → "result 11",
'bad → "bad_string"
) foreach { case (key, str) ⇒
val response = Future(str)
val handled = handleResponse(response)
val result = Await.result(handled.run, Duration.Inf)
println(s"for $key response we have $result")
}
}
}

Executing a code block even after catching an exception in Scala

In my current method, I am trying to make a series of calls and if any of them fail, I want to be able to continue running the remainder (while capturing the Exception that was thrown). I am having a hard time figuring this out in Scala.
So in this example, I want to kick off each of these calls - RunA, RunB and RunC but if RunB throws an exception, I want to print that and continue kicking off RunC after that.
var result = Try {
new RunA()
new RunB()
new RunC()
} catch {
case e: Throwable => e.printStackTrace()
false
}
Outside of having them all individually wrapped in a Try/Catch, I am sure there are better ways to do this which is why I am hoping someone can help with this.
I looked at the 'Ignoring' exception but it appears to completely ignore the exception which I want to atleast log.
Thanks!
First, don't mix try { ... } catch { ... } up with scala.util.Try{ ... }.
You can
import scala.util._
val runA = Try{ new RunA }
val runB = Try{ new RunB }
val runC = Try{ new RunC }
and then deal with the exceptions as you see fit. For instance, if you want to print and continue, you could deal with the try statements right there:
def getOrPrint[A](f: => A): Option[A] = Try{ f } match {
case Success(x) => Some(x)
case Failure(e) => e.printStackTrace; None
}
getOrPrint{ new RunA }
...
There can be more elegant ways for such things with scalaz (e.g. read an article here for some inspiration: http://johnkurkowski.com/posts/accumulating-multiple-failures-in-a-ValidationNEL/), but with "only" Scala you can do something like this:
import scala.reflect.ClassTag
import scala.util.{Try, Success, Failure}
def tryAndLog[T: ClassTag] = Try {
implicitly[ClassTag[T]].runtimeClass.newInstance.asInstanceOf[T] // new instance
} match {
case Success(_) => true
case Failure(ex) => ex.printStackTrace ; false
}
def tryRunAll = {
val A = tryAndLog[RunA]
val B = tryAndLog[RunB]
val C = tryAndLog[RunC]
A && B && C // returns true if all invocations succeeded, false otherwise
}
You are mixing scala.util.Try with try {} catch {} which are different concepts. Try wraps function into Success(result) or Failure(error) class, and try-catch is like Java try-catch. I suggest you something like this:
class RunA
class RunB
class RunC
class Result(a: RunA, b: RunB, c: RunC)
implicit class LogFailure[T](t: Try[T]) {
def logFailure: Try[T] = t match {
case scala.util.Failure(err) => err.printStackTrace(); t
case _ => t
}
}
val tryA= Try(new RunA())
val tryB= Try(new RunB())
val tryC = Try(new RunC())
val result: Try[Result] = for {
a <- tryA.logFailure
b <- tryB.logFailure
c <- tryC.logFailure
} yield {
// do smth with a, b, c
new Result(a, b, c)
}
If A, B, C will be successful you'll get Success(Result) if one of them failure you'll get Failure with first exception, however all of them will be logged (printed stack trace)