onComplete method in Scala's Fututre monod - scala

I am learning Future in Scala and have the following code snippet. I am generating random marks based on the first letter in the name.
For the following scenario, I expect a list to be printed by the onComplete method. But it does not print anything.
def randomMark(name:String) = name(0) match {
case 'R'|'A'|'J'|'S' => Thread.sleep(500); 99
case _ => Thread.sleep(500); 80
}
import scala.concurrent._
import concurrent.ExecutionContext.Implicits.global
val returns = Future sequence List( Future(randomMark("Rob")), Future(randomMark("Andy")), Future(randomMark("George")) )
Thread.sleep(550)
returns onComplete { e => { val y1 = e.getOrElse("Error"); println(y1) } }
//This println statement does not execute. I expect a list List(99,99,80) to be printed
Can someone please help me to understand why the functional literal I supply for onComplete method does not execute?
Thanks!

Most likely because you need to wait for the result because onComplete is an async operation.
import scala.concurrent.duration._
import scala.concurrent._
def randomMark(name:String) = name(0) match {
case 'R'|'A'|'J'|'S' => Thread.sleep(500); 99
case _ => Thread.sleep(500); 80
}
import scala.concurrent._
import concurrent.ExecutionContext.Implicits.global
val returns = Future sequence List( Future(randomMark("Rob")), Future(randomMark("Andy")), Future(randomMark("George")) )
Thread.sleep(550)
returns onComplete { e => { val y1 = e.getOrElse("Error"); println(y1) } }
println("Waiting futures to be completed")
Await.ready(returns, 5.seconds)
println("Futures to be completed")
Print's out:
Waiting futures to be completed
List(99, 99, 80)
Futures to be completed
Scatie: https://scastie.scala-lang.org/SWv18p8RTtuo7nMNHNHMoQ

Since callbacks are executed asynchronously, you'll need to wait for your callback to complete before the program exits. In this case, if you want to wait for an onComplete callback, you need to signal its completion somehow. In the following example I'm using a Promise:
def randomMark(name:String) = name(0) match {
case 'R'|'A'|'J'|'S' => Thread.sleep(500); 99
case _ => Thread.sleep(500); 80
}
import scala.concurrent._
import scala.concurrent.duration._
import ExecutionContext.Implicits.global
val returns = Future sequence List( Future(randomMark("Rob")), Future(randomMark("Andy")), Future(randomMark("George")) )
val returns: scala.concurrent.Future[List[Int]] = Future(<not completed>)
val p = Promise[Unit]()
returns onComplete { e => { val y1 = e.getOrElse("Error"); println(y1); p.success(()) } }
Await.ready(p.future, 5.seconds)
But, you could, instead use andThen as follows:
def randomMark(name:String) = name(0) match {
case 'R'|'A'|'J'|'S' => Thread.sleep(500); 99
case _ => Thread.sleep(500); 80
}
import scala.concurrent._
import scala.concurrent.duration._
import ExecutionContext.Implicits.global
val returns =
Future sequence List( Future(randomMark("Rob")), Future(randomMark("Andy")), Future(randomMark("George")) ) andThen
{ case e => { val y1 = e.getOrElse("Error"); println(y1) } }
Await.ready(returns, 5.seconds)

Related

Does cats mapN run all the futures in parallel?

As mentioned in the jump-start guide, mapN will run all the futures in parallel, so I created the below simple Scala program, but a sample run shows diff to be 9187 ms and diffN to be 9106 ms. So it looks like that the mapN is also running the futures sequentially, isn't it? Please let me know if I am missing something?
package example
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import java.time.LocalDateTime
import java.time.Duration
import scala.util.Failure
import scala.util.Success
import java.time.ZoneOffset
import cats.instances.future._
import cats.syntax.apply._
object FutureEx extends App {
val before = LocalDateTime.now()
val sum = for {
a <- getA
b <- getB
c <- getC
} yield (a + b + c)
sum onComplete {
case Failure(ex) => println(s"Error: ${ex.getMessage()}")
case Success(value) =>
val after = LocalDateTime.now()
println(s"before: $before")
println(s"after: $after")
val diff = getDiff(before, after)
println(s"diff: $diff")
println(s"sum: $value")
}
// let the above finish
Thread.sleep(20000)
val beforeN = LocalDateTime.now()
val usingMapN = (getA, getB, getC).mapN(add)
usingMapN onComplete {
case Failure(ex) => println(s"Error: ${ex.getMessage()}")
case Success(value) =>
val afterN = LocalDateTime.now()
println(s"beforeN: $beforeN")
println(s"afterN: $afterN")
val diff = getDiff(beforeN, afterN)
println(s"diffN: $diff")
println(s"sum: $value")
}
def getA: Future[Int] = {
println("inside A")
Thread.sleep(3000)
Future.successful(2)
}
def getB: Future[Int] = {
println("inside B")
Thread.sleep(3000)
Future.successful(3)
}
def getC: Future[Int] = {
println("inside C")
Thread.sleep(3000)
Future.successful(4)
}
def add(a: Int, b: Int, c: Int) = a + b + c
def getDiff(before: LocalDateTime, after: LocalDateTime): Long = {
Duration.between(before.toInstant(ZoneOffset.UTC), after.toInstant(ZoneOffset.UTC)).toMillis()
}
}
Because you have sleep outside Future it should be like:
def getA: Future[Int] = Future {
println("inside A")
Thread.sleep(3000)
2
}
So you start async Future with apply - Future.successful on the other hand returns pure value, meaning you execute sleep in same thread.
The time is going before mapN is ever called.
(getA, getB, getC).mapN(add)
This expression is creating a tuple (sequentially) and then calling mapN on it. So it is calling getA then getB then getC and since each of them has a 3 second delay, it takes 9 seconds.

Sequence of two Future.sequence doesn't work

The objective of this code is to take a sequence of futures, process them with Future.sequence, generate another sequence of futures and process them again with another Future.sequence.
The problem is that it doesn't print anything. What's wrong with this code?
object TestFutures extends App {
def future (i:Int) = Future { i }
def future2 (i:Int) = Future { i * 20 }
val futureResult = (1 to 10).map {
x => future(x)
}
var futureInts = Seq[Future[Int]]()
Future.sequence(futureResult).map{ list =>
list.foreach( y => futureInts = futureInts :+ future2(y))
}
Future.sequence(futureInts).map { list2 =>
list2.foreach( z => println(z))
}
Thread.sleep(5000)
}
It does work, you just have a race condition between the Thread.sleep and the futures finishing execution. You can wait on the futures completion with Await.result:
import scala.concurrent.{Await, Future}
import scala.concurrent.ExecutionContext.Implicits._
import scala.concurrent.duration._
def future(i: Int) = Future.successful(i)
def future2(i: Int) = Future.successful(i * 20)
val futureResult = (1 to 10).map(x => future(x))
val firstResult =
Future
.sequence(futureResult)
.flatMap(list => Future.sequence(list.map(future2))
val sequence = Await.result(firstResult, 10 seconds)
sequence.foreach(println)
Note you should not be synchronously blocking on futures in production code, this is merely for the demonstration that the future does do what you want it to.

unable to compile a code with list of Future

There are some suggestions on StackOverflow on how to handle a list of Futures but I want to try my own approach. But I am unable to compile the following code
I have a list of Futures.
I want to count how many of them passed or failed. I should get (2,1)
I store this in a tuple
The approach I want to take is go through each element of the list. The element of the list is Future[Int]. for each element, I call flatMap which calls the next cycle of recursion (I am assuming that if flatMap gets called then that particular future would have been successful so I increment pass count). Similarly, I want to call next cycle of recursion in recover and increment fail count but I am getting compilation error in it.
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success, Try}
import scala.concurrent.duration._
import scala.language.postfixOps
object ConcurrencyExample extends App {
type pass = Int
type fail = Int
val time = System.currentTimeMillis()
//use recursion to process each Future in the list
def segregate(l:List[Future[Int]]):Future[Tuple2[pass,fail]] = {
def go(l:List[Future[Int]],t:Tuple2[pass,fail]):Future[Tuple2[pass,fail]] = {
l match {
case Nil => Future{t}
//l is List of Future[Int]. flatMap each successful Future
//recover each failed Future
case l::ls => {
l flatMap (x => go(ls, (t._1 + 1, t._2)))
**l.recover({ case e => go(ls, (t._1 + 1, t._2))})**//I get error here
}
}
}
go(l,(0,0))
}
//hardcoded future
val futures2: List[Future[Int]] = List(Future {
1
}, Future {
2
}, Future {
throw new Exception("error")
})
val result = segregate(futures2)
result onComplete {
case Success(v) => println("pp:" + v)
case Failure(v) => println("fp:" + v)
}
Await.result(result,1000 millis)
}
#evan058 is correct about the signature of recover. But you can fix your program by changing recover to recoverWith.
recoverWith is to recover as flatMap is to map.
Here's the complete solution (with minor stylistic improvements):
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success, Try}
import scala.concurrent.duration._
import scala.language.postfixOps
object ConcurrencyExample extends App {
type pass = Int
type fail = Int
val time = System.currentTimeMillis()
//use recursion to process each Future in the list
def segregate[T](fs:List[Future[T]]):Future[(pass,fail)] = {
def go(fs:List[Future[T]],r:Future[(pass,fail)]):Future[(pass,fail)] = fs match {
case Nil => r
case l::ls =>
val fx = l.transform({_ => (1, 0)}, identity).recoverWith[(pass,fail)]({case _: Exception => Future(0, 1) })
for (x <- fx; t <- r; g <- go(ls, Future(t._1+x._1,t._2+x._2))) yield g
}
go(fs,Future((0,0)))
}
//hardcoded future
val futures2 = List(Future(1), Future(2), Future(throw new Exception("error")))
val result = segregate(futures2)
result onComplete {
case Success(v) => println(s"successes: ${v._1}, failures: ${v._2}")
case Failure(v) => v.printStackTrace()
}
Await.result(result,1000 millis)
}
If you look at the docs, the signature of recover is:
def recover[U >: T](pf: PartialFunction[Throwable, U])(implicit executor: ExecutionContext): Future[U]
You are calling recover on l which is a Future[Int] so recover is expecting a U >: Int.
However you are calling go again which has the return type Future[(pass, fail)] which is not >: Int.

on scala, how to wait for a Future.onFailure to complete?

Scala futures:
With Await.result I can wait for the future to complete before terminating the program.
How can I wait for a Future.onFailure to complete before terminating the program?
package test
import scala.concurrent.duration.Duration
import scala.concurrent.{Await, Future}
import scala.concurrent.ExecutionContext.Implicits.global
object Test5 extends App {
def step(i: Int): Future[String] = Future {
Thread.sleep(1000)
if (i == 3) throw new Exception("exception on t3")
Thread.sleep(1000); val s = s"done: $i"; println(s); s
}
val task: Future[String] = step(1).flatMap(_ => step(2).flatMap(_ => step(3)))
task.onFailure {
case e: Throwable => println("task.onFailure. START: " + e); Thread.sleep(3000); println("task.onFailure. END.")
}
try {
val result: String = Await.result(task, Duration.Inf)
println("RESULT: " + result)
} catch {
case e: Throwable => println("Await.result.try catch: " + e)
}
// without this, "task.onFailure. END." does not get printed.
// How can I replace this line with something such as Await.result([the task.onFailure I set previously])
Thread.sleep(5000)
println("END")
}
Note: Instead of using task.onFailure, I could catch the exception on Await.result (as in the example). But I would prefer to use task.onFailure.
Update
A solution is to use transform or recover as proposed by Ryan. In my case, I needed something more, and I've added this alternative answer: on scala, how to wait for a Future.onFailure to complete?
If all you want to do is to sequence a side-effect such that you know that it has been executed I'd recommend using 'andThen'.
onFailure returns Unit. There's no way to block on that.
Either catch the exception on Await.result or use transform or recover to side-effect on the exception.
You can't. Future#onFailure returns Unit so there is no handle for which you can grab to block execution.
Since you're blocking on the completion of task anyway, the catch clause in your code is equivalent to what you would get from onFailure. That is, if the Future fails, the body of catch will be executed before the program exits because it is running on the main thread (whereas onComplete is not).
We can enrich Future with onComplete2 and onCompleteWith2 as follows:
import scala.concurrent.{ExecutionContext, Future, Promise}
import scala.util.Try
object FutureUtils {
implicit class RichFuture[T](val self: Future[T]) {
def onComplete2[U](f: (Try[T]) => U)(implicit executor: ExecutionContext): Future[T] = {
val p = Promise[T]()
self.onComplete { r: Try[T] => f(r); p.complete(r) }
p.future
}
def onCompleteWith2[U](f: (Try[T]) => Future[U])(implicit executor: ExecutionContext): Future[T] = {
val p = Promise[T]()
self.onComplete { r: Try[T] => f(r).onComplete(_ => p.complete(r)) }
p.future
}
}
}
then I case use it as follows:
import scala.concurrent.duration.Duration
import scala.concurrent.{Await, Future}
import scala.concurrent.ExecutionContext.Implicits.global
import FutureUtils._
import scala.util.{Failure, Success, Try}
object Test5 extends App {
def stepf(i: Int): Future[String] = Future { step(i) }
def step(i: Int): String = {
Thread.sleep(1000)
if (i == 3) throw new Exception("exception on t3")
Thread.sleep(1000); val s = s"done: $i"; println(s); s
}
val task1: Future[String] =
stepf(1).flatMap(_ => stepf(2).flatMap(_ => stepf(3)))
// the result of step(10) and step(11) is ignored
val task2: Future[String] = task1.onComplete2((r: Try[String]) => r match {
case Success(s) => step(10)
case Failure(e) => step(11)
})
/*
// If I want to recover (and so, to avoid an exception on Await.result:
val task3 = task2.recover {case e: Throwable => step(12) }
// I can use onCompleteWith2 instead of onComplete2
val task2: Future[String] = task1.onCompleteWith2((r: Try[String]) => r match {
case Success(s) => stepf(10)
case Failure(e) => stepf(11)
})
*/
try {
val result = Await.result(task2, Duration.Inf)
println("RESULT: " + result)
} catch {
// see my comment above to remove this
case e: Throwable => println("Await.result.try catch: " + e)
}
println("END.")
}
The execution is as follows:
done: 1
done: 2
done: 11
Await.result.try catch: java.lang.Exception: exception on t3
END.
If we don't throw the exception at step(3), the execution is as follows. Note that RESULT is "done: 3", not "done: 10":
done: 1
done: 2
done: 3
done: 10
RESULT: done: 3
END.

How to fail promise in Scala

In Scala documentation, there is an example how to pick future which succeeds faster by using promises.
http://docs.scala-lang.org/overviews/core/futures.html#promises
def first[T](f: Future[T], g: Future[T]): Future[T] = {
val p = promise[T]
f onSuccess {
case x => p.trySuccess(x)
}
g onSuccess {
case x => p.trySuccess(x)
}
p.future
}
This function returns the future which succeeds first and if either one of them fails, it never completes.
Is it possible to modify this in a way that even if other future fails, then the second is returned if it's successful and if both of them are successful, then the faster one is picked like the code does now.
You can add this:
f onFailure {
case e =>
g onFailure {
case _ =>
p.failure(e)
}
}
When both futures are failed, this will fail the promise with the same exception as f. You can elaborate on this to create an exception that records the 2 exceptions coming from f and g if necessary.
I recommend you to follow the advice of Alvin Alexander for futures and promises in Scala here
I believe that this is the best way to work with futures
package futures
import scala.concurrent.{Future => ConcurrentTask} // rename
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success}
import Utils.sleep
object FutureAsConcurrentTask extends App {
// run some long-running task (task has type Future[Int] in this example)
val task = ConcurrentTask {
Cloud.executeLongRunningTask
}
// whenever the task completes, execute this code
task.onComplete {
case Success(value) => println(s"Got the callback, value = $value")
case Failure(e) => println(s"D'oh! The task failed: ${e.getMessage}")
}
// do your other work
println("A ..."); sleep(100)
println("B ..."); sleep(100)
println("C ..."); sleep(100)
println("D ..."); sleep(100)
println("E ..."); sleep(100)
println("F ..."); sleep(100)
}
Here is a basic pattern for picking the fastest future or timing out if they all are too slow:
import scala.concurrent._
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{ Success, Failure }
import akka.actor._
import akka.pattern.after
object GetFastestFutureOrTimeout extends App {
val e = new TimeoutException("TimeoutException")
val system = ActorSystem("GetFastestFutureOrTimeout")
val f1 = Future { Thread.sleep(200); "this is f1" }
val f2 = Future { Thread.sleep(100); "this is f2" }
val timeoutFuture = after(500.milliseconds, using = system.scheduler) { Future.failed(e) }
val f = Future.firstCompletedOf(f1 :: f2 :: timeoutFuture :: Nil)
f onComplete {
case Success(msg) => println(msg)
case Failure(err) => println("An error occured: " + err.getMessage)
}
}
This prints "this is f2". If the timeout of timeoutFuture were changed to 50, it would print "An error occured: TimeoutException".
Under the hood firstCompletedOf uses a Promise to return the value of the first Future that is completed, see https://github.com/scala/scala/blob/v2.11.6/src/library/scala/concurrent/Future.scala#L503.
This one is a basic implementation to pick up the fastest successful response or fail if they all failed:
def getFirstSuccessfulResultOrFail[T](requests: List[Future[T]]): Future[T] = {
val p = Promise[T]()
val countDownLatch = AtomicInt(0)
requests.foreach { f =>
f.onComplete {
case Failure(e) => if (countDownLatch.addAndGet(1) == requests.size) p.tryFailure(e)
case Success(s) => p.trySuccess(s)
}
}
p.future
}