I am a newbie to scala futures and I have a doubt regarding the return value of scala futures.
So, generally syntax for a scala future is
def downloadPage(url: URL) = Future[List[Int]] {
}
I want to know how to access the List[Int] from some other method which calls this method.
In other words,
val result = downloadPage("localhost")
then what should be the approach to get List[Int] out of the future ?
I have tried using map method but not able to do this successfully.`
The case of Success(listInt) => I want to return the listInt and I am not able to figure out how to do that.
The best practice is that you don't return the value. Instead you just pass the future (or a version transformed with map, flatMap, etc.) to everyone who needs this value and they can add their own onComplete.
If you really need to return it (e.g. when implementing a legacy method), then the only thing you can do is to block (e.g. with Await.result) and you need to decide how long to await.
You need to wait for the future to complete to get the result given some timespan, here's something that would work:
import scala.concurrent.duration._
def downloadPage(url: URL) = Future[List[Int]] {
List(1,2,3)
}
val result = downloadPage("localhost")
val myListInt = result.result(10 seconds)
Ideally, if you're using a Future, you don't want to block the executing thread, so you would move your logic that deals with the result of your Future into the onComplete method, something like this:
result.onComplete({
case Success(listInt) => {
//Do something with my list
}
case Failure(exception) => {
//Do something with my error
}
})
I hope you already solved this since it was asked in 2013 but maybe my answer can help someone else:
If you are using Play Framework, it support async Actions (actually all Actions are async inside). An easy way to create an async Action is using Action.async(). You need to provide a Future[Result]to this function.
Now you can just make transformations from your Future[List[Int]] to Future[Result] using Scala's map, flatMap, for-comprehension or async/await. Here an example from Play Framework documentation.
import play.api.libs.concurrent.Execution.Implicits.defaultContext
def index = Action.async {
val futureInt = scala.concurrent.Future { intensiveComputation() }
futureInt.map(i => Ok("Got result: " + i))
}
You can do something like that. If The wait time that is given in Await.result method is less than it takes the awaitable to execute, you will have a TimeoutException, and you need to handle the error (or any other error).
import scala.concurrent._
import ExecutionContext.Implicits.global
import scala.util.{Try, Success, Failure}
import scala.concurrent.duration._
object MyObject {
def main(args: Array[String]) {
val myVal: Future[String] = Future { silly() }
// values less than 5 seconds will go to
// Failure case, because silly() will not be done yet
Try(Await.result(myVal, 10 seconds)) match {
case Success(extractedVal) => { println("Success Happened: " + extractedVal) }
case Failure(_) => { println("Failure Happened") }
case _ => { println("Very Strange") }
}
}
def silly(): String = {
Thread.sleep(5000)
"Hello from silly"
}
}
The best way I’ve found to think of a Future is a box that will, at some point, contain the thing that you want. The key thing with a Future is that you never open the box. Trying to force open the box will lead you to blocking and grief. Instead, you put the Future in another, larger box, typically using the map method.
Here’s an example of a Future that contains a String. When the Future completes, then Console.println is called:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
object Main {
def main(args:Array[String]) : Unit = {
val stringFuture: Future[String] = Future.successful("hello world!")
stringFuture.map {
someString =>
// if you use .foreach you avoid creating an extra Future, but we are proving
// the concept here...
Console.println(someString)
}
}
}
Note that in this case, we’re calling the main method and then… finishing. The string’s Future, provided by the global ExecutionContext, does the work of calling Console.println. This is great, because when we give up control over when someString is going to be there and when Console.println is going to be called, we let the system manage itself. In constrast, look what happens when we try to force the box open:
val stringFuture: Future[String] = Future.successful("hello world!")
val someString = Future.await(stringFuture)
In this case, we have to wait — keep a thread twiddling its thumbs — until we get someString back. We’ve opened the box, but we’ve had to commandeer the system’s resources to get at it.
It wasn't yet mentioned, so I want to emphasize the point of using Future with for-comprehension and the difference of sequential and parallel execution.
For example, for sequential execution:
object FuturesSequential extends App {
def job(n: Int) = Future {
Thread.sleep(1000)
println(s"Job $n")
}
val f = for {
f1 <- job(1)
f2 <- job(2)
f3 <- job(3)
f4 <- job(4)
f5 <- job(5)
} yield List(f1, f2, f3, f4, f5)
f.map(res => println(s"Done. ${res.size} jobs run"))
Thread.sleep(6000) // We need to prevent main thread from quitting too early
}
And for parallel execution (note that the Future are before the for-comprehension):
object FuturesParallel extends App {
def job(n: Int) = Future {
Thread.sleep(1000)
println(s"Job $n")
}
val j1 = job(1)
val j2 = job(2)
val j3 = job(3)
val j4 = job(4)
val j5 = job(5)
val f = for {
f1 <- j1
f2 <- j2
f3 <- j3
f4 <- j4
f5 <- j5
} yield List(f1, f2, f3, f4, f5)
f.map(res => println(s"Done. ${res.size} jobs run"))
Thread.sleep(6000) // We need to prevent main thread from quitting too early
}
Related
I'm converting Future code to IO. I have code similar to this
def doSomething: Future[Foo] = {
Future {
//some code the result of which we don't care about
}
Future {
//Foo
}
}
And then at the end of the program, I doSomething.unsafeRunSync. How do I convert these Futures to IOs while maintaining the fire-and-forget functionality of the first Future? In using IO's async API, I am worried about accidentally blocking the thread when I later call unsafeRunSync on doSomething.
A solution that uses only cats-effect could use IO.start. This, combined with the fact that you will then never join the resulting Fiber, will look something like this:
import cats.effect._
import cats.implicits._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
object ExampleApp extends App{
val fireAndForget =
IO(println("Side effect pre-sleep")) *>
IO.sleep(2.seconds) *>
IO(println("Side effect post-sleep"))
val toBeUsed = IO{
println("Inside second one")
42
}
val result = for {
fiber <- IO.shift *> fireAndForget.start
res <- toBeUsed.handleErrorWith { error =>
// This is just in case you 'toBeUsed' can actually fail,
// and you might want to cancel the original side-effecting IO
fiber.cancel *> IO.raiseError(error) }
} yield res
println(result.unsafeRunSync())
println("Waiting 3 seconds...")
IO.sleep(3.seconds).unsafeRunSync()
println("Done")
}
This will print (most of the times) something similar to:
Side effect pre-sleep
Inside second one
42 // Up until here, will be printed right away
Waiting 3 seconds... // It will then be waiting a while
Side effect post-sleep // ...at which point the side effecting code terminates
Done
Finally, here are more details about Fiber and IO.shift
I believe that you need to wrap the first Future in such a way that it completes immediately. We ignore exeptions, or catch them however, but they are contained within its own thread. The parameter cb is the promise that needs to complete; so we short-circuit the completion by providing a value immediately.
def firstFuture(implicit ec: ExecutionContext): IO[Unit] = {
IO.async[Unit] { cb =>
ec.execute(() => {
try {
//some code the result of which we don't care about
} catch {
}
})
cb(Right(()))
}
}
In the for-comprehension, the firstFuture will complete immediately even though its thread will have a long-running task active on it.
def doSomething(implicit ec: ExecutionContext): IO[Foo] = {
for {
_ <- firstFuture
IO.async[Foo] { fb =>
// Foo
}
}
}
My code which uses mapAync(1) doesn't work what I want it to do. But when I changed the mapAsync(1) to map by using Await.result, it works. So I have a question.
Does the following (A) Use map and (B) use mapAsync(1) yield the same result at anytime?
// (A) Use map
someSource
.map{r =>
val future = makeFuture(r) // returns the same future if r is the same
Await.result(future, Duration.Inf)
}
// (B) Use mapAsync(1)
someSource
.mapAsync(1){r =>
val future = makeFuture(r) // returns the same future if r is the same
future
}
Actually, I want to paste my real code, but it is too long to paste and has some dependencies of my original stages.
While semantically the type of both streams ends up being the same (Source[Int, NotUsed]), the style displayed in example (A) is very very bad – please don't block (Await) inside streams.
Such cases are exactly the use case for mapAsync. Your operation returns a Future[T], and you want to push that value downwards through the stream once the future completes. Please note that there is no blocking in mapAsync, it schedules a callback to push the value of the future internally and does so once it completes.
To answer your question about "do they do the same thing?", technically yes but the first one will cause performance issues in the threadpool you're running on, avoid map+blocking when mapAsync can do the job.
These calls are semantically very similar, although blocking by using Await is probably not a good idea. The type signature of both these calls is, of course, the same (Source[Int, NotUsed]), and in many cases these calls will produce the same results (blocking aside). The following, for example, which includes scheduled futures and a non-default supervision strategy for failures, gives the same results for both map with an Await inside and mapAsync:
import akka.actor._
import akka.stream.ActorAttributes.supervisionStrategy
import akka.stream.Supervision.resumingDecider
import akka.stream._
import akka.stream.scaladsl._
import scala.concurrent._
import scala.concurrent.duration._
import scala.language.postfixOps
object Main {
def main(args: Array[String]) {
implicit val system = ActorSystem("TestSystem")
implicit val materializer = ActorMaterializer()
import scala.concurrent.ExecutionContext.Implicits.global
import system.scheduler
def makeFuture(r: Int) = {
akka.pattern.after(2 seconds, scheduler) {
if (r % 3 == 0)
Future.failed(new Exception(s"Failure for input $r"))
else
Future(r + 100)
}
}
val someSource = Source(1 to 20)
val mapped = someSource
.map { r =>
val future = makeFuture(r)
Await.result(future, Duration.Inf)
}.withAttributes(supervisionStrategy(resumingDecider))
val mappedAsync = someSource
.mapAsyncUnordered(1) { r =>
val future = makeFuture(r)
future
}.withAttributes(supervisionStrategy(resumingDecider))
mapped runForeach println
mappedAsync runForeach println
}
}
It is possible that your upstream code is relying on the blocking behaviour in your map call in some way. Can you produce a concise reproduction of the issue that you are seeing?
I was hoping code like follows would wait for both futures, but it does not.
object Fiddle {
val f1 = Future {
throw new Throwable("baaa") // emulating a future that bumped into an exception
}
val f2 = Future {
Thread.sleep(3000L) // emulating a future that takes a bit longer to complete
2
}
val lf = List(f1, f2) // in the general case, this would be a dynamically sized list
val seq = Future.sequence(lf)
seq.onComplete {
_ => lf.foreach(f => println(f.isCompleted))
}
}
val a = FuturesSequence
I assumed seq.onComplete would wait for them all to complete before completing itself, but not so; it results in:
true
false
.sequence was a bit hard to follow in the source of scala.concurrent.Future, I wonder how I would implement a parallel that waits for all original futures of a (dynamically sized) sequence, or what might be the problem here.
Edit: A related question: https://worldbuilding.stackexchange.com/questions/12348/how-do-you-prove-youre-from-the-future :)
One common approach to waiting for all results (failed or not) is to "lift" failures into a new representation inside the future, so that all futures complete with some result (although they may complete with a result that represents failure). One natural way to get that is lifting to a Try.
Twitter's implementation of futures provides a liftToTry method that makes this trivial, but you can do something similar with the standard library's implementation:
import scala.util.{ Failure, Success, Try }
val lifted: List[Future[Try[Int]]] = List(f1, f2).map(
_.map(Success(_)).recover { case t => Failure(t) }
)
Now Future.sequence(lifted) will be completed when every future is completed, and will represent successes and failures using Try.
And so, a generic solution for waiting on all original futures of a sequence of futures may look as follows, assuming an execution context is of course implicitly available.
import scala.util.{ Failure, Success, Try }
private def lift[T](futures: Seq[Future[T]]) =
futures.map(_.map { Success(_) }.recover { case t => Failure(t) })
def waitAll[T](futures: Seq[Future[T]]) =
Future.sequence(lift(futures)) // having neutralized exception completions through the lifting, .sequence can now be used
waitAll(SeqOfFutures).map {
// do whatever with the completed futures
}
A Future produced by Future.sequence completes when either:
all the futures have completed successfully, or
one of the futures has failed
The second point is what's happening in your case, and it makes sense to complete as soon as one of the wrapped Future has failed, because the wrapping Future can only hold a single Throwable in the failure case. There's no point in waiting for the other futures because the result will be the same failure.
This is an example that supports the previous answer. There is an easy way to do this using just the standard Scala APIs.
In the example, I am creating 3 futures. These will complete at 5, 7, and 9 seconds respectively. The call to Await.result will block until all futures have resolved. Once all 3 futures have completed, a will be set to List(5,7,9) and execution will continue.
Additionally, if an exception is thrown in any of the futures, Await.result will immediately unblock and throw the exception. Uncomment the Exception(...) line to see this in action.
try {
val a = Await.result(Future.sequence(Seq(
Future({
blocking {
Thread.sleep(5000)
}
System.err.println("A")
5
}),
Future({
blocking {
Thread.sleep(7000)
}
System.err.println("B")
7
//throw new Exception("Ha!")
}),
Future({
blocking {
Thread.sleep(9000)
}
System.err.println("C")
9
}))),
Duration("100 sec"))
System.err.println(a)
} catch {
case e: Exception ⇒
e.printStackTrace()
}
Even though it is quite old question But this is how I got it running in recent time.
object Fiddle {
val f1 = Future {
throw new Throwable("baaa") // emulating a future that bumped into an exception
}
val f2 = Future {
Thread.sleep(3000L) // emulating a future that takes a bit longer to complete
2
}
val lf = List(f1, f2) // in the general case, this would be a dynamically sized list
val seq = Future.sequence(lf)
import scala.concurrent.duration._
Await.result(seq, Duration.Inf)
}
This won't get completed and will wait till all the future gets completed. You can change the waiting time as per your use case. I have kept it to infinite and that was required in my case.
We can enrich Seq[Future[T]] with its own onComplete method through an implicit class:
def lift[T](f: Future[T])(implicit ec: ExecutionContext): Future[Try[T]] =
f map { Success(_) } recover { case e => Failure(e) }
def lift[T](fs: Seq[Future[T]])(implicit ec: ExecutionContext): Seq[Future[Try[T]]] =
fs map { lift(_) }
implicit class RichSeqFuture[+T](val fs: Seq[Future[T]]) extends AnyVal {
def onComplete[U](f: Seq[Try[T]] => U)(implicit ec: ExecutionContext) = {
Future.sequence(lift(fs)) onComplete {
case Success(s) => f(s)
case Failure(e) => throw e // will never happen, because of the Try lifting
}
}
}
Then, in your particular MWE, you can do:
val f1 = Future {
throw new Throwable("baaa") // emulating a future that bumped into an exception
}
val f2 = Future {
Thread.sleep(3000L) // emulating a future that takes a bit longer to complete
2
}
val lf = List(f1, f2)
lf onComplete { _ map {
case Success(v) => ???
case Failure(e) => ???
}}
This solution has the advantage of allowing you to call an onComplete on a sequence of futures as you would on a single future.
Create the Future with a Try to avoid extra hoops.
implicit val ec = ExecutionContext.global
val f1 = Future {
Try {
throw new Throwable("kaboom")
}
}
val f2 = Future {
Try {
Thread.sleep(1000L)
2
}
}
Await.result(
Future.sequence(Seq(f1, f2)), Duration("2 sec")
) foreach {
case Success(res) => println(s"Success. $res")
case Failure(e) => println(s"Failure. ${e.getMessage}")
}
I'm confused about how Scala Future works. Can someone explain what's wrong with code:
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
val numbers = List(1,2,3,4,5,6,7,8,9,10)
def showNumbers() = {
numbers
}
val futueNumber = future {
showNumbers.filter((a: Int) => a % 2 == 0)
}
futueNumber onSuccess {
case resultList => resultList
}
futueNumber
I expect to receive a result:
List(1,2,3,4,5,6,7,8,9,10)
But REPL outputs:
res1: scala.concurrent.Future[List[Int]] = scala.concurrent.impl.Promise$DefaultPromise#3644febc
Please, give me a short hint, how to fix this
The simple answer is that you need to await the future for you example to work:
val result = Await.result(futureNumber, 1.second)
However, the purpose of future's is not to block on asynchronous work, as Await does, but instead to use the result of the future when becomes available.
The key to using future is understanding that it does not actually hold the value of your computation, but rather is promise that at some point in the future, it will provide that value.
The usual use case is a request coming into a server that requires some work, which may take some time and itself wait on IO, like this:
def receiveMsg(msg: String, caller: Caller) = {
doSomeAsyncWork(msg).map(response => caller.send(response)
}
I.e. you are called with an object that has a send method that you want to call when doSomeAsyncWork is done, but you don't want to block the current thread until that future completes. Instead you map the future, which means that response => caller.send(response) will be called for you when doSomeAsyncWork is done.
futureNumber is of type Future[List[Int]], since you defined it as such in
val futureNumber = future {
showNumbers.filter((a: Int) => a % 2 == 0)
}
Therefore, when you print futureNumber, that is what gets printed. The variable does not get reassigned to the future value! If you want to print the value of the future, you need to do it inside the onSucess callback:
futureNumber onSuccess {
case resultList => println(resultList)
}
Alternatively, you can block waiting for the future with
import scala.concurrent.ExecutionContext.Implicits.global
import duration._
val theList = scala.concurrent.Await.result(futureNumber, 10 seconds)
println(theList)
Although this might not work if done in the REPL.
The whole point of the Future type is that it acts as a marker that alerts you to the presence of concurrency and should help you handle it better.
I am trying to use a divide-and-conquer (aka fork/join) approach for a number crunching problem. Here is the code:
import scala.actors.Futures.future
private def compute( input: Input ):Result = {
if( pairs.size < SIZE_LIMIT ) {
computeSequential()
} else {
val (input1,input2) = input.split
val f1 = future( compute(input1) )
val f2 = future( compute(input2) )
val result1 = f1()
val result2 = f2()
merge(result1,result2)
}
}
It runs (with a nice speed-up) but the the future apply method seems to block a thread and the thread pool increases tremendously. And when too many threads are created, the computations is stucked.
Is there a kind of react method for futures which releases the thread ? Or any other way to achieve that behavior ?
EDIT: I am using scala 2.8.0.final
Don't claim (apply) your Futures, since this forces them to block and wait for an answer; as you've seen this can lead to deadlocks. Instead, use them monadically to tell them what to do when they complete. Instead of:
val result1 = f1()
val result2 = f2()
merge(result1,result2)
Try this:
for {
result1 <- f1
result2 <- f2
} yield merge(result1, result2)
The result of this will be a Responder[Result] (essentially a Future[Result]) containing the merged results; you can do something effectful with this final value using respond() or foreach(), or you can map() or flatMap() it to another Responder[T]. No blocking necessary, just keep scheduling computations for the future!
Edit 1:
Ok, the signature of the compute function is going to have to change to Responder[Result] now, so how does that affect the recursive calls? Let's try this:
private def compute( input: Input ):Responder[Result] = {
if( pairs.size < SIZE_LIMIT ) {
future(computeSequential())
} else {
val (input1,input2) = input.split
for {
result1 <- compute(input1)
result2 <- compute(input2)
} yield merge(result1, result2)
}
}
Now you no longer need to wrap the calls to compute with future(...) because they're already returning Responder (a superclass of Future).
Edit 2:
One upshot of using this continuation-passing style is that your top-level code--whatever calls compute originally--doesn't block at all any more. If it's being called from main(), and that's all the program does, this will be a problem, because now it will just spawn a bunch of futures and then immediately shut down, having finished everything it was told to do. What you need to do is block on all these futures, but only once, at the top level, and only on the results of all the computations, not any intermediate ones.
Unfortunately, this Responder thing that's being returned by compute() no longer has a blocking apply() method like the Future did. I'm not sure why flatMapping Futures produces a generic Responder instead of a Future; this seems like an API mistake. But in any case, you should be able to make your own:
def claim[A](r:Responder[A]):A = {
import java.util.concurrent.ArrayBlockingQueue
import scala.actors.Actor.actor
val q = new ArrayBlockingQueue[A](1)
// uses of 'respond' need to be wrapped in an actor or future block
actor { r.respond(a => q.put(a)) }
return q.take
}
So now you can create a blocking call to compute in your main method like so:
val finalResult = claim(compute(input))