If I have some computation that takes a while I might place it in a scala.concurrent.Future:
val f = Future { someLongRunningFunction() }
and let's say I want to do something else asynchronously once that computation is completed:
f.flatMap{ _ => anotherLongRunningFunction() }
In the event that f's initial block fails, how do I "idiomatically" handle this when using flatMap or other combinators? Is it merely a case of using recover or onFailure before the flatMap?
I like the elegance and simplicity of using a flatMap but it seems failure scenarios get in the way of it.
Edit: the second future is reliant on the first, hence the flatMap. I'm looking for a solution that'll elegantly let me chain like I would with flatMap but also handle failures of the first.
To quote the scaladoc for flatMap:
Creates a new future by applying a function to the successful result
of this future, and returns the result of the function as the new
future. If this future is completed with an exception then the new
future will also contain this exception.
Notice the bold, meaning that whatever you pass to flatMap will only be executed if the initial future completes successfully.
So you should only handle the result of the entire execution:
val result = future1.flatMap {
result => functionReturningFuture2(result)
}
and then:
result.onFailure // or
result.onSuccess // or
result.recover
If you have several futures you can put them in a for comprehension.
val futureResult = for {
result1 <- future1
result2 <- future2
...
} yield {
//computation with results
}
You can add a recover at the end in case you want to process any exception you may find:
futureResult.recover{
case exceptionResult: Throwable => // Process exception
}
I think this is more clean that using flatMap.
I'm only starting with Future, but here are some ideas.
If you really want to use flatMap, you have to turn the failure into a success.
for{ a <- f recover r
b <- another(a)
} yield b
This works if the return type of r is :> the result type of f.
Or you can pass the problem of what to do with the failure on to the next process
for{ a <- f map (x => Success(x)) recover (ex => Failure(ex))
b <- another(a)
} yield b
Here the argument type of another would be Try[T] where the type of f is Future[T].
Related
I am running into an issue with Scala and multiple futures in a for yield scenario. Both f1 and f2 are futures. f2 future is based on a value obtained from f1 future.
val result = for {
f1 <- Await.result(dao.findNode("nodeA"), 5 seconds) // This returns back a MyNode class
f2 <- if (f1 != None && f1.isUpAndRunning)
Future { LookupResult(true, f1.varA, f1.varB) }
else
lk ? Lookup(sm.id, sm.("address"))
} yield(f1, f2)
Dependent on the result of f1 I either do a Lookup() when my if statement evaluates to false (which takes some time and returns back a LookupResult) or I mimic a default LookupResult.
Getting the following error back:
Type mismatch. Required: Option[B_], found: Future[(MyNode, Any)]
Am I just not mapping the result correctly? e.g. should I use asInstanceOf somewhere as whatever I do I cannot get this to compile.
Many thanks guys.
Chances are your dao.findNode method results in a Future[Option[MyNode]], so the Await.result makes f1 an Option[MyNode]. It's not that clear what type you want, but you can get a Future[(Option[MyNode], LookupResult)] with this
dao.findNode("nodeA"), 5.seconds)
.flatMap { nodeOpt: Option[MyNode] =>
nodeOpt.filter(_.isUpAndRunning)
.map { node =>
// node exists and is up and running
val finalResult = nodeOpt -> LookupResult(true, node.varA, node.varB)
// it's much more efficient in this scenario to use
// Future.successful than Future { }
Future.successful(finalResult)
}
.getOrElse {
// node didn't exist, or wasn't up and running
val lookupFut = lk ? Lookup(sm.id, sm.("address"))
lookupFut.map { lookupResult =>
nodeOpt -> lookupResult
}
}
}
The rough outline of what's happening is:
use dao to find the node
schedule a callback for when we get the (possibly nonexistent) node to check if it's up and running (in which case we short-circuit to our now-known final result), otherwise perform the lookup and build a final result if that lookup succeeds
Future.successful is used because Future {} will almost certainly incur more overhead scheduling a task on a thread and completing the future than constructing the final result.
I've got what seems like a fairly simple situation here:
I create a sequence of futures with yield of signature Future[(String, Date)]
I use Future.sequence to turn the IndexedSeq[Future[(String, Date)]] into a Future[IndexedSeq[(String, Date)]]
I call onComplete on that future to do something with each future after all of them have completed.
I'm doing this because the first wave are database inserts, I can't do the updateCalculatedField call until all inserts are completed, because updateCalculatedField involves aggregate functions and I don't want to introduce a race condition. Here's the code:
val initialInserts = for(i <- 0 until responseData.numValues()) yield { ... }
val res = Future.sequence(initialInserts)
res.onComplete{
case Success(seq) =>
for((i, d) <- seq)
updateCalculatedField(i, d)
case Failure(e) => e.printStackTrace()
}
What exactly am I doing wrong here? The code within the onComplete call never executes, and the program exits with no complaints. I've checked, and res.size is equal to the amount of rows in my database after the program exits... so the callback should have been executed. I'm at a loss here.
Thanks.
I have the following snippet in Scala using Scalaz and Task/EitherT:
def myMethod(request: MyRequest) : EitherT[Task, Failure, Success] =
EitherT[Task, Failure, Success](
for {
aamOpt <- async.findUser(request)
aamEith <- Task.now(aamOpt.\/>(NoUserFound()))
result <- aamEith.fold(err => Task.now(-\/(err)), aam => process(aam)).run)
} yield result)
Where async.findUser returns a Task[Option[MyUser]] and process(aam) returns EitherT[Task, Failure, Success]. These return types are needed as these methods interact (down the line) with external services.
Is there any way to simplify the last line of the for comprehension into something nicer? I mean this one:
result <- aamEith.fold(err => Task.now(-\/(err)), aam => process(aam)).run)
I would probably immediately lift the result from async.findUser into the relevant EitherT monad and then just flatMap:
def myMethod(request: MyRequest): EitherT[Task, Failure, Success] =
EitherT.eitherT[Task, Failure, MyUser](
async.findUser(request).map(_.\/>(NoUserFound()))
).flatMap(process)
This is untested but something like it should more or less work.
I am struggling to understand how this code 'strips out' the Future.
getFutureResult() returns a Future[String]. So how come the return from the yield is only a String?
def getFutureResult:Future[String] = { ... }
def getMyResult:String = {
for {
myFutureResult <- getFutureResult()
} yield {
myFutureResult
}
}
It is translated according to Scala for-comprehension translation rules:
for (x <- <expr>) yield f(x)
becomes:
<expr>.map(x => f(x))
This is a desugaring done by the compiler irregardless if <expr> has the type of a collection, a future or something else.
Future in Scala has a method map, so it uses the myFutureResult String from above to construct another Future[String].
You never know if the resulting future getMyResult is completed -- you should install an onComplete callback that will be called by the resulting future once it completes:
getMyResult onComplete {
case Success(x) => println("Yes! " + x)
case Failure(t) => println("An error: " + t)
}
The code for the onComplete method is executed asynchronously -- it might happen on a different thread, much later or simultaneously.
If you really need to know if the Future has completed, use the:
Await(getMyResult, 0 nanos)
pattern to block the calling thread until the getMyResult future is completed. In general, you should avoid this and compose your Future code using for-comprehensions and callback.
This is a follow-up to my previous question
Suppose I want to create a future with my function but don't want to start it immediately (i.e. I do not want to call val f = Future { ... // my function}.
Now I see it can be done as follows:
val p = promise[Unit]
val f = p.future map { _ => // my function here }
Is it the only way to create a future with my function w/o executing it?
You can do something like this
val p = Promise[Unit]()
val f = p.future
//... some code run at a later time
p.success {
// your function
}
LATER EDIT:
I think the pattern you're looking for can be encapsulated like this:
class LatentComputation[T](f: => T) {
private val p = Promise[T]()
def trigger() { p.success(f) }
def future: Future[T] = p.future
}
object LatentComputation {
def apply[T](f: => T) = new LatentComputation(f)
}
You would use it like this:
val comp = LatentComputation {
// your code to be executed later
}
val f = comp.future
// somewhere else in the code
comp.trigger()
You could always defer creation with a closure, you'll not get the future object right ahead, but you get a handle to call later.
type DeferredComputation[T,R] = T => Future[R]
def deferredCall[T,R](futureBody: T => R): DeferredComputation[T,R] =
t => future {futureBody(t)}
def deferredResult[R](futureBody: => R): DeferredComputation[Unit,R] =
_ => future {futureBody}
If you are getting too fancy with execution control, maybe you should be using actors instead?
Or, perhaps, you should be using a Promise instead of a Future: a Promise can be passed on to others, while you keep it to "fulfill" it at a later time.
It's also worth giving a plug to Promise.completeWith.
You already know how to use p.future onComplete mystuff.
You can trigger that from another future using p completeWith f.
You can also define a function that creates and returns the Future, and then call it:
val double = (value: Int) => {
val f = Future { Thread.sleep(1000); value * 2 }
f.onComplete(x => println(s"Future return: $x"))
f
}
println("Before future.")
double(2)
println("After future is called, but as the future takes 1 sec to run, it will be printed before.")
I used this to executes futures in batches of n, something like:
// The functions that returns the future.
val double = (i: Int) => {
val future = Future ({
println(s"Start task $i")
Thread.sleep(1000)
i * 2
})
future.onComplete(_ => {
println(s"Task $i ended")
})
future
}
val numbers = 1 to 20
numbers
.map(i => (i, double))
.grouped(5)
.foreach(batch => {
val result = Await.result( Future.sequence(batch.map{ case (i, callback) => callback(i) }), 5.minutes )
println(result)
})
Or just use regular methods that return futures, and fire them in series using something like a for comprehension (sequential call-site evaluation)
This well known problem with standard libraries Future: they are designed in such a way that they are not referentially transparent, since they evaluate eagerly and memoize their result. In most use cases, this is totally fine and Scala developers rarely need to create non-evaluated future.
Take the following program:
val x = Future(...); f(x, x)
is not the same program as
f(Future(...), Future(...))
because in the first case the future is evaluated once, in the second case it is evaluated twice.
The are libraries which provide the necessary abstractions to work with referentially transparent asynchronous tasks, whose evaluation is deferred and not memoized unless explicitly required by the developer.
Scalaz Task
Monix Task
fs2
If you are looking to use Cats, Cats effects works nicely with both Monix and fs2.
this is a bit of a hack, since it have nothing to do with how future works but just adding lazy would suffice:
lazy val f = Future { ... // my function}
but note that this is sort of a type change as well, because whenever you reference it you will need to declare the reference as lazy too or it will be executed.