I've got what seems like a fairly simple situation here:
I create a sequence of futures with yield of signature Future[(String, Date)]
I use Future.sequence to turn the IndexedSeq[Future[(String, Date)]] into a Future[IndexedSeq[(String, Date)]]
I call onComplete on that future to do something with each future after all of them have completed.
I'm doing this because the first wave are database inserts, I can't do the updateCalculatedField call until all inserts are completed, because updateCalculatedField involves aggregate functions and I don't want to introduce a race condition. Here's the code:
val initialInserts = for(i <- 0 until responseData.numValues()) yield { ... }
val res = Future.sequence(initialInserts)
res.onComplete{
case Success(seq) =>
for((i, d) <- seq)
updateCalculatedField(i, d)
case Failure(e) => e.printStackTrace()
}
What exactly am I doing wrong here? The code within the onComplete call never executes, and the program exits with no complaints. I've checked, and res.size is equal to the amount of rows in my database after the program exits... so the callback should have been executed. I'm at a loss here.
Thanks.
Related
I'm a beginner in scala and I'm trying to understand this syntax :
val f = Future {
Thread.sleep(Random.nextInt(500))
42
}
println("2- before onComplete")
f.onComplete {
case Success(value) => println(s"Got the callback, meaning = $value")
case Failure(e) => e.printStackTrace
}
Please explain these 2 blocks inside Future{...} and onComplete{...}
Thanks a lot
The first block is equal to something like that:
def code(): Int = {
Thread.sleep(Random.nextInt(500))
42
}
val f = Future(code)
Block acts like an anonymous function but it not reuqires to declare a function, it is just a syntax sugar.
So future f eagerly evaluates contents of a block asynchronously starting at the point of declaration. I.e. if random will yield a number n which is 0 < n < 500, future execution will be suspended for n milliseconds and then resumed and number 42 will be returned and set as a result of a Future.
As futures are non-blocking, while execution context of your future working on your future f, the main thread will run line
println("2- before onComplete")
and you will see this line in the console.
Then, the second block uses pattern matching to create a function that acts from Try[Int] to Unit, as it defined on both Success and Failure constructors of Try, it is an equivalent to:
f.onComplete { ttry =>
ttry match {
case Success(value) => println(s"Got the callback, meaning = $value")
case Failure(e) => e.printStackTrace
}
}
Note that you must await your future till application termination to ensure onComplete block execution.
In scala every statement is expression. Blocks are expressions also. Every expression has return value. These blocks will do something in blocks and return some values.
In Future block, it creates Future object witch contains 2 operations: sleep for random milliseconds and returns 42. In onComplete method block means handling two cases of Future evaluation, on success and on failure statements. If future will throw an error or evaluation will interrupted, Failure case will evaluate. If future evaluated successfully Success case will evaluate. Return value in both cases is Unit it's analogue of void.
I want to create a list of Future, each of which could pass or fail and collate results from successful Future. How can I do this?
val futures2:List[Future[Int]] = List(Future{1}, Future{2},Future{throw new Exception("error")})
Questions
1) I want to wait for each future to finish
2) I want to collect sum of return values from each success future and ignore the ones which failed (so I should get 3).
One thing that you need to understand is that... Avoid trying to "get" values from a Future or Futures.
You can keep on operating in the Futuristic land.
val futureList = List(
Future(1),
Future(2),
Future(throw new Exception("error"))
)
// addd 1 to futures
// map will propagate errors to transformed futures
// only successful futures will result in +1, rest will stay with errors
val tranformedFutureList = futureList
.map(future => future.map(i => i + 1))
// print values of futures
// simimlar to map... for each will work only with successful futures
val unitFutureList = futureList
.map(future => future.foreach(i => println(i)))
// now lets give you sum of your "future" values
val sumFuture = futureList
.foldLeft(Future(0))((facc, f) => f.onComplete({
case Success(i) => facc.map(acc => acc + i)
case Failure(ex) => facc
})
And since OP (#Manu Chanda) asked about "getting" a value from a Promise, I am adding some bits about what Promise are in Scala.
So... first lets talk how to think about a Future in Scala.
If you see a Future[Int] then try to think of it as an ongoing computation which is "supposed to produce" an Int. Now that computation can successfully complete and result in a Success[Int] or a throw an exception and result in a Failure[Throwable]. And thus you see the functions such as onComplete, recoverWith, onFailure which seem like talking about a computation.
val intFuture = Future {
// all this inside Future {} is going to run in some other thread
val i = 5;
val j = i + 10;
val k = j / 5;
k
}
Now... what is a Promise.
Well... as the name indicates... a Promise[Int] is a promise of an Int value... nothing more.
Just like when a parent promises a certain toy to their child. Note that in this case... the parent has not necessarily started working on getting that toy, they have just promised that they will.
To complete the promise... they will first have to start working to complete it... got to market... buy from shop... come back home.Or... sometimes... they are busy so... they will ask someone else to bring that toy and keep doing their work... that other guy will try to bring that toy to parent (he may fail to buy it) and then they will complete the promise with whatever result they got from him.
So... basically a Promise wraps a Future inside of it. And that "wrapped" Future "value" can be considered as the value of the Promise.
so...
println("Well... The program wants an 'Int' toy")
// we "promised" our program that we will give it that int "toy"
val intPromise = Promise[Int]()
// now we can just move on with or life
println("Well... We just promised an 'Int' toy")
// while the program can make plans with how will it play with that "future toy"
val intFuture = intPromise.future
val plusOneIntFuture = intFuture.map(i => i + 1)
plusOneIntFuture.onComplete({
case Success(i) => println("Wow... I got the toy and modified it to - " + i)
case Failure(ex) => println("I did not get they toy")
})
// but since we at least want to try to complete our promise
println("Now... I suppose we need to get that 'Int' toy")
println("But... I am busy... I can not stop everything else for that toy")
println("ok... lets ask another thread to get that")
val getThatIntFuture = Future {
println("Well... I am thread 2... trying to get the int")
val i = 1
println("Well... I am thread 2... lets just return this i = 1 thingy")
i
}
// now lets complete our promise with whatever we will get from this other thread
getThatIntFuture.onComplete(intTry => intPromise.complete(intTry))
The above code will result in following output,
Well... The program wants an 'Int' toy
Well... We just promised an 'Int' toy
Now... I suppose we need to get that 'Int' toy
But... I am busy... I can not stop everything else for that toy
Well... I am thread 2... trying to get the int
Well... I am thread 2... lets just return this i = 1 thingy
Wow... I got the toy and modified it to - 2
Promise don't help you in "getting" a value from a Future. Asynchronous processes (or Future in Scala) are just running in another timeline... you can not "get" their "value" in your time-line unless you work on aligning your timeline with the process's time-line itself.
If I have some computation that takes a while I might place it in a scala.concurrent.Future:
val f = Future { someLongRunningFunction() }
and let's say I want to do something else asynchronously once that computation is completed:
f.flatMap{ _ => anotherLongRunningFunction() }
In the event that f's initial block fails, how do I "idiomatically" handle this when using flatMap or other combinators? Is it merely a case of using recover or onFailure before the flatMap?
I like the elegance and simplicity of using a flatMap but it seems failure scenarios get in the way of it.
Edit: the second future is reliant on the first, hence the flatMap. I'm looking for a solution that'll elegantly let me chain like I would with flatMap but also handle failures of the first.
To quote the scaladoc for flatMap:
Creates a new future by applying a function to the successful result
of this future, and returns the result of the function as the new
future. If this future is completed with an exception then the new
future will also contain this exception.
Notice the bold, meaning that whatever you pass to flatMap will only be executed if the initial future completes successfully.
So you should only handle the result of the entire execution:
val result = future1.flatMap {
result => functionReturningFuture2(result)
}
and then:
result.onFailure // or
result.onSuccess // or
result.recover
If you have several futures you can put them in a for comprehension.
val futureResult = for {
result1 <- future1
result2 <- future2
...
} yield {
//computation with results
}
You can add a recover at the end in case you want to process any exception you may find:
futureResult.recover{
case exceptionResult: Throwable => // Process exception
}
I think this is more clean that using flatMap.
I'm only starting with Future, but here are some ideas.
If you really want to use flatMap, you have to turn the failure into a success.
for{ a <- f recover r
b <- another(a)
} yield b
This works if the return type of r is :> the result type of f.
Or you can pass the problem of what to do with the failure on to the next process
for{ a <- f map (x => Success(x)) recover (ex => Failure(ex))
b <- another(a)
} yield b
Here the argument type of another would be Try[T] where the type of f is Future[T].
I am struggling to understand how this code 'strips out' the Future.
getFutureResult() returns a Future[String]. So how come the return from the yield is only a String?
def getFutureResult:Future[String] = { ... }
def getMyResult:String = {
for {
myFutureResult <- getFutureResult()
} yield {
myFutureResult
}
}
It is translated according to Scala for-comprehension translation rules:
for (x <- <expr>) yield f(x)
becomes:
<expr>.map(x => f(x))
This is a desugaring done by the compiler irregardless if <expr> has the type of a collection, a future or something else.
Future in Scala has a method map, so it uses the myFutureResult String from above to construct another Future[String].
You never know if the resulting future getMyResult is completed -- you should install an onComplete callback that will be called by the resulting future once it completes:
getMyResult onComplete {
case Success(x) => println("Yes! " + x)
case Failure(t) => println("An error: " + t)
}
The code for the onComplete method is executed asynchronously -- it might happen on a different thread, much later or simultaneously.
If you really need to know if the Future has completed, use the:
Await(getMyResult, 0 nanos)
pattern to block the calling thread until the getMyResult future is completed. In general, you should avoid this and compose your Future code using for-comprehensions and callback.
Blocking bad, async good, but is blocking within a future still blocking? This is something that I keep coming back to; consider following pseudo-code:
def queryName(id:Id):Future[String]
def queryEveryonesNames:Future[Seq[String]] = {
val everyonesIds:Future[Seq[Id]] = getIds
val everyonesNames:Future[Seq[Future[String]]] = {
everyonesIds.map(seq.map(id=>queryName(id)))
}
// I'm trying to understand the impact of what I'll do below
everyonesNames.map(seq=>seq.map(fut=>blocking(fut, 1 s)))
}
queryEveryonesNames
In the last line I turned Future[Seq[Future[String]]] (notice future within future) into Future[Seq[String]] by blocking on the inner future.
Blocking on a future within a future feels redundant, at least here, yet having a future within a future feels redundant as well.
Can you propose a smarter way of getting rid of the inner future?
Do you think blocking on a future inside a future is bad? If so why and under what circumstances?
Yes, future blocking is blocking, you should avoid that, as resources will be blocked to wait for a result, even if they are in another thread.
If I understood correctly, your question is how to convert Future[Seq[Future[String]]] into Future[Seq[String]] in a non-blocking way.
You can do that with for-comprehensions:
val in = Future[Seq[Future[String]]]
val m = for( a <- in ) // a is Seq[Future[String]]
yield ( Future.sequence(a)) // yields m = Future[Future[Seq[String]]]
val result = for(a <- m; b <- a) yield (b) // yields Future[Seq[String]]
EDIT:
Or just:
val result = in.flatMap(a => Future.sequence(a))