I want to create a list of Future, each of which could pass or fail and collate results from successful Future. How can I do this?
val futures2:List[Future[Int]] = List(Future{1}, Future{2},Future{throw new Exception("error")})
Questions
1) I want to wait for each future to finish
2) I want to collect sum of return values from each success future and ignore the ones which failed (so I should get 3).
One thing that you need to understand is that... Avoid trying to "get" values from a Future or Futures.
You can keep on operating in the Futuristic land.
val futureList = List(
Future(1),
Future(2),
Future(throw new Exception("error"))
)
// addd 1 to futures
// map will propagate errors to transformed futures
// only successful futures will result in +1, rest will stay with errors
val tranformedFutureList = futureList
.map(future => future.map(i => i + 1))
// print values of futures
// simimlar to map... for each will work only with successful futures
val unitFutureList = futureList
.map(future => future.foreach(i => println(i)))
// now lets give you sum of your "future" values
val sumFuture = futureList
.foldLeft(Future(0))((facc, f) => f.onComplete({
case Success(i) => facc.map(acc => acc + i)
case Failure(ex) => facc
})
And since OP (#Manu Chanda) asked about "getting" a value from a Promise, I am adding some bits about what Promise are in Scala.
So... first lets talk how to think about a Future in Scala.
If you see a Future[Int] then try to think of it as an ongoing computation which is "supposed to produce" an Int. Now that computation can successfully complete and result in a Success[Int] or a throw an exception and result in a Failure[Throwable]. And thus you see the functions such as onComplete, recoverWith, onFailure which seem like talking about a computation.
val intFuture = Future {
// all this inside Future {} is going to run in some other thread
val i = 5;
val j = i + 10;
val k = j / 5;
k
}
Now... what is a Promise.
Well... as the name indicates... a Promise[Int] is a promise of an Int value... nothing more.
Just like when a parent promises a certain toy to their child. Note that in this case... the parent has not necessarily started working on getting that toy, they have just promised that they will.
To complete the promise... they will first have to start working to complete it... got to market... buy from shop... come back home.Or... sometimes... they are busy so... they will ask someone else to bring that toy and keep doing their work... that other guy will try to bring that toy to parent (he may fail to buy it) and then they will complete the promise with whatever result they got from him.
So... basically a Promise wraps a Future inside of it. And that "wrapped" Future "value" can be considered as the value of the Promise.
so...
println("Well... The program wants an 'Int' toy")
// we "promised" our program that we will give it that int "toy"
val intPromise = Promise[Int]()
// now we can just move on with or life
println("Well... We just promised an 'Int' toy")
// while the program can make plans with how will it play with that "future toy"
val intFuture = intPromise.future
val plusOneIntFuture = intFuture.map(i => i + 1)
plusOneIntFuture.onComplete({
case Success(i) => println("Wow... I got the toy and modified it to - " + i)
case Failure(ex) => println("I did not get they toy")
})
// but since we at least want to try to complete our promise
println("Now... I suppose we need to get that 'Int' toy")
println("But... I am busy... I can not stop everything else for that toy")
println("ok... lets ask another thread to get that")
val getThatIntFuture = Future {
println("Well... I am thread 2... trying to get the int")
val i = 1
println("Well... I am thread 2... lets just return this i = 1 thingy")
i
}
// now lets complete our promise with whatever we will get from this other thread
getThatIntFuture.onComplete(intTry => intPromise.complete(intTry))
The above code will result in following output,
Well... The program wants an 'Int' toy
Well... We just promised an 'Int' toy
Now... I suppose we need to get that 'Int' toy
But... I am busy... I can not stop everything else for that toy
Well... I am thread 2... trying to get the int
Well... I am thread 2... lets just return this i = 1 thingy
Wow... I got the toy and modified it to - 2
Promise don't help you in "getting" a value from a Future. Asynchronous processes (or Future in Scala) are just running in another timeline... you can not "get" their "value" in your time-line unless you work on aligning your timeline with the process's time-line itself.
Related
I've got what seems like a fairly simple situation here:
I create a sequence of futures with yield of signature Future[(String, Date)]
I use Future.sequence to turn the IndexedSeq[Future[(String, Date)]] into a Future[IndexedSeq[(String, Date)]]
I call onComplete on that future to do something with each future after all of them have completed.
I'm doing this because the first wave are database inserts, I can't do the updateCalculatedField call until all inserts are completed, because updateCalculatedField involves aggregate functions and I don't want to introduce a race condition. Here's the code:
val initialInserts = for(i <- 0 until responseData.numValues()) yield { ... }
val res = Future.sequence(initialInserts)
res.onComplete{
case Success(seq) =>
for((i, d) <- seq)
updateCalculatedField(i, d)
case Failure(e) => e.printStackTrace()
}
What exactly am I doing wrong here? The code within the onComplete call never executes, and the program exits with no complaints. I've checked, and res.size is equal to the amount of rows in my database after the program exits... so the callback should have been executed. I'm at a loss here.
Thanks.
Let's say I have a ListBuffer[Int] and I iterate it with a foreach loop, and each loop will modify this list from inside a Future (removing the current element), and will do something special when the list is empty. Example code:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.collection.mutable.ListBuffer
val l = ListBuffer(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
l.foreach(n => Future {
println(s"Processing $n")
Future {
l -= n
println(s"Removed $n")
if (l.isEmpty) println("List is empty!")
}
})
This is probably going to end very badly. I have a more complex code with similar structure and same needs, but I do not know how to structure it so I can achieve same functionality in a more reliable way.
The way you present your problem is really not in the functional paradigm that scala is intended for.
What you seem to want, is to do a list of asynchronous computations, do something at the end of each one, and something else when every one is finished. This is pretty simple if you use continuations, which are simple to implement with map and flatMap methods on Future.
val fa: Future[Int] = Future { 1 }
// will apply the function to the result when it becomes available
val fb: Future[Int] = fa.map(a => a + 1)
// will start the asynchronous computation using the result when it will become available
val fc: Future[Int] = fa.flatMap(a => Future { a + 2 })
Once you have all this, you can easily do something when each of your Future completes (successfully):
val myFutures: List[Future[Int]] = ???
myFutures.map(futInt => futInt.map(int => int + 2))
Here, I will add 2 to each value I get from the different asynchronous computations in the List.
You can also choose to wait for all the Futures in your list to complete by using Future.sequence:
val myFutureList: Future[List[Int]] = Future.sequence(myFutures)
Once again, you get a Future, which will be resolved when each of the Futures inside the input list are successfully resolved, or will fail whenever one of your Futures fails. You'll then be able to use map or flatMap on this new Future, to use all the computed values at once.
So here's how I would write the code you proposed:
val l = 1 to 10
val processings: Seq[Future[Unit]] = l.map {n =>
Future(println(s"processing $n")).map {_ =>
println(s"finished processing $n")
}
}
val processingOver: Future[Unit] =
Future.sequence(processings).map { (lu: Seq[Unit]) =>
println(s"Finished processing ${lu.size} elements")
}
Of course, I would recommend having real functions rather than procedures (returning Unit), so that you can have values to do something with. I used println to have a code which will produce the same output as yours (except for the prints, which have a slightly different meaning, since we are not mutating anything anymore).
I'm slowly wrapping my brain around Futures in Scala, and have a bit of a layer cake going on that I'm trying to unravel.
The specific use case is a DeferredResolver in sangria-graphql + akka. I've stolen their demo code, which looks like this
Future.fromTry(Try(
friendIds map (id => CharacterRepo.humans.find(_.id == id) orElse CharacterRepo.droids.find(_.id == id))))
and added my own modification to it. Theirs does an in-memory lookup, whereas mine asks something of another actor:
Future.fromTry(Try(
accountIds match {
case h :: _ =>
val f = sender ? TargetedMessage(h)
val resp = Await.result(f, timeout.duration).asInstanceOf[TargetedMessage]
marshallAccount(resp.body)
case _ => throw new Exception("Not found")
}
))
The pertinent piece here is that I pick the first element in the list, send it to an ActorRef that I got elsewhere and wait for the result. This works. What I'd like to do, however, is not have to wait for the result here, but return the whole thing as a Future
Future.fromTry(Try(
accountIds match {
case h :: _ =>
sender ? TargetedMessage(h) map {
case resp:TargetedMessage => marshallAccount(resp.body)
}
case _ => throw new Exception("Not found")
}
))
This doesn't work. When this is consumed, instead of being of type Account (the return type of function marshallAccount, it's of type Promise. If I understand correctly, it's because instead of having a return type of Future[Account], this has a type of Future[Future[Account]]
How do I flatten this?
You are looking at the wrong API method. Future.fromTry is used to create an immediately resolved Future, meaning the call is not actually asynchronous. Dive into the implementation of Future.fromTry which will take you to:
def fromTry[T](result: Try[T]): Promise[T] = new impl.Promise.KeptPromise[T](result)
A promise kept is basically something that has already happened, so just like Future.successful this is just used to ensure the right return type or similar, it's not actually a way to make something async.
The reason why the return type is Future[Future[Something]] is because you are trying to wrap something that already returns a future into another future.
The ask pattern, namely sender ? TargetMessage(h) is a way to ask something of an actor and await for a result, which will return a future.
The correct way to approach this:
val future: Future[Account] = accountIds match {
case h :: _ => sender ? TargetedMessage(h) map (marshallAccount(_.body)
case _ => Future.failed(throw new Exception("Not found"))
}
Basically you need to use Future.failed to return a failed future from an exception if you want to keep the return type consistent. It's worth reviewing this tutorial to learn a bit more about Futures and how to write application logic with them.
If I have some computation that takes a while I might place it in a scala.concurrent.Future:
val f = Future { someLongRunningFunction() }
and let's say I want to do something else asynchronously once that computation is completed:
f.flatMap{ _ => anotherLongRunningFunction() }
In the event that f's initial block fails, how do I "idiomatically" handle this when using flatMap or other combinators? Is it merely a case of using recover or onFailure before the flatMap?
I like the elegance and simplicity of using a flatMap but it seems failure scenarios get in the way of it.
Edit: the second future is reliant on the first, hence the flatMap. I'm looking for a solution that'll elegantly let me chain like I would with flatMap but also handle failures of the first.
To quote the scaladoc for flatMap:
Creates a new future by applying a function to the successful result
of this future, and returns the result of the function as the new
future. If this future is completed with an exception then the new
future will also contain this exception.
Notice the bold, meaning that whatever you pass to flatMap will only be executed if the initial future completes successfully.
So you should only handle the result of the entire execution:
val result = future1.flatMap {
result => functionReturningFuture2(result)
}
and then:
result.onFailure // or
result.onSuccess // or
result.recover
If you have several futures you can put them in a for comprehension.
val futureResult = for {
result1 <- future1
result2 <- future2
...
} yield {
//computation with results
}
You can add a recover at the end in case you want to process any exception you may find:
futureResult.recover{
case exceptionResult: Throwable => // Process exception
}
I think this is more clean that using flatMap.
I'm only starting with Future, but here are some ideas.
If you really want to use flatMap, you have to turn the failure into a success.
for{ a <- f recover r
b <- another(a)
} yield b
This works if the return type of r is :> the result type of f.
Or you can pass the problem of what to do with the failure on to the next process
for{ a <- f map (x => Success(x)) recover (ex => Failure(ex))
b <- another(a)
} yield b
Here the argument type of another would be Try[T] where the type of f is Future[T].
let's say we have a list of states and we want to sequence them:
import cats.data.State
import cats.instances.list._
import cats.syntax.traverse._
trait MachineState
case object ContinueRunning extends MachineState
case object StopRunning extends MachineState
case class Machine(candy: Int)
val addCandy: Int => State[Machine, MachineState] = amount =>
State[Machine, MachineState] { machine =>
val newCandyAmount = machine.candy + amount
if(newCandyAmount > 10)
(machine, StopRunning)
else
(machine.copy(newCandyAmount), ContinueRunning)
}
List(addCandy(1),
addCandy(2),
addCandy(5),
addCandy(10),
addCandy(20),
addCandy(50)).sequence.run(Machine(0)).value
Result would be
(Machine(10),List(ContinueRunning, ContinueRunning, ContinueRunning, StopRunning, StopRunning, StopRunning))
It's obvious that 3 last steps are redundant. Is there a way to make this sequence stop early? Here when StopRunning gets returned I would like to stop. For example a list of Either's would fail fast and stop sequence early if needed (because it acts like a monad).
For the record - I do know that it is possible to simply write a tail recursion that checks each state that is being runned and if some condition is satisfied - stop the recursion. I just want to know if there is a more elegant way of doing this? The recursion solution seems like a lot of boilerplate to me, am I wrong or not?
Thank you!:))
There are 2 things here needed to be done.
The first is understanding what is actually happening:
State takes some state value, threads in between many composed calls and in the process produces some output value as well
in your case Machine is the state threaded between calls, while MachineState is the output of a single operation
sequence (usually) takes a collection (here List) of some parametric stuff here State[Machine, _] and turns nesting on the left side (here: List[State[Machine, _]] -> State[Machine, List[_]]) (_ is the gap that you'll be filling with your type)
the result is that you'll thread state (Machine(0)) through all the functions, while you combine the output of each of them (MachineState) into list of outputs
// ammonite
// to better see how many times things are being run
# {
val addCandy: Int => State[Machine, MachineState] = amount =>
State[Machine, MachineState] { machine =>
val newCandyAmount = machine.candy + amount
println("new attempt with " + machine + " and " + amount)
if(newCandyAmount > 10)
(machine, StopRunning)
else
(machine.copy(newCandyAmount), ContinueRunning)
}
}
addCandy: Int => State[Machine, MachineState] = ammonite.$sess.cmd24$$$Lambda$2669/1733815710#25c887ca
# List(addCandy(1),
addCandy(2),
addCandy(5),
addCandy(10),
addCandy(20),
addCandy(50)).sequence.run(Machine(0)).value
new attempt with Machine(0) and 1
new attempt with Machine(1) and 2
new attempt with Machine(3) and 5
new attempt with Machine(8) and 10
new attempt with Machine(8) and 20
new attempt with Machine(8) and 50
res25: (Machine, List[MachineState]) = (Machine(8), List(ContinueRunning, ContinueRunning, ContinueRunning, StopRunning, StopRunning, StopRunning))
In other words, what you want is circuit breaking then .sequence might not be what you want.
As a matter of the fact, you probably want something else - combine a list of A => (A, B) functions into one function which stops next computation if the result of a computation is StopRunning (in your code nothing tells the code what is the condition of circuit break and how it should be performed). I would suggest doing it explicitly with some other function, e.g.:
# {
List(addCandy(1),
addCandy(2),
addCandy(5),
addCandy(10),
addCandy(20),
addCandy(50))
.reduce { (a, b) =>
a.flatMap {
// flatMap and map uses MachineState
// - the second parameter is the result after all!
// we are pattern matching on it to decide if we want to
// proceed with computation or stop it
case ContinueRunning => b // runs next computation
case StopRunning => State.pure(StopRunning) // returns current result without modifying it
}
}
.run(Machine(0))
.value
}
new attempt with Machine(0) and 1
new attempt with Machine(1) and 2
new attempt with Machine(3) and 5
new attempt with Machine(8) and 10
res23: (Machine, MachineState) = (Machine(8), StopRunning)
This will eliminate the need for running code within addCandy - but you cannot really get rid of code that combines states together, so this reduce logic will be applied on runtime n-1 times (where n is the size of your list) and that cannot be helped.
BTW If you take a closer look at Either you will find that it also computes n results and only then combines them so that it looks like it's circuit breaking, but in fact isn't. Sequence is combining a result of "parallel" computations but won't interrupt them if any of them failed.