I have a code block like this:
val await1: List[Int] = await(futureMethod(id))
val mapped = await1.map(entry => {
(pq.id, await(anotherFutureMethod(entry.id)))
})
This fails because of "await must not be used under a nested function" How could I get around this? Why should this be a problem?
I had to guess the signatures of your functions, but an example could look like this:
def futureMethod(id: Int): Future[List[Int]] = Future.successful(0 to id toList)
def anotherFutureMethod(id: Int): Future[String] = Future.successful(id.toString)
def finalFuture(id: Int) = async {
val await1 = await(futureMethod(id))
val mapped = Future.sequence(await1 map anotherFutureMethod)
await(mapped)
}
Using Future.sequence could be a possible, non-blocking solution to avoid using nested await calls.
You'll want to chain the future calls, not block for them. Blocking
futures defies the purpose of a future.
val future1: Future[List[Int]] = futureMethod(id)
val mapped = future1.map(_.flatMap(anotherFutureMethod)
.map(entry => {
(pq.id, entry.id)
}))
Related
I have two scala futures. I want to perform an action once both are completed, regardless of whether they were completed successfully. (Additionally, I want the ability to inspect those results at that time.)
In Javascript, this is Promise.allSettled.
Does Scala offer a simple way to do this?
One last wrinkle, if it matters: I want to do this in a JRuby application.
You can use the transform method to create a Future that will always succeed and return the result or the error as a Try object.
def toTry[A](future: Future[A])(implicit ec: ExecutionContext): Future[Try[A]] =
future.transform(x => Success(x))
To combine two Futures into one, you can use zip:
def settle2[A, B](fa: Future[A], fb: Future[B])(implicit ec: ExecutionContext)
: Future[(Try[A], Try[B])] =
toTry(fa).zip(toTry(fb))
If you want to combine an arbitrary number of Futures this way, you can use Future.traverse:
def allSettled[A](futures: List[Future[A]])(implicit ec: ExecutionContext)
: Future[List[Try[A]]] =
Future.traverse(futures)(toTry(_))
Normally in this case we use Future.sequence to transform a collection of a Future into one single Future so you can map on it, but Scala short circuit the failed Future and doesn't wait for anything after that (Scala considers one failure to be a failure for all), which doesn't fit your case.
In this case you need to map failed ones to successful, then do the sequence, e.g.
val settledFuture = Future.sequence(List(future1, future2, ...).map(_.recoverWith { case _ => Future.unit }))
settledFuture.map(//Here it is all settled)
EDIT
Since the results need to be kept, instead of mapping to Future.unit, we map the actual result into another layer of Try:
val settledFuture = Future.sequence(
List(Future(1), Future(throw new Exception))
.map(_.map(Success(_)).recover(Failure(_)))
)
settledFuture.map(println(_))
//Output: List(Success(1), Failure(java.lang.Exception))
EDIT2
It can be further simplified with transform:
Future.sequence(listOfFutures.map(_.transform(Success(_))))
Perhaps you could use a concurrent counter to keep track of the number of completed Futures and then complete the Promise once all Futures have completed
def allSettled[T](futures: List[Future[T]]): Future[List[Future[T]]] = {
val p = Promise[List[Future[T]]]()
val length = futures.length
val completedCount = new AtomicInteger(0)
futures foreach {
_.onComplete { _ =>
if (completedCount.incrementAndGet == length) p.trySuccess(futures)
}
}
p.future
}
val futures = List(
Future(-11),
Future(throw new Exception("boom")),
Future(42)
)
allSettled(futures).andThen(println(_))
// Success(List(Future(Success(-11)), Future(Failure(java.lang.Exception: boom)), Future(Success(42))))
scastie
I'm trying to call persistenceService.find() returning a Future[List[Scan]] foreach email passed to my method findByEmails which should return the merged Future[List[Scan]] of every calls to persistenceService.find()
So currently, I have this:
def findByEmails(emails: Set[String]): Future[List[Scan]] = {
val results: Set[Future[List[Scan]]] = emails.map(email => persistenceService.find("report.commitsStats.contributors." + email -> BSONDocument("$exists" -> true)))
}
I can't figure out how to merge each Future[List[Scan]] in the results Set so my method return one Future[List[Scan]].
Any thoughts?
You can use Future.sequence to combine a list of futures, then a flatten on the list should be enough, the code might look similar to this not tested code:
def f(str: String): Future[List[Scan]] = ???
def g(emails: List[String]): Future[List[Scan] = {
val futures: List[Future[list[Scan]]] = emails.map(f)
val futureListList: Future[List[List[Scan]] = Future.sequence(futures)
futureListList.map(_.flatten)
}
I have been struck with a piece on how to obtain a listbuffer of strings in the case where the listbuffer happens to be constructed in a scala future called in a loop.
Here is a kiss example
def INeedThatListBuffer(): ListBuffer[String] = {
var myCollections: ListBuffer[String] = new ListBuffer[String]()
for (day <- daysInaWeek) {
val myFuture: Future[String] = Future {
// use 'day' do some stuff and get me a result
???
}
myFuture.onComplete {
case Success(result) =>
myCollections += result
}
}
myCollections
}
My problem is that sometimes listBuffer is empty list and sometimes the content that I expected. Clearly, this method is complete before the future is evaluated.
Just to Add
I don't want to used future.await
Passing myCollections as Future obj does not work as there is no binding that myFuture must be complete before myCollections is evaluated.
Kindly help me out.
Thanks
This returns a future. If you don't care about waiting for it to be complete, you can always access the underlying value using .value:
def INeedThatListBuffer(): Future[ListBuffer[String]] = {
def buildFutureFromDay(day: String): Future[String] = Future { ??? }
Future
.sequence(daysInAWeek.map(buildFutureFromDay))
.map(_.foldLeft(ListBuffer[String]())(_ += _))
}
You need to await at some point. Either await for each of the futures to get resolved or change the ListBuffer[String] to ListBuffer[Future[String]] and await on the whole buffer later.
val myFutureCollection: ListBuffer[Future[String]] = new ListBuffer[Future[String]]()
val myFuture: Future[String] = Future(???)
myFutureCollection += myFuture
val eventualBuffer: Future[ListBuffer[String]] = Future.sequence(myFutureCollection)
val buffer: ListBuffer[String] = Await.result(eventualBuffer, 2 seconds)
P.S: You can val instead of var for the list buffer as it is already mutable.
I have a Future lazy val that obtains some object and a function which submits operations in the Future.
class C {
def printLn(s: String) = println(s)
}
lazy val futureC: Future[C] = Future{Thread.sleep(3000); new C()}
def func(s: String): Unit = {
futureC.foreach{c => c.printLn(s)}
}
The problem is when Future is completed it executes operations in reverse order than they have been submited. So for example if I execute sequentialy
func("A")
func("B")
func("C")
I get after Future completion
scala> C
B
A
This order is important for me. Is there a way to preserve this order?
Of course I can use an actor who asks for future and stashing strings while future is not ready, but it seems redundant for me.
lazy val futureC: Future[C]
lazy vals in scala will be compiled in to the code which uses a synchronized block for thread safety.
Here when the func(A) is called, it will obtain the lock for the lazy val and that thread will go to sleep.
Therefore func(B) & func(C) will blocked by the lock.
When those blocked threads are run, the order cannot be guaranteed.
If you do it like below, you'll have the order as you expect. This is because the for comprehension creates a flatMap, & map based chain that gets executed sequentially.
lazy val futureC: Future[C] = Future {
Thread.sleep(1000)
new C()
}
def func(s: String) : Future[Unit] = {
futureC.map { c => c.printLn(s) }
}
val x = for {
_ <- func("A")
_ <- func("B")
_ <- func("C")
} yield ()
The order preserves even without the lazy keyword. You can remove the lazy keyword unless it is really necessary.
Hope this helps.
You can use Future.traverse to ensure the order of execution.
Something like this.. Im not sure how your func has a reference to the correct futureC, so I moved it inside.
def func(s: String): Future[Unit] = {
lazy val futureC = Future{Thread.sleep(3000); new C()}
futureC.map{c => c.printLn(s)}
}
def traverse[A,B](xs: Seq[A])(fn: A => Future[B]): Future[Seq[B]] =
xs.foldLeft(Future(Seq[B]())) { (acc, item) =>
acc.flatMap { accValue =>
fn(item).map { itemValue =>
accValue :+ itemValue
}
}
}
traverse(Seq("A","B","C"))(func)
How to implement cache using functional programming
A few days ago I came across callbacks and proxy pattern implementation using scala.
This code should only apply inner function if the value is not in the map.
But every time map is reinitialized and values are gone (which seems obivous.
How to use same cache again and again between different function calls
class Aggregator{
def memoize(function: Function[Int, Int] ):Function[Int,Int] = {
val cache = HashMap[Int, Int]()
(t:Int) => {
if (!cache.contains(t)) {
println("Evaluating..."+t)
val r = function.apply(t);
cache.put(t,r)
r
}
else
{
cache.get(t).get;
}
}
}
def memoizedDoubler = memoize( (key:Int) => {
println("Evaluating...")
key*2
})
}
object Aggregator {
def main( args: Array[String] ) {
val agg = new Aggregator()
agg.memoizedDoubler(2)
agg.memoizedDoubler(2)// It should not evaluate again but does
agg.memoizedDoubler(3)
agg.memoizedDoubler(3)// It should not evaluate again but does
}
I see what you're trying to do here, the reason it's not working is that every time you call memoizedDoubler it's first calling memorize. You need to declare memoizedDoubler as a val instead of def if you want it to only call memoize once.
val memoizedDoubler = memoize( (key:Int) => {
println("Evaluating...")
key*2
})
This answer has a good explanation on the difference between def and val. https://stackoverflow.com/a/12856386/37309
Aren't you declaring a new Map per invocation ?
def memoize(function: Function[Int, Int] ):Function[Int,Int] = {
val cache = HashMap[Int, Int]()
rather than specifying one per instance of Aggregator ?
e.g.
class Aggregator{
private val cache = HashMap[Int, Int]()
def memoize(function: Function[Int, Int] ):Function[Int,Int] = {
To answer your question:
How to implement cache using functional programming
In functional programming there is no concept of mutable state. If you want to change something (like cache), you need to return updated cache instance along with the result and use it for the next call.
Here is modification of your code that follows that approach. function to calculate values and cache is incorporated into Aggregator. When memoize is called, it returns tuple, that contains calculation result (possibly taken from cache) and new Aggregator that should be used for the next call.
class Aggregator(function: Function[Int, Int], cache:Map[Int, Int] = Map.empty) {
def memoize:Int => (Int, Aggregator) = {
t:Int =>
cache.get(t).map {
res =>
(res, Aggregator.this)
}.getOrElse {
val res = function(t)
(res, new Aggregator(function, cache + (t -> res)))
}
}
}
object Aggregator {
def memoizedDoubler = new Aggregator((key:Int) => {
println("Evaluating..." + key)
key*2
})
def main(args: Array[String]) {
val (res, doubler1) = memoizedDoubler.memoize(2)
val (res1, doubler2) = doubler1.memoize(2)
val (res2, doubler3) = doubler2.memoize(3)
val (res3, doubler4) = doubler3.memoize(3)
}
}
This prints:
Evaluating...2
Evaluating...3