Suppose,there are 2 future sequences of type Future[Seq[A,B]]. How can I combine into one?
You combined Futures using flatMap. So like:
futureA.flatMap(firstSequence =>
futureB.map(secondSequence => firstSequence ++ secondSequence))
For comprehensions are syntax sugar for this:
for {
firstSequence <- futureA
secondSequence <- futureB
} yield firstSequence ++ secondSequence
This code will run your Futures sequentially if they've been lazy up until this point. So you may wish to allow them to run in parallel by assigning them to a val before the for comprehension.
val executingFutureA = futureA
val executingFutureB = futureB
for {
firstSequence <- executingFutureA
secondSequence <- executingFutureB
} yield firstSequence ++ secondSequence
You can use Future.sequence to convert a sequence of Future into a single Future containing a sequence of the results of each Future. So in your case you can do this:
val a: Future[Seq[Either[A,B]]] = ???
val b: Future[Seq[Either[A,B]]] = ???
Future.sequence(Seq(a, b)).map(_.flatten) // => Seq[Either[A,B]]
The flatten operations converts the Seq[Seq[Either[A,B]]] into Seq[Either[A,B]], but the results could be combined in other ways if required.
This solution is very flexible, but for a fixed number of Seq[Future] it is often better to use flatMap/for as explained in another answer.
Related
I have a flow that returns List[Future[List[String]]] and I want to convert it to List[String] .
Here's what I am doing currently to achieve it -
val functionReturnedValue: List[Future[List[String]]] = functionThatReturnsListOfFutureList()
val listBuffer = new ListBuffer[String]
functionReturnedValue.map{futureList =>
val list = Await.result(futureList, Duration(10, "seconds"))
list.map(string => listBuffer += string)
}
listBuffer.toList
Waiting inside loop is not good, also need to avoid use of ListBuffer.
Or, if it is possible to get Future[List[String]] from List[Future[List[String]]]
Could someone please help with this?
There is no way to get a value from an asynchronus context to the synchronus context wihtout blocking the sysnchronus context to wait for the asynchronus context.
But, yes you can delay that blocking as much as you can do get better results.
val listFutureList: List[Future[List[String]]] = ???
val listListFuture: Future[List[List[String]]] = Future.sequence(listFutureList)
val listFuture: Future[List[String]] = listListFuture.map(_.flatten)
val list: List[String] = Await.result(listFuture, Duration.Inf)
Using Await.result invokes a blocking operation, which you should avoid if you can.
Just as a side note, in your code you are using .map but as you are only interested in the (mutable) ListBuffer you can just use foreach which has Unit as a return type.
Instead of mapping and adding item per item, you can use .appendAll
functionReturnedValue.foreach(fl =>
listBuffer.appendAll(Await.result(fl, Duration(10, "seconds")))
)
As you don't want to use ListBuffer, another way could be using .sequence is with a for comprehension and then .flatten
val fls: Future[List[String]] = for (
lls <- Future.sequence(functionReturnedValue)
) yield lls.flatten
You can transform List[Future[In]] to Future[List[In]] safetly as follows:
def aggregateSafeSequence[In](futures: List[Future[In]])(implicit ec: ExecutionContext): Future[List[In]] = {
val futureTries = futures.map(_.map(Success(_)).recover { case NonFatal(ex) => Failure(ex)})
Future.sequence(futureTries).map {
_.foldRight(List[In]()) {
case (curr, acc) =>
curr match {
case Success(res) => res :: acc
case Failure(ex) =>
println("Failure occurred", ex)
acc
}
}
}
}
Then you can use Await.result In order to wait if you like but it's not recommended and you should avoid it if possible.
Note that in general Future.sequence, if one the futures fails all the futures will fail together, so i went to a little different approach.
You can use the same way from List[Future[List[String]]] and etc.
I have a list of fileNames and I want to load the correlated pages in batches (and not all at once). In order to do so, I'm using FoldLeft and I'm writing an aggregate function which aggregates a Future[Map[T1,T2]].
def loadPagesInBatches[T1, T2](fileNames: Set[FileName]): Future[Map[T1, T2]] = {
val fileNameToPageId: Map[FileName, PageId] = ... //invokes a function that returns the pageId correlated to the fileName.
val batches: Iterator[Set[FileName]] = fileNames.grouped(10) //batches of 10;
batches.foldLeft(Future(Map.empty[T1, T2]))(aggregate(fileNameToPageId))
}
And the signature of aggregate is as follows:
def aggregate(fileNameToPageId: Map[FileName, PageId]): (Future[Map[T1, T2]], Set[FileName]) => Future[Map[T1, T2]] = {..}
I'm trying to make sure what is the best way to merge these Future[Map]s.
Thanks ahead!
P.S: FileName and PageId are just Types of string.
In case you have exactly 2 futures, zipWith would probably be the most idiomatic.
val future1 = ???
val future2 = ???
future1.zipWith(future2)(_ ++ _)
Which is a shorter way of writing a for comprehension:
for {
map1 <- future1
map2 <- future2
} yield map1 ++ map2
Although zipWith could potentially implement some kind of optimization.
My solution was putting the two maps into a list and using Future.reduceLeft.
def aggregate(fileNameToPageId: Map[FileName, PageId]): (Future[Map[T1, T2]], Set[FileName]) => Future[Map[T1, T2]] = {
case (all, filesBatch) =>
val mapOfPages: Future[Map[NodeId, T]] = for {
... //Some logic
} yield "TheBatchMap"
Future.reduceLeft(List(all, mapOfPages))(_ ++ _)
}
I'm kind of new to Scala/functional so I'm not yet able to use technical language.
I'm experiencing problems with a for-comprehension
val queries =
for {
_ <- createBanco
_ <- createBancoMedio
bankInsertions <- Update[Banco](insertStr).updateMany(NonEmptyList.fromList(createBankList(1, maxBanks)).get)
mediumInsertions <- Update[BancoMedio](mediumInsert).updateMany(NonEmptyList.fromList(mediumList).get)
bankCount <- BancoStatements.getCount().unique
bankGetIds <- BancoStatements.getIds(0, maxBanks).to[List]
bankSome <- BancoStatements.getSome(halfBanks).to[List]
} yield (bankCount, bankGetIds, bankSome)
//Execute database queries, saves them on tuple
val transactionResults : (Int, List[String], List[Banco]) =
queries.transact(h2Transactor).unsafeRunSync()
I'm trying to refactor the _ <- createBanco & _ <- createBancoMedio, which are both a ConnectionIO[Int] object.
Id like to convert those to a single List(createBanco, createBancoMedio) and then execute transact.
However, i'd be altering the return type of the for-comprehension by doing that. I'd like to know if there is any way on doing that without affecting the for output value
Basically, treat the list as if I was writing multiple anonymous parameters manually.
You can use .sequence to turn a List[G[A]] into a G[List[A]] if G has an Applicative instance, which ConnectionIO does:
val queries =
for {
_ <- List(createBanco, createBancoMedio).sequence
...
Just solved it, did another for comprehension for the List
val createList = for {
m <- createBancoMedio
b <- createBanco
} yield List(b, m)
val queries =
for {
_ <- createList ....
This way i had a ConnectionIO[List[Int]]
I'm looking for a way to convert a Vector[(Future[TypeA], TypeB)] to a Future[Vector[(TypeA, TypeB)]].
I'm aware of the conversion of a collection of futures to a future of a collection using Future.sequence(...) but cannot find out a way to manage the step from the tuple with a future to a future of tuple.
So I'm looking for something that implements the desired functionality of the dummy extractFutureFromTuple in the following.
val vectorOfTuples: Vector[(Future[TypeA], TypeB)] = ...
val vectorOfFutures: Vector[Future[(TypeA, TypeB)]] = vectorOfTuples.map(_.extractFutureFromTuple)
val futureVector: Future[Vector[(TypeA, TypeB)]] = Future.sequence(vectorOfFutures)
Note that you can do this with a single call to Future.traverse:
val input: Vector[(Future[Int], Long)] = ???
val output: Future[Vector[(Int, Long)]] = Future.traverse(input) {
case (f, v) => f.map(_ -> v)
}
I have a
object radExtractor{
def unapplySeq(row:HtmlTableRow):Option[List[String]]={
val lista = (for{
a<-row.getByXPath("td/span/a")
ah= a.asInstanceOf[DomNode]
if(ah.getFirstChild!=null)
} yield a.asInstanceOf[DomNode].getFirstChild.toString).toList
lista match{
case Nil=>None
case l # List(duns,companyname,address,city,postal,_bs,orgnummer, _*) =>Some(l)
case _ =>println("WTF");None
}
}
}
and I want to use it in a list comprehension like:
val toReturn = for{
rad<-rader
val radExtractor(duns,companyname,address,city,postal,_,orgnummer,_*)=rad
} yield Something(duns,companyname,address,city,postal,orgnummer)
But when a "rad" in "rader" fails because the extractor returns None I get a MatchError.
Isn't the extractor for comprehension supposed to handle/ignore the None cases or did I just miss something?
I could do
val toReturn = rader.collect{case radExtractor(duns,companyname,address,city,postal,_,orgnummer, _*)=>
Something(companyname=companyname,address=address,city=city,postalcode=postal,orgnummer=orgnummer,duns=duns.toInt)
}
But that would not be as sexy ;)
Thank you
Because you are performing the pattern match in an assignment to a val:
val radExtractor(duns,companyname,address,city,postal,_,orgnummer,_*)=rad
... the match must succeed, or you will encounter an error. The above syntax is valid outside a for-comprehension and Scala does not provide any special behaviour for non-matching cases.
To filter out non-matching values in a for-comprehension, use the pattern directly to the left of the <-:
val toReturn = for {
radExtractor(duns,companyname,address,city,postal,_,orgnummer,_*) <- rader
} yield Something(duns,companyname,address,city,postal,orgnummer)