What is the best way to go from
Map[String, Future[A]]
to
Map[String, A]
where A is the result of the corresponding future's execution?
This won't compile:
val results = for {
(key, future) <- myMap
result <- future
} yield (key, result)
as I can't mix futures and iterables in the same for comprehension.
If you convert it into a Seq[Future[(String,A)]], you can then use Future.fold to get it back a single Future[Map[...]]:
def transform[A](m: Map[String, Future[A]]): Future[Map[String, A]] = {
val seq: Seq[Future[(String, A)]] = m.toSeq.map { case (key, f) =>
f.map(i => key -> i)
}
Future.fold(seq)(Map.empty[String, A])(_ + _)
}
Then redeem the single future as normal.
Something like this perhaps:
map.mapValues { Await.result(_, 5 seconds) }
Dima already gave an answer using Await. However it will raise an exception when the Future fails.
You can further wrap the types inside as a Try and then do a .collect to filter only for the successful Futures (check out the official API for it).
import scala.util.{ Try, Success }
val results = myMap
.map {
case (key, value) => key => Try(Await.result(value, 5.seconds))
}
.collect {
case (key, Success(value)) => key -> value
}
With the call above, you automatically discard failing futures and only collect successful ones.
Related
I have a flow that returns List[Future[List[String]]] and I want to convert it to List[String] .
Here's what I am doing currently to achieve it -
val functionReturnedValue: List[Future[List[String]]] = functionThatReturnsListOfFutureList()
val listBuffer = new ListBuffer[String]
functionReturnedValue.map{futureList =>
val list = Await.result(futureList, Duration(10, "seconds"))
list.map(string => listBuffer += string)
}
listBuffer.toList
Waiting inside loop is not good, also need to avoid use of ListBuffer.
Or, if it is possible to get Future[List[String]] from List[Future[List[String]]]
Could someone please help with this?
There is no way to get a value from an asynchronus context to the synchronus context wihtout blocking the sysnchronus context to wait for the asynchronus context.
But, yes you can delay that blocking as much as you can do get better results.
val listFutureList: List[Future[List[String]]] = ???
val listListFuture: Future[List[List[String]]] = Future.sequence(listFutureList)
val listFuture: Future[List[String]] = listListFuture.map(_.flatten)
val list: List[String] = Await.result(listFuture, Duration.Inf)
Using Await.result invokes a blocking operation, which you should avoid if you can.
Just as a side note, in your code you are using .map but as you are only interested in the (mutable) ListBuffer you can just use foreach which has Unit as a return type.
Instead of mapping and adding item per item, you can use .appendAll
functionReturnedValue.foreach(fl =>
listBuffer.appendAll(Await.result(fl, Duration(10, "seconds")))
)
As you don't want to use ListBuffer, another way could be using .sequence is with a for comprehension and then .flatten
val fls: Future[List[String]] = for (
lls <- Future.sequence(functionReturnedValue)
) yield lls.flatten
You can transform List[Future[In]] to Future[List[In]] safetly as follows:
def aggregateSafeSequence[In](futures: List[Future[In]])(implicit ec: ExecutionContext): Future[List[In]] = {
val futureTries = futures.map(_.map(Success(_)).recover { case NonFatal(ex) => Failure(ex)})
Future.sequence(futureTries).map {
_.foldRight(List[In]()) {
case (curr, acc) =>
curr match {
case Success(res) => res :: acc
case Failure(ex) =>
println("Failure occurred", ex)
acc
}
}
}
}
Then you can use Await.result In order to wait if you like but it's not recommended and you should avoid it if possible.
Note that in general Future.sequence, if one the futures fails all the futures will fail together, so i went to a little different approach.
You can use the same way from List[Future[List[String]]] and etc.
I feel like this is not that difficult but I'm struggling with the futures and adding objects or Ints to an immutable Seq.
def createCopyOfProcessTemplate(processTemplateId: Int): Action[AnyContent] = Action.async {
//val copies = Seq()
processTemplateDTO.createCopyOfProcessTemplate(processTemplateId).flatMap { process =>
processTemplateDTO.getProcessStepTemplates(processTemplateId).map { steps =>
steps.foreach(processStep =>
copy: Future[Option[ProcessTemplateModel] = processTemplateDTO.createCopyOfStepTemplates(processTemplateId, process.get.id.get, processStep))
//Seq should look something like this: [{processStep.id, copy.id},{processStep.id, copy.id},...] or [[processStep.id, copy.id],[processStep.id, copy.id],...]
}
Ok(Json.obj("copies" -> copies))
}
Where do I have to define the seq and how should I return it since it's handling Futures ?
Any ideas? Thanks in advance!
You can use Future.sequence to convert List[Future[A]] into Future[List[A]] and return as result. First, do not use steps.foreach with copy variable defined, instead use steps.map to get ProcessTemplateModel as result from processTemplateDTO.createCopyOfStepTemplates, and map will return List of future result - List[Future[Option[ProcessTemplateModel]]. Then you can convert the result with Future.sequence and finally return as Json object.
val copies:List[Future[Option[ProcessTemplateModel]] = processTemplateDTO.createCopyOfProcessTemplate(processTemplateId).flatMap {
process =>processTemplateDTO.getProcessStepTemplates(processTemplateId).map { steps =>
steps.map(processStep =>
processTemplateDTO.createCopyOfStepTemplates(processTemplateId, process.get.id.get, processStep))
}
Future.sequence(copies).map{ result =>
Ok(Json.obj("copies" -> result))
}
Hoping someone can offer an opinion on a solution for this issue I'm having.
I'll try to simplify the issue so save bringing in domain issues, etc.
I have a list of Optional strings. I'm using the collect method to basically filter out strings that don't exist.
names collect {
case Some(value) => value
}
Simple enough. I'm homing to actually go one further. If a value is a None I'd like to call a function and use its response in place of the None. For example
names collect {
case Some(value) => value
case _ => getData(_)
}
The catch is the getData method returns a future. I understand that conventions for futures advise accessing the value within a callback, so something like the map method or on complete, but the issue is that I don't know if I need to call the getData method until I'm in the collect and have the value, so I can't simply wrap all my logic in a map method on getData. It doesn't feel like using Await and blocking is a good idea.
Any idea how I could reasonably handle this would be greatly appreciated. Very new to Scala, so I'd love to hear opinions and options.
EDIT:
I was trying to simplify the problem but I think I've instead missed out on key information.
Below is the actual implementation of my method:
def calculateTracksToExport()(
implicit exportRequest: ExportRequest,
lastExportOption: Option[String]
): Future[List[String]] = {
val vendorIds = getAllFavouritedTracks().flatMap { favTracks =>
Future.sequence {
favTracks.map { track =>
musicClient.getMusicTrackDetailsExternalLinks(
track,
exportRequest.vendor.toString.toLowerCase
).map { details =>
details.data.flatMap { data =>
data.`external-links`.map { link =>
link.map(_.value).collect {
case Some(value) => value
case None => getData(track)
}
}
}.getOrElse(List())
}
}
}.map(_.flatten)
}
vendorIds
}
You can use Future.sequence for collecting values:
def collect(list:List[Option[String]]):Future[List[String]] = Future.sequence(
list.map {
case Some(item) => Future.successful(item)
case _ => getData()
}
)
If something can be in future, you will have to always treat it like future. So have sequence of Futures as return value:
def resolve[T](input: Seq[Option[T]], supplier: => Future[T]): Seq[Future[T]] = {
input.map(option => option.map(Future.successful).getOrElse(supplier))
}
Usage example:
// Input to process
val data = Seq(Some(1), None, Some(2), None, Some(5))
//Imitates long-running background process producing data
var count = 6
def getData: Future[Int] = Future( {
Thread sleep (1000)
count += 1
count
})
resolve(data, getData) // Resolve Nones
.map(Await.result(_, 10.second)).foreach( println ) // Use result
Outputs:
1
8
2
7
5
http://ideone.com/aa8nJ9
I am using play2 and reactivemongo to fetch a result from mongodb. Each item of the result needs to be transformed to add some metadata. Afterwards I need to apply some sorting to it.
To deal with the transformation step I use enumerate():
def ideasEnumerator = collection.find(query)
.options(QueryOpts(skipN = page))
.sort(Json.obj(sortField -> -1))
.cursor[Idea]
.enumerate()
Then I create an Iteratee as follows:
val processIdeas: Iteratee[Idea, Unit] =
Iteratee.foreach[Idea] { idea =>
resolveCrossLinks(idea) flatMap { idea =>
addMetaInfo(idea.copy(history = None))
}
}
Finally I feed the Iteratee:
ideasEnumerator(processIdeas)
And now I'm stuck. Every example I saw does some println inside foreach, but seems not to care about a final result.
So when all documents are returned and transformed how do I get a Sequence, a List or some other datatype I can further deal with?
Change the signature of your Iteratee from Iteratee[Idea, Unit] to Iteratee[Idea, Seq[A]] where A is the type. Basically the first param of Iteratee is Input type and second param is Output type. In your case you gave the Output type as Unit.
Take a look at the below code. It may not compile but it gives you the basic usage.
ideasEnumerator.run(
Iteratee.fold(List.empty[MyObject]) { (accumulator, next) =>
accumulator + resolveCrossLinks(next) flatMap { next =>
addMetaInfo(next.copy(history = None))
}
}
) // returns Future[List[MyObject]]
As you can see, Iteratee is a simply a state machine. Just extract that Iteratee part and assign it to a val:
val iteratee = Iteratee.fold(List.empty[MyObject]) { (accumulator, next) =>
accumulator + resolveCrossLinks(next) flatMap { next =>
addMetaInfo(next.copy(history = None))
}
}
and feel free to use it where ever you need to convert from your Idea to List[MyObject]
With the help of your answers I ended up with
val processIdeas: Iteratee[Idea, Future[Vector[Idea]]] =
Iteratee.fold(Future(Vector.empty[Idea])) { (accumulator: Future[Vector[Idea]], next:Idea) =>
resolveCrossLinks(next) flatMap { next =>
addMetaInfo(next.copy(history = None))
} flatMap (ideaWithMeta => accumulator map (acc => acc :+ ideaWithMeta))
}
val ideas = collection.find(query)
.options(QueryOpts(page, perPage))
.sort(Json.obj(sortField -> -1))
.cursor[Idea]
.enumerate(perPage).run(processIdeas)
This later needs a ideas.flatMap(identity) to remove the returning Future of Futures but I'm fine with it and everything looks idiomatic and elegant I think.
The performance gained compared to creating a list and iterate over it afterwards is negligible though.
I'm trying to implement a system that caches requests to an external API. If the response is in cache, then no request should be made to the external site.
I have two methods:
// Check to see if the response is in the database
def checkCache(searchParameters: JsValue): Future[Option[JsValue]]
// Call the external API and get the JSON response
def getResponse(path: String): Future[JsValue]
And then I try to do the following:
val json: Future[JsValue] = for {
databaseJson <- checkCache(searchParameters)
externalJson <- getResponse(path)
} yield databaseJson match {
case None => externalJson
case Some(x) => x
}
This works, but a request to the external API is made all the time, even when the cached result is returned. This is obviously not what I want because it's slow.
How do I fix this?
The for comprehension maps over the futures, not the Option within it. Your code would translate to this
checkCache(searchParameters) flatMap { databaseJson =>
getResponse(path) map { externalJson =>
databaseJson match {
case None => externalJson
case Some(x) => x
}
}
}
So you are always calling getResponse() obviously.
You need something along the lines of this (not tested):
checkCache(searchParameters) flatMap { databaseJson =>
databaseJson match {
case None => getResponse(path)
case Some(x) => Future.successful(x)
}
}
You could also give this a shot:
val json: Future[JsValue] = for {
databaseJson <- checkCache(searchParameters)
json <- databaseJson.fold(getResponse(path))(js =>Future.successful(js))
} yield json
Similar in spirit to Marius Soutier's answer, but doing the Option checking via a fold right in the for comprehension.
This is a little cheesy, but still:
checkCache(params) map (_.get) fallbackTo getResponse(path)
Also not tested.
Update:
I didn't like the conversion to failure by None.get, but actually this isn't cheesy at all but is very natural. More naturally:
checkCache(params) filter (_.nonEmpty) map (_.get) fallbackTo getResponse(path)
The Some.get is now just a wart due to the asymmetry in the two futures (that they are not both Option).