I have a function f which returns a Future[Unit]. I want to apply the function f onto a sequence of Strings, such that I get a Seq[Future[Unit]].
To convert it into a single Future, we are using Future.sequence which converts it into a Future[Seq[Unit]]. Now, since the function f can either fail or pass, we also convert it into a Try (to better handle the failures) using FutureUtil.toTry which gives us a Future[Try[Seq[Unit]]].
Now, the deal is that we don't want to know which Futures passed or not, but the main task is to realise if all passed or not. If either of them fails, we stop the execution.
So, I was wondering if there was some "elegant" way to find this and we could simply remove the Seq from the final Future and have something like Future[Try[Unit]].
A code example (which should help understand the problem in a much better way)
def f(s: String): Future[Unit] = {
if(s.isEmpty)
Future.failed(new Throwable("lalala"))
else
Future.successful()
}
val strings: Seq[String] = Seq[String]("abc", "xyz", "lol")
val stringsFuture: Seq[Future[Unit]] = strings.map({ s =>
f(s)
})
val futureStrings: Future[Seq[Unit]] = Future.sequence(stringsFuture)
val futureStringsTry: Future[Try[Seq[Unit]]] = FutureUtil.toTry(futureStrings)
Is there a way where we can convert futureStringsTry to a simple Future[Try[Unit]].
A naive solution would be to flatmap futureStringsTry, something like this:
val finalFuture: Future[Try[Unit]] = futureStringsTry.map({
case Success(_) => Success()
case Failure(exception) => Failure(exception)
})
But, is there some other way where we can "elegantly" evaluate whether the whole Sequence passed or not?
Related
I'm writing an authentication client that takes an Option[Credentials] as a parameter. This Credentials object has a .token method on it which I will then use to construct an HTTP request to post to an endpoint. This returns a Future[HttpResponse], which I then need to validate, unmarshal, and then convert back to my return type, which is an Option[String].
My first thought was to use a for comprehension like this:
val resp = for {
c <- creds
req <- buildRequest(c.token)
resp <- Http().singleRequest(req)
} yield resp
but then I found out that monads cannot be composed like that. My next thought is to do something like this:
val respFut = Http().singleRequest(buildRequest(token))
respFut.onComplete {
case Success(resp) => Some("john.doe")//do stuff
case Failure(_) => None
}
Unfortunately onComplete returns a unit, and map leaves me with a Future[Option[String]], and the only way I currently know to strip off the future wrapper is using the pipeTo methods in the akka framework. How can I convert this back to just an option string?
Once you've got a Future[T], it's usually good practice to not try to unbox it until you absolutely have to. Can you change your method to return a Future[Option[String]]? How far up the call stack can you deal with futures? Ideally it's all the way.
Something like this will give you a Future[Option[String]] as a result:
val futureResult = creds map {
case Some(c) => {
val req = buildRequest(c.token)
val futureResponse = Http().singleRequest(req)
futureResponse.map(res => Some(convertResponseToString(res)))
}
case None => Future(None)
}
If you really need to block and wait on the result, you can do Await.result as described here.
And if you want to do it in a more monadic style (in a for-comprehension, like you tried), cats has an OptionT type that will help with that, and I think scalaz does as well. But whether you want to get into either of those libraries is up to you.
It's easy to "upgrade" an Option to a Future[Option[...]], so use Future as your main monad. And deal with the simpler case first:
val f: Future[Option[String]] =
// no credential? just wrap a `None` in a successful future
credsOpt.fold(Future.successful(Option.empty[String])) {creds =>
Http()
.singleRequest(buildRequest(creds.token))
.map(convertResponseToString)
.recover {case _ => Option.empty[String]}
}
The only way to turn that future into Option[String] is to wait for it with Await.result(...)... but it's better if that future can be passed along to the next caller (no blocking).
I'm not 100% certain about what all your types are, but it seems like you want a for comprehension that mixes option and futures. I've often been in that situation and I find I can just chain my for comprehensions as a way to make the code look a bit better.
val resp = for {
c <- creds
req <- buildRequest(c.token)
} yield for {
resp <- Http().singleRequest(req)
} yield resp
resp becomes an Option[Future[HttpResponse]] which you can match / partial func around with None meaning the code never got to execute because it failed its conditions. This is a dumb little trick I use to make comprehensions look better and I hope it gives you a hint towards your solution.
How can I remove the option so it is just Try[Int] and not Try[Option[Int]]?
val m = Map("a" -> "1a", "b" -> "2")
Try(m.get("a").map(_.trim.toInt))
>>es17: scala.util.Try[Option[Int]] = Failure(java.lang.NumberFormatException: For input string: "1a")
Map#get returns an Option[String], but you can use Map#apply instead, which will return String, in this case.
scala> Try(m("a").trim.toInt)
res3: scala.util.Try[Int] = Failure(java.lang.NumberFormatException: For input string: "1a")
scala> Try(m("b").trim.toInt)
res4: scala.util.Try[Int] = Success(2)
apply throws an exception if the key you're looking for doesn't exist, but Try will catch it, anyway.
This answer goes in more detail about the comment:
I was wondering if there was a way to use flapmap? Your solution works for me, just want to learn of other alternatives.
As you've probably heard, Option and Try are monad instances and while monads are handy to represent sequence of computations, they don't compose with other monads. In other words, we can't compose Option and Try. We need to find a common ground.
The difference in semantics between Option and Try is that Try contains information about the case when a result is absent.
We can go from Try to Option using Try#toOption effectively loosing any failure information we may have.
If we wanted to go the other way, we need to add this information back: ne need to provide a failure reason when a value is absent in an Option. Something like this:
import scala.util.{Try, Success, Failure}
def optionToTry[T](opt:Option[T], failure: => Throwable): Try[T] = opt match {
case Some(v) => Success(v)
case None => Failure(failure)
}
With the help of that function, we can rewrite the original expression as:
val res: Try[Int] = for {
strValue <- optionToTry(m.get("a"), new NoSuchElementException("a"))
value <- Try(strValue.trim.toInt)
} yield value
which uses flatMap behind the scenes to compose the two Try instances like this:
val res = optionToTry(m.get("a"), new NoSuchElementException("a"))
.flatMap(strValue => Try(strValue.trim.toInt))
Note that we could save ourselves a bit of coding by using the unsafe map getter like so:
val res: Try[Int] = for {
strValue <- Try(m("a"))
value <- Try(strValue.trim.toInt)
} yield value
but this version would be computationally more expensive given the cost of handling exceptions in the JVM.
I have a sequence of parameters. For each parameter I have to perform DB query, which may or may not return a result. Simply speaking, I need to stop after the first result is non-empty. Of course, I would like to avoid doing unnecessary calls. The caveat is - I need to have this operation(s) contained as a another Future - or any "most reactive" approach.
Speaking of code:
//that what I have
def dbQuery(p:Param): Future[Option[Result]] = {}
//my list of params
val input = Seq(p1,p2,p3)
//that what I need to implements
def getFirstNonEmpty(params:Seq[Param]): Future[Option[Result]]
I know I can possibly just wrap entire function in yet another Future and execute code sequentially (Await? Brrr...), but that not the cleanest solution.
Can I somehow create lazy initialized collection of futures, like
params.map ( p => FutureWhichWontStartUnlessAskedWhichWrapsOtherFuture { dbQuery(p) }).findFirst(!_.isEmpty())
I believe it's possible!
What do you think about something like this?
def getFirstNonEmpty(params: Seq[Param]): Future[Option[Result]] = {
params.foldLeft(Future.successful(Option.empty[Result])) { (accuFtrOpt, param) =>
accuFtrOpt.flatMap {
case None => dbQuery(param)
case result => Future.successful(result)
}
}
}
This might be overkill, but if you are open to using scalaz we can do this using OptionT and foldMap.
With OptionT we sort of combine Future and Option into one structure. We can get the first of two Futures with a non-empty result using OptionT.orElse.
import scalaz._, Scalaz._
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
val someF: Future[Option[Int]] = Future.successful(Some(1))
val noneF: Future[Option[Int]] = Future.successful(None)
val first = OptionT(noneF) orElse OptionT(someF)
first.run // Future[Option[Int]] = Success(Some(1))
We could now get the first non-empty Future from a List with reduce from the standard library (this will however run all the Futures) :
List(noneF, noneF, someF).map(OptionT.apply).reduce(_ orElse _).run
But with a List (or other collection) we can't be sure that there is at least one element, so we need to use fold and pass a start value. Scalaz can do this work for us by using a Monoid. The Monoid[OptionT[Future, Int]] we will use will supply the start value and combine the Futures with the orElse used above.
type Param = Int
type Result = Int
type FutureO[x] = OptionT[Future, x]
def query(p: Param): Future[Option[Result]] =
Future.successful{ println(p); if (p > 2) Some(p) else None }
def getFirstNonEmpty(params: List[Param]): Future[Option[Result]] = {
implicit val monoid = PlusEmpty[FutureO].monoid[Result]
params.foldMap(p => OptionT(query(p))).run
}
val result = getFirstNonEmpty(List(1,2,3,4))
// prints 1, 2, 3
result.foreach(println) // Some(3)
This is an old question, but if someone comes looking for an answer, here is my take. I solved it for a use case that required me to loop through a limited number of futures sequentially and stop when the first of them returned a result.
I did not need a library for my use-case, a light-weight combination of recursion and pattern matching was sufficient. Although the question here does not have the same problem as a sequence of futures, looping through a sequence of parameters would be similar.
Here would be the pseudo-code based on recursion.
I have not compiled this, fix the types being matched/returned.
def getFirstNonEmpty(params: Seq[Param]): Future[Option[Result]] = {
if (params.isEmpty) {
Future.successful(None)
} else {
val head = params.head
dbQuery(head) match {
case Some(v) => Future.successful(Some(v))
case None => getFirstNonEmpty(params.tail)
}
}
}
I am trying to incorporate a database into my http-microservice.
The microservice has a function getValueFromInternet(val: Foo): Future[Value] which was being called by my microservice on a GET request. Now, I want it to happen such that, a function getValue(val: Foo): Future[Value] would first query a db and if the database returns no results, call getValueFromInternet. The database query returns a Future[Seq[Value2]] where I can convert Value2 to Value using a function. And if no entry is found corresponding to that value, an empty Vector is returned.
This is what I have tried so far:
def getValue(val: Foo): Future[Value] = {
val resultFuture = db.getValue(val)
// 1st attempt. Clearly wrong
resultFuture onComplete {
case Success(Vector()) => getValueFromInternet(val)
case Success(vec) => convertValue2to1(vec.head)
}
// 2nd attempt. This is also wrong
resultFuture match {
case Future(Success(Vector())) => getValueFromInternet(val)
case Future(Success(vec)) => convertValue2to1(vec.head)
}
}
I would be grateful for any help suggesting how I can do this.
I have implemented the database and microservice independently and you can find them here and here
You have to use flatMap, since the thing you want to do if the first operation does not return a result also returns a future.
This is as close to your code as possible while still compiling. Note that you can't have identifiers called val in scala, since that is a keyword.
def getValue(v: Foo)(implicit ec: ExecutionContext): Future[Value] = {
val resultFuture: Future[Seq[Value2]] = db.getValue(v)
resultFuture.flatMap { vec =>
if(vec.isEmpty)
getValueFromInternet(v)
else
Future.successful(convertValue2to1(vec.head))
}
}
I have to get a list of issues for each file of a given list from a REST API with Scala. I want to do the requests in parallel, and use the Dispatch library for this. My method is called from a Java framework and I have to wait at the end of this method for the result of all the futures to yield the overall result back to the framework. Here's my code:
def fetchResourceAsJson(filePath: String): dispatch.Future[json4s.JValue]
def extractLookupId(json: org.json4s.JValue): Option[String]
def findLookupId(filePath: String): Future[Option[String]] =
for (json <- fetchResourceAsJson(filePath))
yield extractLookupId(json)
def searchIssuesJson(lookupId: String): Future[json4s.JValue]
def extractIssues(json: org.json4s.JValue): Seq[Issue]
def findIssues(lookupId: String): Future[Seq[Issue]] =
for (json <- searchIssuesJson(componentId))
yield extractIssues(json)
def getFilePathsToProcess: List[String]
def thisIsCalledByJavaFramework(): java.util.Map[String, java.util.List[Issue]] = {
val finalResultPromise = Promise[Map[String, Seq[Issue]]]()
// (1) inferred type of issuesByFile not as expected, cannot get
// the type system happy, would like to have Seq[Future[(String, Seq[Issue])]]
val issuesByFile = getFilePathsToProcess map { f =>
findLookupId(f).flatMap { lookupId =>
(f, findIssues(lookupId)) // I want to yield a tuple (String, Seq[Issue]) here
}
}
Future.sequence(issuesByFile) onComplete {
case Success(x) => finalResultPromise.success(x) // (2) how to return x here?
case Failure(x) => // (3) how to return null from here?
}
//TODO transform finalResultPromise to Java Map
}
This code snippet has several issues. First, I'm not getting the type I would expect for issuesByFile (1). I would like to just ignore the result of findLookUpId if it is not able to find the lookUp ID (i.e., None). I've read in various tutorials that Future[Option[X]] is not easy to handle in function compositions and for expressions in Scala. So I'm also curious what the best practices are to handle these properly.
Second, I somehow have to wait for all futures to finish, but don't know how to return the result to the calling Java framework (2). Can I use a promise here to achieve this? If yes, how can I do it?
And last but not least, in case of any errors, I would just like to return null from thisIsCalledByJavaFramework but don't know how (3).
Any help is much appreciated.
Thanks,
Michael
Several points:
The first problem at (1) is that you don't handle the case where findLookupId returns None. You need to decide what to do in this case. Fail the whole process? Exclude that file from the list?
The second problem at (1) is that findIssues will itself return a Future, which you need to map before you can build the result tuple
There's a shortcut for map and then Future.sequence: Future.traverse
If you cannot change the result type of the method because the Java interface is fixed and cannot be changed to support Futures itself you must wait for the Future to be completed. Use Await.ready or Await.result to do that.
Taking all that into account and choosing to ignore files for which no id could be found results in this code:
// `None` in an entry for a file means that no id could be found
def entryForFile(file: String): Future[(String, Option[Seq[Issue]])] =
findLookupId(file).flatMap {
// the need for this kind of pattern match shows
// the difficulty of working with `Future[Option[T]]`
case Some(id) ⇒ findIssues(id).map(issues ⇒ file -> Some(issues))
case None ⇒ Future.successful(file -> None)
}
def thisIsCalledByJavaFramework(): java.util.Map[String, java.util.List[Issue]] = {
val issuesByFile: Future[Seq[(String, Option[Seq[Issue]])]] =
Future.traverse(getFilePathsToProcess)(entryForFile)
import scala.collection.JavaConverters._
try
Await.result(issuesByFile, 10.seconds)
.collect {
// here we choose to ignore entries where no id could be found
case (f, Some(issues)) ⇒ f -> issues
}
.toMap.mapValues(_.asJava).asJava
catch {
case NonFatal(_) ⇒ null
}
}