I used traverse to execute a collection of futures like this:
val result: Future[List[Either[Error, Int]]] = Future.traverse(urls)(foo(_))
I end up with a Future[List[Either[Error, Int]]]. How can I check that one of these futures resulted in an Error?
I tried to do this but I think it is wrong because I am reading that you cannot substitute variables for futures?
val check: Future[Boolean] = result.map{
fut => fut.exists(c => c.isLeft)
}
check.map{
b => b match {
case true => // do something
case false => // do something
}
}
You can convert the result to a list of errors like this:
val errors: Future[List[Error]] = result.map(_.collect{ case Left(err) => err })
It is then possible to use Await.result to extract these error values, but that is nearly always a bad idea because it blocks the current thread.
It is better to ask "What do I want to do once the Future is complete but returns errors?". Then implement that behaviour in a map or foreach on the errors Future.
Related
I have a function f which returns a Future[Unit]. I want to apply the function f onto a sequence of Strings, such that I get a Seq[Future[Unit]].
To convert it into a single Future, we are using Future.sequence which converts it into a Future[Seq[Unit]]. Now, since the function f can either fail or pass, we also convert it into a Try (to better handle the failures) using FutureUtil.toTry which gives us a Future[Try[Seq[Unit]]].
Now, the deal is that we don't want to know which Futures passed or not, but the main task is to realise if all passed or not. If either of them fails, we stop the execution.
So, I was wondering if there was some "elegant" way to find this and we could simply remove the Seq from the final Future and have something like Future[Try[Unit]].
A code example (which should help understand the problem in a much better way)
def f(s: String): Future[Unit] = {
if(s.isEmpty)
Future.failed(new Throwable("lalala"))
else
Future.successful()
}
val strings: Seq[String] = Seq[String]("abc", "xyz", "lol")
val stringsFuture: Seq[Future[Unit]] = strings.map({ s =>
f(s)
})
val futureStrings: Future[Seq[Unit]] = Future.sequence(stringsFuture)
val futureStringsTry: Future[Try[Seq[Unit]]] = FutureUtil.toTry(futureStrings)
Is there a way where we can convert futureStringsTry to a simple Future[Try[Unit]].
A naive solution would be to flatmap futureStringsTry, something like this:
val finalFuture: Future[Try[Unit]] = futureStringsTry.map({
case Success(_) => Success()
case Failure(exception) => Failure(exception)
})
But, is there some other way where we can "elegantly" evaluate whether the whole Sequence passed or not?
I have the following pattern:
Some play route with some optional parameters defined
GET /my/route controllers.something.Class.method(maybeA: Optional[Int], maybeB: Optional[Int])
In the method, I'd like to be able to use those parameters. I feel like I don't understand the proper pattern to use. What I'd like to do is something like:
blah match {
case Some(someCase) => // do something
case _ => for {
a <- maybeA
b <- maybeB
} yield {
somethingThatReturnsAFuture(a, b)
}
}
The problem is that the for comprehension returns an Option[Future[Int]], but I need it to return Future[Int]. I'd like to be able to just throw a get on the end of the comprehension, but that smells funny (and I don't actually think it's possible to just throw get at the end of the comprehension).
Any suggestions?
If something is Option then the result of the direct processing of this will be Option as well. The only way to get rid of Option is to define the None case.
val resultOpt: Option[Future[Int]] = blah match { ...
val result: Future[Int] = resultOpt match {
case Some(f) => f
case None => Future.successful(0)
}
Or just use getOrElse. The same code, different style:
val resultOpt: Option[Future[Int]] = blah match { ...
val result: Future[Int] = resultOpt.getOrElse(Future.successful(0))
A little bit more about this:
http://alvinalexander.com/scala/how-to-use-multiple-options-for-loop-comprehension
In Spark, I can handle exceptions this way:
val myRDD = sc.textFile(path)
.map(line => Try {
// do something dangerous
// if(condition)
// raise isFailure;
}).filter(_.isSuccess).map(_.get)
I'd like the element read by the first map function to raise failure as well, under some conditions. How do I do it?
If you're ignoring the actual errors, you can just filter for the condition (given that the condition depends entirely on the line):
def condition(line: String): Boolean = ???
val myRDD = sc.textFile(path)
.map(line => Try { // do something dangerous })
.filter(_.isSuccess)
.filter(condition)
.map(_.get)
You could use Either
myRDD.map(line=> if (condition) Left("some Error") else Right(somevalue))
.collect { case Right(v) => v }
This gives a result of whatever was in the successful mappings. But you probably want to do something with the failures if using the above code (with for example a partition { }. If not use the Option approach:
Or use a straightforward Option:
myRDD.map(line=> if (condition) None else Some(somevalue))
.flatten()
If you wanted to combine last option with your Try, use something like
myRDD.map(line=> Try{
/*something dangerous*/
if (condition) None else Some(somevalue)
})
.collect{ case Success(r) => r } // successful results
.flatten() // flatten out the Nones
IMHO (and many others), refrain from throwing exceptions in scala yourself. I see them as fancy goto-statements that don't jump through code within one dimension, but through different dimensions (i.e, the stack). They are part of the Java platform, so we need to deal with them when we access Java code (plus you might have to throw some if you are building something which will be used by Java developers)
Hoping someone can offer an opinion on a solution for this issue I'm having.
I'll try to simplify the issue so save bringing in domain issues, etc.
I have a list of Optional strings. I'm using the collect method to basically filter out strings that don't exist.
names collect {
case Some(value) => value
}
Simple enough. I'm homing to actually go one further. If a value is a None I'd like to call a function and use its response in place of the None. For example
names collect {
case Some(value) => value
case _ => getData(_)
}
The catch is the getData method returns a future. I understand that conventions for futures advise accessing the value within a callback, so something like the map method or on complete, but the issue is that I don't know if I need to call the getData method until I'm in the collect and have the value, so I can't simply wrap all my logic in a map method on getData. It doesn't feel like using Await and blocking is a good idea.
Any idea how I could reasonably handle this would be greatly appreciated. Very new to Scala, so I'd love to hear opinions and options.
EDIT:
I was trying to simplify the problem but I think I've instead missed out on key information.
Below is the actual implementation of my method:
def calculateTracksToExport()(
implicit exportRequest: ExportRequest,
lastExportOption: Option[String]
): Future[List[String]] = {
val vendorIds = getAllFavouritedTracks().flatMap { favTracks =>
Future.sequence {
favTracks.map { track =>
musicClient.getMusicTrackDetailsExternalLinks(
track,
exportRequest.vendor.toString.toLowerCase
).map { details =>
details.data.flatMap { data =>
data.`external-links`.map { link =>
link.map(_.value).collect {
case Some(value) => value
case None => getData(track)
}
}
}.getOrElse(List())
}
}
}.map(_.flatten)
}
vendorIds
}
You can use Future.sequence for collecting values:
def collect(list:List[Option[String]]):Future[List[String]] = Future.sequence(
list.map {
case Some(item) => Future.successful(item)
case _ => getData()
}
)
If something can be in future, you will have to always treat it like future. So have sequence of Futures as return value:
def resolve[T](input: Seq[Option[T]], supplier: => Future[T]): Seq[Future[T]] = {
input.map(option => option.map(Future.successful).getOrElse(supplier))
}
Usage example:
// Input to process
val data = Seq(Some(1), None, Some(2), None, Some(5))
//Imitates long-running background process producing data
var count = 6
def getData: Future[Int] = Future( {
Thread sleep (1000)
count += 1
count
})
resolve(data, getData) // Resolve Nones
.map(Await.result(_, 10.second)).foreach( println ) // Use result
Outputs:
1
8
2
7
5
http://ideone.com/aa8nJ9
I have a Scala future that calls an api and returns future, if the result isn't correct, then another api call will be submitted with first future's result and returned as future.
This is what I have so far.
val fut1 = Future(queryFirst)
val fut2 = fut1 map {
case y if y.isInstanceOf[NoResult] => Future(queryAgainWithFut1Result)
case x => x
}
But if I access fut2 result, it gives something like this:
scala.concurrent.Await.result(fut2, 5 seconds)
warning: there was one feature warning; re-run with -feature for details
fut2: Any = scala.concurrent.impl.Promise$DefaultPromise#61ab71c2
Is there a way that I can choose to return fut2 if fut1 result is not accurate?
Edit:
Second future has to use first future to continue api call. This is what I have so far.
val fut1 = Future("queryFirst")
val fut2 = fut1 flatMap {
case y if y.isInstanceOf[Int] => Future("queryAgainWithResult(y)")
case x => Future(x)
}
You can filter the first Future to cause it to fail when its contained result doesn't match what you want, then use recoverWith to recover the value to another Future.
Future(queryFirst)
.filter(result => /* logic that returns true if result is correct */)
.recoverWith { case NonFatal(e) => Future(querySecond) }
If queryFirst is successful and makes it through the filter, then it's result will be returned (wrapped in the Future).
If queryFirst is successful and does not have a correct value according to the filter or simply fails, then the result of Future(querySecond) will be returned, success or fail.
Edit: In light of the new information from the OP, if you need the result of the incorrect first Future, then you don't really have a choice but to use flatMap.
Future(queryFirst).flatMap {
case result if(/* failed condition */) => Future(querySecond(result))
case result => Future.successful(result)
}