I have the following code snippet that I use to read a record from the database and I'm using ReactiveMongo for this.
val futureList: Future[Option[BSONDocument]] = collection.find(query).cursor[BSONDocument].headOption
val os: Future[Option[Exam]] = futureList.map {
(list: Option[BSONDocument]) => list match {
case Some(examBSON) => {
val id = examBSON.getAs[Int]("id").get
val text = examBSON.getAs[String]("text").get
val description = examBSON.getAs[String]("description").get
val totalQuestions = examBSON.getAs[Int]("totalQuestions").get
val passingScore = examBSON.getAs[Int]("passingScore").get
Some(Exam(id, text, description, totalQuestions, passingScore))
}
case None => None
}
}.recover {
case t: Throwable => // Log exception
None
}
I do not want to change my method signature to return a Future. I want to get the value inside the Future and return it to the caller.
You need then to block using the awaitable object:
import scala.concurrent.duration._
val os: Future[Option[Exam]] = ???
val result = Await.result(os, 10 seconds)
result.getOrElse(/* some default */)
Note that blocking will block the thread until the future is completed or the timeout expires and an exception is thrown, note also that this kinda defeats the purpose of having async computation, but it may be ok depending on your use case.
If you don't need the result immediately you can attach a callback using onComplete
os onComplete {
case Success(someOption) => myMethod(someOption)
case Failure(t) => println("Error)
}
Note that onComplete will be fired only when the future is completed so the result is not immediately accessible, also the return type is Unit.
Related
I am kind of failing this weird behaviour not sure where i am wrong exactly. So the situation is that tester2 function is returning a Future[Boolean]] now I want to wait for this to complete and when it gets completed I want it to return a List[String] based on different cases inside reset function, now the problem is instead of returning up a List[String] it is returning Future[List[String]] , not able to understand why match function behaving like this
I am getting this error to be exact
val les = Await.ready(tester2(5),Duration.Inf).map(reset).forEach(println)
object HelloWorld {
def main(args: Array[String]) {
val exp = tester2(5).map(reset)
val les = Await.ready(tester2(5),Duration.Inf).map(reset).forEach(println)
println(s"what do you say ${les}")
}
def reset (x: Option[Boolean]): List[String] =
x match {
case None => List("abc","def")
case Some(false) => List("abc","def")
case Some(true) => List("def","abc")
}
def tester():Future[Option[Message]]={
Future{
Thread.sleep(5000)
Option(Message("abc","def","ghi"))
}
}
def tester2(param:Int):Future[Option[Boolean]]={
Future{
Thread.sleep(5000)
if(param>10){
Some(true)
}else{
Some(false)
}
}
}
If tester2 returns a Future of an Option of a Boolean
def tester2(param:Int):Future[Option[Boolean] = ???
and you want to change the value to a string you need to say "when this future completes and there is a real Option[Boolean].. then do this thing. This is what "map" does on a future. It says "once the future completes, run this code". So you can do this:
def reset (in :Future[Option[Boolean]]) = in.map { optionOfBoolean :Option[Boolean] =>
optionOfBoolean match {
case None => ...
case Some(true) ...
}
}
Scala also allows you to short cut having the map and match together and just write:
def reset (in :Future[Option[Boolean]]) = in map {
case None => List("abc", "bcd")
case Some(true) => List("d3", "d4")
case Some(false) => List("sds", "dssds")
}
Since I can't see your error I can't help you further but something like this should work.
val booleanResult :Future[Option[Boolean]] = tester2(...)
val futureListStr :Future[List[String]] = reset(booleanResult)
val answer :List[String] = Await.result(futureListStr, scala.concurrent.duration.Duration.Inf)
Use Await.result to extract the result value.
final def result[T](awaitable: Awaitable[T], atMost: Duration): T
Await and return the result (of type T) of an Awaitable.
awaitable the Awaitable to be awaited
atMost maximum wait time, which may be negative (no waiting is done), >Duration.Inf for unbounded waiting, or a finite positive duration
returns the result value if awaitable is completed within the specific maximum wait time
I am trying to read incremental data from my data source using Scala-Spark. Before hitting the source tables, I am trying to calculate the min & max of partition column that I use in my code in a Future which is present in a class: GetSourceMeta as given below.
def getBounds(keyIdMap:scala.collection.mutable.Map[String, String]): Future[scala.collection.mutable.Map[String, String]] = Future {
var boundsMap = scala.collection.mutable.Map[String, String]()
keyIdMap.keys.foreach(table => if(!keyIdMap(table).contains("Invalid")) {
val minMax = s"select max(insert_tms) maxTms, min(insert_tms) minTms from schema.${table} where source='DB2' and key_id in (${keyIdMap(table)})"
println("MinMax: " + minMax)
val boundsDF = spark.read.format("jdbc").option("url", con.getConUrl()).option("dbtable", s"(${minMax}) as ctids").option("user", con.getUserName()).option("password", con.getPwd()).load()
try {
val maxTms = boundsDF.select("minTms").head.getTimestamp(0).toString + "," + boundsDF.select("maxTms").head.getTimestamp(0).toString
println("Bounds: " + maxTms)
boundsMap += (table -> maxTms)
} catch {
case np: java.lang.NullPointerException => { println("No data found") }
case e: Exception => { println(s"Unknown exception: $e") }
}
}
)
boundsMap.foreach(println)
boundsMap
}
I am calling the above method in my main method as:
object LoadToCopyDB {
val conf = new SparkConf().setAppName("TEST_YEAR").set("some parameters")
def main(args: Array[String]): Unit = {
val spark = SparkSession.builder().config(conf).master("yarn").enableHiveSupport().config("hive.exec.dynamic.partition", "true").config("hive.exec.dynamic.partition.mode", "nonstrict").getOrCreate()
val gsm = new GetSourceMeta()
val minMaxKeyMap = gsm.getBounds(keyIdMap).onComplete {
case Success(values) => values.foreach(println)
case Failure(f) => f.printStackTrace
}
.
.
.
}
Well, the onComplete didn't print any values so I used andThen as below and that didn't help as well.
val bounds: Future[scala.collection.mutable.Map[String, String]] = gpMetaData.getBounds(incrementalIds) andThen {
case Success(outval) => outval.foreach(println)
case Failure(e) => println(e)
}
Earlier the main thread exits without letting the Future: getBounds execute. Hence I couldn't find any println statements from the Future displayed on the terminal. I found out that I need to keep the main thread Await inorder to complete the Future. But when I use Await in main along with onComplete:
Await.result(bounds, Duration.Inf)
The compiler gives an error:
Type mismatch, expected: Awaitable[NotInferedT], actual:Unit
If I declare the val minMaxKeyMap as Future[scala.collection.mutable.Map[String, String] the compiler says: Expression of type Unit doesn't conform to expected type Future[mutable.map[String,String]]
I tried to print the values of bounds after the Await statement but that just prints an empty Map.
I couldn't understand how can to fix this. Could anyone let me know what do I do to make the Future run properly ?
In this kind of cases, is always better to follow the types. The method onComplete only returns Unit, it won´t return a future hence it can´t be passed using Await.
In case you want to return a Future of any type you will have to map or flatmap the value and return an option, for example. In this case, does not matter what you return, you only want Await method to wait for this result and print a trace. You can treat the possible exception in the recover. It would be like that in your code:
val minMaxKeyMap:Future[Option[Any] = gsm.getBounds(keyIdMap).map { values =>
values.foreach(println)
None
}.recover{
case e: Throwable =>
e. printStackTrace
None
}
Note that the recover part has to return an instance of the type.
After that, you can apply the Await to the Future, and you will get the results printed. Is not the prettiest solution but it will work in your case.
I wrote a simple method that returns List of Int based on a service call i perform in it.
my problem is that the service call im making is returning a future of some value, so I thought i can do onComplete and return what I want on complete of the future, but onComplete returns Unit...
this is how I wanted it to work:
def getListOfInts(str: String): List[Int] = {
myDaoService.findSomethingInDb(str).onComplete {
case Success(res) =>
res.map(line => line.amount.toInt*100).distinct.filter(num => num != 0).toList
case Failure(exception) => logger.error(s"finding in db has failed with exeption: ", exception)
}
}
what is the right way to return the wanted value of a future when it completed?
i know usually its map/flatmap but this will keep it wrapped in the future and I want my method to return the actual value and not future of the value...
thanks!
Await.result is the way to go, but Its not recommended.
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
import scala.concurrent.Await
def getListOfInts(str: String): List[Int] = {
Await.result(myDaoService.findSomethingInDb(str), 10 seconds).map(line => line.amount.toInt*100).distinct.filter(num => num != 0).toList
}
The above way is not recommended until and unless you want to make the thread of execution wait(block) for the result. The above code does not scale.
Recommended way
def getListOfInts(str: String): Future[List[Int]] =
myDaoService.findSomethingInDb(str).map { res =>
res.map(line => line.amount.toInt*100).distinct.filter(num => num != 0).toList
}
The recommended way is to compose the future (use map and flatmap to transform the output type of the database call into the required result) and then finally doing waiting to get the final transformed result.
http://docs.scala-lang.org/overviews/core/futures.html
onComplete, onSuccess and onFailure are the callbacks you are looking for
I was hoping code like follows would wait for both futures, but it does not.
object Fiddle {
val f1 = Future {
throw new Throwable("baaa") // emulating a future that bumped into an exception
}
val f2 = Future {
Thread.sleep(3000L) // emulating a future that takes a bit longer to complete
2
}
val lf = List(f1, f2) // in the general case, this would be a dynamically sized list
val seq = Future.sequence(lf)
seq.onComplete {
_ => lf.foreach(f => println(f.isCompleted))
}
}
val a = FuturesSequence
I assumed seq.onComplete would wait for them all to complete before completing itself, but not so; it results in:
true
false
.sequence was a bit hard to follow in the source of scala.concurrent.Future, I wonder how I would implement a parallel that waits for all original futures of a (dynamically sized) sequence, or what might be the problem here.
Edit: A related question: https://worldbuilding.stackexchange.com/questions/12348/how-do-you-prove-youre-from-the-future :)
One common approach to waiting for all results (failed or not) is to "lift" failures into a new representation inside the future, so that all futures complete with some result (although they may complete with a result that represents failure). One natural way to get that is lifting to a Try.
Twitter's implementation of futures provides a liftToTry method that makes this trivial, but you can do something similar with the standard library's implementation:
import scala.util.{ Failure, Success, Try }
val lifted: List[Future[Try[Int]]] = List(f1, f2).map(
_.map(Success(_)).recover { case t => Failure(t) }
)
Now Future.sequence(lifted) will be completed when every future is completed, and will represent successes and failures using Try.
And so, a generic solution for waiting on all original futures of a sequence of futures may look as follows, assuming an execution context is of course implicitly available.
import scala.util.{ Failure, Success, Try }
private def lift[T](futures: Seq[Future[T]]) =
futures.map(_.map { Success(_) }.recover { case t => Failure(t) })
def waitAll[T](futures: Seq[Future[T]]) =
Future.sequence(lift(futures)) // having neutralized exception completions through the lifting, .sequence can now be used
waitAll(SeqOfFutures).map {
// do whatever with the completed futures
}
A Future produced by Future.sequence completes when either:
all the futures have completed successfully, or
one of the futures has failed
The second point is what's happening in your case, and it makes sense to complete as soon as one of the wrapped Future has failed, because the wrapping Future can only hold a single Throwable in the failure case. There's no point in waiting for the other futures because the result will be the same failure.
This is an example that supports the previous answer. There is an easy way to do this using just the standard Scala APIs.
In the example, I am creating 3 futures. These will complete at 5, 7, and 9 seconds respectively. The call to Await.result will block until all futures have resolved. Once all 3 futures have completed, a will be set to List(5,7,9) and execution will continue.
Additionally, if an exception is thrown in any of the futures, Await.result will immediately unblock and throw the exception. Uncomment the Exception(...) line to see this in action.
try {
val a = Await.result(Future.sequence(Seq(
Future({
blocking {
Thread.sleep(5000)
}
System.err.println("A")
5
}),
Future({
blocking {
Thread.sleep(7000)
}
System.err.println("B")
7
//throw new Exception("Ha!")
}),
Future({
blocking {
Thread.sleep(9000)
}
System.err.println("C")
9
}))),
Duration("100 sec"))
System.err.println(a)
} catch {
case e: Exception ⇒
e.printStackTrace()
}
Even though it is quite old question But this is how I got it running in recent time.
object Fiddle {
val f1 = Future {
throw new Throwable("baaa") // emulating a future that bumped into an exception
}
val f2 = Future {
Thread.sleep(3000L) // emulating a future that takes a bit longer to complete
2
}
val lf = List(f1, f2) // in the general case, this would be a dynamically sized list
val seq = Future.sequence(lf)
import scala.concurrent.duration._
Await.result(seq, Duration.Inf)
}
This won't get completed and will wait till all the future gets completed. You can change the waiting time as per your use case. I have kept it to infinite and that was required in my case.
We can enrich Seq[Future[T]] with its own onComplete method through an implicit class:
def lift[T](f: Future[T])(implicit ec: ExecutionContext): Future[Try[T]] =
f map { Success(_) } recover { case e => Failure(e) }
def lift[T](fs: Seq[Future[T]])(implicit ec: ExecutionContext): Seq[Future[Try[T]]] =
fs map { lift(_) }
implicit class RichSeqFuture[+T](val fs: Seq[Future[T]]) extends AnyVal {
def onComplete[U](f: Seq[Try[T]] => U)(implicit ec: ExecutionContext) = {
Future.sequence(lift(fs)) onComplete {
case Success(s) => f(s)
case Failure(e) => throw e // will never happen, because of the Try lifting
}
}
}
Then, in your particular MWE, you can do:
val f1 = Future {
throw new Throwable("baaa") // emulating a future that bumped into an exception
}
val f2 = Future {
Thread.sleep(3000L) // emulating a future that takes a bit longer to complete
2
}
val lf = List(f1, f2)
lf onComplete { _ map {
case Success(v) => ???
case Failure(e) => ???
}}
This solution has the advantage of allowing you to call an onComplete on a sequence of futures as you would on a single future.
Create the Future with a Try to avoid extra hoops.
implicit val ec = ExecutionContext.global
val f1 = Future {
Try {
throw new Throwable("kaboom")
}
}
val f2 = Future {
Try {
Thread.sleep(1000L)
2
}
}
Await.result(
Future.sequence(Seq(f1, f2)), Duration("2 sec")
) foreach {
case Success(res) => println(s"Success. $res")
case Failure(e) => println(s"Failure. ${e.getMessage}")
}
I've a Future[List[Person]][1] and I want to get the List[Person] from it. How can I do it ?
import scala.concurrent.Future
val futPersons : Future[List[Person]] = ....
There are multiple ways:
futPersons.map { personList =>
....
}
This map returns another Future composed with whatever you return from the map. The map will execute only if the future completes successfully. If you need to handle failure you can use onComplete
futPersons.onComplete {
case Success(personList) => ...
case Failure(exception) => ...
}
Or you can wait for the future to complete (this is blocking):
val personList: List[Person] = Await.result(futPersons, 1 minutes)
Blocking way (pauses your thread until you get the value back) using Await.result:
scala.concurrent.Await.result(futPersons, timeout)
Or, using a callback with onSuccess:
futPersons onSuccess {
case persons => // do something with persons
}