An exercise on forComprehension, doesn't compile since returns Unit instead of Future[User]
(the task is to read using DAO and create new user using 2 parents)
I'm trying to return Future[User] from onComplete but fails (returns Unit).
def unbornUsingForComp(fatherId: String, motherId: String): Future[User] = {
val fatherUser: Future[Option[User]] = usersDao.read(fatherId)
val motherUser: Future[Option[User]] = usersDao.read(motherId)
val combinedLastName = for {
r1 <- fatherUser
r2 <- motherUser
} yield (r1, r2)
combinedLastName.onComplete {
case Success(x) => {
(x._1, x._2) match {
case (Some(u1), Some(u2)) => usersDao.create(User("JustRandomId", "JustRandomFirstName", u1.lastName + " - " +u2.lastName))
case _ => throw new Exception("One or more of the parents not Found")
}
}
case Failure(_) => {
throw new Exception("Exception raised inside the for comp.")
}
}
}
onComplete by design returns Unit which executes a side-effect and then discards
// Note that the returned value of `f` will be discarded.
def onComplete[U](f: Try[T] => U)(implicit executor: ExecutionContext): Unit
You are likely after something like
combinedLastName
.map { x => // do stuff with x }
.recover { case e => // handle exception }
Related
I'm not sure whether I chose the right title for my question..
I'm interested as to why the collection in the companion object is defined. Am I mistaken that this collection will have only one f in it? What I am seeing is a collection with exactly one element.
Here's the Future I'm dealing with:
trait Future[+T] { self =>
def onComplete(callback: Try[T] => Unit): Unit
def map[U](f: T => U) = new Future[U] {
def onComplete(callback: Try[U] => Unit) =
self onComplete (t => callback(t.map(f)))
}
def flatMap[U](f: T => Future[U]) = new Future[U] {
def onComplete(callback: Try[U] => Unit) =
self onComplete { _.map(f) match {
case Success(fu) => fu.onComplete(callback)
case Failure(e) => callback(Failure(e))
} }
}
def filter(p: T => Boolean) =
map { t => if (!p(t)) throw new NoSuchElementException; t }
}
Its companion object:
object Future {
def apply[T](f: => T) = {
val handlers = collection.mutable.Buffer.empty[Try[T] => Unit]
var result: Option[Try[T]] = None
val runnable = new Runnable {
def run = {
val r = Try(f)
handlers.synchronized {
result = Some(r)
handlers.foreach(_(r))
}
}
}
(new Thread(runnable)).start()
new Future[T] {
def onComplete(f: Try[T] => Unit) = handlers.synchronized {
result match {
case None => handlers += f
case Some(r) => f(r)
}
}
}
}
}
In my head I was imagining something like the following instead of the above companion object (notice how I replaced the above val handlers .. with var handler ..):
object Future {
def apply[T](f: => T) = {
var handler: Option[Try[T] => Unit] = None
var result: Option[Try[T]] = None
val runnable = new Runnable {
val execute_when_ready: Try[T] => Unit = r => handler match {
case None => execute_when_ready(r)
case Some(f) => f(r)
}
def run = {
val r = Try(f)
handler.synchronized {
result = Some(r)
execute_when_ready(r)
}
}
}
(new Thread(runnable)).start()
new Future[T] {
def onComplete(f: Try[T] => Unit) = handler.synchronized {
result match {
case None => handler = Some(f)
case Some(r) => f(r)
}
}
}
}
}
So why does the function execute_when_ready leads to stackoverflow, but that's not the case with handlers.foreach? what is the collection is offering me which I can't do without it? And is it possible to replace the collection with something else in the companion object?
The collection is not in the companion object, it is in the apply method, so there is a new instance for each Future. It is there because there can be multiple pending onComplete handlers on the same Future.
Your implementation only allows a single handler and silently removes any existing handler in onComplete which is a bad idea because the caller has no idea if a previous function has added an onComplete handler or not.
As noted in the comments, the stack overflow is because execute_when_ready calls itself if handler is None with no mechanism to stop the recursion.
I am trying to write a function that returns a future. Within this future 3 other future functions are called, the third depends on the results of the first two. My instinct is to use a for comphrension, but when I try to match on the tuple I get this error:
value map is not a member of Object
Which refers to the match of (result1, result2) below
def future1 = Future { ... }
def future2 = Future { ... }
def future3(a, b): Future[T] = Future { .... }
def future4: Future[T] = {
for {
result1 <- future1
result2 <- future2
result3 <- (result1, result2) match {
case (x, y) => future3(result1, result2)
case (_, _) => ???
}
} yield result3
}
It can be solved by making whatever is under ??? return Future[T]
Suppose I have the following code:
val futureInt1 = getIntAsync1();
val futureInt2 = getIntAsync2();
val futureSum = for {
int1 <- futureInt1
int2 <- futureInt2
} yield (int1 + int2)
val sum = Await.result(futureSum, 60 seconds)
Now suppose one of getIntAsync1 or getIntAsync2 takes a very long time, and it leads to Await.result throwing exception:
Caused by: java.util.concurrent.TimeoutException: Futures timed out after [60 seconds]
How am I supposed to know which one of getIntAsync1 or getIntAsync2 was still pending and actually lead to the timeout?
Note that here I'm merging 2 futures with zip, and this is a simple example for the question, but in my app I have this kind of code at different level (ie getIntAsync1 itself can use Future.zip or Future.sequence, map/flatMap/applicative)
Somehow what I'd like is to be able to log the pending concurrent operation stacktraces when a timeout occur on my main thread, so that I can know where are the bottlenexts on my system.
I have an existing legacy API backend which is not fully reactive yet and won't be so soon. I'm trying to increase response times by using concurrency. But since using this kind of code, It's become way more painful to understand why something takes a lot of time in my app. I would appreciate any tip you can provide to help me debugging such issues.
The key is realizing is that a the Future doesn't time out in your example—it's your calling thread which pauses for at most X time.
So, if you want to model time in your Futures you should use zipWith on each branch and zip with a Future which will contain a value within a certain amount of time. If you use Akka then you can use akka.pattern.after for this, together with Future.firstCompletedOf.
Now, even if you do, how do you figure out why any of your futures weren't completed in time, perhaps they depended on other futures which didn't complete.
The question boils down to: Are you trying to do some root-cause analysis on throughput? Then you should monitor your ExecutionContext, not your Futures. Futures are only values.
The proposed solution wraps each future from for block into TimelyFuture which requires timeout and name. Internally it uses Await to detect individual timeouts.
Please bear in mind this style of using futures is not intended for production code as it uses blocking. It is for diagnostics only to find out which futures take time to complete.
package xxx
import java.util.concurrent.TimeoutException
import scala.concurrent.{Future, _}
import scala.concurrent.duration.Duration
import scala.util._
import scala.concurrent.duration._
class TimelyFuture[T](f: Future[T], name: String, duration: Duration) extends Future[T] {
override def onComplete[U](ff: (Try[T]) => U)(implicit executor: ExecutionContext): Unit = f.onComplete(x => ff(x))
override def isCompleted: Boolean = f.isCompleted
override def value: Option[Try[T]] = f.value
#scala.throws[InterruptedException](classOf[InterruptedException])
#scala.throws[TimeoutException](classOf[TimeoutException])
override def ready(atMost: Duration)(implicit permit: CanAwait): TimelyFuture.this.type = {
Try(f.ready(atMost)(permit)) match {
case Success(v) => this
case Failure(e) => this
}
}
#scala.throws[Exception](classOf[Exception])
override def result(atMost: Duration)(implicit permit: CanAwait): T = {
f.result(atMost)
}
override def transform[S](ff: (Try[T]) => Try[S])(implicit executor: ExecutionContext): Future[S] = {
val p = Promise[S]()
Try(Await.result(f, duration)) match {
case s#Success(_) => ff(s) match {
case Success(v) => p.success(v)
case Failure(e) => p.failure(e)
}
case Failure(e) => e match {
case e: TimeoutException => p.failure(new RuntimeException(s"future ${name} has timed out after ${duration}"))
case _ => p.failure(e)
}
}
p.future
}
override def transformWith[S](ff: (Try[T]) => Future[S])(implicit executor: ExecutionContext): Future[S] = {
val p = Promise[S]()
Try(Await.result(f, duration)) match {
case s#Success(_) => ff(s).onComplete({
case Success(v) => p.success(v)
case Failure(e) => p.failure(e)
})
case Failure(e) => e match {
case e: TimeoutException => p.failure(new RuntimeException(s"future ${name} has timed out after ${duration}"))
case _ => p.failure(e)
}
}
p.future
}
}
object Main {
import scala.concurrent.ExecutionContext.Implicits.global
def main(args: Array[String]): Unit = {
val f = Future {
Thread.sleep(5);
1
}
val g = Future {
Thread.sleep(2000);
2
}
val result: Future[(Int, Int)] = for {
v1 <- new TimelyFuture(f, "f", 10 milliseconds)
v2 <- new TimelyFuture(g, "g", 10 milliseconds)
} yield (v1, v2)
val sum = Await.result(result, 1 seconds) // as expected, this throws exception : "RuntimeException: future g has timed out after 10 milliseconds"
}
}
If you are merely looking for informational metrics on which individual future was taking a long time (or in combination with others), your best bet is to use a wrapper when creating the futures to log the metrics:
object InstrumentedFuture {
def now() = System.currentTimeMillis()
def apply[T](name: String)(code: => T): Future[T] = {
val start = now()
val f = Future {
code
}
f.onComplete {
case _ => println(s"Future ${name} took ${now() - start} ms")
}
f
}
}
val future1 = InstrumentedFuture("Calculator") { /*...code...*/ }
val future2 = InstrumentedFuture("Differentiator") { /*...code...*/ }
You can check if a future has completed by calling its isComplete method
if (futureInt1.isComplete) { /*futureInt2 must be the culprit */ }
if (futureInt2.isComplete) { /*futureInt1 must be the culprit */ }
As a first approach i would suggest to lift your Future[Int] into Future[Try[Int]]. Something like that:
object impl {
def checkException[T](in: Future[T]): Future[Try[T]] =
in.map(Success(_)).recover {
case e: Throwable => {
Failure(new Exception("Error in future: " + in))
}
}
implicit class FutureCheck(s: Future[Int]) {
def check = checkException(s)
}
}
Then a small function to combine the results, something like that:
object test {
import impl._
val futureInt1 = Future{ 1 }
val futureInt2 = Future{ 2 }
def combine(a: Try[Int], b: Try[Int])(f: (Int, Int) => (Int)) : Try[Int] = {
if(a.isSuccess && b.isSuccess) {
Success(f(a.get, b.get))
}
else
Failure(new Exception("Error adding results"))
}
val futureSum = for {
int1 <- futureInt1.check
int2 <- futureInt2.check
} yield combine(int1, int2)(_ + _)
}
In futureSum you will have a Try[Int] with the integer or a Failure with the exception corresponding with the possible error.
Maybe this can be useful
val futureInt1 = getIntAsync1();
val futureInt2 = getIntAsync2();
val futureSum = for {
int1 <- futureInt1
int2 <- futureInt2
} yield (int1 + int2)
Try(Await.result(futureSum, 60 seconds)) match {
case Success(sum) => println(sum)
case Failure(e) => println("we got timeout. the unfinished futures are: " + List(futureInt1, futureInt2).filter(!_.isCompleted)
}
I have a Future[T] and I want to map the result, on both success and failure.
Eg, something like
val future = ... // Future[T]
val mapped = future.mapAll {
case Success(a) => "OK"
case Failure(e) => "KO"
}
If I use map or flatmap, it will only map successes futures. If I use recover, it will only map failed futures. onComplete executes a callback but does not return a modified future. Transform will work, but takes 2 functions rather than a partial function, so is a bit uglier.
I know I could make a new Promise, and complete that with onComplete or onSuccess/onFailure, but I was hoping there was something I was missing that would allow me to do the above with a single PF.
Edit 2017-09-18: As of Scala 2.12, there is a transform method that takes a Try[T] => Try[S]. So you can write
val future = ... // Future[T]
val mapped = future.transform {
case Success(_) => Success("OK")
case Failure(_) => Success("KO")
}
For 2.11.x, the below still applies:
AFAIK, you can't do this directly with a single PF. And transform transforms Throwable => Throwable, so that won't help you either. The closest you can get out of the box:
val mapped: Future[String] = future.map(_ => "OK").recover{case _ => "KO"}
That said, implementing your mapAll is trivial:
implicit class RichFuture[T](f: Future[T]) {
def mapAll[U](pf: PartialFunction[Try[T], U]): Future[U] = {
val p = Promise[U]()
f.onComplete(r => p.complete(Try(pf(r))))
p.future
}
}
Since Scala 2.12 you can use transform to map both cases:
future.transform {
case Success(_) => Try("OK")
case Failure(_) => Try("KO")
}
You also have transformWith if you prefer to use a Future instead of a Try. Check the documentation for details.
In a first step, you could do something like:
import scala.util.{Try,Success,Failure}
val g = future.map( Success(_):Try[T] ).recover{
case t => Failure(t)
}.map {
case Success(s) => ...
case Failure(t) => ...
}
where T is the type of the future result. Then you can use an implicit conversion to add this structure the Future trait as a new method:
implicit class MyRichFuture[T]( fut: Future[T] ) {
def mapAll[U]( f: PartialFunction[Try[T],U] )( implicit ec: ExecutionContext ): Future[U] =
fut.map( Success(_):Try[T] ).recover{
case t => Failure(t)
}.map( f )
}
which implements the syntax your are looking for:
val future = Future{ 2 / 0 }
future.mapAll {
case Success(i) => i + 0.5
case Failure(_) => 0.0
}
Both map and flatMap variants:
implicit class FutureExtensions[T](f: Future[T]) {
def mapAll[Target](m: Try[T] => Target)(implicit ec: ExecutionContext): Future[Target] = {
val promise = Promise[Target]()
f.onComplete { r => promise success m(r) }(ec)
promise.future
}
def flatMapAll[Target](m: Try[T] => Future[Target])(implicit ec: ExecutionContext): Future[Target] = {
val promise = Promise[Target]()
f.onComplete { r => m(r).onComplete { z => promise complete z }(ec) }(ec)
promise.future
}
}
Here is the problem, I have a library which has a blocking method return Try[T]. But since it's a blocking one, I would like to make it non-blocking using Future[T]. In the future block, I also would like to compute something that's depend on the origin blocking method's return value.
But if I use something like below, then my nonBlocking will return Future[Try[T]] which is less convince since Future[T] could represent Failure[U] already, I would rather prefer propagate the exception to Future[T] is self.
def blockMethod(x: Int): Try[Int] = Try {
// Some long operation to get an Int from network or IO
throw new Exception("Network Exception") }
}
def nonBlocking(x: Int): Future[Try[Int]] = future {
blockMethod(x).map(_ * 2)
}
Here is what I tried, I just use .get method in future {} block, but I'm not sure if this is the best way to do that.
def blockMethod(x: Int): Try[Int] = Try {
// Some long operation to get an Int from network or IO
throw new Exception("Network Exception") }
}
def nonBlocking(x: Int): Future[Int] = future {
blockMethod(x).get * 2
}
Is this correct way to do that? Or there is a more scala idiomatic way to convert t Try[T] to Future[T]?
Here's an example that doesn't block, note that you probably want to use your own execution context and not scala's global context:
import scala.util._
import scala.concurrent._
import scala.concurrent.duration._
import ExecutionContext.Implicits.global
object Main extends App {
def blockMethod(x: Int): Try[Int] = Try {
// Some long operation to get an Int from network or IO
Thread.sleep(10000)
100
}
def tryToFuture[A](t: => Try[A]): Future[A] = {
future {
t
}.flatMap {
case Success(s) => Future.successful(s)
case Failure(fail) => Future.failed(fail)
}
}
// Initiate long operation
val f = tryToFuture(blockMethod(1))
println("Waiting... 10 seconds to complete")
// Should return before 20 seconds...
val res = Await.result(f, 20 seconds)
println(res) // prints 100
}
In my opinion: Try & Future is a monadic construction and idiomatic way to is monadic composition (for-comprehension):
That you need to define monad transformer for Future[Try[_]] (code for your library):
case class FutureT[R](run : Future[Try[R]])(implicit e: ExecutionContext) {
def map[B](f : R => B): FutureT[B] = FutureT(run map { _ map f })
def flatMap[B](f : R => FutureT[B]): FutureT[B] = {
val p = Promise[Try[B]]()
run onComplete {
case Failure(e) => p failure e
case Success(Failure(e)) => p failure e
case Success(Success(v)) => f(v).run onComplete {
case Failure(e) => p failure e
case Success(s) => p success s
}
}
FutureT(p.future)
}
}
object FutureT {
def futureTry[R](run : => Try[R])(implicit e: ExecutionContext) =
new FutureT(future { run })
implicit def toFutureT[R](run : Future[Try[R]]) = FutureT(run)
implicit def fromFutureT[R](futureT : FutureT[R]) = futureT.run
}
and usage example:
def blockMethod(x: Int): Try[Int] = Try {
Thread.sleep(5000)
if(x < 10) throw new IllegalArgumentException
else x + 1
}
import FutureT._
// idiomatic way :)
val async = for {
x <- futureTry { blockMethod(15) }
y <- futureTry { blockMethod(25) }
} yield (x + y) * 2 // possible due to using modan transformer
println("Waiting... 10 seconds to complete")
val res = Await.result(async, 20 seconds)
println(res)
// example with Exception
val asyncWithError = for {
x <- futureTry { blockMethod(5) }
y <- futureTry { blockMethod(25) }
} yield (x + y) * 2 // possible due to using modan transformer
// Can't use Await because will get exception
// when extract value from FutureT(Failure(java.lang.IllegalArgumentException))
// no difference between Failure produced by Future or Try
asyncWithError onComplete {
case Failure(e) => println(s"Got exception: $e.msg")
case Success(res) => println(res)
}
// Output:
// Got exception: java.lang.IllegalArgumentException.msg