This is a followup to my previous question.
Suppose I have a task, which executes an interruptible blocking call. I would like to run it as a Future and cancel it with failure method of Promise.
I would like the cancel to work as follows:
If one cancels the task before it finished I would like the task to finish "immediately", interrupting the blocking call if it has already started and I would like the Future to invoke onFailure.
If one cancels the task after the task finished I would like to get a status saying that the cancel failed since the task already finished.
Does it make sense? Is it possible to implement in Scala? Are there any examples of such implementations?
scala.concurrent.Future is read-only, so one reader cannot mess things up for the other readers.
It seems like you should be able to implement what you want as follows:
def cancellableFuture[T](fun: Future[T] => T)(implicit ex: ExecutionContext): (Future[T], () => Boolean) = {
val p = Promise[T]()
val f = p.future
p tryCompleteWith Future(fun(f))
(f, () => p.tryFailure(new CancellationException))
}
val (f, cancel) = cancellableFuture( future => {
while(!future.isCompleted) continueCalculation // isCompleted acts as our interrupted-flag
result // when we're done, return some result
})
val wasCancelled = cancel() // cancels the Future (sets its result to be a CancellationException conditionally)
Here is the interruptable version of Victor's code per his comments (Victor, please correct me if I misinterpreted).
object CancellableFuture extends App {
def interruptableFuture[T](fun: () => T)(implicit ex: ExecutionContext): (Future[T], () => Boolean) = {
val p = Promise[T]()
val f = p.future
val aref = new AtomicReference[Thread](null)
p tryCompleteWith Future {
val thread = Thread.currentThread
aref.synchronized { aref.set(thread) }
try fun() finally {
val wasInterrupted = (aref.synchronized { aref getAndSet null }) ne thread
//Deal with interrupted flag of this thread in desired
}
}
(f, () => {
aref.synchronized { Option(aref getAndSet null) foreach { _.interrupt() } }
p.tryFailure(new CancellationException)
})
}
val (f, cancel) = interruptableFuture[Int] { () =>
val latch = new CountDownLatch(1)
latch.await(5, TimeUnit.SECONDS) // Blocks for 5 sec, is interruptable
println("latch timed out")
42 // Completed
}
f.onFailure { case ex => println(ex.getClass) }
f.onSuccess { case i => println(i) }
Thread.sleep(6000) // Set to less than 5000 to cancel
val wasCancelled = cancel()
println("wasCancelled: " + wasCancelled)
}
With Thread.sleep(6000) the output is:
latch timed out
42
wasCancelled: false
With Thread.sleep(1000) the output is:
wasCancelled: true
class java.util.concurrent.CancellationException
Twitter's futures implement cancellation. Have a look here:
https://github.com/twitter/util/blob/master/util-core/src/main/scala/com/twitter/util/Future.scala
Line 563 shows the abstract method responsible for this. Scala's futures currently do not support cancellation.
You can use Monix library instead of Future
https://monix.io
Related
I'm converting Future code to IO. I have code similar to this
def doSomething: Future[Foo] = {
Future {
//some code the result of which we don't care about
}
Future {
//Foo
}
}
And then at the end of the program, I doSomething.unsafeRunSync. How do I convert these Futures to IOs while maintaining the fire-and-forget functionality of the first Future? In using IO's async API, I am worried about accidentally blocking the thread when I later call unsafeRunSync on doSomething.
A solution that uses only cats-effect could use IO.start. This, combined with the fact that you will then never join the resulting Fiber, will look something like this:
import cats.effect._
import cats.implicits._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
object ExampleApp extends App{
val fireAndForget =
IO(println("Side effect pre-sleep")) *>
IO.sleep(2.seconds) *>
IO(println("Side effect post-sleep"))
val toBeUsed = IO{
println("Inside second one")
42
}
val result = for {
fiber <- IO.shift *> fireAndForget.start
res <- toBeUsed.handleErrorWith { error =>
// This is just in case you 'toBeUsed' can actually fail,
// and you might want to cancel the original side-effecting IO
fiber.cancel *> IO.raiseError(error) }
} yield res
println(result.unsafeRunSync())
println("Waiting 3 seconds...")
IO.sleep(3.seconds).unsafeRunSync()
println("Done")
}
This will print (most of the times) something similar to:
Side effect pre-sleep
Inside second one
42 // Up until here, will be printed right away
Waiting 3 seconds... // It will then be waiting a while
Side effect post-sleep // ...at which point the side effecting code terminates
Done
Finally, here are more details about Fiber and IO.shift
I believe that you need to wrap the first Future in such a way that it completes immediately. We ignore exeptions, or catch them however, but they are contained within its own thread. The parameter cb is the promise that needs to complete; so we short-circuit the completion by providing a value immediately.
def firstFuture(implicit ec: ExecutionContext): IO[Unit] = {
IO.async[Unit] { cb =>
ec.execute(() => {
try {
//some code the result of which we don't care about
} catch {
}
})
cb(Right(()))
}
}
In the for-comprehension, the firstFuture will complete immediately even though its thread will have a long-running task active on it.
def doSomething(implicit ec: ExecutionContext): IO[Foo] = {
for {
_ <- firstFuture
IO.async[Foo] { fb =>
// Foo
}
}
}
I have a Future lazy val that obtains some object and a function which submits operations in the Future.
class C {
def printLn(s: String) = println(s)
}
lazy val futureC: Future[C] = Future{Thread.sleep(3000); new C()}
def func(s: String): Unit = {
futureC.foreach{c => c.printLn(s)}
}
The problem is when Future is completed it executes operations in reverse order than they have been submited. So for example if I execute sequentialy
func("A")
func("B")
func("C")
I get after Future completion
scala> C
B
A
This order is important for me. Is there a way to preserve this order?
Of course I can use an actor who asks for future and stashing strings while future is not ready, but it seems redundant for me.
lazy val futureC: Future[C]
lazy vals in scala will be compiled in to the code which uses a synchronized block for thread safety.
Here when the func(A) is called, it will obtain the lock for the lazy val and that thread will go to sleep.
Therefore func(B) & func(C) will blocked by the lock.
When those blocked threads are run, the order cannot be guaranteed.
If you do it like below, you'll have the order as you expect. This is because the for comprehension creates a flatMap, & map based chain that gets executed sequentially.
lazy val futureC: Future[C] = Future {
Thread.sleep(1000)
new C()
}
def func(s: String) : Future[Unit] = {
futureC.map { c => c.printLn(s) }
}
val x = for {
_ <- func("A")
_ <- func("B")
_ <- func("C")
} yield ()
The order preserves even without the lazy keyword. You can remove the lazy keyword unless it is really necessary.
Hope this helps.
You can use Future.traverse to ensure the order of execution.
Something like this.. Im not sure how your func has a reference to the correct futureC, so I moved it inside.
def func(s: String): Future[Unit] = {
lazy val futureC = Future{Thread.sleep(3000); new C()}
futureC.map{c => c.printLn(s)}
}
def traverse[A,B](xs: Seq[A])(fn: A => Future[B]): Future[Seq[B]] =
xs.foldLeft(Future(Seq[B]())) { (acc, item) =>
acc.flatMap { accValue =>
fn(item).map { itemValue =>
accValue :+ itemValue
}
}
}
traverse(Seq("A","B","C"))(func)
I was hoping code like follows would wait for both futures, but it does not.
object Fiddle {
val f1 = Future {
throw new Throwable("baaa") // emulating a future that bumped into an exception
}
val f2 = Future {
Thread.sleep(3000L) // emulating a future that takes a bit longer to complete
2
}
val lf = List(f1, f2) // in the general case, this would be a dynamically sized list
val seq = Future.sequence(lf)
seq.onComplete {
_ => lf.foreach(f => println(f.isCompleted))
}
}
val a = FuturesSequence
I assumed seq.onComplete would wait for them all to complete before completing itself, but not so; it results in:
true
false
.sequence was a bit hard to follow in the source of scala.concurrent.Future, I wonder how I would implement a parallel that waits for all original futures of a (dynamically sized) sequence, or what might be the problem here.
Edit: A related question: https://worldbuilding.stackexchange.com/questions/12348/how-do-you-prove-youre-from-the-future :)
One common approach to waiting for all results (failed or not) is to "lift" failures into a new representation inside the future, so that all futures complete with some result (although they may complete with a result that represents failure). One natural way to get that is lifting to a Try.
Twitter's implementation of futures provides a liftToTry method that makes this trivial, but you can do something similar with the standard library's implementation:
import scala.util.{ Failure, Success, Try }
val lifted: List[Future[Try[Int]]] = List(f1, f2).map(
_.map(Success(_)).recover { case t => Failure(t) }
)
Now Future.sequence(lifted) will be completed when every future is completed, and will represent successes and failures using Try.
And so, a generic solution for waiting on all original futures of a sequence of futures may look as follows, assuming an execution context is of course implicitly available.
import scala.util.{ Failure, Success, Try }
private def lift[T](futures: Seq[Future[T]]) =
futures.map(_.map { Success(_) }.recover { case t => Failure(t) })
def waitAll[T](futures: Seq[Future[T]]) =
Future.sequence(lift(futures)) // having neutralized exception completions through the lifting, .sequence can now be used
waitAll(SeqOfFutures).map {
// do whatever with the completed futures
}
A Future produced by Future.sequence completes when either:
all the futures have completed successfully, or
one of the futures has failed
The second point is what's happening in your case, and it makes sense to complete as soon as one of the wrapped Future has failed, because the wrapping Future can only hold a single Throwable in the failure case. There's no point in waiting for the other futures because the result will be the same failure.
This is an example that supports the previous answer. There is an easy way to do this using just the standard Scala APIs.
In the example, I am creating 3 futures. These will complete at 5, 7, and 9 seconds respectively. The call to Await.result will block until all futures have resolved. Once all 3 futures have completed, a will be set to List(5,7,9) and execution will continue.
Additionally, if an exception is thrown in any of the futures, Await.result will immediately unblock and throw the exception. Uncomment the Exception(...) line to see this in action.
try {
val a = Await.result(Future.sequence(Seq(
Future({
blocking {
Thread.sleep(5000)
}
System.err.println("A")
5
}),
Future({
blocking {
Thread.sleep(7000)
}
System.err.println("B")
7
//throw new Exception("Ha!")
}),
Future({
blocking {
Thread.sleep(9000)
}
System.err.println("C")
9
}))),
Duration("100 sec"))
System.err.println(a)
} catch {
case e: Exception ⇒
e.printStackTrace()
}
Even though it is quite old question But this is how I got it running in recent time.
object Fiddle {
val f1 = Future {
throw new Throwable("baaa") // emulating a future that bumped into an exception
}
val f2 = Future {
Thread.sleep(3000L) // emulating a future that takes a bit longer to complete
2
}
val lf = List(f1, f2) // in the general case, this would be a dynamically sized list
val seq = Future.sequence(lf)
import scala.concurrent.duration._
Await.result(seq, Duration.Inf)
}
This won't get completed and will wait till all the future gets completed. You can change the waiting time as per your use case. I have kept it to infinite and that was required in my case.
We can enrich Seq[Future[T]] with its own onComplete method through an implicit class:
def lift[T](f: Future[T])(implicit ec: ExecutionContext): Future[Try[T]] =
f map { Success(_) } recover { case e => Failure(e) }
def lift[T](fs: Seq[Future[T]])(implicit ec: ExecutionContext): Seq[Future[Try[T]]] =
fs map { lift(_) }
implicit class RichSeqFuture[+T](val fs: Seq[Future[T]]) extends AnyVal {
def onComplete[U](f: Seq[Try[T]] => U)(implicit ec: ExecutionContext) = {
Future.sequence(lift(fs)) onComplete {
case Success(s) => f(s)
case Failure(e) => throw e // will never happen, because of the Try lifting
}
}
}
Then, in your particular MWE, you can do:
val f1 = Future {
throw new Throwable("baaa") // emulating a future that bumped into an exception
}
val f2 = Future {
Thread.sleep(3000L) // emulating a future that takes a bit longer to complete
2
}
val lf = List(f1, f2)
lf onComplete { _ map {
case Success(v) => ???
case Failure(e) => ???
}}
This solution has the advantage of allowing you to call an onComplete on a sequence of futures as you would on a single future.
Create the Future with a Try to avoid extra hoops.
implicit val ec = ExecutionContext.global
val f1 = Future {
Try {
throw new Throwable("kaboom")
}
}
val f2 = Future {
Try {
Thread.sleep(1000L)
2
}
}
Await.result(
Future.sequence(Seq(f1, f2)), Duration("2 sec")
) foreach {
case Success(res) => println(s"Success. $res")
case Failure(e) => println(s"Failure. ${e.getMessage}")
}
I have a class with some methods in Scala, each method has an execution time, for example methodA takes 10 seconds and methodB takes 5 seconds. and each method calls asynchronously.when I call methodB, it should cancel the thread that is running another methods. I first call methodA, and 2 seconds later I call methodB. What is the best solution for this problem?
def methodA()={
async{
// a job that takes 10 seconds
}
}
def methodB()={
async{
// other methods should stop their job
// a job that takes 5 second
}
}
def async[T](fn: => Unit): Unit = scala.actors.Actor.actor {
fn
}
.....
methodA()
methodB()
Here is an idea, based on the assumption that your method is actively checking whether it should still run or cancel:
import concurrent.{ExecutionContext, Future, Promise, blocking, future, promise}
case class Cancelled() extends RuntimeException
object Job {
def apply[A](fun: (() => Boolean) => A)(implicit ctx: ExecutionContext): Job[A] =
new Job[A] {
private val p = promise[A]
def result = p.future
def cancel(): Unit = p.tryFailure(Cancelled())
p tryCompleteWith future {
fun(() => !p.isCompleted)
}
}
}
trait Job[A] {
def result: Future[A]
def cancel(): Unit
}
So Job embodies a future along with a cancel() method. Your example could be similar to this:
import ExecutionContext.Implicits.global
val a = Job { active =>
for (i <- 1 to 100 if active()) {
blocking {
Thread.sleep(1000) // doing actual heavy work here
println(s"A $i")
}
}
}
val b = Job { active =>
for (i <- 1 to 20 if active()) {
blocking {
Thread.sleep(1000) // doing actual heavy work here
println(s"B $i")
}
}
println("Goodbye A...")
a.cancel()
}
I have also seen a rather harsh variant that calls Thread.interrupt.
suppose I have a sleep function:
def sleep(delay:Int) : Unit #suspendable = {
....
}
is it possible to have a function future that creates an async version of the sleep function that can be awaited on synchronously.
def future(targetFunc: (Int => Unit #suspendable)) : (Int => Future) = {
....
}
class Future {
def await : Unit #suspendable = {
....
}
}
you should be able to do something like this:
reset {
val sleepAsync = future(sleep)
val future1 = sleepAsync(2000)
val future2 = sleepAsync(3000)
future1.await
future2.await
/* finishes after a delay of 3000 */
}
the two calls to sleepAsync should appear to return straight away and the two calls to Future#await should appear to block. of course they all really fall off the end of reset and the code after is responsible for calling the continuation after the delay.
otherwise is there an alternative method to running two #suspendable functions in parallel and wait for both of them to complete?
I have a compilable gist with a skeleton of what i want to do: https://gist.github.com/1191381
object Forks {
import scala.util.continuations._
case class Forker(forks: Vector[() => Unit #suspendable]) {
def ~(block: => Unit #suspendable): Forker = Forker(forks :+ (() => block))
def joinIf(pred: Int => Boolean): Unit #suspendable = shift { k: (Unit => Unit) =>
val counter = new java.util.concurrent.atomic.AtomicInteger(forks.size)
forks foreach { f =>
reset {
f()
if (pred(counter.decrementAndGet)) k()
}
}
}
def joinAll() = joinIf(_ == 0)
def joinAny() = joinIf(_ == forks.size - 1)
}
def fork(block: => Unit #suspendable): Forker = Forker(Vector(() => block))
}
using fork(), we can now wait many "suspendables". use ~() to chain together suspendables. use joinAll() to wait for all suspendables and joinAny() to wait for just one. use joinIf() to customize join strategy.
object Tests extends App {
import java.util.{Timer, TimerTask}
import scala.util.continuations._
implicit val timer = new Timer
def sleep(ms: Int)(implicit timer: Timer): Unit #suspendable = {
shift { k: (Unit => Unit) =>
timer.schedule(new TimerTask {
def run = k()
}, ms)
}
}
import Forks._
reset {
fork {
println("sleeping for 2000 ms")
sleep(2000)
println("slept for 2000 ms")
} ~ {
println("sleeping for 4000 ms")
sleep(4000)
println("slept for 4000 ms")
} joinAll()
println("and we are done")
}
println("outside reset")
readLine
timer.cancel
}
and this is the output. program starts at time T:
sleeping for 2000 ms
sleeping for 4000 ms
outside reset <<<<<< T + 0 second
slept for 2000 ms <<<<<< T + 2 seconds
slept for 4000 ms <<<<<< T + 4 seconds
and we are done <<<<<< T + 4 seconds
I'm not sure that I completely understand the question, but here's a try:
import scala.util.continuations._
class Future(thread: Thread) {
def await = thread.join
}
object Future {
def sleep(delay: Long) = Thread.sleep(delay)
def future[A,B](f: A => B) = (a: A) => shift { k: (Future => Unit) =>
val thread = new Thread { override def run() { f(a) } }
thread.start()
k(new Future(thread))
}
def main(args:Array[String]) = reset {
val sleepAsync = future(sleep)
val future1 = sleepAsync(2000) // returns right away
val future2 = sleepAsync(3000) // returns right away
future1.await // returns after two seconds
future2.await // returns after an additional one second
// finished after a total delay of three seconds
}
}
Here, a Future instance is nothing more than a handle on a Thread, so you can use its join method to block until it finishes.
The future function takes a function of type A => B, and returns a function which, when supplied with an A will kick off a thread to run the "futured" function, and wrap it up in a Future, which is injected back into the continuation, thereby assigning it to val future1.
Is this anywhere close to what you were going for?