Zio run blocking backwardscompatible code - scala

(Hopefully) simple question on Scalaz Zio.
I have some old code that I refactored to Zio. I want one path of that code to keep behaving exactly as it was:
synchronous
blocking
on the current thread (this is a hard requirement)
How can I run an IO such that it behaves like the old blocking code?
I currently use:
private lazy val blockingRts = new RTS {}
def runBlocking[E, A](io: IO[E, A]): Either[E, A] = {
blockingRts.unsafeRun(io.attempt)
}
It seems to do the trick, but I am far from sure that this is correct. Is this 100% backwards compatible with the old code?

Okay, I finally looked under the hood and implemented something that seems to be fulfilling my requirements:
/**
* Executes the IO synchronous and blocking on the current thread, thus running an IO
* without any of the advantages of IO. This can be useful for maintaining backwards compatibility.
* Rethrows any exception that was not handled by the IO's error handling.
*/
#throws
def runLegacy[E, A](io: IO[E, A]): Either[E, A] = {
syncBlockingRunTimeSystem.unsafeRunSync[Nothing, Either[E, A]](io.either) match {
case Exit.Success(v) => v
case Exit.Failure(Cause.Die(exception)) => throw exception
case Exit.Failure(Cause.Interrupt) => throw new InterruptedException
case Exit.Failure(fail) => throw FiberFailure(fail)
}
}
private lazy val syncBlockingRunTimeSystem = Runtime(
(),
PlatformLive.fromExecutor(new Executor {
override def yieldOpCount: Int = Int.MaxValue
override def metrics: Option[ExecutionMetrics] = None
override def submit(runnable: Runnable): Boolean = {
runnable.run()
true
}
override def here: Boolean = true
})
)
I also wrote a couple of tests:
"runLegacy" should {
"run synchronous code in blocking fashion on current thread" in {
var runCount = 0
val io = IO.succeedLazy { runCount += 1 }
.map { _ => runCount +=1 }
.flatMap { _ =>
runCount += 1
IO.effect {
runCount += 1
Thread.currentThread()
}
}
runCount shouldBe 0
runLegacy(io) shouldBe Right(Thread.currentThread())
runCount shouldBe 4
}
"run parallel code sequentially on current thread" in {
val ios = (1 to 500).map { i => IO.succeedLazy { i } }
runLegacy(IO.reduceAll(IO.succeed(0), ios) {
case (a, b) => a + b
}) shouldBe Right((500 * 501) / 2)
}
"run many flatMaps without overflowing" in {
var runCount = 0
val io = IO.succeedLazy { runCount += 1 }
val manyIo = (1 to 9999).foldLeft(io) { case (acc, _) => acc.flatMap { _ => io } }
runLegacy(manyIo)
runCount shouldBe 10000
}
case object TestException extends Throwable
"handle sync blocking errors" in {
case object TestException extends Throwable
runLegacy(IO.effect(throw TestException)) shouldBe Left(TestException)
}
"rethrow unhandled exceptions" in {
assertThrows[TestException.type] {
runLegacy(IO.succeedLazy(throw TestException))
}
}
}

Related

Get partial result on Scala time limited best effort computation

Trying to execute a function in a given time frame, but if computation fails by TimeOut get a partial result instead of an empty exception.
The attached code solves it.
The timedRun function is from Computation with time limit
Any better approach?.
package ga
object Ga extends App {
//this is the ugly...
var bestResult = "best result";
try {
val result = timedRun(150)(bestEffort())
} catch {
case e: Exception =>
print ("timed at = ")
}
println(bestResult)
//dummy function
def bestEffort(): String = {
var res = 0
for (i <- 0 until 100000) {
res = i
bestResult = s" $res"
}
" " + res
}
//This is the elegant part from stackoverflow gruenewa
#throws(classOf[java.util.concurrent.TimeoutException])
def timedRun[F](timeout: Long)(f: => F): F = {
import java.util.concurrent.{ Callable, FutureTask, TimeUnit }
val task = new FutureTask(new Callable[F]() {
def call() = f
})
new Thread(task).start()
task.get(timeout, TimeUnit.MILLISECONDS)
}
}
I would introduce a small intermediate class for more explicitly communicating the partial results between threads. That way you don't have to modify non-local state in any surprising ways. Then you can also just catch the exception within the timedRun method:
class Result[A](var result: A)
val result = timedRun(150)("best result")(bestEffort)
println(result)
//dummy function
def bestEffort(r: Result[String]): Unit = {
var res = 0
for (i <- 0 until 100000) {
res = i
r.result = s" $res"
}
r.result = " " + res
}
def timedRun[A](timeout: Long)(initial: A)(f: Result[A] => _): A = {
import java.util.concurrent.{ Callable, FutureTask, TimeUnit }
val result = new Result(initial)
val task = new FutureTask(new Callable[A]() {
def call() = { f(result); result.result }
})
new Thread(task).start()
try {
task.get(timeout, TimeUnit.MILLISECONDS)
} catch {
case e: java.util.concurrent.TimeoutException => result.result
}
}
It's admittedly a bit awkward since you don't usually have the "return value" of a function passed in as a parameter. But I think it's the least-radical modification of your code that makes sense. You could also consider modeling your computation as something that returns a Stream or Iterator of partial results, and then essentially do .takeWhile(notTimedOut).last. But how feasible that is really depends on the actual computation.
First, you need to use one of the solution to recover after the future timed out which are unfortunately not built-in in Scala:
See: Scala Futures - built in timeout?
For example:
def withTimeout[T](fut:Future[T])(implicit ec:ExecutionContext, after:Duration) = {
val prom = Promise[T]()
val timeout = TimeoutScheduler.scheduleTimeout(prom, after)
val combinedFut = Future.firstCompletedOf(List(fut, prom.future))
fut onComplete{case result => timeout.cancel()}
combinedFut
}
Then it is easy:
var bestResult = "best result"
val expensiveFunction = Future {
var res = 0
for (i <- 0 until 10000) {
Thread.sleep(10)
res = i
bestResult = s" $res"
}
" " + res
}
val timeoutFuture = withTimeout(expensiveFunction) recover {
case _: TimeoutException => bestResult
}
println(Await.result(timeoutFuture, 1 seconds))

scala - cache and reload a future which will become invalid later

private class FutMemorizer[T](valid: T => Boolean)(f: () => Future[T]) {
private val ref = new AtomicReference[Promise[T]]
#scala.annotation.tailrec
final def get(): Future[T] = {
val nullableRef = ref.get()
val valid = checkPromise(ref.get())
if(valid) {
nullableRef.future
} else {
val p = Promise[T]
val success = ref.compareAndSet(nullableRef, p)
if(success) {
p.completeWith(f())
p.future
} else {
get()
}
}
}
private def checkPromise(nullable: Promise[T]) = {
nullable != null && {
nullable.future.value match {
case None => true // future is not complete all caller should wait
case Some(Success(v)) => valid(v)
case _ => false
}
}
}
}
I am implementing an Future memorizer that only cache a valid future value.
It must meet following requirements
Futures created by f never executed paralleled
get never return a invalid value (once invalid recall f() to reload)
Is my implementation correct ?
Is there a more functional or simpler way to do this (because I hardly prove correntness of mime)?
As far as I understand this is wrong:
p.completeWith(f())
The caller gets a future the value of which is (or will be sometimes) that of the future returned by f(), but it's not checked anywhere that this value satisfies or will satisfy valid(...); same for other callers that came while the resulting future returned by f() is in progress if it takes time. It's only when result of f() completes will the next caller probably start "fixing" it.
I would probably go about fixing this problem the following way (see the fixed method), with some stylistic changes:
class FutMemorizer[T](valid: T => Boolean)(f: () => Future[T]) {
private val ref = new AtomicReference[Future[T]]
#tailrec
final def get: Future[T] = {
val current = ref.get
if (current != null && isValid(current)) current
else {
val p = Promise[T]
val pf = p.future
if (ref.compareAndSet(current, pf)) {
p.completeWith(fixed(f))
pf
} else get
}
}
private def fixed(f: () => Future[T]): Future[T] =
f() flatMap { t =>
if (valid(t)) Future.successful(t) else fixed(f)
}
private def isValid(future: Future[T]) =
future.value match {
case None => true // future is not complete all caller should wait
case Some(Success(v)) => valid(v)
case _ => false
}
}
As for your question about a more functional way of doing it I guess f and valid having effects on the external state and basing their computations on it (which I guess is the point of having a memorizer with invalidation) would seriously hinder it.
Just find spray-cache already have this feature

Why does Future.firstCompletedOf not invoke callback on timeout?

I am doing Exercises from Learning Concurrent Programming in Scala.
For an exercise question in code comment.
Program prints proper output of HTML contents for proper URL and timeout sufficiently enough.
Program prints "Error occured" for proper URL and low timeout.
However for invalid URL "Error occured" is not printed. What is the problem with the code below?
/*
* Implement a command-line program that asks the user to input a URL of some website,
* and displays the HTML of that website. Between the time that the user hits ENTER and
* the time that the HTML is retrieved, the program should repetitively print a . to the
* standard output every 50 milliseconds, with a two seconds timeout. Use only futures
* and promises, and avoid the synchronization primitives from the previous chapters.
* You may reuse the timeout method defined in this chapter.
*/
object Excersices extends App {
val timer = new Timer()
def timeout(t: Long = 1000): Future[Unit] = {
val p = Promise[Unit]
val timer = new Timer(true)
timer.schedule(new TimerTask() {
override def run() = {
p success ()
timer cancel()
}
}, t)
p future
}
def printDot = println(".")
val taskOfPrintingDot = new TimerTask {
override def run() = printDot
}
println("Enter a URL")
val lines = io.Source.stdin.getLines()
val url = if (lines hasNext) Some(lines next) else None
timer.schedule(taskOfPrintingDot, 0L, 50.millisecond.toMillis)
val timeOut2Sec = timeout(2.second.toMillis)
val htmlContents = Future {
url map { x =>
blocking {
Source fromURL (x) mkString
}
}
}
Future.firstCompletedOf(Seq(timeOut2Sec, htmlContents)) map { x =>
timer cancel ()
x match {
case Some(x) =>
println(x)
case _ =>
println("Error occured")
}
}
Thread sleep 5000
}
As #Gábor Bakos said exception produces Failure which doesn't handled by map:
val fut = Future { Some(Source fromURL ("hhhttp://google.com")) }
scala> fut map { x => println(x) } //nothing printed
res12: scala.concurrent.Future[Unit] = scala.concurrent.impl.Promise$DefaultPromise#5e025724
To process failure - use recover method :
scala> fut recover { case failure => None } map { x => println(x) }
None
res13: scala.concurrent.Future[Unit] = scala.concurrent.impl.Promise$DefaultPromise#578afc83
In your context it's something like:
Future.firstCompletedOf(Seq(timeOut2Sec, htmlContents)) recover {case x => println("Error:" + x); None} map { x => ...}
The Complete Code after using recover as advised by #dk14:
object Exercises extends App {
val timer = new Timer()
def timeout(t: Long = 1000): Future[Unit] = {
val p = Promise[Unit]
val timer = new Timer(true)
timer.schedule(new TimerTask() {
override def run() = {
p success ()
timer cancel ()
}
}, t)
p future
}
def printDot = println(".")
val taskOfPrintingDot = new TimerTask {
override def run() = {
printDot
}
}
println("Enter a URL")
val lines = io.Source.stdin.getLines()
val url = if (lines hasNext) Some(lines next) else None
timer.schedule(taskOfPrintingDot, 0L, 50.millisecond.toMillis)
val timeOut2Sec = timeout(2.second.toMillis)
val htmlContents = Future {
url map { x =>
blocking {
Source fromURL (x) mkString
}
}
}
Future.firstCompletedOf(Seq(timeOut2Sec, htmlContents))
.recover { case x => println("Error:" + x); None }
.map { x =>
timer cancel ()
x match {
case Some(x) =>
println(x)
case _ =>
println("Timeout occurred")
}
}
Thread sleep 5000
}

How to communicate between threads in Scala?

I have a class with some methods in Scala, each method has an execution time, for example methodA takes 10 seconds and methodB takes 5 seconds. and each method calls asynchronously.when I call methodB, it should cancel the thread that is running another methods. I first call methodA, and 2 seconds later I call methodB. What is the best solution for this problem?
def methodA()={
async{
// a job that takes 10 seconds
}
}
def methodB()={
async{
// other methods should stop their job
// a job that takes 5 second
}
}
def async[T](fn: => Unit): Unit = scala.actors.Actor.actor {
fn
}
.....
methodA()
methodB()
Here is an idea, based on the assumption that your method is actively checking whether it should still run or cancel:
import concurrent.{ExecutionContext, Future, Promise, blocking, future, promise}
case class Cancelled() extends RuntimeException
object Job {
def apply[A](fun: (() => Boolean) => A)(implicit ctx: ExecutionContext): Job[A] =
new Job[A] {
private val p = promise[A]
def result = p.future
def cancel(): Unit = p.tryFailure(Cancelled())
p tryCompleteWith future {
fun(() => !p.isCompleted)
}
}
}
trait Job[A] {
def result: Future[A]
def cancel(): Unit
}
So Job embodies a future along with a cancel() method. Your example could be similar to this:
import ExecutionContext.Implicits.global
val a = Job { active =>
for (i <- 1 to 100 if active()) {
blocking {
Thread.sleep(1000) // doing actual heavy work here
println(s"A $i")
}
}
}
val b = Job { active =>
for (i <- 1 to 20 if active()) {
blocking {
Thread.sleep(1000) // doing actual heavy work here
println(s"B $i")
}
}
println("Goodbye A...")
a.cancel()
}
I have also seen a rather harsh variant that calls Thread.interrupt.

Converting Scala #suspendable Method into a Future

suppose I have a sleep function:
def sleep(delay:Int) : Unit #suspendable = {
....
}
is it possible to have a function future that creates an async version of the sleep function that can be awaited on synchronously.
def future(targetFunc: (Int => Unit #suspendable)) : (Int => Future) = {
....
}
class Future {
def await : Unit #suspendable = {
....
}
}
you should be able to do something like this:
reset {
val sleepAsync = future(sleep)
val future1 = sleepAsync(2000)
val future2 = sleepAsync(3000)
future1.await
future2.await
/* finishes after a delay of 3000 */
}
the two calls to sleepAsync should appear to return straight away and the two calls to Future#await should appear to block. of course they all really fall off the end of reset and the code after is responsible for calling the continuation after the delay.
otherwise is there an alternative method to running two #suspendable functions in parallel and wait for both of them to complete?
I have a compilable gist with a skeleton of what i want to do: https://gist.github.com/1191381
object Forks {
import scala.util.continuations._
case class Forker(forks: Vector[() => Unit #suspendable]) {
def ~(block: => Unit #suspendable): Forker = Forker(forks :+ (() => block))
def joinIf(pred: Int => Boolean): Unit #suspendable = shift { k: (Unit => Unit) =>
val counter = new java.util.concurrent.atomic.AtomicInteger(forks.size)
forks foreach { f =>
reset {
f()
if (pred(counter.decrementAndGet)) k()
}
}
}
def joinAll() = joinIf(_ == 0)
def joinAny() = joinIf(_ == forks.size - 1)
}
def fork(block: => Unit #suspendable): Forker = Forker(Vector(() => block))
}
using fork(), we can now wait many "suspendables". use ~() to chain together suspendables. use joinAll() to wait for all suspendables and joinAny() to wait for just one. use joinIf() to customize join strategy.
object Tests extends App {
import java.util.{Timer, TimerTask}
import scala.util.continuations._
implicit val timer = new Timer
def sleep(ms: Int)(implicit timer: Timer): Unit #suspendable = {
shift { k: (Unit => Unit) =>
timer.schedule(new TimerTask {
def run = k()
}, ms)
}
}
import Forks._
reset {
fork {
println("sleeping for 2000 ms")
sleep(2000)
println("slept for 2000 ms")
} ~ {
println("sleeping for 4000 ms")
sleep(4000)
println("slept for 4000 ms")
} joinAll()
println("and we are done")
}
println("outside reset")
readLine
timer.cancel
}
and this is the output. program starts at time T:
sleeping for 2000 ms
sleeping for 4000 ms
outside reset <<<<<< T + 0 second
slept for 2000 ms <<<<<< T + 2 seconds
slept for 4000 ms <<<<<< T + 4 seconds
and we are done <<<<<< T + 4 seconds
I'm not sure that I completely understand the question, but here's a try:
import scala.util.continuations._
class Future(thread: Thread) {
def await = thread.join
}
object Future {
def sleep(delay: Long) = Thread.sleep(delay)
def future[A,B](f: A => B) = (a: A) => shift { k: (Future => Unit) =>
val thread = new Thread { override def run() { f(a) } }
thread.start()
k(new Future(thread))
}
def main(args:Array[String]) = reset {
val sleepAsync = future(sleep)
val future1 = sleepAsync(2000) // returns right away
val future2 = sleepAsync(3000) // returns right away
future1.await // returns after two seconds
future2.await // returns after an additional one second
// finished after a total delay of three seconds
}
}
Here, a Future instance is nothing more than a handle on a Thread, so you can use its join method to block until it finishes.
The future function takes a function of type A => B, and returns a function which, when supplied with an A will kick off a thread to run the "futured" function, and wrap it up in a Future, which is injected back into the continuation, thereby assigning it to val future1.
Is this anywhere close to what you were going for?