The andThen meaning I learned from this answer is a function composer.
Say that
f andThen g andThen h
will equal to
h(g(f(x)))
This implies the h function will receive input from g(f(x))
But for the andThen in Future, all the closure of the following andThen always receives the result from the original Future.
Future{
1
}.andThen{ case Success(x) =>
println(x) // print 1
Thread.sleep(2000)
x * 2
}.andThen{ case Success(x) =>
println(x) // print 1
Thread.sleep(2000)
x * 2
}
compare to
val func: Function1[Int, Int] = { x: Int =>
x
}.andThen { y =>
println(y) // print 1
y * 2
}.andThen { z =>
println(z) // print 2
z * 2
}
func(1)
What is the reason to make Future::andThen(s) receive all the same result from original Future instead of chaining Future? I've observed that these chained andThen will be executed sequentially, so the reason may not be for parallel purpose.
scala.concurrent.Future is designed as compromise of two asynchronous approaches:
Object-oriented observer which allows binding of asynchronous handlers
Functional monad which offers rich functional composition capabilities.
Reading Future.andThen's docs:
Applies the side-effecting function to the result of this future, and
returns a new future with the result of this future.
So andThen is most likely from OOP universe. To gain similar similar result to Function1.andThen you could use map method :
Future(1).map {_ * 2}.map {_ * 2}
andThen differs from onComplete with one little thing: resulting Future of andThen still returning same result, but will wait until supplied observer will return or throw something. That's why there is written in the docs:
This method allows one to enforce that the callbacks are executed in a
specified order.
Also note third line from docs:
Note that if one of the chained andThen callbacks throws an exception,
that exception is not propagated to the subsequent andThen callbacks.
Instead, the subsequent andThen callbacks are given the original value
of this future.
So it' completely do nothing with new Future's result. Could not even spoil it with it's ownt exception. This andThen and onComplete just sequential and parallel binding of observers.
Let me sum up this nice discussion.
Say, we have tf: Future[T] =..., and two functions, f: T => U and g: U => V
We can do vf: Future[V] = tf map f map g, same asvf: Future[V] = tf map (f andThen g)
In another use case, having fp: PartialFunction[T, U] and gp: PartialFunction[U, V],
we can run tf1: Future[T] = tf andThen fp andThen gp - these partial functions will be called on the value that tf produces, with no outside effect - only side effects happen. This sequence waits for fp before calling gp.
Yet another future operation, onComplete, works like this: having f: Try[T] => U, the call tf onComplete f will call f even if the future ended with an error; the result of tf onComplete f is of type Unit.
Also, if your function f produces a Future, you will need to use flatMap.
Related
I'm looking for a way to lazily compose two effects without first executing their results in Zio. My program takes the following form:
/**
* Returns a reference to an effectful singleton cron scheduler backed by akka
* See https://github.com/philcali/cronish for more info on the API
*/
def scheduled: UManaged[Ref[Scheduled]] = ???
def schedule[R, E, A](e: => ZIO[R, E, A], crondef: String) =
(for {
resource <- scheduled
task <- ZManaged.fromEffect(e) // I need to lift the underlying effect here, not access its result
} yield resource.modify(schedule => schedule(job(task), crondef.cron) -> schedule)).flattenM
def scheduleEffect[A](e: => A, description: String = "")(crondef: String) =
(for {
resource <- scheduled
} yield resource.modify(schedule => schedule(job(e), crondef.cron) -> schedule)).flattenM
// Program which schedules cron jobs to increment/decrement x and y, respectively
def run(args: List[String]): URIO[ZEnv, ExitCode] = {
var x = 0
var y = 100
(for {
_ <- Scheduler.schedule(UIO({ x += 1; println(x) }), "every second")
_ <- Scheduler.scheduleEffect({ y -= 1; println(y) }, "every second")
} yield ())
.provideCustomLayer(???)
.as(ExitCode.success)
.useForever
}
In this current formulation, the decrementing of y runs every second until the program terminates, while the incrementing of x only runs once. I know that ZIO provides a Schedule utility, but for legacy compatibility reasons I have to stick with the effectful singleton used by the Cronish library. Basically job takes a pass-by-reference effect of type A and suspends it in a CronTask for execution within the Scheduled singleton according to the schedule defined by crondef.
What I am wondering is if it is possible to compose the effects themselves, rather than their results in the context of ZIO? I've basically wrapped the legacy cron scheduler in ZIO data types to manage the concurrency properly, but I still need the suspended effect from other ZIO-signature methods in my code to be available for me to pass down into the scheduler.
I ultimately found the solution by reading through the source code for ZIO.effectAsyncM, specifically noting its reference to ZIO.runtime[R]:
/**
* Imports an asynchronous effect into a pure `ZIO` value. This formulation is
* necessary when the effect is itself expressed in terms of `ZIO`.
*/
def effectAsyncM[R, E, A](
register: (ZIO[R, E, A] => Unit) => ZIO[R, E, Any]
): ZIO[R, E, A] =
for {
p <- Promise.make[E, A]
r <- ZIO.runtime[R] // This line right here!
a <- ZIO.uninterruptibleMask { restore =>
val f = register(k => r.unsafeRunAsync_(k.to(p)))
restore(f.catchAllCause(p.halt)).fork *> restore(p.await)
}
} yield a
Though I was not able to find a direct reference to this method in the ZIO docs, the Scaladocs were clear enough:
Returns an effect that accesses the runtime, which can be used to (unsafely) execute tasks. This is useful for integration with legacy code that must call back into ZIO code.
With this, my new implementation works beautifully as follows:
def schedule[R, E, A](e: => ZIO[R, E, A], crondef: String) =
(for {
resource <- scheduled
runtime <- ZManaged.fromEffect(ZIO.runtime[R])
} yield resource.modify({
schedule => schedule(job(runtime.unsafeRun(e)), crondef.cron) -> schedule
})).flattenM
def computeComplexNumber: () ⇒ Future[Int] =
() ⇒
Future {
println("Computing complex number ...")
Int.MaxValue
}
def convert(f: () ⇒ Future[Int]): Future[() ⇒ Int] = ???
convert(computeComplexNumber)
Is it possible to implement convert method with above signature with condition that when convert(computeComplexNumber) is called, nothing gets printed unless you do this
Await.result(convert(computeComplexNumber), 5.seconds)()
I don't see any way to do it without blocking inside convert. There is simply no way to get hold of an instance of Future[X] inside of convert, without invoking f. But once you invoke f, the Future { ... } will eagerly start to evaluate the block with the println, and produce an output.
You probably want to take a look at scalaz.concurrent.Task instead.
If you want to convert () => Future[Int] to () => Int you have to wait for the result of the Future like this:
def convert(f: () ⇒ Future[Int]): Future[() ⇒ Int] =
Future{
() => Await.result(f(), Duration.Inf)
}
Then
val f = convert(computeComplexNumber) // Get Future
val g = Await.result(f, Duration.Inf) // Get Function
g() // Call function
The output will not appear until you call g.
Explanation in response to comments
The question
At the core of this question is this function signature:
def convert(f: () ⇒ Future[Int]): Future[() ⇒ Int]
There is also a requirement that (in paraphrase)
f is not invoked until the returned function is invoked
Also, I am assuming that the Int in the result is the value returned by the Future in the argument, not just some random number!
The issues
The return type of the input function is Future[Int] and the return type of the output function is Int. The first is an asynchronous result and the second is a synchronous result. Any solution must therefore convert from asynchronous Future[Int] to synchronous Int. This means Await.result or something similar.
There is also a requirement that the input function is not executed until the output function is executed. This means that it must be called inside the returned function which, in turn, means that the Await.result must be called inside the returned function.
The solution
The solution above has the function signature of convert that is required by the question. It also does not invoke f until the function returned by convert is executed, which was the other requirement.
This implementation of convert is not blocking; it returns immediately with a Future value. It does not invoke f, it just returns a function that does.
The comments
There is some dispute about what it means to block inside a function, or for one function to invoke another. Blocking and invoking are processes that that happen during the execution of a function. Therefore a sensible definition of these things happening "inside" a function would be that they happen between the execution of the entry point of the function and the execution of the exit point of the function. Using this definition it is clear that the solution does not block inside the function or invoke f inside the function.
The questioner has also commented that Await.result should not be called anywhere. Unfortunately, as explained above, this is not possible because the requirement is to convert an asynchronous result into a synchronous result.
Maybe something like this would work:
def computeComplexNumber: () => Future[Int] = () =>
Future {
delay(4.seconds.fromNow)
println("Computing complex number ...")
Int.MaxValue
}
def convert(f: () => Future[Int]): Future[() => Int] = {
f().map( m => () => m)
}
def delay(dur:Deadline) = {
Try(Await.ready(Promise().future, dur.timeLeft))
}
I am trying to build a new control structure which create a thread for each of its argument, and run them in parallel. The code seems to be fine when I built two future manually for each input, because I see the fast thread finishes before the slow thread.
Here is output:
fast
slow
However, if I use List(a, b).map(f => Future {f}) then I always see fast thread is executed after slow is done. Here is the output:
slow
fast
Can someone explain this?
code pasted here:
import java.util.concurrent.Executors
import scala.concurrent.{ExecutionContext, Future}
object ExecInParallel extends App {
def run(a: => Unit, b: => Unit): Unit = {
val executorService = Executors.newFixedThreadPool(2)
implicit val executionContext =
ExecutionContext.fromExecutorService(executorService)
// af and bf are executed in parallel
val af = Future(a)
val bf = Future(b)
// however, the following code is not parallel
List(a, b).map(f => Future(f))
Thread.sleep(3000)
executorService.shutdown
}
run(
{
Thread.sleep(2000)
println("slow")
},
{
Thread.sleep(1000)
println("fast")
}
)
}
That's because a and b are evaluated every time they are referenced in a non-by-name position and List(a, b) arguments are not by-name. From https://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_name:
Call by name is an evaluation strategy where the arguments to a function are not evaluated before the function is called—rather,... then left to be evaluated whenever they appear in the function. If an argument is not used in the function body, the argument is never evaluated; if it is used several times, it is re-evaluated each time it appears.
Effectively that is equivalent to this code:
List({
Thread.sleep(2000)
println("slow")
},
{
Thread.sleep(1000)
println("fast")
}).map(f => Future(f))
Since the List constructor doesn't take by-name arguments, these values are evaluated before the list itself is even constructed
This happens because you first create list of two call-by name values:
List(a, b)...
and until a and b not fully computed the map operation is not executed.
When List(a,b) is ready you wrap it in Futures:
List(a, b).map(f => Future(f))
Your by-name a and b is executed (sequentially) in List(a, b), before the construction of Future in map. If you check the inferred type of List(a, b) you'd see it's a List[Unit].
To achieve what you intented, you need a list of functions rather than list of results.
The following will work:
List(a _, b _).map(f => Future(f()))
If I have some computation that takes a while I might place it in a scala.concurrent.Future:
val f = Future { someLongRunningFunction() }
and let's say I want to do something else asynchronously once that computation is completed:
f.flatMap{ _ => anotherLongRunningFunction() }
In the event that f's initial block fails, how do I "idiomatically" handle this when using flatMap or other combinators? Is it merely a case of using recover or onFailure before the flatMap?
I like the elegance and simplicity of using a flatMap but it seems failure scenarios get in the way of it.
Edit: the second future is reliant on the first, hence the flatMap. I'm looking for a solution that'll elegantly let me chain like I would with flatMap but also handle failures of the first.
To quote the scaladoc for flatMap:
Creates a new future by applying a function to the successful result
of this future, and returns the result of the function as the new
future. If this future is completed with an exception then the new
future will also contain this exception.
Notice the bold, meaning that whatever you pass to flatMap will only be executed if the initial future completes successfully.
So you should only handle the result of the entire execution:
val result = future1.flatMap {
result => functionReturningFuture2(result)
}
and then:
result.onFailure // or
result.onSuccess // or
result.recover
If you have several futures you can put them in a for comprehension.
val futureResult = for {
result1 <- future1
result2 <- future2
...
} yield {
//computation with results
}
You can add a recover at the end in case you want to process any exception you may find:
futureResult.recover{
case exceptionResult: Throwable => // Process exception
}
I think this is more clean that using flatMap.
I'm only starting with Future, but here are some ideas.
If you really want to use flatMap, you have to turn the failure into a success.
for{ a <- f recover r
b <- another(a)
} yield b
This works if the return type of r is :> the result type of f.
Or you can pass the problem of what to do with the failure on to the next process
for{ a <- f map (x => Success(x)) recover (ex => Failure(ex))
b <- another(a)
} yield b
Here the argument type of another would be Try[T] where the type of f is Future[T].
Being writing a completely async library to access a remote service (using Play2.0), I'm using Promise and Validation to create non-blocking call, which has a type presenting fail and valid result at once.
Promise comes from Play2-scala, where Validation comes from scalaz.
So here is the type of examples of such functions
f :: A => Promise[Validation[E, B]]
g :: B => Promise[Validation[E, C]]
So far, so good, now if I want to compose them, I can simple use the fact that Promise present a flatMap, so I can do it with a for-comprehension
for (
x <- f(a);
y <- g(b)
) yield y
Ok, I took a shortcut to my problem here because I didn't reused the Validation results within the for-comprehension. So if I want to reuse x in g, here is how I could do
for (
x <- f(a); // x is a Validation
y <- x.fold(
fail => Promise.pure(x),
ok => g(ok)
)
) yield y
Fair enough, but this kind of boilerplate will go to pollute my code over and over again. The problem here is that I've a kind of two-levels Monadic structure like M[N[_]].
At this stage, is there any structure in f° programming that enables working with such structure by skipping easily the secong level:
for (
x <- f(a); //x is a B
y <- g(b)
) yield y
Now, below is how I achieved something similar.
I created kind of Monadic structure that wraps the two level in one, let say ValidationPromised which pimped the Promise type with two methods:
def /~> [EE >: E, B](f: Validation[E, A] => ValidationPromised[EE, B]): ValidationPromised[EE, B] =
promised flatMap { valid =>
f(valid).promised
}
def /~~>[EE >: E, B](f: A => ValidationPromised[EE, B]): ValidationPromised[EE, B] =
promised flatMap { valid =>
valid.fold (
bad => Promise.pure(KO(bad)),
good => f(good).promised
)
}
This allows me to do such things
endPoint.service /~~> //get the service
(svc => //the service
svc.start /~~> (st => //get the starting elt
svc.create(None) /~~> //svc creates a new elt
(newE => //the created one
newEntry.link(st, newE) /~~> //link start and the new
(lnk => Promise.pure(OK((st, lnk, newE)))) //returns a triple => hackish
)
)
)
As we can see /~~> is pretty similar to flatMap but skips one level. The problem is the verbosity (that's why "for-comprehension" exists in Scala and "do" in Haskell).
Another point, I've the /~> that stands like a map also but works on the second level (instead of the Valid type -- third level)
So my second question is corollary to the former... Am I approching a sustainable solution with this construction ?
sorry to be that long
The concept you are looking for here is monad transformers. In brief, monad transformers compensate for monads not composing by allowing you to "stack" them.
You didn't mention the version of Scalaz you are using, but if you look in the scalaz-seven branch, you'll find ValidationT. This can be used to wrap any F[Validation[E, A]] into a ValidationT[F, E, A], where in your case F = Promise. If you change f and g to return ValidationT, then you can leave your code as
for {
x ← f(a)
y ← g(b)
} yield y
This will give you a ValidationT[Promise, E, B] as a result.