Difference between Promise.failure and throwing exception? - scala

Is there any difference between these two ways of completing a failed Future? If so, which way is considered to be more "correct"?
Calling Promise.failure:
def functionThatFinishesLater: Future[String] = {
val myPromise = Promise[String]
Future {
// Do something that might fail
if (failed) {
myPromise.failure(new RuntimeException("message")) // complete with throwable
} else {
myPromise.success("yay!")
}
} (aDifferentExecutionContext)
myPromise.future
}
Or just throwing an exception
def functionThatFinishesLater: Future[String] = {
val myPromise = Promise[String]
Future {
// Do something that might fail
if (failed) {
throw new RuntimeException("message") // throw the exception
} else {
myPromise.success("yay!")
}
} (aDifferentExecutionContext)
myPromise.future
}

It looks to me like you're mixing paradigms. A Promise is an imperative way of completing a Future, but a Future can also be made completed by wrapping the computation in a Future constructor. You're doing both, which is probably not what you want. The second statement in both code fragments is of type Future[Promise[String]], and I'm almost certain you really want just Future[String].
If you're using using the Future.apply constructor, you should just treat the value produced as the Future, rather than using it to resolve a separate Promise value:
val myFuture = Future {
// Do some long operation that might fail
if (failed) {
throw new RuntimeException("message")
} else {
"yay!"
}
}
The way to use the Promise is to create the Promise, give its Future to some other piece of code that cares, and then use .success(...) or .failure(...) to complete it after some long running operation. So to recap, the big difference is that Future has to wrap the whole computation, but you can pass a Promise around and complete it elsewhere if you need to.

Related

Multiple finally clauses

Assume you have two different objects requiring cleanup after execution using a call to method cleanup(). I would do this using a try-finally construct.
val objA = new ObjA()
val objB = new ObjB()
try {
// do failure prone important stuff
} finally {
objA.cleanup()
objB.cleanup()
}
However, the cleanup function might also throw an exception. Is there a more elegant way of solving this than having another try-finally in the finally block?
// cluttering working solution:
val objA = new ObjA()
val objB = new ObjB()
try {
// do failure prone important stuff
} finally {
try {
objA.cleanup()
} finally {
objB.cleanup()
}
}
// Is there an option to do something more readable like:
try {
//do failure prone important stuff
} finally {
objA.cleanup()
} finally {
objB.cleanup()
}
I concede that having a cleanup function that may throw exceptions breaks the contract of a cleanup function. However, you do not always control external APIs.
How about using scala.util.Try to wrap the cleanup calls ?
import scala.util.Try
try {
// do failure prone important stuff
} finally {
Try(objA.cleanup())
Try(objB.cleanup())
}
How about this - two blocks, outer dealing with objA, inner dealing with objB, and exceptions, no matter where they get thrown ('important stuff' or any of two cleanup calls), can be caught 'outside' of both of them:
val objA = new ObjA()
try {
val objB = new ObjB()
try {
// do failure prone important stuff
} finally {
objB.cleanup()
}
} finally {
objA.cleanup()
}

Can Spark ForEachPartitionAsync be async on worker nodes?

I write a custom spark sink. In my addBatch method I use ForEachPartitionAsync which if I'm not wrong only makes the driver work asynchronously, returning a future.
val work: FutureAction[Unit] = rdd.foreachPartitionAsync { rows =>
val sourceInfo: StreamSourceInfo = serializeRowsAsInputStream(schema, rows)
val ackIngestion = Future {
ingestRows(sourceInfo) } andThen {
case Success(ingestion) => ackIngestionDone(partitionId, ingestion)
}
Await.result(ackIngestion, timeOut) // I would like to remove this line..
}
work onSuccess {
case _ => // move data from temporary table, report success of all workers
}
work onFailure{
//delete tmp data
case t => throw t.getCause
}
I can't find a way to run the worker nodes without blocking on the Await call, as if I remove them a success is reported to the work future object although the future didn't really finish.
Is there a way to report to the driver that all the workers finished
their asynchronous jobs?
Note: I looked at the foreachPartitionAsync function and it has only one implementation that expects a function that returns a Unit (i would've expected it to have another one returning a future or maybe a CountDownLatch..)

Repeat Single based on onSuccess() value

I want to repeat a Single based on the single value emitted in onSuccess(). Here is a working example
import org.reactivestreams.Publisher;
import io.reactivex.Flowable;
import io.reactivex.Single;
import io.reactivex.functions.Function;
public class Temp {
void main() {
Job job = new Job();
Single.just(job)
.map(this::processJob)
.repeatWhen(new Function<Flowable<Object>, Publisher<?>>() {
#Override
public Publisher<?> apply(Flowable<Object> objectFlowable) throws Exception {
// TODO repeat when Single emits false
return null;
}
})
.subscribe();
}
/**
* returns true if process succeeded, false if failed
*/
boolean processJob(Job job) {
return true;
}
class Job {
}
}
I understand how repeatWhen works for Observables by relying on the "complete" notification. However since Single doesn't receive that notification I'm not sure what the Flowable<Object> is really giving me. Also why do I need to return a Publisher from this function?
Instead of relying on a boolean value, you could make your job throw an exception when it fails:
class Job {
var isSuccess: Boolean = false
}
fun processJob(job: Job): String {
if (job.isSuccess) {
return "job succeeds"
} else {
throw Exception("job failed")
}
}
val job = Job()
Single.just(job)
.map { processJob(it) }
.retry() // will resubscribe until your job succeeds
.subscribe(
{ value -> print(value) },
{ error -> print(error) }
)
i saw a small discrepancy in the latest docs and your code, so i did a little digging...
(side note - i think the semantics of retryWhen seem like the more appropriate operator for your case, so i've substituted it in for your usage of repeatWhen. but i think the root of your problem remains the same in either case).
the signature for retryWhen is:
retryWhen(Function<? super Flowable<Throwable>,? extends Publisher<?>> handler)
that parameter is a factory function whose input is a source that emits anytime onError is called upstream, giving you the ability to insert custom retry logic that may be influenced through interrogation of the underlying Throwable. this begins to answer your first question of "I'm not sure what the Flowable<Object> is really giving me" - it shouldn't be Flowable<Object> to begin with, it should be Flowable<Throwable> (for the reason i just described).
so where did Flowable<Object> come from? i managed to reproduce IntelliJ's generation of this code through it's auto-complete feature using RxJava version 2.1.17. upgrading to 2.2.0, however, produces the correct result of Flowable<Throwable>. so, see if upgrading to the latest version generates the correct result for you as well.
as for your second question of "Also why do I need to return a Publisher from this function?" - this is used to determine if re-subscription should happen. if the factory function returns a Publisher that emits a terminal state (ie calls onError() or onComplete()) re-subscription will not happen. however, if onNext() is called, it will. (this also explains why the Publisher isn't typed - the type doesn't matter. the only thing that does matter is what kind of notification it publishes).
another way to rewrite this, incorporating the above, might be as follows:
// just some type to use as a signal to retry
private class SpecialException extends RuntimeException {}
// job processing results in a Completable that either completes or
// doesn't (by way of an exception)
private Completable rxProcessJob(Job job) {
return Completable.complete();
// return Completable.error(new SpecialException());
}
...
rxProcessJob(new Job())
.retryWhen(errors -> {
return errors.flatMap(throwable -> {
if(throwable instanceof SpecialException) {
return PublishProcessor.just(1);
}
return PublishProcessor.error(throwable);
});
})
.subscribe(
() -> {
System.out.println("## onComplete()");
},
error -> {
System.out.println("## onError(" + error.getMessage() + ")");
}
);
i hope that helps!
The accepted answer would work, but is hackish. You don't need to throw an error; simply filter the output of processJob which converts the Single to a Maybe, and then use the repeatWhen handler to decide how many times, or with what delay, you may want to resubscribe. See Kotlin code below from a working example, you should be able to easily translate this to Java.
filter { it }
.repeatWhen { handler ->
handler.zipWith(1..3) { _, i -> i }
.flatMap { retryCount -> Flowable.timer(retryDelay.toDouble().pow(retryCount).toLong(), TimeUnit.SECONDS) }
.doOnNext { log.warn("Retrying...") }
}

How to do: Try {block of code} for X duration and catch {Exceptions, such as Timeout}?

Alright so the title is a little obscure since I was unsure how to word it, but essentially I am trying to do a try-catch statement that will timeout... Here's some pseudo-code that may help describe what I'm trying to do:
try (10 seconds) {
*make some connection and do some things*
} catch {
case ex1: TimeoutException => *do something*
case ex2: Exception => *do something else*
}
Currently there is a bug in the hardware I'm working with where the request for a connection never gets a response back, so it just sits there and doesn't catch any exceptions. Since it's only a bug (that should be temporary), I don't want to manipulate the architecture of the application (specifically I do not want to create a new actor just to account for something small) and it would be very ideal if I could implement this pseudo-code within the scope of the class.
So essentially my question is how do I implement the pseudo-code above within the scope of the class it's in?
Let me know if anything is unclear! Thank you!
Try:
import scala.concurrent._
import ExecutionContext.Implicits.global
val f = future {
// make some connection and do some things
}
try {
Await.result(f, 10 seconds);
} catch {
case e: TimeoutException => // do something
case _ => // do something else
}
More info: Futures and Promises

Play slick and Async - is it a race condition?

Reading Play-Slick DBAction code, I thought that this code might contain a race condition:
object DBAction{
// snip
def apply(r: (RequestWithDbSession) => Result)(implicit app:Application) = {
Action { implicit request =>
AsyncResult {
DB.withSession{ s:scala.slick.session.Session =>
Future(r( RequestWithDbSession(request,s) ))(executionContext)
}
}
}
}
The function r runs at a future time, after withSession has returned a Future[Result], and called session.close(). Is there a race condition in this code?
I am not sure if that is called a race condition. However to me it seems that you are correct that something is wrong here. The session might no longer be valid when the future executes the code.
It would be better to invert the execution and request a database session from within the future:
Async {
Future {
DB.withSession{ s:scala.slick.session.Session =>
r( RequestWithDbSession(request, s) )
}
}
}
I think your are right and fix suggested by EECOLOR looks correct. We are tracking this in a ticket: https://github.com/freekh/play-slick/issues/81
Thx