play - how to wrap a blocking code with futures - scala

I am trying to understand the difference between the 2 methods, in terms of functionality.
class MyService (blockService: BlockService){
def doSomething1(): Future[Boolean] = {
//do
//some non blocking
//stuff
val result = blockService.block()
Future.successful(result)
}
def doSomething2(): Future[Boolean] = {
Future{
//do
//some non blocking
//stuff
blockService.block()
}
}
}
To my understanding the difference between the 2 is which thread is the actual thread that will be blocked.
So if there is a thread: thread_1 that execute something1, thread_1 will be the one that is blocked, while if a thread_1 executed something2a new thread will run it - thread_2, and thread_2 is the one to be blocked.
Is this true?
If so, than there is no really a preferred way to write this code? if I don't care which thread will eventually be blocked, then the end result will be the same.
dosomething1 seems like a weird way to write this code, I would choose dosomething2.
Make sense?

Yes, doSomething1 and doSomething2 blocks different threads, but depending on your scenario, this is an important decision.
As #AndreasNeumann said, you can have different execution contexts in doSomething2. Imagine that the main execution context is the one receiving HTTP requests from your users. Block threads in this context is bad because you can easily exhaust the execution context and impact requests that have nothing to do with doSomething.
Play docs have a better explanation about the possible problems with having blocking code:
If you plan to write blocking IO code, or code that could potentially do a lot of CPU intensive work, you need to know exactly which thread pool is bearing that workload, and you need to tune it accordingly. Doing blocking IO without taking this into account is likely to result in very poor performance from Play framework, for example, you may see only a few requests per second being handled, while CPU usage sits at 5%. In comparison, benchmarks on typical development hardware (eg, a MacBook Pro) have shown Play to be able to handle workloads in the hundreds or even thousands of requests per second without a sweat when tuned correctly.
In your case, both methods are being executed using Play default thread pool. I suggest you to take a look at the recommended best practices and see if you need a different execution context or not. I also suggest you to read Akka docs about Dispatchers and Futures to gain a better understanding about what executing Futures and have blocking/non-blocking code.

This approach makes sense if you make use of different execution contexts in the second method.
So having for example one for answering requests and another for blocking requests.
So you would use the normal playExecutionContext to keep you application running and answering and separate blocking operation in a different one.
def doSomething2(): Future[Boolean] = Future{
blocking { blockService.block() }
}( mySpecialExecutionContextForBlockingOperations )
For a little more information: http://docs.scala-lang.org/overviews/core/futures.html#blocking

You are correct. I don't see a point in doSomething1. It simply complicates the interface for the caller while not providing the benefits of an asynchronous API.

Does BlockService handle blocking operation?
Normally, use blocking ,as #Andreas remind,to make blocking operation into another thread is meanful.

Related

Background task in reactive pipeline (Fire-and-forget)

I have a reactive pipeline to process incoming requests. For each request I need to call a business-relevant function (doSomeRelevantProcessing).
After that is done, I need to notify some external service about what happened. That part of the pipeline should not increase the overall response time.
Also, notifying this external system is not business critical: giving a quick response after the main part of the pipeline is finished is more important than making sure the notification is successful.
As far as I learned, the only way to run something in the background without slowing down the overall process is to subscribe to in directly in the pipeline, thus achieving a fire-and-forget mentality.
Is there a good alternative to subscribing inside the flatmap?
I am a little worried about what might happen if notifying the external service takes longer than the original processing and a lot of requests are coming in at once. Could this lead to a memory exhaustion or the overall process to block?
fun runPipeline(incoming: Mono<Request>) = incoming
.flatMap { doSomeRelevantProcessing(it) } // this should not be delayed
.flatMap { doBackgroundJob(it) } // this can take a moment, but is not super critical
fun doSomeRelevantProcessing(request: Request) = Mono.just(request) // do some processing
fun doBackgroundJob(request: Request) = Mono.deferContextual { ctx: ContextView ->
val notification = "notification" // build an object from context
// this uses non-blocking HTTP (i.e. webclient), so it can take a second or so
notifyExternalService(notification).subscribeOn(Schedulers.boundedElastic()).subscribe()
Mono.just(Unit)
}
fun notifyExternalService(notification: String) = Mono.just(Unit) // might take a while
I'm answering this assuming that you notify the external service using purely reactive mechanisms - i.e. you're not wrapping a blocking service. If you are then the answer would be different as you're bound by the size of your bounded elastic thread pool, which could quickly become overwhelmed if you have hundreds of requests a second incoming.
(Assuming you're using reactive mechanisms, then there's no need for .subscribeOn(Schedulers.boundedElastic()) as you give in your example, as that's not buying you anything - it's designed for wrapping legacy blocking services.)
Could this lead to a memory exhaustion
It's only a possibility in really extreme cases, the memory used by each individual request will be tiny. It's almost certainly not worth worrying about, if you start seeing memory issues here then you'll almost certainly be hit by other issues elsewhere.
That being said, I'd probably recommend adding .timeout(Duration.ofSeconds(5)) or similar before your inner subscribe method to make sure the requests are killed off after a while if they haven't worked for any reason - this will prevent them building up.
...or [can this cause] the overall process to block?
This one is easier - a short no, it can't.

Safety of running assertions in a separate execution context

How isolated are different execution contexts from each other? Say we have two execution contexts ec1 and ec2 both used on the same code path implementing some user journey. If, say, starvation and crashing starts happening in ec2, would ec1 remain unaffected?
For example, consider the following scenario where we want to make sure user was charged only once by running an assertion inside a Future
chargeUserF andThen { case _ =>
getNumberOfChargesF map { num => assert(num == 0) }
.andThen { case Failure(e) => logger.error("User charged more than once! Fix ASAP!", e) }
}
Here getNumberOfChargesF is not necessary to fulfil user's request, it is just a side-concern where we assert on the expected state of the database after it was mutated by chargeUserF. Because it is not necessary I feel uneasy adding it to the main business logic out of fear it could break the main logic in some way. If I run getNumberOfChargesF on a different execution context from the one chargeUserF uses, can I assume issues such as starvation, blocking etc. caused by getNumberOfChargesF will not affect the main business logic?
Each execution context has its own thread pool, so, yeah ... kinda.
They are "independent" in the sense that if one runs out of threads, the other one might still keep going, however, they do use the same resource (cpu), so if that gets maxed out by one, the other will obviously be affected.
They are also affected by each other's side effects. For example, the way your code is written, chargeUser and getNumberOfCharges are happening in parallel, and there is no saying which one will finish first, so, if I am guessing the semantics right, the number of charges may end up being either 0 or 1 fairly randomly, depending on whether the previous future has completed or not.

How to choose in which ExecutorService the play ws is executed?

1)
I would like to execute specific Play WS in an isolated thread pool. Because I'm going to have a lot of HTTP call to do in background and I don't want that it overload my main executor service.
Note : I also found an information that I don't understand here : https://groups.google.com/forum/#!topic/play-framework/ibDf2vL3sT0
It explains that PlayWs already have it owns thread pool. Is it still right in Play 2.6 ? I don't understand things like that when reading the play documentation (see : https://www.playframework.com/documentation/2.5.x/ThreadPools#Using-the-default-thread-pool)
I created my own context :
call-to-db-context {
fork-join-executor {
parallelism-factor = 1
parallelism-max = 24
}
}
But I don't know how to specify that ws request use this context.
ws.url("http://127.0.0.1:8080/b")
.get() // How to specify executorContext here ?
2)
Also, this call-to-db-context must have a low priority because it's background task. I would like that Akka handle user request have higher priority and my default executorContext too. What is the best way to do it ?
In early play it was easier, you could just configure client yourself (as soon as WS was wrapper on top of ning).
It looks bit more complicated in 2.6. In mature products when something is not easy to change, than, most probably, you do not need to change it.
So, I do not think that you need to specify thread pool for WS methods. For post-processing may be, if it is long. But play client is asynchronous, means, that it will not block thread while wait for response. If you do requests using unreliable network, just use timeouts.
See here more about play client
Not sure that understand your requirement about priority. Did you know about akka deployment configuration? If no, you need to read here
So, you can specify thread pools for your actors. Having different dispatchers for actors and ws post-processing (processing of data from DB?) you will separate these functionalities. If ws-calls postprocessing is heavy, limit amount of threads for your call-to-db-context dispatcher
Update after comment
Lots of ws-calls in your case (if you think that their amount could affect perormance) need to be limited in two places: limit amount of calls started, limit amount of concurrent post-processing. You need to understand. Setting specific dispatcher for ws itself will not limit anything: as soon as it is async, it can start 1000 requests having only one thread.
So, I would say you can wrap your ws-calls into actor. Actor will handle message to start the request, and post-processing e.g.
receive: Receive = {
…
case Get(url) =>
ws.url(url).get().onComplete {
case Success(response) => self ! GetSuccess(response)
case Failure(exception) => self ! GetFailure(exception)
}
case GetSuccess(response) => …..
case GetFailure(exception) => ……
}
you can deploy this actor on specific dispatcher with round robin pool (set amount of workers). This solution does not limit start of requests (so you can get long queue of responses). You can add become with disabling accepting Get, while not received GetSuccess or GetFailure (so worker need to process request completely before start next).

Clarification about Scala Future that never complete and its effect on other callbacks

While re-reading scala.lan.org's page detailing Future here, I have stumbled up on the following sentence:
In the event that some of the callbacks never complete (e.g. the callback contains an infinite loop), the other callbacks may not be executed at all. In these cases, a potentially blocking callback must use the blocking construct (see below).
Why may the other callbacks not be executed at all? I may install a number of callbacks for a given Future. The thread that completes the Future, may or may not execute the callbacks. But, because one callback is not playing footsie, the rest should not be penalized, I think.
One possibility I can think of is the way ExecutionContext is configured. If it is configured with one thread, then this may happen, but that is a specific behaviour and a not generally expected behaviour.
Am I missing something obvious here?
Callbacks are called within an ExecutionContext that has an eventually limited number of threads - if not by the specific context implementation, then by the underlying operating system and/or hardware itself.
Let's say your system's limit is OS_LIMIT threads. You create OS_LIMIT + 1 callbacks. From those, OS_LIMIT callbacks immediately get a thread each - and none ever terminate.
How can you guarantee that the remaining 1 callback ever gets a thread?
Sure, there could be some detection mechanisms built into the Scala library, but it's not possible in the general case to make an optimal implementation: maybe you want the callback to run for a month.
Instead (and this seems to be the approach in the Scala library), you could provide facilities for handling situations that you, the developer, know are risky. This removes the element of surprise from the system.
Perhaps most importantly - it enables the developer to "bake in" the necessary information about handler/task characteristics directly into his/her program, rather than relying on some obscure piece of language functionality (which may change from version to version).

Practical use of futures? Ie, how to kill them?

Futures are very convenient, but in practice, you may need some guarantees on their execution. For example, consider:
import scala.actors.Futures._
def slowFn(time:Int) = {
Thread.sleep(time * 1000)
println("%d second fn done".format(time))
}
val fs = List( future(slowFn(2)), future(slowFn(10)) )
awaitAll(5000, fs:_*)
println("5 second expiration. Continuing.")
Thread.sleep(12000) // ie more calculations
println("done with everything")
The idea is to kick off some slow running functions in parallel. But we wouldn't want to hang forever if the functions executed by the futures don't return. So we use awaitAll() to put a timeout on the futures. But if you run the code, you see that the 5 second timer expires, but the 10 second future continues to run and returns later. The timeout doesn't kill the future; it just limits the join wait.
So how do you kill a future after a timeout period? It seems like futures can't be used in practice unless you're certain that they will return in a known amount of time. Otherwise, you run the risk of losing threads in the thread pool to non-terminating futures until there are none left.
So the questions are: How do you kill futures? What are the intended usage patterns for futures given these risks?
Futures are intended to be used in settings where you do need to wait for the computation to complete, no matter what. That's why they are described as being used for slow running functions. You want that function's result, but you have other stuff you can be doing meanwhile. In fact, you might have many futures, all independent of each other that you may want to run in parallel, while you wait until all complete.
The timer just provides a wait to get partial results.
I think the reason Future can't simply be "killed" is exactly the same as why java.lang.Thread.stop() is deprecated.
While Future is running, a Thread is required. In order to stop a Future without calling stop() on the executing Thread, application specific logic is needed: checking for an application specific flag or the interrupted status of the executing Thread periodically is one way to do it.