I have a Scala application that relies on an external RESTful webservice for some part of its functionality. We'd like to do some performance tests on the application, so we stub out the webservice with an internal class that fakes the response.
One thing we would like to keep in order to make the performance test as realistic as possible is the network lag and response time from the remote host. This is between 50 and 500 msec (we measured).
Our first attempt was to simply do a Thread.sleep(random.nextInt(450) + 50), however I don't think that's accurate - we use NIO, which is non-blocking, and Thread.sleep is blocking and locks up the whole thread.
Is there a (relatively easy / short) way to stub a method that contacts an external resource, then returns and calls a callback object when ready? The bit of code we would like to replace with a stub implementation is as follows (using Sonatype's AsyncHttpClient), where we wrap its completion handler object in one of our own that does some processing:
def getActualTravelPlan(trip: Trip, completionHandler: AsyncRequestCompletionHandler) {
val client = clientFactory.getHttpClient
val handler = new TravelPlanCompletionHandler(completionHandler)
// non-blocking call here.
client.prepareGet(buildApiURL(trip)).setRealm(realm).execute(handler)
}
Our current implementation does a Thread.sleep in the method, but that's, like I said, blocking and thus wrong.
Use a ScheduledExecutorService. It will allow you to schedule things to run at some time in the future. Executors has factory methods for creating them fairly simply.
Related
I have a reactive pipeline to process incoming requests. For each request I need to call a business-relevant function (doSomeRelevantProcessing).
After that is done, I need to notify some external service about what happened. That part of the pipeline should not increase the overall response time.
Also, notifying this external system is not business critical: giving a quick response after the main part of the pipeline is finished is more important than making sure the notification is successful.
As far as I learned, the only way to run something in the background without slowing down the overall process is to subscribe to in directly in the pipeline, thus achieving a fire-and-forget mentality.
Is there a good alternative to subscribing inside the flatmap?
I am a little worried about what might happen if notifying the external service takes longer than the original processing and a lot of requests are coming in at once. Could this lead to a memory exhaustion or the overall process to block?
fun runPipeline(incoming: Mono<Request>) = incoming
.flatMap { doSomeRelevantProcessing(it) } // this should not be delayed
.flatMap { doBackgroundJob(it) } // this can take a moment, but is not super critical
fun doSomeRelevantProcessing(request: Request) = Mono.just(request) // do some processing
fun doBackgroundJob(request: Request) = Mono.deferContextual { ctx: ContextView ->
val notification = "notification" // build an object from context
// this uses non-blocking HTTP (i.e. webclient), so it can take a second or so
notifyExternalService(notification).subscribeOn(Schedulers.boundedElastic()).subscribe()
Mono.just(Unit)
}
fun notifyExternalService(notification: String) = Mono.just(Unit) // might take a while
I'm answering this assuming that you notify the external service using purely reactive mechanisms - i.e. you're not wrapping a blocking service. If you are then the answer would be different as you're bound by the size of your bounded elastic thread pool, which could quickly become overwhelmed if you have hundreds of requests a second incoming.
(Assuming you're using reactive mechanisms, then there's no need for .subscribeOn(Schedulers.boundedElastic()) as you give in your example, as that's not buying you anything - it's designed for wrapping legacy blocking services.)
Could this lead to a memory exhaustion
It's only a possibility in really extreme cases, the memory used by each individual request will be tiny. It's almost certainly not worth worrying about, if you start seeing memory issues here then you'll almost certainly be hit by other issues elsewhere.
That being said, I'd probably recommend adding .timeout(Duration.ofSeconds(5)) or similar before your inner subscribe method to make sure the requests are killed off after a while if they haven't worked for any reason - this will prevent them building up.
...or [can this cause] the overall process to block?
This one is easier - a short no, it can't.
I would like to clear up what it means to be a asynchronous/synchronous and blocking/non-blocking operations in Scala Play using Scala's Future.
So we can have:
Asynchronous and non-blocking
Asynchronous and blocking
Synchronous and non-blocking
Synchronous and blocking
In the Play documentation it says that all operations are asynchronous. But I would like to know how the following would be classed:
If I call an external web service and I have to wait for the response then is this considered to be a asynchronous and blocking operation? For example:
def externalWebCall(url : String): Future[SomeData]
def storeData(data : String) : Future[Unit]
for {
data <- externalWebCall(url)
_ <- storeData(data)
} yield(data)
In this example, I don't want storeData to execute until the externalWebCall service has completed. Is the above code the right way to achieve this? If so I think this is asynchronous and blocking.
The initial request that comes to Play service is treated in an asynchronous and non-blocking way according to the documentation. My understanding of this is that when the Play server receives the request it will not block the main thread. It will then service the request and then make a call to the callback of this request and then return the result. Is that correct?
When people use the term "asynchronous", they usually mean "non-blocking", in the sense that the current execution thread will not stop and wait for the result. In general, synchronous means blocking, asynchronous means non-blocking.
It seems you are confusing "blocking" with "ordering the execution". Your example with the Futures is completely asynchronous, and it equivalent to
val result: Future[Unit] = externalWebCall(url).flatMap(storeData)
storeData can only execute when externalWebCall is done, but any code that would come after val result = ... could also execute before either externalWebCall or storeData. The execution of these methods is wrapped in a Future and run in a different thread, so you have no way of knowing when it will happen. That is what asynchronous means.
The terms blocking and asynchronous have different meanings, but are often confused or used interchangeably.
blocking means that the current thread waits for another thread to do something before executing further instructions. The current thread is blocked until the other thread completes and then it can continue.
asynchronous means that an operation happens on another thread while the current thread continues to execute. The current thread initiates an operation but then continues to execute. When the asynchronous operation completes it notifies the current thread by raising an event or calling some event handler function.
Blocking functions look like normal functions and there is not necessarily any indication that the code being called is blocking or non blocking. So a println statement may be blocking for stderr but non-blocking for stdout.
Asynchronous functions will always have a mechanism for being notified of completion of the operation and this will be visible in the interface. In Scala this usually means returning Future[T] rather than T.
In your example, both externalWebCall and storeData are asynchronous because each method returns Future.
storeData takes a data argument that is only available when externalWebCall completes, so these two operations must by definition operate sequentially.
I am trying to understand the difference between the 2 methods, in terms of functionality.
class MyService (blockService: BlockService){
def doSomething1(): Future[Boolean] = {
//do
//some non blocking
//stuff
val result = blockService.block()
Future.successful(result)
}
def doSomething2(): Future[Boolean] = {
Future{
//do
//some non blocking
//stuff
blockService.block()
}
}
}
To my understanding the difference between the 2 is which thread is the actual thread that will be blocked.
So if there is a thread: thread_1 that execute something1, thread_1 will be the one that is blocked, while if a thread_1 executed something2a new thread will run it - thread_2, and thread_2 is the one to be blocked.
Is this true?
If so, than there is no really a preferred way to write this code? if I don't care which thread will eventually be blocked, then the end result will be the same.
dosomething1 seems like a weird way to write this code, I would choose dosomething2.
Make sense?
Yes, doSomething1 and doSomething2 blocks different threads, but depending on your scenario, this is an important decision.
As #AndreasNeumann said, you can have different execution contexts in doSomething2. Imagine that the main execution context is the one receiving HTTP requests from your users. Block threads in this context is bad because you can easily exhaust the execution context and impact requests that have nothing to do with doSomething.
Play docs have a better explanation about the possible problems with having blocking code:
If you plan to write blocking IO code, or code that could potentially do a lot of CPU intensive work, you need to know exactly which thread pool is bearing that workload, and you need to tune it accordingly. Doing blocking IO without taking this into account is likely to result in very poor performance from Play framework, for example, you may see only a few requests per second being handled, while CPU usage sits at 5%. In comparison, benchmarks on typical development hardware (eg, a MacBook Pro) have shown Play to be able to handle workloads in the hundreds or even thousands of requests per second without a sweat when tuned correctly.
In your case, both methods are being executed using Play default thread pool. I suggest you to take a look at the recommended best practices and see if you need a different execution context or not. I also suggest you to read Akka docs about Dispatchers and Futures to gain a better understanding about what executing Futures and have blocking/non-blocking code.
This approach makes sense if you make use of different execution contexts in the second method.
So having for example one for answering requests and another for blocking requests.
So you would use the normal playExecutionContext to keep you application running and answering and separate blocking operation in a different one.
def doSomething2(): Future[Boolean] = Future{
blocking { blockService.block() }
}( mySpecialExecutionContextForBlockingOperations )
For a little more information: http://docs.scala-lang.org/overviews/core/futures.html#blocking
You are correct. I don't see a point in doSomething1. It simply complicates the interface for the caller while not providing the benefits of an asynchronous API.
Does BlockService handle blocking operation?
Normally, use blocking ,as #Andreas remind,to make blocking operation into another thread is meanful.
I need to add a WebSocket-to-TCP proxy to my Play 2.3 application, but while the outgoing TCP connection using Akka I/O supports back-pressure, I don't see anything for the WebSocket. There's clearly no support in the actor-based API, but James Roper says:
Iteratees handle this by design, you can't feed a new element into an
iteratee until last future it returns has been redeemed, because you
don't have a reference to it until then.
However, I don't see what he's referring to. Iteratee.foreach, as used in the examples, seems too simple. The only futures I see in the iteratee API are for completing the result of the computation. Should I be completing a Future[Unit] for each message or what?
Iteratee.foldM lets to pass a state along to each step, much like the regular fold operation, and return a future. If you do not have such a state you can just pass Unit and it will behave as a foreach that will not accept the next step until the future completes.
Here is an example of a utility function that does exactly that:
def foreachM[E](f: E => Future[Unit])(implicit ec: ExecutionContext): Iteratee[E, Unit] =
Iteratee.foldM[E, Unit](Unit)((_, e) => f(e))
Iteratee is not the same as Iterator. An Iteratee does indeed inherently support back-pressure (in fact you'll find yourself with the opposite problem - by default they don't do any buffering (at least within the pipeline - of course async sockets still have receive buffers), so you sometimes have to add an explicit buffering step to an enumerator/iteratee pipeline to get reasonable performance). The examples look simple but that just means the framework is doing what a framework does and making things easy. If you're doing a significant amount of work, or making async calls, in your handlers, then you shouldn't use the simple Iteratee.foreach, but instead use an API that accepts a Future-based handler; if you're blocking within an Iteratee then you block the whole thing, waste your threads, and defeat the point of using them at all.
Have been through a lot of stuff explaining the fine line of difference between non-blocking and asynchronous I/O, but most seem to pertain to server side programming. And it seems to make sense as well. But could non-blocking vs. asynchronous I/O have a relevance in terms of a Rest Client?
Have spent a lot of time on the net but am afraid, am still unable to grasp the difference or rather the significance of having a non-blocking Rest Client. Am more concerned with Jersey Client. Yes, the API says that it supports Asynchronous Client and that a blocking "get" call on the Future object could be avoided using Future.isDone() (ref: https://jersey.java.net/documentation/latest/async.html) but if my Rest Client makes a Post request how is that handled? Is it just asynchronous or is it asynchronous and non-blocking as well? Would be much grateful if anyone could help me with an insight.
Thanks and regards
I recommend reading this article from Microsoft on the topic.
The term "non-blocking" really means the same thing whether it is happening on the server side or the client side. As the article above points out, blocking means that the thread you make the call on stops processing until the call has finished. The Task async pattern described in the article makes it possible to await a Rest call instead of blocking the thread. Generally, if a method returns a Task, and the method is suffixed with async, it will be a non-blocking call. This means that it can be ran concurrently with other calls so that calls do not need to be ran in a sequence. They can be ran in parallel.
Here is an example where non-blocking rest calls are made in parallel and awaited so that they do not block each other. A simple for loop without the parallel would achieve the same result because it would kick the calls off in a sequence but would not block on each call.
var tasks = new List<Task<Response<Person>>>();
var client = clientFactory.CreateClient();
for (var i = 0; i < 100; i++)
{
tasks.Add(client.GetAsync<Person>(new Uri("JsonPerson", UriKind.Relative)));
}
var results = await Task.WhenAll(tasks);
Full code