I would like to utilize key update notification mechanism of redis in a Http based rest api implemented in java.
Once a request is received in http rest api, it publishes details to be handled by an async process and waits for associated unique key notification from redis.
Async process after computation will create an entry in redis db with same unique key.
rest api receives unique key notification and replies back with http response.
Is this possible with redis, or there is a better option to get notified inside http request/reply implementation?
This approach is fine as long as you make sure your async process that receives the message and provides the result is fast enough to not exceed any configured request timeouts (especially under foreseen load). When you cannot guarantee that - you can consider using polling strategy:
return an async job identifier and let the client ask for its result, or
define a timeout for the async job to complete - if it provides the result in that time - return it, otherwise return async job identifier as above.
Related
Let's say I have a micro service which just registers a user into the database and we expose it to our client. I want to understand what's the better way of handling the following scenario,
What if the user sends multiple requests in parallel(say 10 requests within the 1 second) with same request body. Should I keep the requests in a queue and register for the very first user and deny all the other 9 requests, or should I classify each request and compare whichever having similar request body and if any of them has different request body shall be picked up one each and rest are rejected? or What's the best thing I can do to handle this scenario?
One more thing I would like to understand, is it recommended to have rate-limiting (say n requests per minute) on a global API level or micro-service level?
Thanks in advance!
The best way is to use an idempotent call. Instead of exposing an endpoint like this :
POST /users + payload
Expose an endpoint like this :
PUT /user/ID + payload
You let the caller generate the id, and you ask for an UUID. With UUID, no matter who generates it. This way, if caller invokes your endpoint multiple times, the first time you will create the user, the following times you will juste update the user with the same payload, which means you'll do nothing. At least you won't generate duplicates.
It's always a good practice to protect your services with rate-limiting. You have to set it at API level. If you define it at microservice level, you will authorize N times the rate if you have N instances, because you will ditribute the requests.
I'm not find any document to clarify how does Postgres handle request-response for async non-block request at the socket protocol level.
As a example, suggest only two query-sql in pg-client which use one socket connection to pg-server. I'm suppose there are two way to handle asyc operation:
client can't send second command before the first request not response.
client socket1 server
-----query1---------->
<----response1--------
------query2--------->
<-----response1-------
socket send two querys at sametime and distinct response by a unique flag.
client socket1 server
-------query1 + uid:msg1----->
-------query2 + uid:msg2----->
<------response2 + uid:msg2---
<------response1 + uid:msg1---
I think it should be the second way to handle request as real async process but I'm not find any resource from document.
Question:
which one is the Postgres handle async socket operation?
If use the first way and why?
On the network protocol level, there is no distinction between synchronous and asynchronous mode. The difference is only in the way the client API works.
There is always at most one statement active at any given time. See for example the documentation for the simple protocol, but it is the same for the extended protocol.
The difference is in the way the client API works:
In synchronous mode, the client thread is blocked until the query result us complete.
In asynchronous mode, control is returned to the client thread immediately after sending the query, and the client can go and do something else while it waits for the server response. It has to poll the socket regularly to check if the result has arrived. Then it can read and process the result.
If you want to run two statements concurrently, you have to use two database sessions.
I am confused between the usage of Action and Action.async. And what is the appropriate condition to use one.
I have wrote the method with Action.async with just a for loop, which takes 12 secs to process:
def asyncIndex() = Action.async {
val time = Calendar.getInstance().get(Calendar.SECOND)
Future {
for(i<- 0 to 20000000) {
print(i)
}
Ok(Json.toJson(time))
}
}
When I simultaneously make two request to this method the second request is blocked until the first one is completed.
PS:- I Think I have not understood the proper concept about async call.
I am confused in Action and Action.async and what is the appropriate condition to use one
From the documentation:
Note: Both Action.apply and Action.async create Action objects that are handled internally in the same way. There is a single kind of Action, which is asynchronous, and not two kinds (a synchronous one and an asynchronous one). The .async builder is just a facility to simplify creating actions based on APIs that return a Future, which makes it easier to write non-blocking code.
when i simultaneously make two request to this method the second request is blocked until the first one is completed
Also from the documentation:
The web client will be blocked while waiting for the response, but nothing will be blocked on the server, and server resources can be used to serve other clients.
If your simultaneous requests are from the same synchronous client, one of the requests on the client side will be blocked until the other is completed. There is no blocking on the server side. To achieve parallel processing of requests to the same endpoint, use distinct clients to make those requests, or use a client that makes asynchronous HTTP calls. Also consider using a separate dispatcher for this endpoint, even if you're wrapping the processing inside a Future (more information on creating a custom dispatcher is in the linked documentation).
Waiting blocks in code created that waiting time: The code written with the body of Future is not fully concurrent because there is a loop in it before sending the Ok response; so obviously when you send the calls it takes some time to get the second response. If you remove the for loop and send number of calls (through curl for example) you will see the app is running without "awaiting" time. Of course this has the limitation; which is the specifications of your machine (cpu, ram, etc.). So, using Action.async just by itself and writing a waiting/blocking inside it; doesn't make the whole code concurrent.
When to use: There is simple rule for it: If your controller's method body has concurrent code in it then the action should be defined as Action.async{...}; if not Action{...}.
Please note that in Play all actions are asynchronous.
I have a systems design challenge that I would like to get some community feedback on.
Basic system structure:
[Client] ---HTTP-POST--> [REST Service] ---> [Queue] ---> [Processors]
[Client] POSTs json to [REST Service] for processing.
Based on request, [Rest Services] sends data to various queues to be picked up by various processors written in various languages and running in different processes.
Work is parallelized in each processor but can still take up to 30 seconds to process. The time to process is a function of the complexity of the data and cannot be speed up.
The result cannot be streamed back to the client as it is completed because there is a final post processing step that can only be completed once all the sub steps are completed.
Key challenge: Once the post processing is complete, the client either needs to:
be sent the results after the client has been waiting
be notified async that the job is completed and passed an id to request the final result
Design requirements
I don't want to block the [REST Service]. It needs to take the incoming request, route the data to the appropriate queues for processing in other processes, and then be immediately available for the next incoming request.
Normally I would have used actors and/or futures/promises so the [REST Service] is not blocked when waiting for background workers to complete. The challenge here is the workers doing the background work are running in separate processes/VMs and written in various technology stacks. In order to pass these messages between heterogeneous systems and to ensure integrity of the request lifetime, a durable queue is being used (not in memory message passing or RPC).
Final point of consideration, in order to scale, there are a load balanced set of [REST Services] and [Processors] in respective pools. Therefore, since the messages from the [REST Service] to the [Processor] need to be sent asynchronously via a queue (and everything is running is separate processes), there is no way to correlate the work done in a background [Processor] back to its original calling [REST Service] instance in order to return the final processed data in a promise or actor message and finally pass the response back to the original client.
So, the question is, how to make this correlation? Once the all the background processing is completed, I need to get the result back to the client either via a long waited response or a notification (I do not want to use something like UrbanAirship as most of the clients are browsers or other services.
I hope this is clear, if not, please ask for clarification.
Edit: Possible solution - thoughts?
I think I pass a spray RequestContext to any actor which can then response back to the client (does not have to be the original actor that received HTTP request). If this is true, can I cache the RequestContext and then use it later to asynchronously send the response to the appropriate client using this cached RequestContext when the processing is completed?
Well, it's not the best because it requires more work from your Client, but it sounds like you want to implement a webhook. So,
[Client] --- POST--> [REST Service] ---> [Calculations] ---> POST [Client]
[Client] --- GET
For explanation:
Client sends a POST request to your service. Your Service then does whatever processing necessary. Upon completion, your service will then send an HTTP-POST to a URL that the Client has already set. With that POST data, the Client will then have the necessary information to then do a GET request for the completed data.
I am designing a REST API which works according to the asynchronous design detailed here. I am using RabbitMQ to enqueue the initial requests - so the client makes a call, receives a 202 Accepted response, and the job is enqueued by the server. In order that clients can get status updates ('percent done') on tasks we have a secondary queue resource, just as in the linked article.
Given that each task has its own queue resource it seems we need one temporary RabbitMQ queue per task. I am wondering whether this is a wise design choice, even though I can't really see any other options. It seems unlikely to be very efficient, and I am uneasy about the possibility of having lots of temporary queues being created like this, especially as I cannot see a way to guarantee that they will all be cleaned up (despite RabbitMQ's auto-delete facility). Prior to RabbitMQ I was using SQS for this, and have painful experience of what can happen in this respect.
I note that a similar type of queue management will be already familiar to those using RabbitMQ in RPC style. Is there a possible alternative, however?
Firs of all, each queue utilize apr. 20k memory, so having a lot of them is up to you and your hardware. But in general, it smells. Really.
For status updates I see nothing wrong to use some key-value database, like redis or even memcache and update percent done there. Thus status check (as well as updating) will be damn fast, simple and lightweight.
Update:
I can suggest further architecture:
Client POST task payload to some endpoint, say /tasks.
Application generate unique task id (uuid aka guid is your friend here), published that task with it id to RabbitMQ queue and then return id to client.
Workers (one or many) consume tasks from RabbitMQ and depends of processing step update Redis key which has task id with some value (step, percentage done, estimated time to receive result). So, it may be looks like SET task:{id} "<some valye>". When task completed by worker it MAY update Redis key with task result or store it somewhere else and then set Redis key represent task is finished.
Client MAY time to time GET /tasks/{id} to receive task status or it result.
When Application receive GET /tasks/{id} it return task status represented by Redis key (GET task:{id}). If key is not set (nil) then task is not yet taken by worker.
P.S.
RPC is something different from what you asked, but i would recommend to read this question for some details.