I have the following clear algorithm:
Client sends a request to my spray application.
Spray receives a request and I see spray receiving load as multiple requests come in.
If loading is high, spray returns HTTP 503; otherwise it starts processing the request.
How can I manage current spray loading?
Also, as I understand spray uses akka internally which can be extended with adding additional nodes, so how can I manage the load with additional nodes?
Spray itself uses reactive I/O and can handle very high loads, probably higher than any custom code "protecting" it could handle. So don't worry about trying to protect the spray system itself. If you've got complex processing logic that might take a while to handle certain requests, it might make sense to put a protective throttle around that processing logic, using something like http://letitcrash.com/post/28901663062/throttling-messages-in-akka-2 . And in the case where the queue is full you can simply complete(StatusCodes.ServiceUnavailable).
Related
Im learning akka streams but obviously its relevant to any streaming framework :)
quoting akka documentation:
Reactive Streams is just to define a common mechanism of how to move
data across an asynchronous boundary without losses, buffering or
resource exhaustion
Now, from what I understand is that if up until before streams, lets take an http server for example, the request would come and when the receiver wasent finished with a request, so the new requests that are coming will be collected in a buffer that will hold the waiting requests, and then there is a problem that this buffer have an unknown size and at some point if the server is overloaded we can loose requests that were waiting.
So then stream processing came to play and they bounded this buffer to be controllable...so we can predefine the number of messages (requests in my example) we want to have in line and we can take care of each at a time.
my question, if we implement that a source in our server can have a 3 messages at most, so if the 4th id coming what happens with it?
I mean when another server will call us and we are already taking care of 3 requests...what will happened to he's request?
What you're describing is not actually the main problem that Reactive Streams implementations solve.
Backpressure in terms of the number of requests is solved with regular networking tools. For example, in Java you can configure a thread pool of a networking library (for example Netty) to some parallelism level, and the library will take care of accepting as much requests as possible. Or, if you use synchronous sockets API, it is even simpler - you can postpone calling accept() on the server socket until all of the currently connected clients are served. In either case, there is no "buffer" on either side, it's just until the server accepts a connection, the client will be blocked (either inside a system call for blocking APIs, or in an event loop for async APIs).
What Reactive Streams implementations solve is how to handle backpressure inside a higher-level data pipeline. Reactive streams implementations (e.g. akka-streams) provide a way to construct a pipeline of data in which, when the consumer of the data is slow, the producer will slow down automatically as well, and this would work across any kind of underlying transport, be it HTTP, WebSockets, raw TCP connections or even in-process messaging.
For example, consider a simple WebSocket connection, where the client sends a continuous stream of information (e.g. data from some sensor), and the server writes this data to some database. Now suppose that the database on the server side becomes slow for some reason (networking problems, disk overload, whatever). The server now can't keep up with the data the client sends, that is, it cannot save it to the database in time before the new piece of data arrives. If you're using a reactive streams implementation throughout this pipeline, the server will signal to the client automatically that it cannot process more data, and the client will automatically tweak its rate of producing in order not to overload the server.
Naturally, this can be done without any Reactive Streams implementation, e.g. by manually controlling acknowledgements. However, like with many other libraries, Reactive Streams implementations solve this problem for you. They also provide an easy way to define such pipelines, and usually they have interfaces for various external systems like databases. In particular, such libraries may implement backpressure on the lowest level, down to to the TCP connection, which may be hard to do manually.
As for Reactive Streams itself, it is just a description of an API which can be implemented by a library, which defines common terms and behavior and allows such libraries to be interchangeable or to interact easily, e.g. you can connect an akka-streams pipeline to a Monix pipeline using the interfaces from the specification, and the combined pipeline will work seamlessly and supporting all of the backpressure features of Reacive Streams.
I'm trying to figure out the best way to implement a real websocket app using akka-http and akka-streams. What I'm mostly looking for is simplicity, which I'm just not getting now.
Assume you have a fairly complex pipeline which needs to discriminate between multiple requests and sometimes send the request to an actor for processing, sometimes issue a mongo query and return the response, sometimes perform a PUT on a REST API, etc.
Unlike the simple chat application examples out there, there are at least 3 problems that arise which seem to not have a standard solution:
Conditionally skipping the response, e.g., because it is not expected by the client that this request will receive a response. If I use the typical Flow from Message to Message, once the request has hit its target, I need to stop it from propagating further back to the websocket. It can be done with a special filter (involves some pain) or using various other ways (e.g., Conditionally skip flow using akka streams), but this adds a lot of boilerplate and complexity. Ideally, I'd like to be able to insert 'Skip' messages that just skip everything else.
Routing incoming messages to the appropriate place (e.g., actor, mongo). Once again, I can find solutions to that which involve a lot of boilerplate (e.g., broadcast and filter out at branches which do not handle this kind of request). Ideally, I should be able to define something like: if the message is X, send it there, if the message is Y, send it there, etc.
Propagating errors back to the client. Very similar to the routing problem described above. For example, if the JSON parse fails, I need to add a separate path (broadcast + merge) along which I send an error message, but I cannot even easily reuse the same path if an error occurs at the next stage and I want to propagate that error to the user. Ideally, I should have one single separate path for error handling that can be used at any arbitrary point in the flow, bypasses the rest of the flow entirely and goes back to the client.
At the moment, I have this insanely complex graph spanning 15 lines with paths going through >20 different stages and I'm really worried about keeping the complexity of this solution in check. The DSL is mostly unreadable at this size. I could of course modularize a bit better, but this feels like an insane amount of trouble for something that should be a lot simpler.
Am I missing something? Am I insane for considering akka-streams for such a task? Any ideas or code examples that could allow me to rein in all that complexity?
Thanks in advance!
This is a very wide-ranging question and may not be answerable in its current form.
Akka HTTP addresses many of these concerns in its HTTP handling layers (e.g. empty responses, routing, returning errors). Could you use some of the lessons learnt there and apply them to your system? Or, perhaps better, could you convert your system from using websocket communication into using HTTP communication and use that code directly?
Writing a one page web application, and knowing that some of the screens would need real-time updates, I am faced to one big general question, whatever API and frontend framework and language I am going to use:
I'll implement data transfer over websocket, should I keep http for any data transfer which would not need real-time updates, or should I just use websocket?
Knowing that websockets are not handled on ALL browsers but most if not all recent ones support it, would it be better for the servers to handle both websocket and http, or should I just use websockets for ALL data transfer?
You will probably end up using both WebSockets and HTTP requests at the end.
WebSockets, because it sounds like you need them (because of the real-time updates) and can afford to require browser support for them (otherwise, you'd be forced to use the older Ajax/Comet based approaches).
HTTP for two possible reasons:
You will sooner or later need a blocking request-response behavior. For example, authenticating a user may need to block for the result before further processing happens, so you need to send a request for authentication and block until you get the result. This can be a bit annoying to handle over WebSockets.
You may need to load heavy data without interrupting the ongoing real-time updates. If you were to load such data over WebSockets (as a single big chunk), it will be queued together with the real-time updates and may delay them.
Both of these issues can be handled over WebSockets, but they are simply easier to solve with simple HTTP Ajax requests.
Use websockets for the following needs,
Server data changes frequently
Multi-user communication
Live feeds etc.,
Refer this for better understanding on websocket usage.
I am trying to develop a simple web API for testing using Racket's web server. The requirements are:
Respond to port requests with a callback in a new thread.
Read the header values and POST data
Write response headers and content to the port.
I do not want to engage the complexity of stateful versus stateless servlets. Essentially I want to avoid the overhead of managing continuations.
By avoiding calls to any send/... function other than send/back the serve/servlet can be used without invoking continuation handling.
Calling (serve/servlet start #:manager web-server/managers/non ...) will cause an error if there is an attempt to use continuations.
Custom headers/content can be created using a "raw" response structure.
Alternatively, using serve\launch\wait with a dispatcher using web-server/dispatchers/dispatch-lift is possible. Raw data may even be written directly to the port.
Reference: Original discussion on Racket discussion list.
I'm using spray-routing to build a simple HTTP server. This server calls out to a number of services that take a while to respond (seconds). We would like to reject requests when the number of concurrent requests becomes to large. Otherwise a large number of concurrent requests bogs down the system to nobody's advantage.
There are a number of layers where this might be solved. I'm not sure how to do any of them precisely, or which is the best one.
I could supply an execution context for spray-routing that has a bounded queue and a rejection policy.
I could limit the mailbox size of my spray http server since it is also an actor.
I could configure a setting in application.conf that addresses this directly for spray.
What is a simple an effective way of implementing such a policy?
I don't know what solution would be the best for your case (I would go for creating my own execution context) but I believe that maybe you should rethink how you want to process your requests.
What do you do with your request? Do you try to handle them in Spray directly? With some help from Futures?
I would suggest creating additional actors, passing the request context to them and then deciding what to do. If you want to process it or maybe you should put it down immediately. This will give you much flexibility in future. You can attach additional servers with now support for clustering in Akka without changing the spray part adding more processing power easily.
I know this doesn't answer your question but I think akka was designed to handle this kind of problems differently and cutting on mailboxes or anything else is not the right choice.