What's a proper HTTP Status code when the server cannot process the response due to the large number of records that are there in the Database (may be due to the large record size and it's going to taking too much of time to process it) as part of the request that user sent? We wanted to tell the client that there were too many records. The request is semantically correct in this case, but since the server is not able to process it due to it's processing power limitation, we have to give an error response.
For example, let's say client tried to generate all the transactions in the past 3 years out of the maximum allowed duration of 5 years. But since there are more transactions done by the client within 3 years, the server would like to communicate to the client to reduce the duration further. The reason being the hardware resources at the server side is limited and cannot process this huge number of records.
It should not be categorised under the 4xx errors since it's not an issue with the client right? If yes, what's the best 5xx STATUS code to be set for this?
First let's decide on whether it should be 4xx or 5xx code. While it's true that the original problem is not on client side, it's the client who should change the request to make it successful. So it needs to be 4xx.
Sure, if it was just a server bug that you didn't expect - it had to be 500 Internal Server Error. But you decide to live with this defect, so now it's a part of your contract. And client needs to know that if such error happens - a narrower period needs to be requested.
In your case since syntax is correct, 422 Unprocessable Entity is probably the best option. It's still a bit off since 422 should be used when client sends an entity that doesn't make sense logically, but it's the closest code that exists there. You can always include more details in response body.
NB: It's a WebDAV extended code, not the original HTTP code. If you want to go with pure HTTP, I'd use 400 and put the reason in the body so that client could parse it out. I don't think there's anything appropriate in pure HTTP.
Related
We have an akka/scala app that has some naively written error handling that I need to fix.
The REST endpoint communicates with an internal actor that makes a remote call to create an order.
It does this using ask, and when the ask times out e.g. because of a network or comms error, we send the client a message over the REST endpoint that the request has failed.
The problem is that the internal actor has its own queuing/retry logic and it will continue to call the remote interface until the request succeeds.
So we have the situation where we've told the client that the request has failed but it's really just queued (and will often eventually succeed). The client resubmits the request and we end up with 100's of duplicate orders.
My question is: does akka support a generic way of rolling back or poisoning an ask message when the ask request times out?
There are a couple of approaches one can take. Neither is generic, as (especially once network communication is involved) any choice made in this area has some cases where it's exactly the wrong thing for the business logic:
If there's something about the orders that can be used to determine that two submitted orders are actually the same (e.g. a client-supplied correlation ID), that can be used in the actor to piggyback on the queuing/retry logic handling the earlier order. This requires some client-visible API changes.
It's also possible to include a stop retrying and ignore if pulled from a queue after this time field in the message; you can set this time based on the ask timeout duration.
The standard says:
10.5.4 503 Service Unavailable. The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay.
REF: https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
If the server is having lock contension at the database access level, should the server return 503 in such cases. Or is the 503 status meant for:
Network overload
DOS type situations
Maxed out load balancing
Explicit maintenance window.
What other circumstances it makes sense to return HTTP status 503.
Any detailed clarification is much appreciated.
The updated spec, for reference.
If the server is having lock contension at the database access level, should the server return 503 in such cases
Keep in mind, in a REST model HTTP status codes aren't about describing the problem in your service; they instead describe an analogous condition in a document store.
503 is the way that a document store says "I'm too busy, try again later."
What other circumstances it makes sense to return HTTP status 503.
Back pressure. From Martin Thompson
Applying back pressure is one effective technique for coping with sustained high-load, such that maximum throughput can be delivered without degrading system performance for the already accepted requests and transactions.
When we need to support synchronous protocols like REST then use back pressure, signalled by our full incoming queue at the gateway, to send a meaningful “server busy” message such as the HTTP 503 status code
Let's say the service experiences a contention for the database. There are no rules as to what a server MUST do. You may however choose to return a 503 if you believe the issue is temporary and will be resolved after some time. You can specify the Retry-After header to inform the caller when he could retry. This is especially useful to allow the caller to automatically recover from the issue. You may use it for some of your other scenarios as well as the callers could automatically retry after the specified period.
The status codes serve 2 purposes
Let the caller know what happened
Let the caller know what he could do next
Sending the caller a 503 with Retry-After provides more options to the caller than say sending a 500. Of course the caller is free to totally ignore the Retry-After and treat it as a 500 but you as a service provider are providing more information and options.
I have an application for group scheduling that serves a Caldav feed to members. It's very simple - on request, just grabs upcoming events for that member, formats the stream, returns it. Some members are using devices that poll much too frequently - things don't update that often - so I'm hitting my database more often than I need to.
I can obviously cache the formatted stream and just update it no more frequently than I feel like it. It would be easier to return some kind of "ask me later" response to someone who has polled recently. Is there such a response? If I just don't return a stream, will the client read that as a "there are no events" response?
thanks!
You should probably fix the original reason why clients are polling 'too frequently'. What does 'too frequently' mean for you? Do they send a proper etag/sync-token so that the polling is cheap?
Regardless, there is indeed an HTTP status for that: HTTP 503.
The server is currently unable to handle the request due to a
temporary overloading or maintenance of the server. The implication is
that this is a temporary condition which will be alleviated after some
delay. If known, the length of the delay MAY be indicated in a
Retry-After header. If no Retry-After is given, the client SHOULD
handle the response as it would for a 500 response.
Now the question is how many HTTP clients actually deal with that properly, but I do know some which do :-)
Let's say we have a REST client with some UI that lists items it GETs from the server. The server also exposes some REST methods to manipulate the items (POST / PUT).
Now the user triggers one of those calls that are supposed to change the data on the server side. The UI will reflect the server state change, if the call was successful.
But what are good strategies to handle the situation when the server is not available?
What is a reasonable timeout lengths (especially in a 3G / Cloud setup)?
How do you handle the timeout in the client, considering the fact that the client can't tell whether the operation succeeded or not?
Are there any common patterns to solve that, other than a complete client termination (and subsequent restart)?
This will be application specific. You need to decide what makes the most sense in your usage case.
Perhaps start with a timeout similar to that of the the default PHP session of 24 minutes. Adjust as necessary based on testing.
Do you have server and client mixed up here? If so the server cannot tell if the client times out other than reaching the end of a session. The client can always query the server for a progress update.
This one is a little general to provide an answer for.
If there is a REST resource that I want to monitor for changes or modifications from other clients, what is the best (and most RESTful) way of doing so?
One idea I've had for doing so is by providing specific resources that will keep the connection open rather than returning immediately if the resource does not (yet) exist. For example, given the resource:
/game/17/playerToMove
a "GET" on this resource might tell me that it's my opponent's turn to move. Rather than continually polling this resource to find out when it's my turn to move, I might note the move number (say 5) and attempt to retrieve the next move:
/game/17/move/5
In a "normal" REST model, it seems a GET request for this URL would return a 404 (not found) error. However, if instead, the server kept the connection open until my opponent played his move, i.e.:
PUT /game/17/move/5
then the server could return the contents that my opponent PUT into that resource. This would both provide me with the data I need, as well as a sort of notification for when my opponent has moved without requiring polling.
Is this sort of scheme RESTful? Or does it violate some sort of REST principle?
Your proposed solution sounds like long polling, which could work really well.
You would request /game/17/move/5 and the server will not send any data, until move 5 has been completed. If the connection drops, or you get a time-out, you simply reconnect until you get a valid response.
The benefit of this is it's very quick - as soon as the server has new data, the client will get it. It's also resilient to dropped connections, and works if the client is disconnected for a while (you could request /game/17/move/5 an hour after it's been moved and get the data instantly, then move onto move/6/ and so on)
The issue with long polling is each "poll" ties up a server thread, which quickly breaks servers like Apache (as it runs out of worker-threads, so can't accept other requests). You need a specialised web-server to serve the long-polling requests.. The Python module twisted (an "an event-driven networking engine") is great for this, but it's more work than regular polling..
In answer to your comment about Jetty/Tomcat, I don't have any experience with Java, but it seems they both use a similar pool-of-worker-threads system to Apache, so it will have that same problem. I did find this post which seems to address exactly this problem (for Tomcat)
I'd suggest a 404, if your intended client is a web browser, as keeping the connection open can actively block browser requests in the client to the same domain. It's up to the client how often to poll.
2021 Edit: The answer above was in 2009, for context.
Today, I would suggest using a WebSocket interface with push notifications.
Alternatively, in the above suggestion, I might suggest holding the connection for 500-1000ms and check twice at the server before returning the 404, to reduce the overhead of creating multiple connections at the client.
I found this article proposing a new HTTP header, "When-Modified-After", that essentially does the same thing--the server waits and keeps the connection open until the resource is modified.
I prefer a version-based approach rather than a timestamp-based approach, since it's less prone to race conditions and gives you a little more information about what it is you're retrieving. Any thoughts to this approach?