Explicitly disable caching for REST services - rest

I am to apply Cache-Control: must-revalidate,no-cache,no-store to all responses from out backend REST services. I have two questions about it:
Is it common to do so? For some reason I was under the impression that it's not necessary, but I have no source to back this claim (yet).
Is the value I mentioned above really sufficient, or should I set more?
Edit: found this: https://devcenter.heroku.com/articles/increasing-application-performance-with-http-cache-headers#cache-prevention. Is says browsers may choose to cache when nothing is explicitly configured, so it means yes, it should be configured if I want to make sure cache is disabled.

Short: yes, caches may cache the response even if no explicit controls are present, you need to explicitly disallow it.
The HTTP caching specification Section 3 lists when the response is forbidden to be cached. It suggests that the response may be cached as long as the response code is cacheable. A list of cacheable response codes is in the HTTP specification section 6.1:
Responses with status codes that are defined as cacheable by default
(e.g., 200, 203, 204, 206, 300, 301, 404, 405, 410, 414, and 501 in
this specification) can be reused by a cache with heuristic
expiration unless otherwise indicated by the method definition or
explicit cache controls...
"Heuristic expiration" is defined as the expiration time assigned when no explicit controls are present. (HTTP caching specification section 4.2.)

I think it's disabled by default. There are mechanisms though to enable caching to enhance performance:
Here's a good explanation with examples how to enable caching:
[Source: Heroku Dev Center]
Time-based cache headers
In HTTP 1.1 the Cache-Control header specifies the resource caching behavior as well as the max age the resource can be cached. As an example, this response would be cached for one day:
HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: private, max-age=86400
Last-Modified: Thu, 07 Feb 2013 11:56 EST
Here is a list of all the available Cache-Control tokens and their
meaning:
private only clients (mostly the browser) and no one else in
the chain (like a proxy) should cache this
public any entity in the chain can cache this
no-cache should not be cached anyway
no-store can be cached but should not be stored on disk (most browsers will hold the resources in memory until they will be quit)
no-transform the resource should not be modified (for example shrink image by proxy)
max-age how long the resource is valid (measured in seconds)
s-maxage same like max-age but this value is just for non clients
And here an example with CDI Cache Control annotation:
[Source: abhirockzz.wordpress.com]
#Path("/testcache")
public class RESTfulResource {
#Inject
#CachControlConfig(maxAge = 20)
CacheControl cc;
#GET
#Produces("text/plain")
public Response find() {
return Response.ok(UUID.randomUUID().toString()).cacheControl(cc).build();
}
}

Related

Best REST practice for responce "GET" method status?

I didn't find useful information about which methods status is correct for absent object in db.
For example I have deleted user with id = 1, but someone try to get it's information thought GET method with query params id=1.
Which status will be right: 404, 204, 400,406 or 410?
I didn't find useful information about which methods status is correct for absent object in db.
Yup, that's right - HTTP status codes don't tell you anything about rows in a database, what they tell you about are documents ( "resources" ) in a document store.
More precisely, the HTTP status code is metadata that tells general purpose components (like a web browser, or a cache) what's in the message-body of the response.
Depending on what document you put into the message-body, the appropriate status code could be any of:
200
404
410
200 announce that the message-body is a document (more broadly, a current representation of the resource). 404 and 410 (and all 4xx and 5xx status codes) announce that the message-body is a representation of the explanation of the error.
404 indicates that the document identified by the effective target uri of the request doesn't exist right now, but it might exist later; you can attach caching metadata to communicate when the might check again.
410 indicates that the document identified by the effective target uri of the request doesn't exist right now, and that condition is likely permanent. That permanence implies that clients can delete bookmarks, and remote links to the document should be removed, and so on.
If you recycle ids, or if deletes ids can be restored, then 410 isn't an appropriate choice.
In some APIs, resources have current representations even when there is no matching information in the database.
In other words, the current representation of the resource might be an empty document
200 OK HTTP/2
Content-Type: text/plain
Content-Length: 0
or it could be a null object
200 OK HTTP/2
Content-Type: application/json
Content-Length: 4
null
or it could be an empty list
200 OK HTTP/2
Content-Type: application/json
Content-Length: 2
[]
or an empty object
200 OK HTTP/2
Content-Type: application/json
Content-Length: 2
{}
or a meme
200 OK HTTP/2
Content-Type: text/plain
Content-Length: 36
This space intentionally left blank.
The status code to use follows from the decision to use a sort of "default" representation of our document when there is no specific information available.
The more common decision, of course, is to choose not to provide default representations, but instead announce that the client has made a mistake (in which case the 4xx class of status code is the correct starting point).
Isn't it write to return 204(NO CONTENT) status or something similar? 'Cause I think 200 is not fully informative status
Maybe - there's some ambiguity in the HTTP standard, and because of that ambiguity I tend to be biased against 204 (today; if you look up some of my older answers, I was much likely to try 204 in the past).
RFC 7231, Section 6.3.1
Aside from responses to CONNECT, a 200 response always has a payload, though an origin server MAY generate a payload body of zero length. If no payload is desired, an origin server ought to send 204 (No Content) instead.
So we have two different ways to send zero bytes of data back to the client; either 200 with Content-Length set to zero, or 204.
Are those two things the same?
The answer seems to be "not quite"; there's a subtle difference documented in section 6.3.5
The 204 response allows a server to indicate that the action has been successfully applied to the target resource, while implying that the user agent does not need to traverse away from its current "document view" (if any).
Now, think about that in the context of a web browser. If I click a link that points to an empty file, a 200 response would mean that the browser would traverse away from the current "document view" to show me the empty file. But the language of 204 suggests that instead the browser should stay put, and just indicate that the empty file was successfully downloaded.
Note: I haven't done any experiments to see if browsers do act that way; my only claim is that staying in place is the specified behavior.
My reading of the specification is that 204 is designed to support a use case that only arises in the context of unsafe actions, like PUT. You can see hints of that as far back as HTTP/1.0
This response is primarily intended to allow input for scripts or other actions to take place without causing a change to the user agent's active document view. The response may include new metainformation in the form of entity headers, which should apply to the document currently in the user agent's active view.
In short, responding with a 204 to a GET request is placing a bet that the authors of general purpose components have interpreted an ambiguous part of the specification the same way that you do -- and I don't like that bet at all. Much more reliable to use the well understood 200 response, and avoid the unnecessary ambiguity.

ReST low latency - how should I reply to a GET while an upload is pending?

I am designing a ReST API which follows the basic CRUD pattern.
My API can receive a request to update a resource which may take a short time to process. Ideally I would like to inform clients that a new version is about to be available and that there is some uncertainty over when the version I have cached actually expires.
So the process I intend to use something like this (improvements welcome):
client: GET /some/item
myapi: 200 OK
last-modified: time-stamp-of-v1
etag: some-hash-relating-to-v1-of-my-item-in-this-format
content: json or whatever
data/for/some/item/v1...
client: PUT /some/item
if-match: some-hash-relating-to-v1-of-my-item-in-this-format
content: json or whatever
data/for/some/item/v2...
myapi: 202 ACCEPTED,
content: json or whatever
time-accepted: time-stamp-after-v1-but-before-v2
your item will be at /some/item
here is a URI /some/taskid to track progress
while upload is pending:
client: GET /some/item
myapi: 200 OK
some/item ...
last-modified: time-stamp-of-v1
etag: some-hash-relating-to-v1-of-my-item-in-this-format
>>>> expires: time-stamp-after-v1-but-before-v2 <<<
>>>> warning: 110 Response is stale <<<<
content: json or whatever
data/for/some/item/v1...
client: GET /some/task/id
myapi: 200 OK
content: json or whatever
time-accepted: time-stamp-after-v1-but-before-v2
your item will be at /some/item
status/of/upload/v2...
after task completed:
client: GET /some/item
myapi: 200 OKAY
some/item/v2 ...
last-modified: time-stamp-of-v2
etag: some-hash-relating-to-v2-of-my-item-in-this-format
content: json or whatever
data/for/some/item/v2...
client: GET /some/task/id
myapi: 303 SEE OTHER
look-here: /some/item
If you are a proxy and know know your content is stale you can put "warning: 110 - response is stale" in the header.
However, in this case the data is not actually invalid yet.
I would like to say that I can guarantee it is valid up until the time I received and passed on the upload request (time-stamp-after-v1-but-before-v2 or later as if I am in contact with the upload server). It hasn't really expired at the time I receive the upload request. I just expect its going to.
(In fact if the request fails it might not be updated at all).
Now the default choice is just to serve the old content and let the client catch up on its own. This has high latency. If possible, I would like to do better.
For example, if the client knows the document is about to expire it could poll more often or it could try to upgrade the connection to a web-socket and get sent an update the moment I get it (would that still count as ReST?)
There is another case where using expired data must be avoided at all costs. For that scenario I think I want to tell the client that the resource is temporarily unavailable. Using the warning and expires fields as I have above seems correct there. Though it might be better to send a 503 with a suitable retry-after header.
So the question is: how should I reply to a GET while the upload of a new version is pending?
In anticipation of answers along the lines of use a messaging framework like AMQP or zeroMQ instead for low latency, I should point out this API is acting as a AMQP gateway/proxy for clients unwilling to use AMQP directly. Information on using webhooks or websockets would be still be interesting.
Some related useful content is:
How to proper design a restful API to invalidate a cache?
https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
HTTP status code for temporarily unavailable pages
http://www.albertoleal.me/posts/how-to-prevent-race-conditions-in-restful-apis.html
(the etag prevents races from simultaneously uploads)
Tl;Dr;
While upload is pending send:
client: GET /some/item
myapi: 200 OK
some/item ...
last-modified: time-stamp-of-v1
etag: some-hash-relating-to-v1-of-my-item-in-this-format
expires: time-stamp-after-v1-but-before-v2
stale-while-revalidate: 100
warning: 110 Response is stale
content: json or whatever
data/for/some/item/v1...
At first sight it looks like using Warning is not correct. See https://www.rfc-editor.org/rfc/rfc7234#section-5.5.0
In this case the server is acting as a proxy (though not an HTTP proxy).
It is not disconnected from AMQP and "A proxy MUST NOT send stale responses" unless it is disconnected.
This is annoying as it looked like the right thing to do here.
4.2.4. Serving Stale Responses
A "stale" response is one that either has explicit expiry
information or is allowed to have heuristic expiry calculated, but
is not fresh according to the calculations in Section 4.2.
A cache MUST NOT generate a stale response if it is prohibited by
an explicit in-protocol directive (e.g., by a "no-store" or
"no-cache" cache directive, a "must-revalidate"
cache-response-directive, or an applicable "s-maxage" or
"proxy-revalidate" cache-response-directive; see Section 5.2.2).
**> A cache MUST NOT send stale responses unless it is disconnected
(i.e., it cannot contact the origin server or otherwise find a
forward path) or doing so is explicitly allowed (e.g., by the
max-stale request directive; see Section 5.2.1).**
A cache SHOULD generate a Warning header field with the 110
warn-code (see Section 5.5.1) in stale responses. Likewise, a
cache SHOULD generate a 112 warn-code (see Section 5.5.3) in stale
responses if the cache is disconnected.
A cache SHOULD NOT generate a new Warning header field when
forwarding a response that does not have an Age header field, even if
the response is already stale. A cache need not validate a response
that merely became stale in transit.
Also
4.4. Invalidation
Because unsafe request methods (Section 4.2.1 of [RFC7231]) such as
PUT, POST or DELETE have the potential for changing state on the
origin server, intervening caches can use them to keep their contents
up to date.
**> A cache MUST invalidate the effective Request URI (Section 5.5 of
[RFC7230]) as well as the URI(s) in the Location and Content-Location
response header fields (if present) when a non-error status code is
received in response to an unsafe request method.**
However a warning is required if stale-while-revalidate is used (see https://www.rfc-editor.org/rfc/rfc5861)
The stale-while-revalidate Cache-Control Extension
When present in an HTTP response, the stale-while-revalidate Cache-
Control extension indicates that caches MAY serve the response in
which it appears after it becomes stale, up to the indicated number
of seconds.
stale-while-revalidate = "stale-while-revalidate" "=" delta-seconds
If a cached response is served stale due to the presence of this
extension, the cache SHOULD attempt to revalidate it while still
serving stale responses (i.e., without blocking).
I thought this was unclear so I submitted an errata. This was rejected (though at the time of writing its still showing as reported) on the grounds that the cache control extensions in rfc5861 override the MUST NOT in rfc7234 ("doing so is explicitly allowed" see above).
It is okay to use expires but its not very helpful as it doesn't imply anything.
5.3. Expires
The "Expires" header field gives the date/time after which the
response is considered stale. See Section 4.2 for further discussion
of the freshness model.
**> The presence of an Expires field does not imply that the original
resource will change or cease to exist at, before, or after that
time.**

REST API Design: Respond with 406 or 404 if a resource is not available in a requested representation

We have a REST API to fetch binary files from the server.
The requests look like
GET /documents/e62dd3f6-18b0-4661-92c6-51c7258f9550 HTTP/1.1
Accept: application/octet-stream
For every response indicating an error, we'd like to give a reason in JSON.
The problem is now, that as the response is not of the same content type as the client requested.
But what kind of response should the server produce?
Currently, it responds with a
HTTP / 1.1 406 Not Acceptable
Content-Type: application/json
{
reason: "blabla"
...
}
Which seems wrong to me, as the underlying issue is, that the resource is not existing and not the client requesting the wrong content type.
But the question is, what would be the right way to deal with such situations?
Is it ok, to respond with 404 + application/json although application/octet-stream was requested
Is it ok, to respond with 406 + application/json, as the client did not specify an application/json as an acceptable type
Should spec been extended so that the client should use the q-param - for example, application/octet-stream, application/json;q=0.1
Other options?
If no representation can be found for the requested resource (because it doesn't exist or because the server wishes to "hide" its existence), the server should return 404.
If the client requests a particular representation in the Accept header and the server is not available to provide such representation, the server could either:
Return 406 along with a list of the available representations. (see note** below)
Simply ignore the Accept header and return a default representation of the resource.
See the following quote from the RFC 7231, the document the defines the content and semantics of the HTTP/1.1 protocol:
A request without any Accept header field implies that the user agent will accept any media type in response. If the header field is present in a request and none of the available representations for the response have a media type that is listed as acceptable, the origin server can either honor the header field by sending a 406 (Not Acceptable) response or disregard the header field by treating the response as if it is not subject to content negotiation.
Mozilla also recommends the following regarding 406:
In practice, this error is very rarely used. Instead of responding using this error code, which would be cryptic for the end user and difficult to fix, servers ignore the relevant header and serve an actual page to the user. It is assumed that even if the user won't be completely happy, they will prefer this to an error code.
** Regarding the list of available representations, see this answer.

REST Api naming convention?

I have a simple question that I cannot find the answer to.
My colleague is currently making a REST Api for an application, and we have a call that simply checks some information, and returns either true or false. However, we do not know what to call this type of request, as it does not retrive any resources or insert anyhing, it simply checks some information passed into the query. As far as I can understand, a GET has to retrive a resource, which this call isn't doing
What I understand is, resource in this case is either true or false. While calling the API you will expect response either true or false on the basis of information processed by API server (status will be always 200). So a GET method is still suitable for this case.
If you are not interested in response body and you want data like response code and header details, go with HEAD.
There might be a different way to express 'checking some information', and it's important to be a bit more specific as to what that means.
So lets take an arbitrary example. You're modelling blog posts and want to know if some blog post is set to 'draft'.
The 'draft' status can be its own resource, for example:
/posts/hello-world/is-draft
Doing a GET request on the is-draft resource can yield:
{
"is-draft": true
}
So to model arbitrary things as resources, the best way to think about this is to look at the result of the operation as the 'representation' and the 'thing you want to know' as the URI.
As far as I can understand, a GET has to retrive a resource, which this call isn't doing
Technically, it is retrieving a resource; see Fielding
The key abstraction of information in REST is a resource. Any information that can be named can be a resource: a document or image, a temporal service (e.g. "today's weather in Los Angeles"), a collection of other resources, a non-virtual object (e.g. a person), and so on. In other words, any concept that might be the target of an author's hypertext reference must fit within the definition of a resource.
The resource, in this case, might not load an entity in your data model, but that's OK. Not all resources have to.
Technically, I think what you have there is a "function"; all of the information that you need to compute the result is present within the URI itself? Which would mean that, if the client knew how to do the computation (and had the compute resources available), then the client would be capable of doing the work for itself.
But there's nothing wrong with having a resource that is "the result of a function".
In some API, you'll see predicates (functions that return true/false) implemented as resources that only exist (more precisely, only have "representations") if the evaluation is true.
GET /predicate?true
204 No Content
GET /predicate?false
404 Not Found
The fact that you don't need to consider the resources "state" to compute the correct response to the query is an implementation detail hidden behind the uniform interface.
It's hard to tell by the level of details you provided your question. But if you need to check whether a resource exists or not, you can use HEAD. It's identical to GET, but it doesn't return a representation in the response payload: it returns just the status code and the response headers.
Consider the following request HEAD request:
HEAD /postal-codes/10001 HTTP/1.1
Host: example.org
Content-Type: application/json
It should return 200 for a resource that exists:
HTTP/1.1 200 OK
Content-Type: application/json
And 404 for a resource that doesn't exists:
HTTP/1.1 404 Not Found
Content-Type: application/json
Depending on your needs, you could address it with POST, which can be seen as a catch all verb.
For example, consider the following request and responses:
POST /postal-codes/validation HTTP/1.1
Host: example.org
Content-Type: application/json
{ "postal-code": "10001" }
HTTP/1.1 200 OK
Content-Type: application/json
{ "postal-code": "10001", "status": "valid" }
HTTP/1.1 200 OK
Content-Type: application/json
{ "postal-code": "10001", "status": "invalid" }

HTTP Status 202 - how to provide information about async request completion?

What is the appropriate way of giving an estimate for request completion when the server returns a 202 - Accepted status code for asynchronous requests?
From the HTTP spec (italics added by me):
202 Accepted
The request has been accepted for processing, but the processing has not been completed. [...]
The entity returned with this response SHOULD include an indication of the request's current status and either a pointer to a status monitor or some estimate of when the user can expect the request to be fulfilled.
Here are some of thoughts:
I have glanced at the max-age directive, but using it would be abusing Cache-Control?
Return the expected wait time in the response body?
Add an application specific X- response header, but the X-headers was deprecated in RFC 6648?
Add a (non X-) specific response header? If so, how should it be named? The SO question Custom HTTP headers : naming conventions gave some ideas, but after the deprecation it only answers how HTTP headers are formatted, not how they should be named.
Other suggestions?
Definitely do not abuse existing HTTP headers for this. Since it's your own server, you get to define what the response looks like. You can (and should) pick whatever response works best for the intended recipient of this information and return the actual information in the response body.
For example, if you are only interested in displaying a human-readable message then you could return text/plain saying "Your request is likely to be processed in the next 30 minutes.".
At the other end of the spectrum, you might want to go all the REST way and return application/json, perhaps formatted like this (I totally made this up on the spot):
{
"status": "pending",
"completion": {
"estimate": "Thu Sep 08 2011 12:00:00 GMT-0400",
"rejected-after": "Fri Sep 09 2011 12:00:00 GMT-0400",
},
"tracking": {
"url": "http://server/status?id=myUniqueId"
}
}
You can use the Location header to specify the URL of the status monitor. Things like current status and estimate can either go in custom headers (which noone but your own software would use), or in the response body (which a web browser would display to a user, at least).
Although not explicitly mentioned specifically for the 202 - Accepted response code, the Retry-After header seems to be a suitable option. From the documentation:
The Retry-After response-header field can be used [...] to indicate how long the service is expected to be unavailable to the requesting client.