I'm designing a RESTful API that involves clients submitting requests to create a resource, let's say a report. For valid reasons not worth getting into, this process can take a minimum of 30 seconds and a maximum of several minutes to complete successfully. I'm trying to ensure this API is easy and intuitive to work with so I just want to get some feedback on what I was thinking of doing.
Client POSTs request body to /reports
Server responds with 202 Accepted and Location: /jobs/{jobId} header
Client GETs /jobs/{jobId} and receives 200 OK and response body like {"status": "pending"} (abbreviated)
Client retries until they get 200 OK (unchanged) and a body like{
"status": "complete",
"location": "/reports/{reportId}",
details": { ... }}
}
Client GETs /reports/{reportId} to retrieve their report
Some things I've though of doing differently from the above:
Having the /jobs/{jobId} resource return 303 See Other with a Location: /reports/{reportId} header when ready. A number of blog posts & SO answers I saw took that approach. I decided against it because we want to retain these jobs as first-class resources, e.g. we want to be able to view all jobs submitted in the past 24 hours, all failed jobs in the past 15 minutes, etc. Also it seems 303 See Other really should not return a body as clients should be expected to redirect to the Location url so they wouldn't see it anyway.
Having client POST to /jobs instead of /reports. I feel like either choice is defensible but it seems less surprising to clients that if they want to create a report they should POST to /reports. That being said, it may be surprising to get a response pointing them to a job instead of a report.
Whether clients POST to /reports or /jobs, since I am immediately creating a job, maybe I should return 201 Created instead of 202 Accepted.
Anyway, that's where my thinking currently is on this. Any confirmation, suggestions, respectfully explained disagreements, etc. are all greatly appreciated.
Thanks.
we want to retain these jobs as first-class resources, e.g. we want to be able to view all jobs submitted in the past 24 hours, all failed jobs in the past 15 minutes, etc.
If both jobs and reports are first-class, then I suggest giving each the usual obvious, boring semantics:
POST /jobs returns 201 Created and a Location: /jobs/{id} header. After all, the job was immediately created (the report is N/A).
Optionally you could have this return an ETag header (see next).
GET /jobs/{id} returns 200 OK; the response body indicates the readiness of the report and (if ready) the URI to /reports/{id}.
Optionally you could have this handle If-None-Match by returning 304 Not Modified until the readiness changes. And of course return an ETag header.
Of course GET /reports/{id} works.
p.s. If I'm not mistaken, Location headers ought to be full URIs e.g. https://example.com/path/to/thing not just relative paths /path/to/thing. If you do that, probably do likewise with any JSON response location values? That way, clients can just use the values as-is when making requests -- which is both more convenient for them, and better for you if they don't hardcode the protocol/host.
Related
A microservice I develop exposes a single endpoint, PUT /deployments/{uuid} . The endpoint is used to initiate a potentially expensive deployment operation, so we only ever want it to happen once, which is why we chose PUT + UUID over POST (for uniqueness). The deployment is immutable, so it can never be updated, so we currently raise an exception if the PUT is called more than once with the same uuid.
As a person who loves bikeshedding and therefore cares deeply about restfulness, this grinds my gears. PUT is supposed to be idempotent, so raising an exception after making the same request multiple times is an antipattern. However, we have a requirement to not allow sequential identical requests to generate new deployments, so the usual POST is out.
While the best solution is one that works, I'd like ours to be a little more elegant, if possible. I've posited a POST with the UUID in the payload, but my team seems to think that's worse than the current solution. I'm considering just returning a 200 OK from a PUT to the same UUID rather than a 201 CREATED, but I'm not sure if that has the same problem as non-idempotent-put in not semantically conveying the information I want.
Is there a "best solution" here? Or am I doomed to be "that guy" on my team if I pursue this further (joke's on you i'm already that guy).
tl;dr What is the correct RESTful API signature for a /deployments endpoint that is immutable, and required to not allow the same request to be processed twice?
Idempotent does not mean "2 identical requests should yield the same response". It means: "The server state after 2 identical requests should be the same as when only 1 is made".
A similar example, if you call DELETE on a resource and get a 204 No Content back, and call DELETE again and get a 404, this doesn't violate the idempotency requirement. After the second delete the resource is still removed, just like it was after the first.
So multiple identical idempotent requests are allowed to give different responses.
That said though, I think it might be nicer to the second identical request to also get a 2xx status back. It doesn't have to be the same as the first.
The use-case is if a client sent a HTTP request but got disconnected before it got a response. The client should retry and if the server detects the request is the same as the first, the server can just give the client a success response (but don't do anything).
This is generally a good idea, because if the client got an error back for the second request, it might be harder to know if the request failed because it succeeded earlier, or for other reasons.
That all being said though, there is also a way here to have your cake and eat it too.
A client could send the following header along with the PUT request:
If-None-Match: *
If the client omits the header, you can always return 424 Precondition Required.
If the resource does not yet exist, it's a success response. If the resource was created earlier, you can return 412 Precondition Failed.
Using this mechanism a client has a standard way to figure out that the request failed because a successful one was made earlier.
Based on the docs here, PUT is the best method to use. The response should be 201 when it triggers the deployment, and either 200 or 204 when nothing is changed. It shouldn't be POST because calling a POST endpoint twice should trigger the effect both times.
I'm working on a REST API that requires multiple steps/validations to create a resource, then making a POST /transfer might not create an actual transfer resource (because it needs an extra steps), but is not failing as it will trigger the second step/validation.
In this answer the response is an incomplete or pending transaction with an id, and the resource includes endpoints to finish the transaction, like /transaction/123/commit or in my case /transaction/123/verification/432 where the client can continue, abort or finish the transaction processing.
But what are the possible response codes on that endpoints? Does /transaction/123/commit actually returns 201 because it create the transaction or the transaction is created when it reach pending state?
201 (Created)
Status code 201 (Created) indicates the creation of (at least one) new HTTP resource. The URL of the created resource is sent in the response’s Location header.
If POST /transfer creates the resource /transaction/123/commit — that is, if requests to /transaction/123/commit might now succeed where previously you’d have 404 (Not Found) — then it is correct to respond to POST /transfer with 201 and Location: /transaction/123/commit.
If POST /transfer creates several resources, then Location must be the “primary” one (in some sense).
If POST /transaction/123/commit does not create any new resource, then it is incorrect to respond with 201, even if it does create something else (like an internal database record).
If you can’t come up with a URL to send in Location, that probably means you’re not creating any new resource, in which case 201 is incorrect.
Note: Location is always relative to the request URL, not to an “API root” or any such concept you might have. For example, if POST /api/v1/foo/bar creates /api/v1/foo/bar/baz, correct values for Location would include bar/baz and /api/v1/foo/bar/baz, but not /foo/bar/baz.
200 (OK)
Status code 200 (OK) indicates general success. It can be used in most successful responses. It’s a kind of a safe fallback: it doesn’t say much, thus it’s guaranteed not to say much wrong and confuse the client.
If POST /transaction/123/commit succeeds without creating a new resource, then it is correct to respond with 200.
204 (No Content)
Except for responses to GET/HEAD, status code 204 (No Content) is mostly the same as 200. If you use 204 to say something different than 200, you’re probably making up a local convention — same as making up your own status code 275.
Other
IANA maintains a registry of standardized status codes. You can look there for a status code that is standardized to mean exactly what you want to say.
You generally don’t want to use a non-standard status code, or to use a standard status code incorrectly, because that would preclude a uniform interface, which is kind of the whole point of REST.
If you find yourself struggling all the time to uphold a uniform interface, it’s possible that you don’t need REST at all, and should be doing RPC instead.
Imagine a request that starts a long running process whose output is a large set of records.
We could start the process with a POST request:
POST /api/v1/long-computation
The output consists of a large sequence of numbered records, that must be sent to the client. Since the output is large, the server does not store everything, and so maintains a window of records with a upper limit on the size of the window. Let's say that it stores upto 1000 records (and pauses computation whenever this many records are available). When the client fetches records, the server may subsequently delete those records and so continue with generating more records (as more slots in the 1000-length window are free).
Let's say we fetch records with:
GET /api/v1/long-computation?ack=213
We can take this to mean that the server should return records starting from index 214. When the server receives this request, it can assume that the (well-behaved) client is acknowledging that records up to number 213 are received by the client and so it deletes them, and then returns records starting from number 214 to whatever is available at that time.
Next if the client requests:
GET /api/v1/long-computation?ack=214
the server would delete record 214 and return records starting from 215.
This seems like a reasonable design until it is noticed that GET requests need to be safe and idempotent (see section 9.1 in the HTTP RFC).
Questions:
Is there a better way to design this API?
Is it OK to keep it as GET even though it appears to violate the standard?
Would it be reasonable to make it a POST request such as:
POST /api/v1/long-computation/truncate-and-fetch?ack=213
One question I always feel like that needs to be asked is, are you sure that REST is the right approach for this problem? I'm a big fan and proponent REST, but try to only apply to to situations where it's applicable.
That being said, I don't think there's anything necessarily wrong with expiring resources after they have been used, but I think it's bad design to re-use the same url over and over again.
Instead, when I call the first set of results (maybe with):
GET /api/v1/long-computation
I'd expect that resource to give me a next link with the next set of results.
Although that particular url design does sort of tell me there's only 1 long-computation on the entire system going on at the same time. If this is not the case, I would also expect a bit more uniqueness in the url design.
The best solution here is to buy a bigger hard drive. I'm assuming you've pushed back and that's not in the cards.
I would consider your operation to be "unsafe" as defined by RFC 7231, so I would suggest not using GET. I would also strongly advise you to not delete records from the server without the client explicitly requesting it. One of the principles REST is built around is that the web is unreliable. Under your design, what happens if a response doesn't make it to the client for whatever reason? If they make another request, any records from the lost response will be destroyed.
I'm going to second #Evert's suggestion that you absolutely must keep this design, you instead pick a technology that's build around reliable delivery of information, such as a messaging queue. If you're going to stick with REST, you need to allow clients to tell you when it's safe to delete records.
For instance, is it possible to page records? You could do something like:
POST /long-running-operations?recordsPerPage=10
202 Accepted
Location: "/long-running-operations/12"
{
"status": "building next page",
"retry-after-seconds": 120
}
GET /long-running-operations/12
200 OK
{
"status": "next page available",
"current-page": "/pages/123"
}
-- or --
GET /long-running-operations/12
200 OK
{
"status": "building next page",
"retry-after-seconds": 120
}
-- or --
GET /long-running-operations/12
200 OK
{
"status": "complete"
}
GET /pages/123
{
// a page of records
}
DELETE /pages/123
// remove this page so new records can be made
You'll need to cap out page size at the number of records you support. If the client request is smaller than that limit, you can background more records while they process the first page.
That's just spitballing, but maybe you can start there. No promises on quality - this is totally off the top of my head. This approach is a little chatty, but it saves you from returning a 404 if the new page isn't ready yet.
I want to create or update an item in one go with the following request:
PUT /items/{id}
Should I return 201 Created if the item has been created and 204 No Content when it has been updated? Or should I return the same status for both actions?
The answer is clearly stated in RFC2616, section 9.6:
If a new resource is created, the origin server MUST inform the user agent via the 201 (Created) response. If an existing resource is modified, either the 200 (OK) or 204 (No Content) response codes SHOULD be sent to indicate successful completion of the request.
TL;DR
Use 200 OK in both cases if the distinction isn't important to clients.
Use 201 Created upon creation, and 200 OK upon update, if the distinction is important.
Details
With RESTful design, there's no single, correct way of doing things, but I've often found that the original HTTP status code specification provides good guidance.
It's always fine to return 200 OK if you don't expect the client to take separate action based on the response, but you can provide more information if you think the client is going to need it. Just be aware that clients may come to expect that of your API, so once you start providing more details, you can't easily change your mind.
That said, if you want to return 201 Created, then according to the specification:
The response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate.
As an example, you can put the location in the Location header, but you can also put it in the body of the response (but then you must document the structure and content-type of that body).
When it comes to 204 No Content responses, they create dead ends for link-following clients, so if you're creating a true level 3 REST API then you should avoid that response type as a courtesy to clients.
I am building a RESTful data store and leveraging Conditional GET and PUT. During a conditional PUT the client can include the Etag from a previous GET on the resource and if the current representation doesn't match the server will return the HTTP status code of 412 (Precondition Failed). Note this is an Atom based server/protocol.
My question is, when I return the 412 status can I also include the new representation of the resource or must the user issue a new GET? The HTTP spec doesn't seem to say yes or no and neither does the Atom spec (although their example shows an empty entity body on the response). It seems pretty wasteful not to return the new representation and make the client specifically GET it. Thoughts?
Although conditional GETs and PUTs are summarized as 'conditional requests' they are very different conceptually. Conditional GETs are a performance optimization and conditional PUTs are a concurrency control mechanism. It is hard to discuss them together.
To your question regarding the conditional GET: If you send GET and include an If-None-Match header the server will send 200 Ok if the resource has changed and 304 Not Modified if it did not (if the condition failed). 412 is only to be used with conditional PUTs.
UPDATE: It seems I misread the question slightly. To your point regarding the 'refresh' of the local copy upon a failed conditional PUT: It might well be that a cache already has the newest version and that your refresh-GET will be served from some cache. Having the server return the current entity with the 412 might actually give you worse performance.
No, technically you should not. Error codes are generally to specify that something has gone wrong. Although nothing would stop you from returning content (and in fact, some errors like a 404 return a pretty page that says You didn't find what you're looking for), the point of the response is not to return other content, but to return something that tells you what was wrong. Technically you also should not return that data because you passed the If-None-Match: etag (I'm assuming that's what you passed?)
On another note, do you really need to optimize away one additional http call?
The more I think about this, the more I'm convinced it's a bad idea - Are you going to return the content on any other errors? PUT semantics are to PUT. GET semantics should be used for GET.
If the number of additional requests incurred, due to an extra request after an update conflict, is significant enough for you to have performance concerns, then I would suggest you might have issues with the granularity of your resources.
Do you really expect millions of times a day multiple users will be editing the same resource simultaneously? Maybe you need to be storing delta changes to the resource instead updating the resource directly. If there really is that much contention for these resources then aren't users going to be constantly working on out of date data.
If your problem was that your resource contains the last-modified date and last-modified user and you had to do a GET after every PUT then I would be more convinced of the need to twist the rules.
However, I think the performance hit of the extra request is worth it for the clarity to the client developer.