I'm using AWS S3 REST API, and after solving some annoying problems with signing it seems to work. However, when I use correct REST verb for creating resource, namely POST, I get 405 method not allowed. Same request works fine with method PUT and creates resource.
Am I doing something wrong, or is AWS S3 REST API not fully REST-compliant?
Yes, you are wrong in mapping CRUD to HTTP methods.
Despite the popular usage and widespread misconception, including high-rated answers here on Stack Overflow, POST is not the "correct method for creating resource". The semantics of other methods are determined by the HTTP protocol, but the semantics of POST are determined by the target media type itself. POST is the method used for any operation that isn't standardized by HTTP, so it can be used for creation, but also can be used for updates, or anything else that isn't already done by some other method. For instance, it's wrong to use POST for retrieval, since you have GET standardized for that, but it's fine to use POST for creating a resource when the client can't use PUT for some reason.
In the same way, PUT is not the "correct method for updating resource". PUT is the method used to replace a resource completely, ignoring its current state. You can use PUT for creation if you have the whole representation the server expects, and you can use PUT for update if you provide a full representation, including the parts that you won't change, but it's not correct to use PUT for partial updates, because you're asking for the server to consider the current state of the resource. PATCH is the method to do that.
In informal language, what each method says to the server is:
POST: take this data and apply it to the resource identified by the given URI, following the rules you documented for the resource media type.
PUT: replace whatever is identified by the given URI with this data, ignoring whatever is in there already, if anything.
PATCH: if the resource identified by the given URI still has the same state it had the last time I looked, apply this diff to it.
Notice that create or update isn't mentioned and isn't part of the semantics of those methods. You can create with POST and PUT, but not PATCH, since it depends on a current state. You can update with any of them, but with PATCH you have an update conditional to the state you want to update from, with PUT you update by replacing the whole entity, so it's an idempotent operation, and with POST you ask the server to do it according to predefined rules.
By the way, I don't know if it makes sense to say that an API is or isn't REST-compliant, since REST is an architectural style, not a spec or a standard, but even considering that, very few APIs who claim to be REST are really RESTful, in most cases because they are not hypertext driven. AWS S3 is definitely not RESTful, although where it bears on your question, their usage of HTTP methods follows the HTTP standard most of the time.
+--------------------------------------+---------------------+
| POST | PUT |
+--------------------------------------+---------------------+
| Neither safe nor idempotent Ex: x++; | Idempotent Ex: x=1; |
+--------------------------------------+---------------------+
To add to #Nicholos
From the http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
POST:
The posted entity is subordinate to the URI in the same way that a
file is subordinate to a directory containing it, a news article is
subordinate to a newsgroup to which it is posted, or a record is
subordinate to a database
The action performed by the POST method might not result in a resource
that can be identified by a URI. In this case, either 200 (OK) or 204
(No Content) is the appropriate response status, depending on whether
or not the response includes an entity that describes the result
If a resource has been created on the origin server, the response
SHOULD be 201 (Created)
PUT:
The PUT method requests that the enclosed entity be stored under the
supplied Request-URI. If the Request-URI refers to an already existing
resource, the enclosed entity SHOULD be considered as a modified
version of the one residing on the origin server. If the Request-URI
does not point to an existing resource, and that URI is capable of
being defined as a new resource by the requesting user agent, the
origin server can create the resource with that URI. If a new resource
is created, the origin server MUST inform the user agent via the 201
(Created) response. If an existing resource is modified, either the
200 (OK) or 204 (No Content) response codes SHOULD be sent to indicate
successful completion of the request
IMO PUT can be used to create or modify/replace the enclosed entity.
In the original HTTP specification, the resource given in the payload of a POST request is "considered to be subordinate to the specified object" (i.e. the request URL). TimBL has said previously (can't find the reference) that it was modelled on the identically-named method in NNTP.
Related
Designing API using JSONAPI 1.0 standard there is no PUT method. There is only POST method for create resource and PATCH for partially update. We have use case where user can send request to the server and if resource doesn't exist then must be created otherwise updated. RFC describe such method as a PUT. Next quoting mentioned RFC 5789 standard for PATCH there is information:
"If the Request-URI does not point to an existing resource, the server MAY create a new resource,
depending on the patch document type (whether it can logically modify a null resource) and permissions, etc."
Is it good idea to have PATCH method for update and create resource? Which standard should be used to support both PUT and PATCH methods (maybe OpenApi)?
How to interpret RFC description?
Best Regards
We have use case where user can send request to the server and if resource doesn't exist then must be created otherwise updated.
The right answer, in this case, is almost certainly going to be to POST your request to the collection resource, and let the server figure out the "right" thing to do.
A resource can be created by sending a POST request to a URL that represents a collection of resources.
Using PUT to create a resource assumes that the client can/should guess what the correct identifier for the new resource should be. If we're not giving the client that authority/control, then the request needs to instead use a stable target-uri, and the server computes the side effects on other resources.
In JSON:API, the server gets to control the choice of URI for the new item.
POST /photos HTTP/1.1
Content-Type: application/vnd.api+json
...
HTTP/1.1 201 Created
Location: http://example.com/photos/550e8400-e29b-41d4-a716-446655440000
If the API were supporting PUT semantics, an equivalent change would look
something like
PUT /photos/550e8400-e29b-41d4-a716-446655440000 HTTP/1.1
Content-Type: application/vnd.api+json
HTTP/1.1 201 Created
But JSON:API has decided that PUT isn't interesting yet. Reading between the lines, the authors decided that PUT should be reserved until more implementations demonstrate that they understand the HTTP spec.
So instead you have POST to the collection for the create, and PATCH on the item to for partial or complete replacement.
That in turn implies that if the client doesn't/cannot know that a resource already exists, that it should POST to the collection. The server, in turn, should be aware that the resource may already exist, and do something sensible (replace the representation of the resource, redirect the client to the resource, etc). How the server achieves that would be an implementation detail.
Looking into Internet deal with REST HTTP methods I have never seen that PATCH can be used for resource creation therefore I am surprised that JsonApi forgo PUT method.
PATCH can certainly be used for resource creation -- see RFC 5789
If the Request-URI does not point to an existing resource, the server MAY create a new resource, depending on the patch document type (whether it can logically modify a null resource) and permissions, etc.
It's an uncommon choice, because PUT semantics are a better fit for that use case. Choosing to support PATCH, but not PUT, is weird.
I am surprised that JsonApi forgo PUT method
I am also surprised.
It might be possible to resolve your concerns by registering a new profile, encouraging the community to adopt a common pattern for the semantics that you need.
According to Postel's law one should be conservative in what you do, be liberal in what you accept from others.
Two common media-types used with PATCH are application/json-patch+json (a.k.a. JSON Patch) and application/json-merge-patch+json (a.k.a MergePatch).
MergePatch defines a couple of rules that determine whether a part needs to be added, removed or updated. The spec defines that a request received of that type needs to be processed by calling a function that takes in two arguments, the target resource and the representation received. The target itself might be either a JSON value or undefined. If the resource does not yet exist it is undefined and will lead to all of the values in the received patch document to be added to the yet undefined resource. This is basically your resource creation then.
JSON Patch, in contrast to MergePatch, is specified to only operate on JSON documents. There is no mention how the patch needs to be applied in case no resource was yet available. This makes somehow sense if you look at the operations is offers, such as test, remove, replace or move that only work if there is a counterpart in the original JSON document availalbe. This media-type is quite close to the actual PATCH definition in that a client sends a set of instructions, which were previously calculate by the client, that need to be applied atomically by the server. Either all or none of the changes are applied. Here a client should have already fetched the current state of the target resource beforehand, otherwise it wont be able to calculate the necessary changes to transform the current representation into the desired one. So applying a representation of that media-type only makes sense if there is already a resource available. If the client saw that no target resource is yet available it simply can use POST then to create the resource. If a client though sends a patch document containing only add operations I'd create a JSON representation and add all of the fields accordingly though.
As you see there are two different takes on how PATCHing can be done in HTTP. One being very close to the original idea of how patching is done in software engineering for decades while the other method being a more pragmatic approch to partial updating remote resources. Which one you use or support (in best case both) is your choice.
Lets suppose that someone performs a PUT request on my endoint:
/resources/{id}
However there is not resource with the given id stored in my PostgreSQL database.
According to the RFC 2616, I should create the resource if I am capable to:
The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI.
Would be okay to create the resource with the provided id? As manually assigning ids on database insert is not the best practice.
Should I return a 404 error if the creation of the resource is not possible?
First of all, you are using an obsolete document: The RFC 2616 is no longer relevant nowadays and anyone using such document as reference should stop right away.
Quoting Mark Nottingham who, at the time of writing, co-chairs the IETF HTTP and QUIC Working Groups:
Don’t use RFC2616. Delete it from your hard drives, bookmarks, and burn (or responsibly recycle) any copies that are printed out.
The old RFC 2616 has been supplanted by the following documents that, together, define the HTTP/1.1 protocol:
RFC 7230: Message Syntax and Routing
RFC 7231: Semantics and Content
RFC 7232: Conditional Requests
RFC 7233: Range Requests
RFC 7234: Caching
RFC 7235: Authentication
If you are looking for methods, status codes and headers definitions, then the RFC 7231 is the document you should refer to.
Having said that, let's go back to your question.
Should HTTP PUT create a resource if it does not exist?
It depends.
But, if your application generates resource identifiers on behalf of the client, as you mentioned in your question, then you should use POST instead of PUT for creating resources.
Some parts of the PUT method definition are quoted below. The last sentence seems to be the most relevant to you (highlight is mine), supporting what I've just mentioned above:
4.3.4. PUT
The PUT method requests that the state of the target resource be created or replaced with the state defined by the representation enclosed in the request message payload. [...]
If the target resource does not have a current representation and the PUT successfully creates one, then the origin server MUST inform the user agent by sending a 201 (Created) response. If the target resource does have a current representation and that representation is successfully modified in accordance with the state of the enclosed representation, then the origin server MUST send either a 200 (OK) or a 204 (No Content) response to indicate successful completion of the request. [...]
Proper interpretation of a PUT request presumes that the user agent knows which target resource is desired. A service that selects a proper URI on behalf of the client, after receiving a state-changing request, SHOULD be implemented using the POST method rather than PUT. [...]
Should I return a 404 error if the creation of the resource is not possible?
That's seems to be an accurate status code to be returned, as no representation has been found for the requested resource:
6.5.4. 404 Not Found
The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists. [...]
Now, for the sake of completeness, find below some relevant quotes on the POST method definition, which should be used to create resources in the scenario described in your question:
4.3.3. POST
The POST method requests that the target resource process the representation enclosed in the request according to the resource's own specific semantics. For example, POST is used for the following functions (among others):
[...]
Creating a new resource that has yet to be identified by the origin server;
[...]
If one or more resources has been created on the origin server as a result of successfully processing a POST request, the origin server SHOULD send a 201 (Created) response containing a Location header field that provides an identifier for the primary resource created and a representation that describes the status of the request while referring to the new resource(s).
While the 201 status code indicates that a new resource has been created, the Location header indicate where the newly created resource is located. If no Location header is provided, then the client should assume that the resource is identified by the effective request URI:
6.3.2. 201 Created
The 201 (Created) status code indicates that the request has been fulfilled and has resulted in one or more new resources being created. The primary resource created by the request is identified by either a Location header field in the response or, if no Location field is received, by the effective request URI. [...]
In short, it depends wheter the payload you want to store violates any constraint the server has for resources or not.
In general I'd say it should attempt it as the client explicitly expresses his intent to store that particular representation at the target URI. The server should though perform constraint checks before! Usually, in a real REST scenario though, the client should use URI that are provided by the server and not just chose any URI on its own. Thereby, a server should be in control of its namespace, as such using PUT to create resources is not recommended here by default.
With that being said, as PUT is idempotent while POST being not, some clients might want to benefit from this property. Here a POST-PUT creation pattern has evolved, where a client is attempting to create a new resource via POST until it receives a confirmation via a Location header in the response and afterwards attempts the update of that resource's state via PUT. This way the client can be sure that in case of transmission problems the representation was only created once. Depending on the stance, some people might consider the actual update of the resource as the actual resource creation, though as the client beforehand received the respective link, this is not quite the case.
Note that a server also has the right to transform the representation to something different if i.e. the server is configured to provide specific representations for certain URI endpoints. Think of uploading an image via PUT to a URI and the server embedds the image into a HTML page
There's two questions embedded here: 1) should PUT try to create the resource and 2) what happens if it cant.
1)
The RFC linked by #cass says https://www.rfc-editor.org/rfc/rfc7231#section-4.3.4:
The PUT method requests that the state of the target resource be
created or replaced with the state defined by the representation
enclosed in the request message payload. A successful PUT of a given
representation would suggest that a subsequent GET on that same
target resource will result in an equivalent representation being
sent in a 200 (OK) response.
Further, Mozilla's text https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/PUT
The HTTP PUT request method creates a new resource or replaces a representation of the target resource with the request payload.
Further, from the original RFC (that was replaced with the above test) https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI.
This is a bit anecdotal, but the Kubernetes API also carefully makes this distinction and informs it's users of PATCH if they really meant update: https://kubernetes.io/docs/reference/using-api/api-concepts/#api-verbs
For PUT requests, Kubernetes internally classifies these as either create or update based on the state of the existing object. An update is different from a patch; the HTTP verb for a patch is PATCH.
2
In terms of "what happens if it fails" I think the code depends on what went wrong:
400: it couldnt be created due to a bad payload
409: it couldnt be created due to a conflict - for example some field in the input JSON has some global uniqueness check on it
502/3 - it couldnt be created because it tried to call the database and it was dead
I'm not sure if 404 is the best code, becuase it doesnt tell the user anything about why.
how to handle resource checking on server side?
For example, my api looks like:
/books/{id}
After googling i found, that i should use HEAD method to check, if resource exists.
https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
I know, that i can use GET endpoint and use HEAD method to fetch information about resource and server does not return body in this case.
But what should i do on server side?
I have two options.
One endpoint marked as GET. I this endpoint i can use GET method to fetch data and HEAD to check if resource is available.
Two endpoints. One marked as GET, second as HEAD.
Why i'm considering second solution?
Let's assume, that GET request fetch some data from database and process them in some way which takes some time, eg. 10 ms
But what i actually need is only to check if data exists in database. So i can run query like
select count(*) from BOOK where id = :id
and immediately return status 200 if result of query is equal to 1. In this case i don't need to process data so i get a faster response time.
But... resource in REST is a object which is transmitted via HTTP, so maybe i should do processing data but not return them when i use HEAD method?
Thanks in advance for your answer!
You could simply delegate the HEAD handler to the existing GET handler and return the status code and headers only (ignoring the response payload).
That's what some frameworks such as Spring MVC and JAX-RS do.
See the following quote from the Spring MVC documentation:
#GetMapping — and also #RequestMapping(method=HttpMethod.GET), are implicitly mapped to and also support HTTP HEAD. An HTTP HEAD request is processed as if it were HTTP GET except but instead of writing the body, the number of bytes are counted and the "
Content-Length header set.
[...]
#RequestMapping method can be explicitly mapped to HTTP HEAD and HTTP OPTIONS, but that is not necessary in the common case.
And see the following quote from the JAX-RS documentation:
HEAD and OPTIONS requests receive additional automated support. On receipt of a HEAD request an implementation MUST either:
Call a method annotated with a request method designator for HEAD or, if none present,
Call a method annotated with a request method designator for GET and discard any returned entity.
Note that option 2 may result in reduced performance where entity creation is significant.
Note: Don't use the old RFC 2616 as reference anymore. It was obsoleted by a new set of RFCs: 7230-7235. For the semantics of the HTTP protocol, refer to the RFC 7231.
Endpoint should be the same and server side script should make decision what to do based on method. If method is HEAD, then just return suitable HTTP code:
204 if content exists but server don't return it
404 if not exists
4xx or 5xx on other error
If method is GET, then process request and return content with HTTP code:
200 if content exists and server return it
404 if not exists
4xx or 5xx on other error
The important thing is that URL should be the same, just method should be different. If URL will be different then we talking about different resources in REST context.
Your reference for HTTP methods is out of date; you should be referencing RFC 7231, section 4.3.2
The HEAD method is identical to GET except that the server MUST NOT send a message body in the response (i.e., the response terminates at the end of the header section).
This method can be used for obtaining metadata about the selected representation without transferring the representation data and is often used for testing hypertext links for validity, accessibility, and recent modification.
You asked:
resource in REST is a object which is transmitted via HTTP, so maybe i should do processing data but not return them when i use HEAD method?
That's right - the primary difference between GET and HEAD is whether the server returns a message-body as part of the response.
But what i actually need is only to check if data exists in database.
My suggestion would be to use a new resource for that. "Resources" are about making your database look like a web site. It's perfectly normal in REST to have many URI that map to a queries that use the same predicate.
Jim Webber put it this way:
The web is not your domain, it's a document management system. All the HTTP verbs apply to the document management domain. URIs do NOT map onto domain objects - that violates encapsulation. Work (ex: issuing commands to the domain model) is a side effect of managing resources. In other words, the resources are part of the anti-corruption layer. You should expect to have many many more resources in your integration domain than you do business objects in your business domain.
When you use PUT endpoint to create resource in REST api, what should the endpoint return for subsequent calls after returning 201(created) for the first call? 403(cannot create since the resource already exist)? 200(updated to the same exact object?) if you change the status code after one call(201-> 200 or 403), isn't that a violation of idempotency? I looked everywhere but all I can find is you can use PUT to create but nowhere it said about status code change after resource creation.
In short my question is that PUT is an idempotent method, but when it is used in resource creation, can it still change it's return status code from the following calls?
p.s.
After first calls, it will be idempotent(constantly 403 or 200). And ideally I want to be able to tell the client that the resource is already created and you shouldn't call this again.(403)
I know using POST is an alternative but as ID is already known to client at the point of creation I wanna use PUT method but want to know the proper REST way in terms of idempotency.
===================================================================
References of Using PUT endpoints for creating resources
http://restcookbook.com/HTTP%20Methods/put-vs-post/
https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
The fundamental difference between the POST and PUT requests is
reflected in the different meaning of the Request-URI. The URI in a
POST request identifies the resource that will handle the enclosed
entity. That resource might be a data-accepting process, a gateway to
some other protocol, or a separate entity that accepts annotations. In
contrast, the URI in a PUT request identifies the entity enclosed with
the request -- the user agent knows what URI is intended
9.6. PUT If a new resource is created, the origin server MUST inform the user agent via the 201 (Created) response. If an existing resource
is modified, either the 200 (OK) or 204 (No Content) response codes
SHOULD be sent to indicate successful completion of the request.
http://zalando.github.io/restful-api-guidelines/http/Http.html
PUT requests are usually robust against non-existence of resources by
implicitly creating before updating
successful PUT requests will usually generate 200 or 204 (if the
resource was updated - with or without actual content returned), and
201 (if the resource was created)
Idempotency is about the server state - not about the responses. E.g. DELETE is idempotent, but after the 2nd try the resource will not be found and you may choose to respond with 404. But the state of the server is going to be the same - the resource is deleted.
Same with PUT - you can invoke it multiple times, but the state of the server will always be the same after the operation is finished.
Ideally though you could reuse PUT for updating the resources. So when the 2nd request is arrived you can use that for updating instead of returning errors. That will probably simplify implementation and the contract.
REST recommends that queries (not resource creation) be done through the GET method. In some cases, the query data is too large or structured in a way that makes it difficult to put in a URL, and to address this, a RESTful API is modified to support queries with bodies.
It seems that the convention for RESTful queries that require bodies is to use POST. Here are a few examples:
Dropbox API
ElasticSearch
O'Reilley's Restful Web Services Cookbook
Queries don't modify the internal state of the system, but POST doesn't support idempotent operations. However, PUT is idempotent. Why don't RESTful APIs use PUT with a body instead of POST for queries that require a body?
NOTE: A popular question asks which (PUT vs POST) is preferred for creating a resource. This question asks why PUT is not used for queries that require bodies.
No. PUT might be idempotent, but it also has a specific meaning. The body of the request in PUT should be used to replace the resource in the URI.
With POST no such assumptions are being made. And note that using a POST request means that the request might not be idempotent, in specific cases it still might be.
However, you could do it with PUT, but it requires you to jump through an extra hoop. Basically, you could create a "query resource" with PUT, and then use GET immediately after to fetch the results of this query resource. Perhaps this was what you were after, but this is the most RESTful, because the resulting query results can still be linked to. (something which is completely missing if you use POST requests).
You should read the standard: http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
The definition of POST:
The POST method is used to request that the origin server accept the
entity enclosed in the request as a new subordinate of the resource
identified by the Request-URI in the Request-Line.
The action performed by the POST method might not result in a resource
that can be identified by a URI. In this case, either 200 (OK) or 204
(No Content) is the appropriate response status, depending on whether
or not the response includes an entity that describes the result.
The definition of PUT
The PUT method requests that the enclosed entity be stored under the
supplied Request-URI. If the Request-URI refers to an already existing
resource, the enclosed entity SHOULD be considered as a modified
version of the one residing on the origin server. If the Request-URI
does not point to an existing resource, and that URI is capable of
being defined as a new resource by the requesting user agent, the
origin server can create the resource with that URI.
If the resource could not be created or modified with the Request-URI, an appropriate error response SHOULD be given that reflects the nature of the problem.
Another thing that PUT is not cacheable while POST is.
Responses to this method are not cacheable, unless the response
includes appropriate Cache-Control or Expires header fields.
e.g. http://www.ebaytechblog.com/2012/08/20/caching-http-post-requests-and-responses/