Roy Fielding writes
A REST API must not define fixed
resource names or hierarchies (an
obvious coupling of client and
server). Servers must have the freedom
to control their own namespace.
Instead, allow servers to instruct
clients on how to construct
appropriate URIs, such as is done in
HTML forms and URI templates, by
defining those instructions within
media types and link relations.
How do you do this for system-to-system interfaces? Say the client wants to create an order in the server at http://my.server.org How is it supposed to learn that for creating an order it is supposed to use the url http://my.server.org/newOrder and not http://my.server.org/nO or anything else?
For a human interface (i.e. browser), I guess the server would provide some form of link (possibly in a form element) and the text around and in that link would tell a user which of the forms on that page is the correct one for creating an order (as supposed to creating a user or navigating to some search result)
What are the mechanics used for implementing this on the client side? And also: are they actually used or does the majority of people just hardwire the urls into the client?
How do you do this for
system-to-system interfaces? Say the
client wants to create an order in the
server at http://my.server.org How is
it supposed to learn that for creating
an order it is supposed to use the url
http://my.server.org/newOrder and not
http://my.server.org/nO or anything
else?
It doesn't learn. Machine clients, generally, can't "learn". Not yet at least, we're still pre-Skynet. You have to "teach" them.
But what the key is that you don't teach them URLs. You teach them relations.
Consider, in HTML...
<a rel="order" href="http://my.server.org/newOrder"/>
and
<a rel="order" href="http://my.server.org/nO"/>
You'll notice that the rel is the same, "order", but the URL is not.
In a "perfect" world, you system will have a single entry point, say, http://my.server.org/ and from there the client can find all of the rels that it needs to know about.
In practice, many systems have several "well known", and defined entry points from which the client can start, just as an expediency so the client does not alway have to start at the root of the system. These well known entry points have an implied commitment from the provider that these URLs won't be changing any time soon. That they're long lived, and the server will support them very well.
But once passed the entry point, any URL you find likely does not have such a promise behind it. The URL can be a one use only URL. It could be directed to different machines for, say, load balancing. Who knows. But as a consumer of the service, you really don't care what the URL is, you only care about the relation. The relation tells you the detail of the URL to use.
The documentation of your hypermedia API explains how to apply the uniform interface to each of the rels that your client will encounter. The client can't "intuit" that either, it has to be taught.
Basically, by teaching the client how to navigate the relations that it will or MAY find in the payloads it processes is how the client manipulates the hypermedia API. The payloads contain sign posts to show the way, but the server dictates where those sign posts go.
As for how often it is used, in the machine to machine world, likely not very much. Most systems aren't large enough where the URLs change enough to matter, and the clients are so few that changing the clients is not a significant burden. So most just hard code away.
But then, in the end, you just have bad clients. Nothing a REST system can do with a bad client. It can't tell them apart at runtime anyway.
No matter how you publish an API (to be consumed by machines), it is possible to make breaking changes.
When wrapping your API behind a UI (such as HTML forms), you have the freedom to change the URI without breaking the user, but that is because the user is consuming an abstraction you provided. Change the URL schema without changing your form, and you'll still break the client.
A couple ways to avoid breaking machine clients (basically, supporting backward-compatibility):
Build in some sort of URL versioning
Do redirection from old URL schemas to your new schema
We've quite sucessfully approached it the following way: expose a WADL file at the very root URL of the application describing the media types as well as where to find links in it and their semantics. I know this (WADL) is something seen critical by some in the REST community but I always felt intimidated by very URL focus of WADL only. Beyond all the religious debates we liked having a well defined way of documenting representations. There is a way to get around the URL focus of WADL and rather point out where links can be found in the representation and then rather document that. See that blog post (currently down because of maintenance so you might want to look at it in the Google cache) for details on the approach.
This results in only a single URL to be known by the client as he can find out about it accessing the WADL, and from then on just learn about the representation and where to find links, what HTTP method needs what parameters when being invoked and so on.
Related
I'm surprised to find so little mention of this dilemma online, and it makes me wonder if I'm totally missing something.
Assume I have a singleton resource called Settings. It is created on init/install of my web server, but certain users can modify it via a REST API, lets say /settings is my URI. I have a GET operation to retrieve the settings (as JSON), and a PATCH operation to set one or more of its values.
Now, I would like to let the user reset this resource (or maybe individual properties of it) to default - the default being "whatever value was used on init", before any PATCH calls were done. I can't seem to find any "best practice" approach for this, but here are the ones I have come up with:
Use a DELETE operation on the resource. It is after all idempotent, and its pretty clear (to me). But since the URI will still exist after DELETE, meaning the resource was neither removed nor moved to an inaccessible location, this contradicts the RESTful definition of DELETE.
Use a POST to a dedicated endpoint such as /settings/reset - I really dislike this one because its the most blatantly non-RESTful, as the verb is in the URI
Use the same PATCH operation, passing some stand-in for "default" such as a null value. The issue I have with this one is the outcome of the operation is different from the input (I set a property to null, then I get it and it has a string value)
Create a separate endpoint to GET the defaults, such as /setings/defaults, and then use the response in a PATCH to set to those values. This doesn't seem to contradict REST in any way, but it does require 2 API calls for seemingly one simple operation.
If one of the above is considered the best practice, or if there is one I haven't listed above, I'd love to hear about it.
Edit:
My specific project has some attributes that simplify this question, but I didn't mention them originally because my aim was for this thread to be used as a reference for anyone in the future trying to solve the same problem. I'd like to make sure this discussion is generic enough to be useful to others, but specific enough to also be useful to me. For that, I will append the following.
In my case, I am designing APIs for an existing product. It has a web interface for the average user, but also a REST (ish) API intended to meet the needs of developers who need to automate certain tasks with said product. In this oversimplified example, I might have the product deployed to a test environment on which i run various automated tests that modify the /settings and would like to run a cleanup script that resets /settings back to normal when I'm done.
The product is not SaaS (yet), and the APIs are not public (as in, anyone on the web can access them freely) - so the audience and thus the potential types of "clients" I may encounter is rather small - developers who use my product, that is deployed in their private data center or AWS EC2 machines, and need to write a script in whatever language to automate some task rather than doing it via UI.
What that means is that some technical considerations like caching are relevant. Human user considerations, like how consistent the API design is across various resources, and how easy it is to learn, are also relevant. But "can some 3rd party crawler identify the next actions it can perform from a given state" isn't so relevant (which is why we don't implement HATEOAS, or the OPTIONS method at all)
Let's discuss your mentioned options first:
1: DELETE does not necessarily need to delete or remove the state contained in the resource targeted by the URI. It just requires that the mapping of target URI to the resource is removed, which means that a consecutive request on the same URI should not return the state of the resource further, if no other operation was performed on that URI in the meantime. As you want to reuse the URI pointing to the client's settings resource, this is probably not the correct approch.
2: REST doesn't care about the spelling of the URI as long as it is valid according to RFC3986. There is no such thing as RESTful or RESTless URI. The URI as a whole is a pointer to a resource and a client should refrain from extracting knowledge of it by parsing and interpreting it. Client and server should though make use of link relation names URIs are attached to. This way URIs can be changed anytime and client will remain to be able to interact with the service further. The presented URI however leaves an RPC kind of smell, which an automated client is totally unaware of.
3: PATCH is actually pretty-similar to patching done by code versioning tools. Here a client should precalculate the steps needed to transform a source document to its desired form and contain these instructions into a so called patch document. If this patch document is applied by someone with the state of a document that matches the version used by the patch document, the changes should be applied correctly. In any other cases the outcome is uncertain. While application/json-patch+json is very similar to the philosophy on a patch-document containing separate instructions, application/merge-patch+json has a slightly different take on it by defining default rules (nulling out a property will lead to a removal, including a property will lead to its adding or update and leaving out properties will ignore these properties in the original document)
4: In this sense first retrieving the latest state from a resource and locally updating it/calculating the changes and then send the outcome to the server is probably the best approach of the ones listed. Here you should make use of conditional requests to guarantee that the changes are only applied on the version you recently downloaded and prevent issues by ruling out any intermediary changes done to that resource.
Basically, in a REST architecture the server offers a bunch of choices to a client that based on his task will chose one of the options and issue a request to the attached URI. Usually, the client is taught everything it needs to know by the server via form representations such as HTML forms, HAL forms or ION.
In such an environment settings is, as you mentioned, a valid resource on its own, so is also a default settings resource. So, in order to allow a client to reset his settings it is just a matter of "copying" the content of the default settings resource to the target settings resource. If you want to be WebDAV compliant, which is just an extension of HTTP, you could use the COPY HTTP operation (also see other registered HTTP operations at IANA). For plain HTTP clients though you might need a different approach so that any arbitrary HTTP clients will be able to reset settings to a desired default one.
How a server wants a client to perform that request can be taught via above mentioned form support. A very simplistic approach on the Web would be to send the client a HTML page with the settings pre-filled into the HTML form, maybe also allow the user to tweak his settings to his wishes beforehand, and then click a submit button to send the request to the URI present in the action attribute of the form, which can be any URI the server wants. As HTML only supports POST and GET in forms, on the Web you are restricted to POST.
One might think that just sending a payload containing the URI of the settings resource to reset and optionally the URI to the default settings to a dedicated endpoint via POST is enough and then let it perform its magic to reset the state to the default one. However, this approach does bypass caches and might let them believe that the old state is still valid. Caching in HTTP works as such that the de-facto URI of a resource is used as key and any non-safe operations performed on that URI will lead to an eviction of that stored content so that any consecutive requests would directly go to the server instead of being served by the cache instead. As you send the unsafe POSTrequest to a dedicated resource (or endpoint in terms of RPC) you miss out on the capability to inform the cache about the modification of the actual settings resource.
As REST is just a generalization of the interaction model used on the human Web, it is no miracle that the same concepts used on the Web also apply onto the application domain level. While you can use HTML here as well, JSON-based formats such as application/hal+json or the above mentioned HAL forms or ION formats are probably more popular. In general, the more media-type your service is able to support, the more likely the server will be to server a multitude of clients.
In contrast to the human Web, where images, buttons and further stuff provide an affordance of the respective control to a user, arbitrary clients, especially automated ones, usually don't coop with such affordances good. As such other ways to hint a client on the purpose of a URI or control element need to be provided, such as link relation names. While <<, <, >, >> may be used on a HTML page link to indicate first, previous, next and last elements in a collection, link relation here provide first, prev, next and last as alternatives. Such link relations should of course be either registered with IANA or at least follow the Web linking extension approach. A client looking up the URI on a prev relation will know the purpose of the URI as well as still be able to interact with the server if the URI ever changes. This is in essence also what HATEOAS is all about, using given controls to navigate the application though the state machine offered by the server.
Some general rules of thumb in designing applications for REST architectures are:
Design the interaction as if you'd interact with a Web page on the human Web, or more formally as a state machine or domain application protocol, as Jim Webber termed it, a client can run through
Let servers teach clients on how requests need to look like via support of different form types
APIs shouldn't use typed resources but instead rely on content type negotiation
The more media type your API or client supports the more likely it will be to interact with other peers
Long story short, in summary, a very basic approach is to offer a client a pre-filled form with all the data that makes up the default settings. The target URI of the action property targets the actual resource and thus also informs caches about the modification. This approach is on top also future-proof that clients will be served automatically with the new structure and properties a resource supports.
... so the audience and thus the potential types of "clients" I may encounter is rather small - developers who use my product, that is deployed in their private data center or AWS EC2 machines, and need to write a script in whatever language to automate some task rather than doing it via UI.
REST in the sense of Fielding's architectural style shines when there are a multitude of different clients interacting with your application and when there needs to be support for future evolution inherently integrated into the design. REST just gives you the flexibility to add new features down the road and well-behaved REST clients will just pick them up and continue. If you are either only interacting with a very limited set of clients, especially ones under your control, of if the likelihood of future changes are very small, REST might be overkill and not justify the additional overhead caused by the careful desing and implementation.
... some technical considerations like caching are relevant. Human user considerations, like how consistent the API design is across various resources, and how easy it is to learn, are also relevant. But "can some 3rd party crawler identify the next actions it can perform from a given state" isn't so relevant ...
The term API design already indicates that a more RPC-like approach is desired where certain operations are exposed user can invoke to perform some tasks. This is all fine as long as you don't call it REST API from Fielding's standpoint. The plain truth here is that there are hardly any applications/systems out there that really follow the REST architectural style but there are tons of "bad examples" who misuse the term REST and therefore indicate a wrong picture of the REST architecture, its purpose as well as its benefits and weaknesses. This is to some part a problem caused by people not reading Fielding's thesis (carefully) and partly due to the overall perference towards pragmatism and using/implementing shortcuts to get the job done ASAP.
In regards to the pragmatic take on "REST" it is hard to give an exact answer as everyone seems to understand different things about it. Most of those APIs rely on external documentation anyway, such as Swagger, OpenAPI and what not and here the URI seems to be the thing to give developers clue about the purpose. So a URI ending with .../settings/reset should be clear to most of the developers. Whether the URI has an RPC-smell to it or whether or not to follow the semantics of the respective HTTP operations, i.e. partial PUT or payloads within GET, is your design choice which you should document.
It is okay to use POST
POST serves many useful purposes in HTTP, including the general purpose of “this action isn’t worth standardizing.”
POST /settings HTTP/x.y
Content-Type: text/plain
Please restore the default settings
On the web, you'd be most likely to see this as a result of submitting a form; that form might be embedded within the representation of the /settings resource, or it might live in a separate document (that would depend on considerations like caching). In that setting, the payload of the request might change:
POST /settings HTTP/x.y
Content-Type: application/x-www-form-urlencoded
action=restoreDefaults
On the other hand: if the semantics of this message were worth standardizing (ie: if many resources on the web should be expected to understand "restore defaults" the same way), then you would instead register a definition for a new method token, pushing it through the standardization process and promoting adoption.
So it would be in this definition that we would specify, for instance, that the semantics of the method are idempotent but not safe, and also define any new headers that we might need.
there is a bit in it that conflicts with this idea of using POST to reset "The only thing REST requires of methods is that they be uniformly defined for all resources". If most of my resources are typical CRUD collections, where it is universally accepted that POST will create a new resource of a given type
There's a tension here that you should pay attention to:
The reference application for the REST architectural style is the world wide web.
The only unsafe method supported by HTML forms was POST
The Web was catastrophically successful
One of the ideas that powered this is that the interface was uniform -- a browser doesn't have to know if some identifier refers to a "collection resource" or a "member resource" or a document or an image or whatever. Neither do intermediate components like caches and reverse proxies. Everybody shares the same understanding of the self descriptive messages... even the deliberately vague ones like POST.
If you want a message with more specific semantics than POST, you register a definition for it. This is, for instance, precisely what happened in the case of PATCH -- somebody made the case that defining a new method with additional constraints on the semantics of the payload would allow a richer, more powerful general purpose components.
The same thing could happen with the semantics of CREATE, if someone were clever enough to sit down and make the case (again: how can general purpose components take advantage of the additional constraints on the semantics?)
But until then, those messages should be using POST, and general purpose components should not assume that POST has create semantics, because RFC 7231 doesn't provide those additional constraint.
I created a few Rest apis right now and I always preferred a solution, where I created an endpoint for each resource.
For example:
GET .../employees/{id}/account
GET .../supervisors/{id}/account
and the same with the other http methods like put, post and delete. This blows up my api pretty much. My rest apis in general preferred redundancy to reduce complexity but in this cases it always feels a bit cumbersome. So I create another approach where I work with inheritance to keep the "dry" principle.
In this case there is a base class User and via inheritance my employee and supervisor model extends from it. Now I only need one endpoint like
GET .../accounts/{id}
and the server decides which object is returned. Also while this thins out my api, it increases complexity and in my api documentation ( where I use spring rest docs ) I have to document two different Objects for the same endpoint.
Now I am not sure about what is the right way to do it ( or at least the better way ). When I think about Rest, I think in resources. So my employees are a seperate resource as well as my supervisors.
Because I always followed this approach, I tink I might be mentally run in it and maybe lost the objectivity.
It would be great if you can give my any objective advice on how this should be handled.
I built an online service that deals with this too. It's called Wirespec:
https://wirespec.dev
The backend automatically creates the url for users and their endpoints dynamically with very little code. The code for handling the frontend is written in Kotlin while the backend for generating APIs for users is written in Node.js. In both cases, the amount of code is very negligible and self-maintaining, meaning that if the user changes the name of their API, the endpoint automatically updates with the name. Here are some examples:
API: https://wirespec.dev/Wirespec/projects/apis/Stackoverflow/apis/getUserDetails
Endpoint: https://api.wirespec.dev/wirespec/stackoverflow/getuserdetails?id=100
So to answer your question, it really doesn't matter where you place the username in the url.
Try signing in to Wirespec with your Github account and you'll see where your Github username appears in the url.
There is, unfortunately, no wright or wrong answer to this one and it soley depends on how you want to design things.
With that being said, you need to distinguish between client and server. A client shouldn't know the nifty details of your API. It is just an arbitrary consumer of your API that is fed all the information it needs in order to make informed choices. I.e. if you want the client to send some data to the server that follows a certain structure, the best advice is to use from-like representations, such as HAL forms, Ion or even HTML. Forms not only teach a client about the respective properties a resource supports but also about the HTTP operation to use, the target URI to send the request to as well as the representation format to send the data in, which in case of HTML is application/x-www-form-urlencoded most of the time.
In regards to receiving data from the server, a client shouldn't attempt to extract knowledge from URIs directly, as they may change over time and thus break clients that rely on such a methodology, but rely on link relation names. Per URI there might be multiple link relation names attached to that URI. A client not knowing the meaning of one should simply ignore it. Here, either one of the standardized link relation names should be used or an extension mechanism as defined by Web linking. While an arbitrary client might not make sense from this "arbitrary string" out of the box, the link relation name may be considered the predicate in a tripple often used in ontologies where the link relation name "connects" the current resource with the one the link relation was annotated for. For a set of URIs and link relation names you might therefore "learn" a semantic graph over all the resources and how they are connected to each other. I.e. you might annotate an URI pointing to a form resource with prefetch to hint a client that it may load the content of the referenced URI if it is IDLE as the likelihood is high that the client will be interested to load that resource next anyway. The same URI might also be annotated with edit-form to hint a client that the resource will provide an edit form to send some data to the server. It might also contain a Web linking extension such as https://acme.org/ref/orderForm that allows clients, that support such a custom extension, to react to such a resource accordingly.
In your accounts example, it is totally fine to return different data for different resources of the same URI-path. I.e. resource A pointing to an employee account might only contain properties name, age, position, salery while resource B pointing to a supervisor could also contain a list of subordinates or the like. To a generic HTTP client these are two totally different resources even though they used a URI structure like /accounts/{id}. Resources in a REST architecture are untyped, meaning they don't have a type ouf of the box per se. Think of some arbitrary Web page you access through your browser. Your browser is not aware of whether the Web page it renders contains details about a specific car or about the most recent local news. HTML is designed to express a multitude of different data in the same way. Different media types though may provide more concrete hints about the data exchanged. I.e. text/vcard, applciation/vcard+xml or application/vcard+json all may respresent data describing an entity (i.e. human person, jusistic entity, animal, ...) while application/mathml+xml might be used to express certain mathematical formulas and so on. The more general a media type is, the more wiedspread usage it may find. With more narrow media types however you can provide more specific support. With content type negotiation you also have a tool at your hand where a client can express its capabilities to servers and if the server/API is smart enough it can respond with a representation the client is able to handle.
This in essence is all what REST is and if followed correctly allow the decoupling of clients from specific servers. While this might sound confusing and burdensome to implement at first, these techniques are intended if you strive for a long-lasting environment that still is able to operate in decateds to come. Evolution is inherently integrated into this phiolosophy and supported by the decoupled design. If you don't need all of that, REST might not be the thing you want to do actually. Buf if you still want something like REST, you for sure should design the interactions between client and server as if you'd intereact with a typical Web server. After all, REST is just a generalization of the concepts used on the Web quite successfully for the past two decades.
We are building set of new REST APIs.
Let's say we have a resource /users with the following fields:
{
id: 1
email: "test#user.com"
}
Clients implement this API and can then update this resource by sending a new resource representation to PUT /users/1.
Now let's say we add a new property name to the model like so:
{
id: 1
email: "test#user.com"
name: "test user"
}
If the models the existing clients are using are to call our API not updated, then calls to PUT /users/1 will remove the new name property since PUT is supposed to replace the resource. I know that the clients could work straight with the raw json to ensure they always receive any new properties that are added in the API, but that is a lot of extra work, and under normal circumstances clients are going to create their own model representations of the API resources on their side. This means that any time any new property is added, all clients need to update the code/models on their side to make sure they aren't accidentally removing properties. This creates unneeded coupling between systems.
As a way to solve this problem, we are considering not implementing PUT operations at all and switching updates to PATCH where properties that aren't passed in are simply not changed. That seems technically correct, but might not be in the spirit of REST. I am also slightly concerned about client support for the PATCH verb.
How are others solving this problem? Was is the best practice here?
You are in a situation where you need some form of API versioning. The most appropriate way is probably using a new media-type every time you make a change.
This way you can support older versions and a PUT would be perfectly legal.
If you don't want this and just stick to PATCH, PATCH is supported everywhere except if you use ancient browsers. Not something to worry about.
Switching from PUT to PATCH will not fix your problem, IMO. The root cause, IMO, is that clients already consider the data being returned for a representation to follow a certain type. According to Fielding
A REST API should never have “typed” resources that are significant to the client.
(Source)
Instead of using typed resources clients should use content-type negotiation to exchange data. Here, media-type formats that are generic enough to gain widespread adoption are for sure beneficial, certain domains may however require a more specific representation format.
Think of a car-vendor Web page where you can retrieve the data from your preferred car. You, as a human, can easily identify that the data depicts a typical car. However, the media-type you most likely received the data in (HTML) does not state by its syntax or the semantics of its elements that the data describes a car, unless some semantic annotation attributes or elements are present, though you might be able to update the data or use the data elsewhere.
This is possible as HTML ships with a rich specification of its elements and attributes, such as Web forms that not only describe the supported or expected input parameters but also the URI where to send the data to, the representation format to use upon sending (implicitly given by application/x-www-form-urlencoded; may be overwritten by the enctype attribute though) or the HTTP method to use, which is fixed to either GET or POST in HTML. Through this, a server is able to teach a client on how a request needs to be built. As a consequence the client does not need to know anything else besides having to understand the HTTP, URI and HTML specifications.
As Web pages are usually filled with all kinds of unrelated stuff, such as adds, styling information or scripts, and the XML(-like) syntax, which is not every ones favourite, as it may increase the size of the actual payload slightly, most so-called "REST" APIs do want to exchange JSON-based documents. While plain JSON is not an ideal representation format, as it does not ship with link-support at all, it is though very popular. Certain additions such as JSON Hyper-Schema (application/schema+json hyper-schema) or JSON Hypertext Application-Language (HAL) (application/hal+json) add support for links and link-relation. These can be used to render data received from the server as-is. However, if you want a response to automatically drive your application state (i.e. to dynamically draw the GUI with the processed data) a more specific representation format is needed, that can be parsed by your client and act accordingly as it understands what the server wants it to do with it (= affordance). If you like to instruct a client on how to build a request support for other media-types such as hal-forms or ion need to be supported. Certain media-types furthermore allow you to use a concept called profiles, that allow you to annotate a resource with a semantic type. HAL JSON i.e. does support something like that where the Content-Type header may now contain a value such as application/hal+json;profile=http://schema.org/Car that hints the media-type processor that the payload follows the definition of the given profile and may thus apply further validity checks.
As the representation format should be generic enough to gain widespread usage, and URIs itself shouldn't hint a client as well what kind of data to expect, an other mechanism needs to be used. Link relation names are basically an annotation for URIs that tell a client about the purpose of a certain link. A pageable collection might return links annotated with first, prev, next and last which are pretty obvious what they do. Other links might be hinted with prefetch, that hint a client that a resource can be loaded right after loading the current resource finished as it is very likely that the client will retrieve this resource next. Such media-types, however, should be either standardized (defined in a proposal or RFC and registerd with IANA) or follow the schema proposed by Web linking, (i.e. as used by Dublin Core). A client that just uses the URI for an invoked link-relation name will still work in case the server changes its URI scheme instead of attempting to parse some parameters from the URI itself.
In regards to de/coupling in a distributed system a certain amount of coupling has to exist otherwise parties wont be able to communicate at all. Though the point here is, the coupling should be based on well-defined and standardized formats that plenty of clients may support instead of exchanging specific representation formats only a very limited number of clients support (in worst case only the own client). Instead of directly coupling to the API and using an undefined JSON-based syntax (maybe with external documentation of the semantics of the respective fields) the coupling should now occur on the media-types parties can use to exchange the format. Here, not the question of which media-type to support should be asked but how many you want to support. The more media-types your client or server supports, the more likely it is to interact with other peers in the distributed system. On the grand-scheme of things, you want a server to be able to server a plethora of clients while a single client should be able to interact with (in best case) every server without the need for constant adoptions.
So, if you really want to decouple clients from servers, you should take a closer look at how the Web actually works and try to mimic its interaction model onto your application layer. As "Uncle Bob" Robert C. Martin mentioned
An architecture is about intent! (Source)
and the intention behind the REST architecture is the decoupling of clients from servers/services. As such, supporting multiple media-types (or defining your own-one that is generic enough to reach widespread adoption), looking up URIs just via their accompanying link-relation names and relying on content-type negotiation as well as relying only on the provided data may help you to achieve the degree of decoupling you are looking for.
All nice and well in theory, but so far every rest api I encountered in my career had predefined contracts that changed over time.
The problem here is, that almost all of those so called "REST APIs" are RPC services at its heart which should not be termed "REST" to start with - this is though a community issue. Usually such APIs ship with external documentation (i.e. Swagger) that just re-introduce the same problems classical RPC solutions, such as CORBA, RMI or SOAP, suffer from. The documentation may be seen as IDL in that process without the strict need for skeleton classes, though most "frameworks" use some kind of typed data classes that will either ignore the recently introduced field (in best case) or totally blow up on invocation.
One of the problems REST suffers from is, that most people haven't read Fieldings thesis and therefore don't see the big picture REST tries to establish but claim to know what REST is and therefore mix up things and call their services RESTful which lead to a situation where REST != REST. The ones pointing out what a REST architecture is and how one might achieve it are called out as dreamers and unworldly when the ones proclaiming the wrong term (RPC over HTTP = REST) continue to do so adding to the confusion of especially the ones just learning the whole matter.
I admit that developing a true REST architecture is really, really hard as it is just too easy to introduce some form of coupling. Hence, a very careful design needs to be done that needs time and also costs money. Money plenty of companies can't or don't want to spend, especially in a domain where new technologies evolve on a regular basis and the ones responsible for developing such solutions often leave the company before the whole process had finished.
Just saying it shouldn’t be ‘typed’ is not really a viable solution
Well, how often did you need to change your browser as it couldn't interact with a Web page? I don't talk about CSS-stuff or browser-specific CSS or JS stuff. How often needed the Web to change in the last 2-3 decades? Similar to the Web, the REST architecture is intended for long-lasting applications for years to come, that supports natural evolution by design. For simple frontend-2-backend systems it is for sure overkill. It starts to shine especially in cases where there are multiple peers not under your control you can interact with.
POST , PUT, PATCH and GET are all different. Idempotent and safety being the key difference makers.
While writing RESTFul APIs , I encountered guidelines on when and where to use one of the HTTP methods. Since I am using Java for the back-end implementation, I can control the behavior of the HTTP methods on the persistent data.
For example , GET v1/book/{id} can be replaced with POST v1/book (with "id" in body) now with that id I can perform a query on db , fetching that particular book. (assuming book with that id already exists).
Similarly , I can achieve the workings of PATCH and PUT with POST itself.
Now, coming to the question , why don't we just use POST instead of GET , PUT and PATCH almost every time, ALMOST, when we can control the idempotent and safety behavior in the back-end?
Or , Is it just a guideline mentioned in RESTFul docs somewhere or stated by Roy fielding and we all are blindly following? Even if the guidelines are so what is the major idea behind them?
https://restfulapi.net/rest-put-vs-post/
https://restful-api-design.readthedocs.io/en/latest/methods.html
https://www.keycdn.com/support/put-vs-post
Above resources just mention either what does all the methods do or their differences. Articles mention the workings as if they were some guidelines , none of the docs online speak about the reason behind them.
None of them says , what if I used POST instead of PUT, PATCH and GET, what would be the side-effects? (as I can control their behaviors in the back-end)
Http methods are designed in the way that each method holds some responsibility. I will say that REST are the standards which are conventions and not the obligation. The convention doesn't stress us to follow the rules but they are designed for our code betterment. You can tweak the things and can use them in your way but that would be a bad idea. Like in this case if you are performing all the three actions with one method it would create great confusion in code (As the simple definition of POST is to create an object and that is what understood by everyone) and also degrade our coding standards.
I strongly discourage to replace three methods with one.
If you do that, you can't say you are "writing RESTFul APIs".
Whoever knows the RESTFul standard, will be confused about the behaviour of your apis.
If you fit the standard, then you will have an easier life.
After all, you have no real benefit in your approach.
HTTP is a transport protocol which as its name suggest is responsible for transfering data such as files or db entries across the wire to or from a remote system. In version 0.9 you basically only had the GET operation at your disposal while in HTTP 1.0 almost all of the current operations were added to the spec.
Each of these methods fulfills its own purpose. POST i.e. does process the payload according to the server's own semantics, whatever they will be. In theory it could be used therefore for retrieving, updating or removing content. Though, to a client it is basically unclear what a server actually does with the payload. There is no guarantee whether invoking a URI with that method is safe (the remote resource being altered) or not. Think of a crawler that is just invoking any URIs it finds and one of the links is an order link or a link where you perform a payment process. Do you really want a crawler to trigger one of your processes? The spec is rather clear that if something like that happens, the client must not made accountable for that. So, if a crawler ordered 10k products as one of your links, did trigger such a process, and the products are created in that process, you can't claim refund from the crawler's maintainer.
In additon to that, a response from a GET operation is cacheable by default. So if you invoke the same resource twice in a certain amount of time, chances are that the resource does not need to be fetched again a second (third, ...) time as it can be reused from the cache. This can reduce the load on the server quite significantly if used propperly.
As you've mentioned Fielding and REST. REST is an architectural style which you should use if you have plenty of different clients connecting to your services that are furthermore not under your control. Plenty of so-called REST APIs aren't adhering to REST as they follow a more simple and pragmatic RPC approach with external documentations such as Swagger and similar. RESTs main focus is on the decoupling of clients from servers which allow the latters to evolve freely without having to fear breaking clients. Clients on the other hand get more robust to changes.
Fielding only added few constraints a REST architecture has to adhere to. One of them is support for caching. Though Fielding later on wrote a well-cited blog-post where he explains what API designers have to consider before calling their API REST. True decoupling can only occur if all of the constraints are followed strictly. If only one clients violates these premises it won't benefit from REST at all.
The main premise in REST is (and should always be): Server teaches clients what they need and clients only use what they are served with. In the browsable Web, the big cousin of REST, a server will teach a client i.e. on what data the server expects via Web Forms through HTML and links are annotated with link-relation names to give the browser some hints on when to invoke that URI. On a Web page a trash bin icon may indicate a delition while a pencil icon may indicate an edit link. Such visual hints are also called affordacne. Such visual hints may not be applicable in machine to machine communication though such affordances may hint on other things they may provide. Think of a stylesheet that is annotated with preload. In HTTP 2 i.e. such a resource could be attempted to be pushed by the server or in HTTP 1.1 the browser could alread load that stylesheet while the page is still parsed to speed things up. In order to gain whitespread knowledge of those meanings, such values should be standardized at IANA. Through custom extensions or certain microformats such as dublin core or the like you may add new relation names that are too specific for common cases but are common to the domain itself.
The same holds true for media-types client and server negotiate about. A general applicable media-type will probably reach wider acceptance than a tailor-made one that is only usable by a single company. The main aim here is to reach a point where the same media-type can be reused for various areas and APIs. REST vision is to have a minimal amount of clients that are able to interact with a plethora of servers or APIs, similar to a browser that is able to interact with almost all Web sites.
Your ultimate goal in REST is that a user is following an interaction protocol you've set up, which could be something similar to following an order process or playing a text game or what not. By giving a client choices it will progress through a certain process which can easily be depicted as state machine. It is following a kind of application-driven protocol by following URIs that caught the clients attention and by returning data that was taught through a form like representation. As, more or less, only standardized representation formats should be used, there is no need for out-of-band information on how to interact with the API necessary.
In reality though, plenty of enterprises don't really care about long-lasting APIs that are free to evolve over the years but in short-term success stories. They usually also don't care that much whether they use the propper HTTP operations at all or stay in bounds with the HTTP spec (i.e. sending payloads with HTTP GET requesst). Their primary intent is to get the job done. Therefore pragmatism usually wins over design and as such plenty of developers follow the way of short success and have to adept their work later on, which is often cumbersome as the API is now the driving factor of their business and therefore they can't change it easily without having to revampt the whole design.
... why don't we just use POST instead of GET , PUT and PATCH almost every time, ALMOST, when we can control the idempotent and safety behavior in the back-end?
A server may know that a request is idempotent, but the client does not. Properties such as safe and idempotency are promisses to the client. Whether the server satisfies these or not is a different story. How should a client know whether a sent payment request reached the server and the response just got lost or the initial request didn't make it to the server at all in case of a temporary connection issue? A PUT requests does guarante idempotency. I.e. you don't want to order the same things twice if you resubmit the same request again in case of a network issue. While the same request could also be sent via POST and the server being smart enough to not process it again, the client doesn't know the server's behavior unless it is externally documented somehwere, which violates REST principles again also somehow. So, to state it differently, such properties are more or less promisses to the client, less to the server.
In learning about REST architecture, I've noticed that Hypermedia seems to be an important part of the uniform interface constraint to become RESTful; however, I'm having a hard time understanding how this concept of hypermedia is to be consumed by a client based off of the definition of hypermedia found across the web.
From what I understand about REST, hypermedia is/are basically the links that are provided to a client in a rest response containing the representation of a resource (so that the client only has to know about the base entry point url to a REST service). The links are essentially there to help the client know what options it has regarding the representation it received (for example, if I request the /children/ resource, I might receive an xml list of children that also contains links to /children/youngest, /children/oldest/, children/create, etc... This is exactly what i have a hard time understanding... Why does the server have to return these "hypermedia" links to the client? Shouldn't the client have already known about those links? The client isn't going to read the links on its own and follow them correctly... Somebody has to write client code (html and/or javascript for example) beforehand. What good do these links do for the client if the client should have already known about them? What am i missing?
Hypermedia is indeed an important part of creating RESTful interfaces. However, I see a full hypermedia client (sometimes referred to as HATEOAS) as just a point on the spectrum of becoming more RESTful. The Richardson Maturity Model describes 3 steps to becoming more RESTful and I've found these as good guidelines when considering how much of a RESTful patten to adopt on particular projects.
IMO, the best example of a full hypermedia client is a web browser. The web browser understands that the html <a href> tag represents a link. The key to a link is that a user can discover functionality as they move through their journey. So, when designing a RESTful API, the same principle can be applied.
Why does the server have to return these "hypermedia" links to the
client?
So, one reason is that clients (if written to expose links) can allow state transitions to be discoverable.
Another benefit to using links is your API becomes responsible for the structure and data of each link. So, the server-side API is generating the links for each request. This means that the server API can make changes (version or structure) to the links without breaking the client.
You can build a RESTful API without links (Richardson Level 2), but then what if you want to change the URL for one of your API requests? You need to either implement a versioning strategy (through the url or headers) or you need to create a new url for the new feature. Through time, this can get out of hand.
Think about how browsing a web-page works. REST is very similar (although you're actually describing a related topic HATEOAS). The intent is that your client will retrieve the initial data, then have some way of deciding whether or not to request the referenced data. In a browser, some references (such as css, javascript, images) are immediately requested, and others, such as hrefs, are simply displayed to the user, and only retrieved if the user chooses to click the link. The same is true of your REST client -- you will decide for yourself, based on your requirements, which references to automatically request, and which to display to a user for decision making.
If you have an app that shows a person, along with their address, then your app would be written to automatically find the address reference in the person data and retrieve it. If your app shows a person and an option to show related data, then you'd show the person along with hyperlinks (or something similar) that allows the user to choose to retrieve and display the address data.
The issue really comes when you think about how the client visually represents the information it is retrieving. For this, the REST style suggests that you should allow the server to return javascript to dynamically parse the new data (for example, the address in my example above). Maybe the returned data also contains css-like references that will help. Maybe it'll contain javascript links to allow dynamic user interaction (which sounds dangerous to me, but what the hell, it works for the web).
Fielding defined the uniform interface constraint in order to decouple the clients from the implementation of the service. This is just the same as defining a new interface in any oo language. You define it to decouple a class from the implementation of its dependency.
To decouple the client from the URI structure you need links in the response, which contain semantic annotation and the URI. So your client can decide which link to follow based on the semantic annotation (e.g. link relation or a term in a related RDF vocab). This way it does not need to know anything about the URI structure.
You need to decouple clients from the implementation details of the services to make them reusable. At the end you can create general clients like a web browsers today. The difference that these clients will understand what they do, so they won't necessary rely on humans.
What good do these links do for the client if the client should have already known about them?
They make state transitions more directly a part of the interface. They tell the client whether these links or state transitions are available to be used (or not if they are not present!) in the current state.
If the client did not take notice of this, it could potentially still try to generate and follow an URL for an option (state transition) that was actually not available. It might be necessary to replicate the logic to determine what is possible in a particular state on both sides to be sure to avoid this.
If an invalid link/state transition is attempted, the server should report back an error.
On the other hand, if the client never allowed generating/following a link not provided by the server, it could have awareness and provide feedback about the problem earlier (normally resulting in a clearer situation or error message), and while the server should handle the invalid request correctly, in case it didn't it would be less of a problem as it is much less likely to happen in the first place.
Something like this.
var discountDetails = new AddDiscountForm()
{
promotionId = 123456,
discountType = "Promotion",
discount = UnitValue.FromFloat(14.00)
};
await journey
.SelectRoot()
.SelectRelation("product-finder", new { barcode = "23409-23097" })
.SelectItem(0)
.SelectRelation("add-discount", discountDetails)
.RunAsync();
In fact I took the idea from my response and built it.
https://github.com/lukepuplett/surfdude-csharp