This is a follow up on a question asking for an explanation of REST.
As you can see from the comments to my answer, we've had a small argument with Darrel Miller on the best media representation of the resources. We've had a further email discussion that resulted in this question.
The main difference between Darrel's and mine understanding of REST is whether the semantic of the data is part of the REST API.
Darrel believes (my interpretation of his words :-)) that the semantic of the data is an essential part of the REST API and as such, the media representation choosen should reflect it. Thus, a proper REST API should choose either:
a well-known media like ATOM to represent the data, so that as many clients could understand the semantic of the resource natively;
an app-specific media type like application/vdn.mycomany.mymedia and expect the client to understand this media type to be able to consume the resources data.
Application/xml is not a good resource representation, as it does not represent the semantic in the media type, yet requires the client to know more about the semantic.
I, on the other hand, believe that the REST API is a separate layer from the actual data representation. The media type exposed by the API is just a container to transfer the resource data. The actual semantic of the data is treated separately. Thus, a client that does not understand the data, can still consume the REST API. Application/xml is a really good data representation, as it allows tight coupling for clients that understand the schema, yet still allows client that don't understand the schema to do some basic processing of the resources.
Thus, the question: is the data semantic part of the REST API? Should we choose only media types for resources representation that actually represent the semantic of the data as well?
I would appreaciate if people post in their answers some citations, preferably from the Roy man himself. :-)
Let's start at the beginning: media types are there to provide the client with a format it can use to decide what to do next. Without an html page, the browser has no links to go to. Without an html renderer, the browser cannot render a page and won't know what to do.
Without a media type, the client has no clue if it will be able to do anything with the byte stream. Indeed, when a client receives the headers specifying application/xml, it has no knowledge of what to do beyond get an xml parser.
So the question really is, should the client be able to make a decision based on the http message without having a look inside the message, or should it go and peek inside the message (or worse, parse the message first) to know what to do.
Lack of media types means that your client will have to do additional peeking work, or worse process the entity body itself, before it can make a decision, be it for rendering or for processing. You now have to add a lot of custom behavior for each of your formats you may want to process, and you loose a bit of coupling in the process.
It's also an http fundamental that intermediaries should be able to process the requests without inspecting the body, and there as well application/xml is problematic.
Now when you say that the semantic of the media types is part or not of the API... What constitutes the API?
From a client perspective, there is no API. There is an initial representation that lets the client make a decision as to what to do next. The media type is indeed where the client gets the information it requires to navigate the "API", and as such there can be no API without representations.
Furthermore, a client should have only three bits of knowledge: a bootstrap location, the HTTP protocol and the media types. The first is only a URI and doesn't convey much beyond the location of a represetnation needed to continue. The second has already very clear semantics. The third is the one where you have control, as it's the contract you have with your client.
That contraact says that whenever you want to do something, the something will have semantics: to add a customer, send a application/vnd.acme.customer+xml to /customers using a POST.
Hence my answer: designing a REST architecture relies on two steps: resource modeling (at the conceptual level) and media type building. Anything else and you're likely doing it wrong.
I don't see the need to be overly pedantic about it. A resource can expose multiple representations; each with its own semantics (and even multiple dimensions of semantics at that). If one representation doesn't provide the semantics required by a particular use case, expose one that does.
Thus, a client that does not
understand the data, can still consume
the REST API.
I'm not sure that's a good litmus test for what does or does not make a decent representation. What good is a client that can consume a document but not understand it well enough to do anything with it? I guess I don't understand how "basic processing of resources" makes application/xml a better choice than some arbitrary blob of 1s and 0s?
Since you asked for references, here's an article from Roy Fielding where he "proposes" a bitmap representation of social network graphs. I can certainly get a machine to display these bitmaps, but of what use is that if I don't understand the underlying social network graph? Would changing the representation to application/xml allow a naive client to extract additional meaning from it that isn't contained in the bitmap? Nope.
Check out this set of slides from Mark Baker for more explanation as to why application/xml does not satisfy the "self-describing" constraint. You can also read a number of posts on his blog including this one where he continues to explain why application/xml + namespace is not equivalent to media-types.
Related
We are building set of new REST APIs.
Let's say we have a resource /users with the following fields:
{
id: 1
email: "test#user.com"
}
Clients implement this API and can then update this resource by sending a new resource representation to PUT /users/1.
Now let's say we add a new property name to the model like so:
{
id: 1
email: "test#user.com"
name: "test user"
}
If the models the existing clients are using are to call our API not updated, then calls to PUT /users/1 will remove the new name property since PUT is supposed to replace the resource. I know that the clients could work straight with the raw json to ensure they always receive any new properties that are added in the API, but that is a lot of extra work, and under normal circumstances clients are going to create their own model representations of the API resources on their side. This means that any time any new property is added, all clients need to update the code/models on their side to make sure they aren't accidentally removing properties. This creates unneeded coupling between systems.
As a way to solve this problem, we are considering not implementing PUT operations at all and switching updates to PATCH where properties that aren't passed in are simply not changed. That seems technically correct, but might not be in the spirit of REST. I am also slightly concerned about client support for the PATCH verb.
How are others solving this problem? Was is the best practice here?
You are in a situation where you need some form of API versioning. The most appropriate way is probably using a new media-type every time you make a change.
This way you can support older versions and a PUT would be perfectly legal.
If you don't want this and just stick to PATCH, PATCH is supported everywhere except if you use ancient browsers. Not something to worry about.
Switching from PUT to PATCH will not fix your problem, IMO. The root cause, IMO, is that clients already consider the data being returned for a representation to follow a certain type. According to Fielding
A REST API should never have “typed” resources that are significant to the client.
(Source)
Instead of using typed resources clients should use content-type negotiation to exchange data. Here, media-type formats that are generic enough to gain widespread adoption are for sure beneficial, certain domains may however require a more specific representation format.
Think of a car-vendor Web page where you can retrieve the data from your preferred car. You, as a human, can easily identify that the data depicts a typical car. However, the media-type you most likely received the data in (HTML) does not state by its syntax or the semantics of its elements that the data describes a car, unless some semantic annotation attributes or elements are present, though you might be able to update the data or use the data elsewhere.
This is possible as HTML ships with a rich specification of its elements and attributes, such as Web forms that not only describe the supported or expected input parameters but also the URI where to send the data to, the representation format to use upon sending (implicitly given by application/x-www-form-urlencoded; may be overwritten by the enctype attribute though) or the HTTP method to use, which is fixed to either GET or POST in HTML. Through this, a server is able to teach a client on how a request needs to be built. As a consequence the client does not need to know anything else besides having to understand the HTTP, URI and HTML specifications.
As Web pages are usually filled with all kinds of unrelated stuff, such as adds, styling information or scripts, and the XML(-like) syntax, which is not every ones favourite, as it may increase the size of the actual payload slightly, most so-called "REST" APIs do want to exchange JSON-based documents. While plain JSON is not an ideal representation format, as it does not ship with link-support at all, it is though very popular. Certain additions such as JSON Hyper-Schema (application/schema+json hyper-schema) or JSON Hypertext Application-Language (HAL) (application/hal+json) add support for links and link-relation. These can be used to render data received from the server as-is. However, if you want a response to automatically drive your application state (i.e. to dynamically draw the GUI with the processed data) a more specific representation format is needed, that can be parsed by your client and act accordingly as it understands what the server wants it to do with it (= affordance). If you like to instruct a client on how to build a request support for other media-types such as hal-forms or ion need to be supported. Certain media-types furthermore allow you to use a concept called profiles, that allow you to annotate a resource with a semantic type. HAL JSON i.e. does support something like that where the Content-Type header may now contain a value such as application/hal+json;profile=http://schema.org/Car that hints the media-type processor that the payload follows the definition of the given profile and may thus apply further validity checks.
As the representation format should be generic enough to gain widespread usage, and URIs itself shouldn't hint a client as well what kind of data to expect, an other mechanism needs to be used. Link relation names are basically an annotation for URIs that tell a client about the purpose of a certain link. A pageable collection might return links annotated with first, prev, next and last which are pretty obvious what they do. Other links might be hinted with prefetch, that hint a client that a resource can be loaded right after loading the current resource finished as it is very likely that the client will retrieve this resource next. Such media-types, however, should be either standardized (defined in a proposal or RFC and registerd with IANA) or follow the schema proposed by Web linking, (i.e. as used by Dublin Core). A client that just uses the URI for an invoked link-relation name will still work in case the server changes its URI scheme instead of attempting to parse some parameters from the URI itself.
In regards to de/coupling in a distributed system a certain amount of coupling has to exist otherwise parties wont be able to communicate at all. Though the point here is, the coupling should be based on well-defined and standardized formats that plenty of clients may support instead of exchanging specific representation formats only a very limited number of clients support (in worst case only the own client). Instead of directly coupling to the API and using an undefined JSON-based syntax (maybe with external documentation of the semantics of the respective fields) the coupling should now occur on the media-types parties can use to exchange the format. Here, not the question of which media-type to support should be asked but how many you want to support. The more media-types your client or server supports, the more likely it is to interact with other peers in the distributed system. On the grand-scheme of things, you want a server to be able to server a plethora of clients while a single client should be able to interact with (in best case) every server without the need for constant adoptions.
So, if you really want to decouple clients from servers, you should take a closer look at how the Web actually works and try to mimic its interaction model onto your application layer. As "Uncle Bob" Robert C. Martin mentioned
An architecture is about intent! (Source)
and the intention behind the REST architecture is the decoupling of clients from servers/services. As such, supporting multiple media-types (or defining your own-one that is generic enough to reach widespread adoption), looking up URIs just via their accompanying link-relation names and relying on content-type negotiation as well as relying only on the provided data may help you to achieve the degree of decoupling you are looking for.
All nice and well in theory, but so far every rest api I encountered in my career had predefined contracts that changed over time.
The problem here is, that almost all of those so called "REST APIs" are RPC services at its heart which should not be termed "REST" to start with - this is though a community issue. Usually such APIs ship with external documentation (i.e. Swagger) that just re-introduce the same problems classical RPC solutions, such as CORBA, RMI or SOAP, suffer from. The documentation may be seen as IDL in that process without the strict need for skeleton classes, though most "frameworks" use some kind of typed data classes that will either ignore the recently introduced field (in best case) or totally blow up on invocation.
One of the problems REST suffers from is, that most people haven't read Fieldings thesis and therefore don't see the big picture REST tries to establish but claim to know what REST is and therefore mix up things and call their services RESTful which lead to a situation where REST != REST. The ones pointing out what a REST architecture is and how one might achieve it are called out as dreamers and unworldly when the ones proclaiming the wrong term (RPC over HTTP = REST) continue to do so adding to the confusion of especially the ones just learning the whole matter.
I admit that developing a true REST architecture is really, really hard as it is just too easy to introduce some form of coupling. Hence, a very careful design needs to be done that needs time and also costs money. Money plenty of companies can't or don't want to spend, especially in a domain where new technologies evolve on a regular basis and the ones responsible for developing such solutions often leave the company before the whole process had finished.
Just saying it shouldn’t be ‘typed’ is not really a viable solution
Well, how often did you need to change your browser as it couldn't interact with a Web page? I don't talk about CSS-stuff or browser-specific CSS or JS stuff. How often needed the Web to change in the last 2-3 decades? Similar to the Web, the REST architecture is intended for long-lasting applications for years to come, that supports natural evolution by design. For simple frontend-2-backend systems it is for sure overkill. It starts to shine especially in cases where there are multiple peers not under your control you can interact with.
My understanding of SOAP vs REST:
REST = JSON, simple consistent interface, gives you CRUD access to 'entities' (Abstractions of things which are not necessarily single DB rows), simpler protocol, no formally enforced 'contract' (e.g. the values an endpoint returns could change, though it shouldn't)
SOAP = XML, more complex interface, gives you access to 'services' (specific operations you can apply to entities, rather than allowing you to CRUD entities directly), formally enforced, pre-stated 'contract' (like a WSDL, where e.g. the return types are predefined and formalized)
Is that a broadly correct assessment?
What about a mixture?
If so, what do I call an API that is a mixture?
For example, If we have what at surface level looks like a REST API (returns JSON, no WSDL or formalized contract defined - but instead of giving you access to the 'entities' that the system manages (User, product, comment, etc) it instead gives you specific access to services and complex operations (/sendUserAnUpdate/1111, /makeCommentTextPurple/3333, /getAllCommentsByUserThisYear/2222) without having full coverage?
The 'services' already exist internally, and the team simply publishes access to them on a request by request basis, through what would otherwise look like a REST API.
Question:
What is the 'mixture' typically referred to as (besides, maybe, a bad API). Is there a word for it? or a concept I can refer to that'll make most developers understand what I'm referring to, without having to say the entire paragraph I did above?
Is it just "JSON SOAP API?", "A Service-based REST API?" - what would you call it?
Thanks!
Thanks!
If you take a look at all those so-called REST-APIs your observation might seem true, though REST actually is something completely different. It describes an architecture or a philosophy whose intent it is to decouple clients from servers, allowing the latter one to evolve in future without breaking clients. It is quite similar to the typical Web page interaction in that a server will teach a client on what it needs and only reacts on client-triggered requests. One has to be pretty careful and pendant when designing REST services as it is too easy to include a coupling that may affect clients when a change is introduced, especially with all the pragmatism around in (commercial) software engineering. Stefan Tilkov gave a great talk on REST back in 2014 that, alongside with Jim Webber or Asbjørn Ulsberg, can be used as introduction lectures to what REST is at its core.
The general premise in REST should always be that a server teaches clients what they need and what a server expects and offers choices to the client via links. If the server expects to receive data from the client it will send a form-esque representation to inform the client about the respective fields it supports and based on the affordance of the respective elements contained in the form a client knows whether to select one or multiple options, enter some free text or enter a date value and such. Unfortunately, most of the media-type formats that attempt to mimic HTML's forms are still in draft versions.
If you take a look at HTML forms in particular you might sense what I'm refering to. Each of the elements that may occur inside a form are well defined to avoid abmiguity and improve interoperability. This is defacto the ultimate goal in REST, having one client that is able to interact with a sheer amount of other services without having to be adapted to each single API explicitely.
The beauty of REST is, it isn't limited to a single representation form, i.e. JSON, in fact there is almost an infinite number of possible representation formats that could be exchanged in a REST environment. Plain application/json is a terrible media-type for REST applications IMO as it doesn't include any defintions in regards to links and forms and doesn't describe the semantics of certain fields that may be shipped in requests and responses. The lack of semantical description usually leads to typed resources where a recipient expects that receiving data from i.e. /api/users returns some specific user data, that may differ from host to host. If you skim through IANA's media type registry you will find a couple of media-type formats you could have used to transfer user-related data and any client supporting these representation formats whold be able to interact with this enpoint without any issues. Fielding himself claimed that
A REST API should spend almost all of its descriptive effort in defining the media type(s) used for representing resources and driving application state, or in defining extended relation names and/or hypertext-enabled mark-up for existing standard media types. Any effort spent describing what methods to use on what URIs of interest should be entirely defined within the scope of the processing rules for a media type (and, in most cases, already defined by existing media types). (Source)
Through content-type negotiation client and server will negotiate about a representation format both support and understand. The question therefore shouldn't be which one to support but how many you want to support. The more media-type your API or client is able to exchange payloads for, the more likely it will be to interact with other participants.
Most of those so-called REST APIs are in reality just RPC services exposed via HTTP that may or may not respect and support certain HTTP operations. HTTP thereby is just a transport layer whose domain is the transfer of files or data over the Web. Plenty of people still believe that you shouldn't put verbs in URIs when in reality a script or process usually doesn't (and shouldn't) care whether a URI contains a verb or not. The URI itself is just a pointer a client will follow and invoke when it is interested in receiving the payload. We humans are also not that much interested in the URI itself in regards to the content it may return after invoking that URI. The same holds true for arbitrary clients. It is more important what you ship along with that URI. On the Web a link can be annotated with certain text and/or link relation names that set the links content in relation to the current page. It may hint a client that certain content may be invoked before the whole response was parsed as it is quite likely that the client will also want to know about that. preload i.e. is such a link-relation name that hints the client about that. If certain domain-specific terms exist one might use an extension scheme as defined by Web linking or reuse common knowlege or special microformats.
The whole interaction in a REST environment is similar to playing a text-based computer game or following a certain process flow (i.e. ordering and paying produts) defined by an application domain protocol, that can be designed as a state machine. The client is therefore guided through the whole process. It basically just follows the orders the server gave it, with some choices to break out of the process (i.e. cancel the order before paying).
SOAP on the otherhand is, as you've stated, an XML-based RPC protocol reusing a subset of HTTP to exchange requests and responses. The likelihood that when you change something within your WSDL plenty of clients have to be adapted and recompiled are quite high. SOAP even defines its own security mechanism instead of reusing TLS, which requires explicit support by the clients therefore. As you have a one-to-one communication model due to the state that may be kept in process, scaling SOAP services isn't that easy. In a REST environment this is just a matter of adding a load-balancer before the server and then mirroring the server n-times. The load-balancer can send the request to any of the servers due to the stateless constraint
What is the 'mixture' typically referred to as (besides, maybe, a bad API). Is there a word for it? or a concept I can refer to that'll make most developers understand what I'm referring to, without having to say the entire paragraph I did above?
Is it just "JSON SOAP API?", "A Service-based REST API?" - what would you call it?
The general term for an API that communicates on top of HTTP would be Web API or HTTP API IMO. This article also uses this term. It also lists XML-RPC and JSON-RPC besides SOAP. I do agree with Voice though that you'll receive 5 answers on asking 4 people about the right term to use. While it would be convenient to have a respective term available everyone would agree upon, the reality shows that people are not that interested in a clear separation. Just look here at SO on the questions taged with rest. There is nothing wrong with not being "RESTful", though one should avoid the term REST for truly RPC services. Though I think we are already in a situation where the term REST can't be rescued from misusage and marketing purposes.
For something that requires external documentation to use and that ships with its own custom, non-standardized representation format or that just exposes CRUD for domain objects I'd add -RPC to it, as this is more or less what it is at its heart. So if the API sends JSON and the representation to expect is documented via Swagger or some other external documentationJSON-RPC would probably the most fitting name IMO.
To sum up this post, I hope I could shed some light on what REST truly is and how your observation is flawed by all those pragmatic attempts that unfortunately are RPC through and through. If you change something within their implementation, how many clients will break? In addition to that you can't reuse the client that you've implemented for API A to interact with API B (of a different company or vendor) out of the box and therefore have to either adapt your client or create a new one solely for that API. This is true RPC and therfore should be reflected in the name somehow to hint developers about future expectations. Unfortunately, the process of naming things propperly, especially in regards to REST, seems already lost. There is a fine but tiny group who attempt to spread the true meaning, like Voice, Cassio and some others, though it is like fighting windmills. The best advice here would be to first discuss the naming conventions and what each participant understand on which term and then agree on a naming scheme everyone agrees on to avoid future confusion.
My understanding of SOAP vs REST
...
Is that a broadly correct assessment?
No.
REST is an "architectural style", which is to say a coordinated collection of architectural constraints. The World Wide Web is an example of an application built using the REST architectural style.
SOAP is a transport agnostic message protocol specification, based on XML Information Set
If so, what do I call an API that is a mixture?
I don't think you are going to find an authoritative terminology here. Colloquially, you are likely to hear the broad umbrella term "web api" to describe an HTTP API that isn't "RESTful".
The whole space is rather polluted by semantic diffusion.
I read that HATEOAS links are the one that separates a REST API from a normal http API. In that case, does REST need a separate name? I wonder what all this hype about REST API is about. It seems to be just a http method with one extra rule in the response.
Q) What other differences exist?
I read that HATEOAS links are the one that separates a REST API from a normal http API.
That's probably a little bit of an understatement. When Leonard Richardson (2008) described the "technology stack" of the web, he listed:
URI
HTTP
HTML
A way of exploring the latter is to consider how HTML, as a media type, differs from a text document with URI in it. To my mind, the key element is links and forms -- standardized ways of encoding into the representation the semantics of a URI (this is a link to another page, this is an embedded image, this is an embedded script, this is a form...).
Mike Admundsen, 2010:
Hypermedia Types are MIME media types that contain native hyper-linking semantics that induce application flow. For example, HTML is a hypermedia type; XML is not.
Atom Syndication/Atom Publishing is a good demonstration for defining a REST API.
Can you throw some light on what REST actually means and how it differs from normal http?
Have you noticed that websites don't normally use plain text for the representations of the information that they share? It's something of a dead end -- raw text doesn't have any hypermedia semantics built into it, so a generic client can't do anything more interesting than search for sequences that might be URI.
On the other hand, with HTML we have link semantics: we can include references to images, to style sheets, to scripts, as well as linking to other documents. We can describe forms, that allow the creation of parameterized HTTP requests.
Additionally, that means that if some relation shouldn't be used by the client, the server can easily change the representation to remove the link.
Furthermore, the use of the hypermedia representation allows the server to use a richer description of which request message should be sent by the client.
Consider, for example, Google. They can use the form to control whether search requests use GET or POST. They can remove the "I Feel Lucky" option, or arrange that it redirects to the main experience. They can embed additional information in to the fields of the form, to track what is going on. They can choose which URI targets are used in the search results, directing the client to send to Google another request which gets redirected to the actual target, with additional meta data embedded in the query parameters, all without requiring any special coordination with the client used.
For further discussion, see Leonard Richardson's slide deck from QCon 2008, or Phil Sturgeon's REST and Hypermedia in 2019.
Does n't think the client need to read the documentation if the HATEOAS link is a POST API? HATEOAS links will only guide you to an API but will not throw any light on how its request body needs to be filled....GET won't have request body. So, not much or a problem. but POST API?
Sort of - here's Fielding writing in 2008:
REST doesn’t eliminate the need for a clue. What REST does is concentrate that need for prior knowledge into readily standardizable forms.
On the web, the common use case is agents assisting human beings; the humans can resolve certain ambiguities on their own. The result is a separation of responsibilities; the humans decode the domain specific semantics of the messages, the clients determine the right way to describe an interaction as an HTTP request.
If we want to easily replace the human with a machine, then we'll need to invest extra design capital in a message schema that expresses the domain specific semantics as clearly as we express the plumbing.
To me, REST is an ideology you want to aim for if you have a system that should last for years to come which has the freedom to evolve freely without breaking stuff on parts you can't control. This is very similar to the Web where a server can't control browsers directly though browsers are able to cooperate with any changes done to Web site representations returned by the server.
I read that HATEOAS links are the one that separates a REST API from a normal http API. In that case, does REST need a separate name?
REST does basically what its name implies, it transfers the state of a resource representation. If so, we should come up with a new name for such "REST" APIs that are truly RPC in the back, to avoid confusion.
If you read through the Richardson Maturity Model (RMM) you might fall under the impression that links or hypermedia controls as Fowler named it, which are mandatory at Level 3, are the feature that separates REST from normal HTTP interaction. However, Level 3 is just not enough to reach the ultimate goal of decoupling.
Most so called "REST APIs" do put a lot of design effort into pretty URIs in a way to express meaning of the target resource to client developer. They come up with fancy documentation generated by their tooling support, such as Swagger or similar stuff, which the client developer has to follow stringent or they wont be able to interact with their API. Such APIs are RPC though. You won't be able to point the same client that interacts with API A to point to API B now and still work out of the box as they might use completely different endpoints and return different types of data for almost the same named resource endpoint. A client that is attempting to use a bit more of dynamic behavior might learn the type from parsing the endpoint and expect a URI such as .../api/users to return users, when all of a sudden now the API changed its URI structure to something like .../api/entities. What would happen now? Most of these clients would break, a clear hint that the whole interaction model doesn't follow the one outline by a REST architecture.
REST puts emphasis on link relation names that should give clients a stable way of learning the URIs intent by allowing a URI to actually change over time. A URI basically is attached to a link relation name and basically represents an affordance, something that is clear what it does. I.e. the affordance of a button could be that you can press it and something would happen as a result. Or the affordance of a light switch would be that a light goes on or off depending on the toggled state of the light switch.
Link relation names now express such an affordance and are a text-based way to represent something like a trash bin or pencil symbol next to table entry on a Web page were you might figure out that on clicking one will delete an entry from the table while the other symbol allows to edit that entry. Such link relation names should be either standardized, use widely accepted ontologies or use custom link-relation extensions as outlined by RFC 8288 (Web Linking)
It is important to note however, that a URI is just a URI which should not convey a semantic meaning to a client. This does not mean that a URI can't have a semantic meaning to the server or API, but a client should not attempt to deduce one from the URI itself. This is what the link-relation name is for, which provides the infrequently changing part of that relation. An endpoint might be referenced by multiple, different URIs, some of which might use different query parameters used for filtering. According to Fielding each of these URIs represent different resources:
The definition of resource in REST is based on a simple premise: identifiers should change as infrequently as possible. Because the Web uses embedded identifiers rather than link servers, authors need an identifier that closely matches the semantics they intend by a hypermedia reference, allowing the reference to remain static even though the result of accessing that reference may change over time. REST accomplishes this by defining a resource to be the semantics of what the author intends to identify, rather than the value corresponding to those semantics at the time the reference is created. It is then left to the author to ensure that the identifier chosen for a reference does indeed identify the intended semantics. (Source 6.2.1)
As URIs are used for caching results, they basically represent the keys used for caching the response payload. As such, it gets obvious that on adding additional query parameters to URIs used in GET requests, you end up bypassing caches as the key is not stored in the cache yet and therefore get the result of a different resource, even though it might be identical (also in response representation) as the URI without that additional parameter.
I wonder what all this hype about REST API is about. It seems to be just a http method with one extra rule in the response.
In short, this is what those self- or marketing-termed pseudo "REST APIs" do convey and many people seem to understand.
The hype for "REST" arose from the inconveniences put onto developers on interacting with other interop-solutions such as Corba, RMI or SOAP where often partly-commercial third-party libraries and frameworks had to be used in order to interact with such systems. Most languages supported HTTP both as client and server out of the box removing the requirement for external libraries or frameworks per se. In addition to that, RPC based solution usually require certain stub- or skeleton-classes to be generated first, which was usually done by the build pipeline automatically. Upon updates of the IDL, such as WSDL linking or including XSD schemata, the whole stub-generation needed to be redone and the whole code needed to looked through in order to spot whether a breaking change was added or not. Usually no obvious changelog was available which made changing or updating such stuff a pain in the ...
In those pseudo "REST" APIs plain JSON is now pretty much the de facto standard, avoiding the step of generating stub classes and the hazzle of analyzing the own code to see whether some of the forced changes had a negative impact on the system. Most of those APIs use some sort of URI based versioning allowing a developer to see based on the URI whether something breaking was introduced or not, mimicking some kind of semantic versioning.
The problem with those solution though is, that not the response representation format itself is versioned but the whole API itself leading to common issues when only a change on a part of the API should be introduced as now the whole API's version needs to be bumped. In addition to that, to URIs such as .../api/v1/users/1234 and .../api/v2/users/1234 may represent the same user and thus the same resource though are in fact different by nature as the URI is different.
Q) What other differences exist?
While REST is just an architecture model that can't force you to implement it stringent, you simply will not benefit from its properties if you ignore some of its constraints. As mentioned above, HATEOAS support is therefore not yet enough to really decouple all clients from an API and thus allow to benefit from the REST architecture.
RMM unfortunately does not talk about media types at all. A media type basically specifies how a received payload should be processed and defines the semantics and constraints of each of the elements used within that payload. I.e. if you look at text/html registered in IANA's media type registry, you can see that it points to the published specification, which always references the most recent version of HTML. HTML is designed in a way to stay backwards compatible so no special versioning stuff is required.
HTML provides, IMO, two important things:
semi-structured content
form support
The former one allows to structure data, giving certain segments or elements the possibility to express different semantics defined in the media type. I.e. a browser will handle an image differently than a div element or an article element. A crawler might favor links and content contained in an article element and ignore script and image elements completely. Based on the existence or absence of certain elements even certain processing differences may occur.
Including support for forms is a very important thing in REST actually as this is the feature which allows a server to teach a client on what a server needs as input. Most so called "REST APIs" just force a developer to go through their documentation, which might be outdated, incorrect or incomplete, and send data to a predefined endpoint according to the documentation. In case of outdated or incomplete documentation, how should a client ever be able to send data to the server? Moreover, a server might never be able to change as basically the documentation is now the truth and the API has to align with the documentation.
Unfortunately, form-support is still a bit in its infancy. Besides HTML, which provides <form>...</form>, you have a couple of JSON based form attempts such as hal-forms, halo-json (halform), Ion or hydra. None of these have yet wide library or framework support yet as some of these form representations still have not finalized their specification on how to support forms more effectively.
Other media-types, unfortunately, might not use semi-structured content or provide support for forms that teach a client on the needs of a server, though they are still valuable to REST in general. First, through Web linking link support can be added to media types that do not naturally support those. Second, the data itself does not really need to be text-based at all in order for an application to use it further. I.e. pictures an videos usually are encoded and byte based anyways still a client can present them to users.
The main point about media-types though is, as Fielding already pointed out in one of his cited blog posts, is, that representations shouldn't be confused with types. Fielding stated that:
A REST API should never have “typed” resources that are significant to the client. Specification authors may use resource types for describing server implementation behind the interface, but those types must be irrelevant and invisible to the client. The only types that are significant to a client are the current representation’s media type and standardized relation names.
Jørn Wildt explained in an excellent blog post what a "typed" resource is and why a REST architecture shouldn't use such types. Basically, to sum the blog post up, a client expecting a ../api/users endpoint to return a pre-assumed data payload might break if the server adds additional, unexpected fields, renames existing fields or leave out expected fields. This coupling can be avoided by using simple content-type negotiation where a client informs a server on which capabilities the client supports and the client will chose the representation that best fits the target resource. If the server can't support the client with a representation the client supports the server should respond with a failure (or a default representation) the client might log to inform the user.
This in essence is exactly what the name REST stands for, the transfer of a resource's state representation where the representation may differ depending on the representation format defined by the selected media type. While HATEOAS may be one of the most obvious changes between REST and a non-REST based HTTP solution, this for sure is not the only factor that makes up a payload in REST. I hope I could shed some light on the decoupling intention and that a server should teach clients what the server expects through forms and that the affordance of URIs is captured by link-relation names. All these tiny aspects in sum make up REST, and you will only benefit from REST, unfortunately, if you respect all of its constraints and not only those that are either easy to obtain or what you have the mood for implementing.
In my API I should return a list of products. In the client (browser) the user can add this products to a cart, increasing, decreasing, or remove this a product from the cart.
In my response with the products list should I put the URLs actions like this:
{
"alias": 'aliasValue',
"removeUrl": 'domain/product/alias/',
"increaseUrl": 'domain/product/alias/increase/',
"decreaseUrl": 'domain/product/alias/decrease/',
...
}
Is this a good practice, I've searched for this but the only thing that I found about URL and API is about the URL structures.
What do you think?
I would advise against reinventing the wheel. There are many ways to use HATEOAS in your API.
Consider, for example, the HAL (Hypertext Application Language) approach, where you have a _links property:
{
"_links": {
"self": {
"href": "http://example.com/api/books/1"
}
},
"id": "1",
"name": "Hypermedia as First-Class Citizen"
}
What you are trying to achieve as a name HATEOAS that stands for Hypermedia as the Engine of Application State.
So if you search for it you will find a lot of formats:
JSON-LD
HAL
SIREN
ION
JSON API
Spring
Ripozo
Source: https://nordicapis.com/tools-to-make-hateoas-compliance-easier/
As REST is just a generatlization of the browsable Web we all know, you can basically apply the same concepts as if you'd implement your system for the browser. In short, a server should provide a client with all the necessary information the client will need to make educated decisions which resources to invoke next.
In a browser-based Web application the interaction model could be similar to invoking the start page, seeing a section in the response that is of interest to you, click a link to learn further details. The next page might list some items where you are provided with further controls to modify this list, i.e. add a new item, remove one and so forth. Upon clicking some "add" button a form is presented to you which teaches you what fields a request should contain and also where this request should be sent to, even if it might not be visible to you. In the back the browser will take care of that for you. The jumping point here is, the server is actually feeding the client with any information needed to proceed through his task, whether it is the choice of available links to explore or the presentation of some guidelines on how a request should look like.
In order to determine whether a link is of interest to a client a server will use meaningful link relation names "attached" to URIs a client should use instead of parsing and interpreting URIs. This allows a server to change URIs anytime without affecting a client. Fielding mentioned the following in one of his blogposts:
A REST API should spend almost all of its descriptive effort in defining the media type(s) used for representing resources and driving application state, or in defining extended relation names and/or hypertext-enabled mark-up for existing standard media types. (Source)
In addtion to that REST APIs shouldn't have typed resources meaningful to clients as this basically couples the client tightly to the API itself and will lead to failures if the serve ever changes anything in regards to the response returned. Instead content-type negotiation should be used in order to let client and server negotiate about the actual representation format used.
A media type defines the processing rules of some payload that is received for a certain Content-Type HTTP header. These rules define both syntax as well as semantics of the document. Servers in a REST architecture are i.e. allowed to reject any messages sent for a certain content type if the rules outlined in the media type are violated.
While the focus on media-types doesn't prevent changes done to the representations received completly, it still is useful in helping to decouple clients from servers further and reduce out-of-band information needed in order to interact with the service. Instead of coupling a client directly to the API, both couple to the media-type actually, and there might be multiple media types they actually couple to. If there ever needs to be a change done to a media-type you can easily introduce a new media type or, similar to HTML, specify within the media-type how backwards compatibility is achived.
In regards to prior, out-of-band information requirements, which REST doesn't completly remove, Fielding stated
Of course the client has prior knowledge. Every protocol, every media type definition, every URI scheme, and every link relationship type constitutes prior knowledge that the client must know (or learn) in order to make use of that knowledge. REST doesn’t eliminate the need for a clue. What REST does is concentrate that need for prior knowledge into readily standardizable forms. That is the essential distinction between data-oriented and control-oriented integration. (Source)
As REST isn't a focus on one single aspect but a conglomeration of all of the aspects mentioned above, your actual question should be tackled from multiple angles.
First, either use existing link relations, i.e. administered by IANA or other microformats, define new ones which you should register with IANA at best or use some semantic web related tags like schema.org. I.e. if you have collections then next, prev, first and last are pretty meaningful (and already registered at IANA) for pagination or item for a decicated item within the list. This collection might have been returned previously as collection before or specified by the respective item in order to go back to the previous collection. If somethin like an add or edit should be done a link-relation like edit-form can teach the client that the URI will return some kind of form that will tell the client how a request to the API will need to look like.
Next, as basic JSON isn't that great in terms of providing help to a client, as all it does is define the syntactical structure of a document but lack support of given the elements meaning, some more advanced media types should be supported. As already mentioned by Cassio and Exadra37 there are a couple of JSON based document types that provide support for HATEOAS (~ URIs and link relation names). Instead of only going for i.e. application/hal+json a multitude of document types would be preferable as this just increases interoperablility with a magnitude of different clients that might come with support for other media types. Note futher that there is nothing wrong with returning a HTML representation as well. REST isn't restricting you to only specific JSON or XML content. I guess most of the time, instead of specifying an own media-type, simply using HTML would be sufficient to carry the meaning of the content from API to server.
You are of course free to create your own media type and use it. In order to increase interoperability you should however standardize it and probably also provide plugins/libraries for third party frameworks so that they are able to use your format as well.
The best advice, though, for sure is to reuse existing standards as the likelihood of other clients already being familiar with such implementations or concepts is way higher than for your custom format. Besides that, it probably also spares you from a lot of work and effort.
I've been convinced by a fellow developer (now left) that the proper way to evolve RESTful web services is by creating custom media types for your services.
For example application/vnd.acme.payroll.v1+json
This way, you can tell your client to specify the encoding to use without changing the URI.
Is this technique a good one? Usually services embed the version into the url:
eg /acme/1.0/payroll/
I've had a lot of difficulty enforcing clients to use this scheme, especially as it seems DELETE does not enforce a media type
There are a few main signaling mechanisms you can use in a RESTful service:
The media type
The rel of a resource you are linking to.
Custom headers, like Accept-Version/Api-Version.
Each of these has distinct uses, and I will outline the ways in which we have come to understand them while designing our API.
Media Types
To signal what operations are possible on a given resource, and what the semantics of these operations are, many use custom media types. In my opinion, this is not quite correct, and a rel is more accurate.
A custom media type should tell you about the type of the data, e.g. its format or the way certain information is embodied or embedded. Having a custom media type means consumers of your API are tightly coupled to that specific representation. Whereas, using something more generic like application/json says "this is just JSON data."
Usually JSON alone is not enough for a RESTful service, since it has no built-in linking or resource-embedding functionality. That is where something like HAL (application/hal+json) comes in. It is a specialization of JSON that is still a generic format, and not application-specific. But it gives just enough to overlay the linking and embedding semantics on top of JSON that is necessary for coherently expressing a RESTful API.
Link Relation Types (rels)
This brings us to rels. To me, a custom rel is a perfect way to signal what type of resource is being dealt with or linked to. For example, a custom rel for a user resource might be http://rel.myapi.com/user, which serves two purposes:
Clients of your API must know this key ahead of time, as it is API-specific knowledge. For example, if it was available on your initial resource and you were using HAL to link to the user resource, clients might find the user link via initialResource._links["http://rel.myapi.com/user"].href.
Developers writing API clients can visit that URI in their web browser, and get an explanation of what that resource represents in your API, including what methods are applicable and what they do. This is a very convenient way to communicate that API-specific knowledge I mentioned. For examples of this, see http://rel.nkstdy.co.
If you combine rels with a standard or semi-standard media type like application/hal+json, you get resources which follow a uniform format specified by their media type, with API-specific semantics defined by their rels. This gets you almost all the way there.
Custom Headers
The remaining question is versioning. How do you allow clients to negotiate different versions of the resource, while not invalidating old URIs?
Our solution, inspired by the Restify Node.js framework, is two custom headers: Accept-Version from the client, which much match X-Api-Version from the server (or Api-Version in the upcoming Restify 2.0 release, as per the new RFC 6648). If they don't match, a 400 Bad Request is the result.
I admit that custom media types are a fairly popular solution here. In my opinion they don't fit very well conceptually, in light of the above considerations, but you would not be doing something weird if you chose them as your versioning mechanism. It has some semantic issues when used with methods other than GET though, as you note.
One thing to keep in mind is that in a truly RESTful system, versioning should not be such an issue. It should only matter in one very specific situation: when the representations of your resources change in backward-incompatible ways, but you still want to keep the same rels. So if the http://rel.myapi.com/friend resource suddenly loses its username field and gains an id field, that would qualify. But if it suddenly gains a nickname field, that's not backward-incompatible, so no versioning is needed. And if the concept of "friends" is completely replaced in your API with the concept of, say, "connection", this is not actually backward-incompatible, because API consumers will simply no longer find http://rel.myapi.com/friend links anywhere in the API for them to follow.
Yes, it's a good option. It clarifies the encoding you'll be using for payloads and lets both sides negotiate a different version of the encoding without changing the URI, as you correctly pointed out.
And yes, there's no need for a client to send a DELETE along with an entity-body. I believe it will simply be ignored by a compliant HTTP server, given that no payload data is transferred in that case. The client issues a DELETE for a URI, and the server returns a response code indicating whether it succeeded. Nice and simple! If the server wishes to return some data after a DELETE then it is free to do so, and should specify the media type of the response when it does.