Is there a proof of concept client(i.e. web application) that represents a real-world application implemented using and taking advantage of the RESTful principles?
All I could find are API browsers but the development of a real world application(i.e. a social network or ecommerce website) is quite different.
I've read Roy's work and related papers but I still can't gasp how to make the most of Restful in the client development. I always end-up storing state on the client or specialise the media/type rendering. For example the same resource(i.e. profile resource) is rendered differently based on context(i.e. on the homepage, on the product page or on the dedicated profile page) so farewell media-type -> code on demand rendering.
I really can't see any advantage(in the way I work) of HATEOAS over an API with well defined/auto-generated IDL(i.e. json hyper-schema).
My current conclusion is that only generic clients(i.e. google) can benefit from HATEOS not real-world/specialised applications. The specialised client development doesn't seem to take any benefit if your API is HATEOS-enabled instead of being IDL described.
While it's true that HATEOAS gives you URI flexibility, and human discovery of flows, the real benefit is using it as an encoding of resource state.
If you have a state machine associated with a resource, you will have some states that permit certain state transitions and not others.
The opportunity to effect a possible state transition is offered to REST clients via operations against resource URIs - using HATEAOS hypermedia, you can define the transitions by a known rel link name, and then include or exclude the rel links, depending on which transitions are permitted by the current state.
This means the logic of determining which transitions are valid is kept server side - the client can choose to hide or disable UI options depending on if the associated rel link is present.
Another reason to include or exclude a particular rel link may be related to the access control permissions offered to the current user. Simply exclude them if the current user isn't permitted to carry out the transition.
If you are not dynamically including or excluding rel links based on resource state and/or state of the authorized user, then your analysis of the pros cons is pretty spot on, because you are not using them for the real reason they were included. After all, the S in REST stands for state! :)
HATEOS is a design philosophy / style / flavor and this is largely a matter of taste or a tradeoff between full-blown code gen and a hand-written API.
The key differentiating aspect of HATEOS is the way references are constructed to other resources in the API (namely, by a full URL). This removes a lot of the documentation burden that you might otherwise encounter if the API response only includes an ID (and not the full URL to the resource).
However, when you use HATEOS with JSON instead of XML you lose some of the other context (e.g. should I PUT or GET or POST to this endpoint?) and so you must supplement this with some other kind of metadata if you want to generate a client, or documentation for humans.
In my experience HATEOS APIs are much easier for humans to consume with simple REST clients (e.g. cURL) compared to a WSDL or IDL which assumes the client is using generated code and will never touch the API directly.
Tradeoffs
So why would you choose HATEOS vs WSDL or some other generated option?
The basic assumption for APIs (which is not always true) is that they will have many flavors of clients / consumers, possibly implemented in different languages. This means that over time, writing and updating clients is more work than writing the service.
If you or your business are going to maintain the API clients yourself then there is a cost tradeoff between generating code for all of the clients (WSDL, SWIG, etc.) or hiring a language-specific developer to maintain one.
Chances are a generated API client is not going to follow the idiomatic style for any given language, and the code is generally ugly. If these things matter to you then you will probably want a human to write the client code. If you don't care about this, then you can stop reading about HATEOS and use a WSDL or similar approach instead.
In case you do want to optimize for a human to consume the API, though, HATEOS succeeds because it conveys contextual information to a human, and this makes it easier to write clients without extensive API documentation.
Example
For an example of a HATEOS-like API take a look at the GitHub API. It is quite easy to browse with a REST client and once you learn how to authenticate you can find most of the things you want by following referenced data URLs. You will still need to reference the documentation for specific details and advanced use-cases (like POSTing data) but it is very easy to write a simple client for GitHub without pulling in a GitHub client library or reading the docs end-to-end.
Related
My understanding of SOAP vs REST:
REST = JSON, simple consistent interface, gives you CRUD access to 'entities' (Abstractions of things which are not necessarily single DB rows), simpler protocol, no formally enforced 'contract' (e.g. the values an endpoint returns could change, though it shouldn't)
SOAP = XML, more complex interface, gives you access to 'services' (specific operations you can apply to entities, rather than allowing you to CRUD entities directly), formally enforced, pre-stated 'contract' (like a WSDL, where e.g. the return types are predefined and formalized)
Is that a broadly correct assessment?
What about a mixture?
If so, what do I call an API that is a mixture?
For example, If we have what at surface level looks like a REST API (returns JSON, no WSDL or formalized contract defined - but instead of giving you access to the 'entities' that the system manages (User, product, comment, etc) it instead gives you specific access to services and complex operations (/sendUserAnUpdate/1111, /makeCommentTextPurple/3333, /getAllCommentsByUserThisYear/2222) without having full coverage?
The 'services' already exist internally, and the team simply publishes access to them on a request by request basis, through what would otherwise look like a REST API.
Question:
What is the 'mixture' typically referred to as (besides, maybe, a bad API). Is there a word for it? or a concept I can refer to that'll make most developers understand what I'm referring to, without having to say the entire paragraph I did above?
Is it just "JSON SOAP API?", "A Service-based REST API?" - what would you call it?
Thanks!
Thanks!
If you take a look at all those so-called REST-APIs your observation might seem true, though REST actually is something completely different. It describes an architecture or a philosophy whose intent it is to decouple clients from servers, allowing the latter one to evolve in future without breaking clients. It is quite similar to the typical Web page interaction in that a server will teach a client on what it needs and only reacts on client-triggered requests. One has to be pretty careful and pendant when designing REST services as it is too easy to include a coupling that may affect clients when a change is introduced, especially with all the pragmatism around in (commercial) software engineering. Stefan Tilkov gave a great talk on REST back in 2014 that, alongside with Jim Webber or Asbjørn Ulsberg, can be used as introduction lectures to what REST is at its core.
The general premise in REST should always be that a server teaches clients what they need and what a server expects and offers choices to the client via links. If the server expects to receive data from the client it will send a form-esque representation to inform the client about the respective fields it supports and based on the affordance of the respective elements contained in the form a client knows whether to select one or multiple options, enter some free text or enter a date value and such. Unfortunately, most of the media-type formats that attempt to mimic HTML's forms are still in draft versions.
If you take a look at HTML forms in particular you might sense what I'm refering to. Each of the elements that may occur inside a form are well defined to avoid abmiguity and improve interoperability. This is defacto the ultimate goal in REST, having one client that is able to interact with a sheer amount of other services without having to be adapted to each single API explicitely.
The beauty of REST is, it isn't limited to a single representation form, i.e. JSON, in fact there is almost an infinite number of possible representation formats that could be exchanged in a REST environment. Plain application/json is a terrible media-type for REST applications IMO as it doesn't include any defintions in regards to links and forms and doesn't describe the semantics of certain fields that may be shipped in requests and responses. The lack of semantical description usually leads to typed resources where a recipient expects that receiving data from i.e. /api/users returns some specific user data, that may differ from host to host. If you skim through IANA's media type registry you will find a couple of media-type formats you could have used to transfer user-related data and any client supporting these representation formats whold be able to interact with this enpoint without any issues. Fielding himself claimed that
A REST API should spend almost all of its descriptive effort in defining the media type(s) used for representing resources and driving application state, or in defining extended relation names and/or hypertext-enabled mark-up for existing standard media types. Any effort spent describing what methods to use on what URIs of interest should be entirely defined within the scope of the processing rules for a media type (and, in most cases, already defined by existing media types). (Source)
Through content-type negotiation client and server will negotiate about a representation format both support and understand. The question therefore shouldn't be which one to support but how many you want to support. The more media-type your API or client is able to exchange payloads for, the more likely it will be to interact with other participants.
Most of those so-called REST APIs are in reality just RPC services exposed via HTTP that may or may not respect and support certain HTTP operations. HTTP thereby is just a transport layer whose domain is the transfer of files or data over the Web. Plenty of people still believe that you shouldn't put verbs in URIs when in reality a script or process usually doesn't (and shouldn't) care whether a URI contains a verb or not. The URI itself is just a pointer a client will follow and invoke when it is interested in receiving the payload. We humans are also not that much interested in the URI itself in regards to the content it may return after invoking that URI. The same holds true for arbitrary clients. It is more important what you ship along with that URI. On the Web a link can be annotated with certain text and/or link relation names that set the links content in relation to the current page. It may hint a client that certain content may be invoked before the whole response was parsed as it is quite likely that the client will also want to know about that. preload i.e. is such a link-relation name that hints the client about that. If certain domain-specific terms exist one might use an extension scheme as defined by Web linking or reuse common knowlege or special microformats.
The whole interaction in a REST environment is similar to playing a text-based computer game or following a certain process flow (i.e. ordering and paying produts) defined by an application domain protocol, that can be designed as a state machine. The client is therefore guided through the whole process. It basically just follows the orders the server gave it, with some choices to break out of the process (i.e. cancel the order before paying).
SOAP on the otherhand is, as you've stated, an XML-based RPC protocol reusing a subset of HTTP to exchange requests and responses. The likelihood that when you change something within your WSDL plenty of clients have to be adapted and recompiled are quite high. SOAP even defines its own security mechanism instead of reusing TLS, which requires explicit support by the clients therefore. As you have a one-to-one communication model due to the state that may be kept in process, scaling SOAP services isn't that easy. In a REST environment this is just a matter of adding a load-balancer before the server and then mirroring the server n-times. The load-balancer can send the request to any of the servers due to the stateless constraint
What is the 'mixture' typically referred to as (besides, maybe, a bad API). Is there a word for it? or a concept I can refer to that'll make most developers understand what I'm referring to, without having to say the entire paragraph I did above?
Is it just "JSON SOAP API?", "A Service-based REST API?" - what would you call it?
The general term for an API that communicates on top of HTTP would be Web API or HTTP API IMO. This article also uses this term. It also lists XML-RPC and JSON-RPC besides SOAP. I do agree with Voice though that you'll receive 5 answers on asking 4 people about the right term to use. While it would be convenient to have a respective term available everyone would agree upon, the reality shows that people are not that interested in a clear separation. Just look here at SO on the questions taged with rest. There is nothing wrong with not being "RESTful", though one should avoid the term REST for truly RPC services. Though I think we are already in a situation where the term REST can't be rescued from misusage and marketing purposes.
For something that requires external documentation to use and that ships with its own custom, non-standardized representation format or that just exposes CRUD for domain objects I'd add -RPC to it, as this is more or less what it is at its heart. So if the API sends JSON and the representation to expect is documented via Swagger or some other external documentationJSON-RPC would probably the most fitting name IMO.
To sum up this post, I hope I could shed some light on what REST truly is and how your observation is flawed by all those pragmatic attempts that unfortunately are RPC through and through. If you change something within their implementation, how many clients will break? In addition to that you can't reuse the client that you've implemented for API A to interact with API B (of a different company or vendor) out of the box and therefore have to either adapt your client or create a new one solely for that API. This is true RPC and therfore should be reflected in the name somehow to hint developers about future expectations. Unfortunately, the process of naming things propperly, especially in regards to REST, seems already lost. There is a fine but tiny group who attempt to spread the true meaning, like Voice, Cassio and some others, though it is like fighting windmills. The best advice here would be to first discuss the naming conventions and what each participant understand on which term and then agree on a naming scheme everyone agrees on to avoid future confusion.
My understanding of SOAP vs REST
...
Is that a broadly correct assessment?
No.
REST is an "architectural style", which is to say a coordinated collection of architectural constraints. The World Wide Web is an example of an application built using the REST architectural style.
SOAP is a transport agnostic message protocol specification, based on XML Information Set
If so, what do I call an API that is a mixture?
I don't think you are going to find an authoritative terminology here. Colloquially, you are likely to hear the broad umbrella term "web api" to describe an HTTP API that isn't "RESTful".
The whole space is rather polluted by semantic diffusion.
I read that HATEOAS links are the one that separates a REST API from a normal http API. In that case, does REST need a separate name? I wonder what all this hype about REST API is about. It seems to be just a http method with one extra rule in the response.
Q) What other differences exist?
I read that HATEOAS links are the one that separates a REST API from a normal http API.
That's probably a little bit of an understatement. When Leonard Richardson (2008) described the "technology stack" of the web, he listed:
URI
HTTP
HTML
A way of exploring the latter is to consider how HTML, as a media type, differs from a text document with URI in it. To my mind, the key element is links and forms -- standardized ways of encoding into the representation the semantics of a URI (this is a link to another page, this is an embedded image, this is an embedded script, this is a form...).
Mike Admundsen, 2010:
Hypermedia Types are MIME media types that contain native hyper-linking semantics that induce application flow. For example, HTML is a hypermedia type; XML is not.
Atom Syndication/Atom Publishing is a good demonstration for defining a REST API.
Can you throw some light on what REST actually means and how it differs from normal http?
Have you noticed that websites don't normally use plain text for the representations of the information that they share? It's something of a dead end -- raw text doesn't have any hypermedia semantics built into it, so a generic client can't do anything more interesting than search for sequences that might be URI.
On the other hand, with HTML we have link semantics: we can include references to images, to style sheets, to scripts, as well as linking to other documents. We can describe forms, that allow the creation of parameterized HTTP requests.
Additionally, that means that if some relation shouldn't be used by the client, the server can easily change the representation to remove the link.
Furthermore, the use of the hypermedia representation allows the server to use a richer description of which request message should be sent by the client.
Consider, for example, Google. They can use the form to control whether search requests use GET or POST. They can remove the "I Feel Lucky" option, or arrange that it redirects to the main experience. They can embed additional information in to the fields of the form, to track what is going on. They can choose which URI targets are used in the search results, directing the client to send to Google another request which gets redirected to the actual target, with additional meta data embedded in the query parameters, all without requiring any special coordination with the client used.
For further discussion, see Leonard Richardson's slide deck from QCon 2008, or Phil Sturgeon's REST and Hypermedia in 2019.
Does n't think the client need to read the documentation if the HATEOAS link is a POST API? HATEOAS links will only guide you to an API but will not throw any light on how its request body needs to be filled....GET won't have request body. So, not much or a problem. but POST API?
Sort of - here's Fielding writing in 2008:
REST doesn’t eliminate the need for a clue. What REST does is concentrate that need for prior knowledge into readily standardizable forms.
On the web, the common use case is agents assisting human beings; the humans can resolve certain ambiguities on their own. The result is a separation of responsibilities; the humans decode the domain specific semantics of the messages, the clients determine the right way to describe an interaction as an HTTP request.
If we want to easily replace the human with a machine, then we'll need to invest extra design capital in a message schema that expresses the domain specific semantics as clearly as we express the plumbing.
To me, REST is an ideology you want to aim for if you have a system that should last for years to come which has the freedom to evolve freely without breaking stuff on parts you can't control. This is very similar to the Web where a server can't control browsers directly though browsers are able to cooperate with any changes done to Web site representations returned by the server.
I read that HATEOAS links are the one that separates a REST API from a normal http API. In that case, does REST need a separate name?
REST does basically what its name implies, it transfers the state of a resource representation. If so, we should come up with a new name for such "REST" APIs that are truly RPC in the back, to avoid confusion.
If you read through the Richardson Maturity Model (RMM) you might fall under the impression that links or hypermedia controls as Fowler named it, which are mandatory at Level 3, are the feature that separates REST from normal HTTP interaction. However, Level 3 is just not enough to reach the ultimate goal of decoupling.
Most so called "REST APIs" do put a lot of design effort into pretty URIs in a way to express meaning of the target resource to client developer. They come up with fancy documentation generated by their tooling support, such as Swagger or similar stuff, which the client developer has to follow stringent or they wont be able to interact with their API. Such APIs are RPC though. You won't be able to point the same client that interacts with API A to point to API B now and still work out of the box as they might use completely different endpoints and return different types of data for almost the same named resource endpoint. A client that is attempting to use a bit more of dynamic behavior might learn the type from parsing the endpoint and expect a URI such as .../api/users to return users, when all of a sudden now the API changed its URI structure to something like .../api/entities. What would happen now? Most of these clients would break, a clear hint that the whole interaction model doesn't follow the one outline by a REST architecture.
REST puts emphasis on link relation names that should give clients a stable way of learning the URIs intent by allowing a URI to actually change over time. A URI basically is attached to a link relation name and basically represents an affordance, something that is clear what it does. I.e. the affordance of a button could be that you can press it and something would happen as a result. Or the affordance of a light switch would be that a light goes on or off depending on the toggled state of the light switch.
Link relation names now express such an affordance and are a text-based way to represent something like a trash bin or pencil symbol next to table entry on a Web page were you might figure out that on clicking one will delete an entry from the table while the other symbol allows to edit that entry. Such link relation names should be either standardized, use widely accepted ontologies or use custom link-relation extensions as outlined by RFC 8288 (Web Linking)
It is important to note however, that a URI is just a URI which should not convey a semantic meaning to a client. This does not mean that a URI can't have a semantic meaning to the server or API, but a client should not attempt to deduce one from the URI itself. This is what the link-relation name is for, which provides the infrequently changing part of that relation. An endpoint might be referenced by multiple, different URIs, some of which might use different query parameters used for filtering. According to Fielding each of these URIs represent different resources:
The definition of resource in REST is based on a simple premise: identifiers should change as infrequently as possible. Because the Web uses embedded identifiers rather than link servers, authors need an identifier that closely matches the semantics they intend by a hypermedia reference, allowing the reference to remain static even though the result of accessing that reference may change over time. REST accomplishes this by defining a resource to be the semantics of what the author intends to identify, rather than the value corresponding to those semantics at the time the reference is created. It is then left to the author to ensure that the identifier chosen for a reference does indeed identify the intended semantics. (Source 6.2.1)
As URIs are used for caching results, they basically represent the keys used for caching the response payload. As such, it gets obvious that on adding additional query parameters to URIs used in GET requests, you end up bypassing caches as the key is not stored in the cache yet and therefore get the result of a different resource, even though it might be identical (also in response representation) as the URI without that additional parameter.
I wonder what all this hype about REST API is about. It seems to be just a http method with one extra rule in the response.
In short, this is what those self- or marketing-termed pseudo "REST APIs" do convey and many people seem to understand.
The hype for "REST" arose from the inconveniences put onto developers on interacting with other interop-solutions such as Corba, RMI or SOAP where often partly-commercial third-party libraries and frameworks had to be used in order to interact with such systems. Most languages supported HTTP both as client and server out of the box removing the requirement for external libraries or frameworks per se. In addition to that, RPC based solution usually require certain stub- or skeleton-classes to be generated first, which was usually done by the build pipeline automatically. Upon updates of the IDL, such as WSDL linking or including XSD schemata, the whole stub-generation needed to be redone and the whole code needed to looked through in order to spot whether a breaking change was added or not. Usually no obvious changelog was available which made changing or updating such stuff a pain in the ...
In those pseudo "REST" APIs plain JSON is now pretty much the de facto standard, avoiding the step of generating stub classes and the hazzle of analyzing the own code to see whether some of the forced changes had a negative impact on the system. Most of those APIs use some sort of URI based versioning allowing a developer to see based on the URI whether something breaking was introduced or not, mimicking some kind of semantic versioning.
The problem with those solution though is, that not the response representation format itself is versioned but the whole API itself leading to common issues when only a change on a part of the API should be introduced as now the whole API's version needs to be bumped. In addition to that, to URIs such as .../api/v1/users/1234 and .../api/v2/users/1234 may represent the same user and thus the same resource though are in fact different by nature as the URI is different.
Q) What other differences exist?
While REST is just an architecture model that can't force you to implement it stringent, you simply will not benefit from its properties if you ignore some of its constraints. As mentioned above, HATEOAS support is therefore not yet enough to really decouple all clients from an API and thus allow to benefit from the REST architecture.
RMM unfortunately does not talk about media types at all. A media type basically specifies how a received payload should be processed and defines the semantics and constraints of each of the elements used within that payload. I.e. if you look at text/html registered in IANA's media type registry, you can see that it points to the published specification, which always references the most recent version of HTML. HTML is designed in a way to stay backwards compatible so no special versioning stuff is required.
HTML provides, IMO, two important things:
semi-structured content
form support
The former one allows to structure data, giving certain segments or elements the possibility to express different semantics defined in the media type. I.e. a browser will handle an image differently than a div element or an article element. A crawler might favor links and content contained in an article element and ignore script and image elements completely. Based on the existence or absence of certain elements even certain processing differences may occur.
Including support for forms is a very important thing in REST actually as this is the feature which allows a server to teach a client on what a server needs as input. Most so called "REST APIs" just force a developer to go through their documentation, which might be outdated, incorrect or incomplete, and send data to a predefined endpoint according to the documentation. In case of outdated or incomplete documentation, how should a client ever be able to send data to the server? Moreover, a server might never be able to change as basically the documentation is now the truth and the API has to align with the documentation.
Unfortunately, form-support is still a bit in its infancy. Besides HTML, which provides <form>...</form>, you have a couple of JSON based form attempts such as hal-forms, halo-json (halform), Ion or hydra. None of these have yet wide library or framework support yet as some of these form representations still have not finalized their specification on how to support forms more effectively.
Other media-types, unfortunately, might not use semi-structured content or provide support for forms that teach a client on the needs of a server, though they are still valuable to REST in general. First, through Web linking link support can be added to media types that do not naturally support those. Second, the data itself does not really need to be text-based at all in order for an application to use it further. I.e. pictures an videos usually are encoded and byte based anyways still a client can present them to users.
The main point about media-types though is, as Fielding already pointed out in one of his cited blog posts, is, that representations shouldn't be confused with types. Fielding stated that:
A REST API should never have “typed” resources that are significant to the client. Specification authors may use resource types for describing server implementation behind the interface, but those types must be irrelevant and invisible to the client. The only types that are significant to a client are the current representation’s media type and standardized relation names.
Jørn Wildt explained in an excellent blog post what a "typed" resource is and why a REST architecture shouldn't use such types. Basically, to sum the blog post up, a client expecting a ../api/users endpoint to return a pre-assumed data payload might break if the server adds additional, unexpected fields, renames existing fields or leave out expected fields. This coupling can be avoided by using simple content-type negotiation where a client informs a server on which capabilities the client supports and the client will chose the representation that best fits the target resource. If the server can't support the client with a representation the client supports the server should respond with a failure (or a default representation) the client might log to inform the user.
This in essence is exactly what the name REST stands for, the transfer of a resource's state representation where the representation may differ depending on the representation format defined by the selected media type. While HATEOAS may be one of the most obvious changes between REST and a non-REST based HTTP solution, this for sure is not the only factor that makes up a payload in REST. I hope I could shed some light on the decoupling intention and that a server should teach clients what the server expects through forms and that the affordance of URIs is captured by link-relation names. All these tiny aspects in sum make up REST, and you will only benefit from REST, unfortunately, if you respect all of its constraints and not only those that are either easy to obtain or what you have the mood for implementing.
Can anyone explain difference between Swagger & HATEOAS. I can Search many time but no buddy can explain the proper detailed answer this two aspects.
The main difference between Swagger and HATEOAS IMO, which is not covered in the accepted answer, is, that Swagger is only needed for RPC'esque APIs. Such APIs, however, have actually hardly anything to do with REST.
There is a further, widespread misconception that anything exchanged via HTTP is automatically RESTful (~ in accordance with the REST archtitectural style), which it is not. REST just defines a set of constraints that are not choices or options but are mandatory. From start to finish. There is nothing wrong from being not RESTful, but it is wrong to term such an architecture REST.
Swagger describe the operations that can be performed on an endpoint and the payload (including headers and the expected representation formats) that needs to be sent to the service and also describe what a client might expect as response. This allows Swagger to be used both as documentation as well as testing-framework for the API. Due to the tight coupling of Swagger to the API it behaves much like a typical RPC service description, i.e. similar to WSDL files in SOAP or stub or skeletton classes in RMI or CORBA. If either the endpoint changes or something in the payload changes, clients implementing against a Swagger documentation will probably break over time just reintroducing the same problems typical RPC implementations have.
REST and HATEOAS, on the other side, are designed for disovery and further development. REST isn't a protocol but an architectural style to start with that describes the interaction flow between a client and server in a distributed system. It basically took the concepts which made the Web so successful and translated it onto the application layer. So the same concepts that apply to the browsable Web also apply to REST. Therefore it is no miracle that also HATEOAS (the usage of and support for links, link relations and link names) behave similar to the Web.
On designing a REST architecture it is benefitial to think of a state machine where a server provides all of the information a client needs to take further actions. Asbjørn Ulsberg held a great talk back in 2016 where he explains affordances and how a state machine might be implemented through HATEOAS. Besides common or standardized media-types and relation names no out-of-band knowledge is necessary to interact with the service further. In the case of the toaster example Asbjørn gave in his talk, a toaster may have the states off, on, heating and idle where turning a toaster on will lead to a state transition from off to on followed by a transition to heating till a certain temperature is reached where the state is transitioned to idle and switches between idle and heating till the toaster is turned off.
HATOAS will provide a client with the information on the current state and include links a client can invoke to transition to the next state, i.e. turning the toaster off again. It's important to stress here, that a client is provided by the server with every action the client might perform next. There is no need for a client implementor to consult any proprietary API documentation in order for a client to be able to interact with a REST service. Further, URIs do not have to be meaningful or designed to convey a semantical-expressive structure as clients will determine whether invoking that URI makes sense via the link-relation name. Such relation names are either specified by IANA, by a common approach such as Dublin Core or schema.org or by absolut URIs acting as extension attributes which might point to a human-readable description, which further might be propagated to the user via mouse-over tooltips or such.
I hope you can see by yourself that Swagger is only needed to describe RPC Web-APIs rather than applications that follow the REST architectural design. Messages exchanged via REST APIs should include all the information needed by a client to make informed choices on the next state transition. As such it is benefitial to design such message flows and interactions as state machine.
Update:
How are Swagger and HATEOAS mutually exclusive? The former documents your endpoints (making auto-generating code possible) and the latter adds meta-information to your endpoints which tell the consumer what they can do (i.e. which other endpoints are available). These are very different things.
I never stated that they are mutually exclusive, just that they serve two different purposes, where if you follow one approach the other gets more or less useless. Using both does not make any sense though.
Let's move the discussion to the Web domain as this is probably more easily understandable and REST is de facto just a generalization of the concepts used on the Web, so doing this step is just natural and also a good recommendation in terms of designing REST architectures in general. Think of a case where you as a user want to send some data to the server. You have never used the service before so you basically don't know how a request has to look like.
In Swagger you would call the endpoint documentation, select the option that most likely might solve your task, read up on how the request needs to look like and hack a test-case into your application that ends up generating a HTTP request that is sent to the respective location. Auto-generating code might spare you some hacking time, though you still need to integrate the stub classes into your application and test the whole thing at least once just to be safe. If you later on need to integrate a second service of that API or of yet an other API in general, you need to start from the beginning and look up the Swagger documentation, generate or hack the interaction code and integrate it into your domain. Plenty of manual steps involved and in cases of API changes you need to update your client as otherwise it might stop working.
In the Web example however, you just start your browser/Web client, invoke the respective URI that allows you to send the data to the server and the server will most likely send you a HTML form you just need to fill out and click the send button which automatically sends the request to the server which will start to process it. This is HATEOAS. You used the given controls to drive your workflow. The server taught your client every little detail it needed to make a valid request. It served your client with the target URI to send the request to, the HTTP method it should use and most often also implicitly the media type the payload should be in. In addition to that it also gave your clients a skeleton of the expected and/or supported elements the payload should contain. I.e. the form may require you to fill out a couple of input fields, select among a given set of choices or use some other controls such as a date or time picker value that is translated to a valid date or time representation for you. All you needed to do was to invoke the respective resource in your Web client. No auto-generation, no integration into your browser/application. Using other services (from the same or different providers) will, most likely, just work the same way so no need to change or update your HTTP client (browser) as long as the media-type request and responses are exchanged are supported.
In the case where you rely on Swagger RPC'esque documentation, that documentation is the truth on how to interact with the service. Mixing in some HATEOAS information doesn't provide you any benefits. In the Swagger case, carrying around additional meta-information that bloat up the request/response for no obvious reasons, as all the required information is given in the reference documentation, will, with some certainty, lead to people starting questioning the sanity of the developers of that service and ask for payload reduction. Just look here at SO for a while and you will find enough question asking on how to optimize the interaction further and further and reducing message size to a minimum as they process every little request and don't make use of response caching at all. In the HATEOAS case, pointing to an external reference is just useless as peers in such an architecture most likely already have support for the required necessities, such as URI, HTTP and the respective media types, implemented into it. In cases where custom media-types are used, support can be added at runtime via plug-ins or add-ons dynamically (if supported).
So, Swagger and HATEOAS are not mutually exclusive but the other gets more or less useless once you decided for one route or the other.
Swagger: Swagger aids in development across the entire API lifecycle, from design and documentation, to test and deployment. (Refer to swagger.io)
HATEOAS: Hypermedia as the Engine of Application State
An Ion Form is a Collection Object where the value member array contains Form Fields. Ion Forms ensure that resource transitions (links) that support data submissions can be discovered automatically (colloquially referred to as HATEOAS). (Refer to https://ionspec.org/)
One is a framework for supporting designing and testing for APIs, the other is an API design architecture.
Building a RESTful API is not a binary concept. That is why we use the Richardson maturity model in order to measure how RESTful an API is.
Based on this maturity model
At level 0 we provide mechanisms for client of the API to call some methods on the server (Simple RPC)
At level 1 we expose resources on the server so the client of the API can have direct access to the resources that it requires (exposing Resources)
At level 2 we provide a uniform way for the client of the api to interact with the API (exposed resources) and the HTTP protocol has these methods (using HTTP verbs to interact with resources).
the ultimate step is to make our api explorable by the client. HATEOAS provides such functionality (over HTTP) meaning that it adds relevant links and affordances (extra methods) that can executed on the resource so the client of the API can understand its behavior.
Based on these definitions in properly designed RESTful API there is no coupling between client and server and client can interact with the exposed endpoints an discover them.
On the other hand, swagger is a tool that helps you document your API along with some extra goodies (code generators).
I believe that Swagger (with the help of swagger Hub) provides services for implementing a RESTful endpoint with maturity levels up to 2. But it does not go any further and it does not provide proper support of HATEOAS.
You can define your resources and HTTP verbs in (json/yml) files. And based on this definition Swagger can generate API documentation and the extra goodies (client stubs and skeletal implementation of the server API).
For all those people who have worked with Java RMI, SOAP,... the extra goodies part is a reminder of old technologies where there was tight coupling between Client and Server because the stubs and skeletal implementations are all built based on the same API definition file.
a basic REST question.. I design a REST API and would like to be able to get a list of book recommendations based on a book id (i.e. client sends book id=w to server and server replies with a list of recommended books, id=x,y,z).
I see two ways to do this:
/recommendation?bookId=thetitle
/recommendation/thetitle
Option 2 seems a bit cleaner to me but I'm not sure if it would be considered good REST design? Because /recommendation/thetitle looks like an element URI, not a collection URI (although in this case it would return a collection). Also, the first part of the resource (/recommendation) would not make any sense by itself.
Thankful for any advice.
URL patterns of this kind have nothing to do with REST. None of the defining properties of REST requires readable URLs.
At the same time, one of the core principles (HATEOAS), if followed properly, allows API clients (applications, not people!) to browse the API and obtain every link required to perform a desired transition of application state or resource state based on a well known message format.
If you feel your API must have readable URLs, it's a good sign that its design probably isn't RESTful at all. This implies the need for a developer to understand the URL structure and hardcode it somewhere in a client application. Something that REST is supposed to avoid by principle.
To quote Roy Fielding's blog post on the subject:
A REST API must not define fixed resource names or hierarchies (an obvious coupling of client and server). Servers must have the freedom to control their own namespace. Instead, allow servers to instruct clients on how to construct appropriate URIs, such as is done in HTML forms and URI templates, by defining those instructions within media types and link relations. [Failure here implies that clients are assuming a resource structure due to out-of band information, such as a domain-specific standard, which is the data-oriented equivalent to RPC’s functional coupling].
Obviously, nothing stops you from actually making URLs meaningful regardless of how RESTful your API actually is. Even if it's for a purpose not dictated by REST itself (viewing the logs left by a client of a properly RESTful API could be easier for a human if they're readable, off the top of my head).
Finally, if you're fine with developing a Web API that's not completely RESTful and you expect developers of clients to read this kind of docs and care about path building, you might actually benefit from comprehensible URLs. This can be very useful in APIs of the so-called levels 0-3, according to Richardson's maturity model.
What's important in terms of REST is how you're leveraging the underlying protocol (HTTP in this case) and what it allows you to do. If we consider your examples from this perspective, /recommendation/thetitle seems preferable. This is because the use of query parameters may prevent responses from being cached by browsers (important if you're writing a JS client) or proxies, making it harder to reuse existing tools and infrastructure.
For a SaaS startup I'm involved in, I am building both a RESTful web API and a couple of client apps on different platforms that consume it. I think I've got the API figured out, but now I'm turning to the clients. As I've been reading about REST, I see that a key part of REST is discovery, but there seems to be a lot of debate between two different interpretations of what discovery really means:
Developer discovery: The developer hard-codes copious amounts of API details into the client, such as resource URI's, query parameters, supported HTTP methods, and other details that they've discovered through browsing the docs and experimenting with the API's responses. This type of discovery IMHO necessitates cool linkage and the API versioning question, and leads to hard coupling of the client code to the API. Not much better than if using a well-documented collection of RPC's it seems.
Runtime discovery - The client app itself is able to figure out everything it needs with little or no out-of-band information (presumably, only a knowledge of the media types the API deals with.) Links can be hot. But to make the API very efficient, a lot of link templating for query parameters seems to be needed, which makes out-of-band info creep back in. There are possibly other difficulties I haven't thought of yet since I haven't gotten to that point in development. But I do like the idea of loose coupling.
Runtime discovery seems to be the holy grail of REST, but I'm seeing precious little discussion about how to implement such a client. Almost all REST sources I've found seem to assume Developer discovery. Anyone know of some Runtime discovery resources? Best practices? Examples or libraries with real code? I'm working in PHP (Zend Framework) for one client. Objective-C (iOS) for the other.
Is Runtime discovery a realistic goal, given the present set of tools and knowledge in the developer community? I can write my client to treat all of the URI's in an opaque manner, but how to do this most efficiently is a question, especially over low-bandwidth connections. Anyway, URI's are only part of the equation. What about link templating in the Runtime context? How about communicating what methods are supported, aside from making a lot of OPTIONS requests?
This is definitely a tough nut to crack. At Google, we've implemented our Discovery Service that all our new APIs are built against. The TL;DR version is we generate a JSON Schema-like spec that our clients can parse - many of them dynamically.
That results means easier SDK upgrades for the developer and easy/better maintenance for us.
By no means the perfect solution, but many of our devs seem to like.
See link for more details (and make sure to watch the vid.)
Fascinating. What you are describing is basically the HATEOAS principle. What is HATEOAS you ask? Read this: http://en.wikipedia.org/wiki/HATEOAS
In layman's terms, HATEOAS means link following. This approach decouples your client from specific URL's and gives you the flexibility to change your API without breaking anyone.
You did your home work and you got to the heart of it: runtime discovery is holy grail. Don't chase it.
UDDI tells a poignant story of runtime discovery: http://en.wikipedia.org/wiki/Universal_Description_Discovery_and_Integration
One of the requirements that should be satisfied before you can call an API 'RESTful' is that it should be possible to write a generic client application on top of that API. With the generic client, a user should be able to access all the API's functionality. A generic client is a client application that does not assume that any resource has a specific structure beyond the structure that is defined by the media type. For example, a web browser is a generic client that knows how to interpret HTML, including HTML forms etc.
Now, suppose we have a HTTP/JSON API for a web shop and we want to build a HTML/CSS/JavaScript client that gives our customers an excellent user experience. Would it be a realistic option to let that client be a generic client application? No. We want to provide a specific look-and-feel for every specific data element and every specific application state. We don't want to include all knowledge about these presentation-specifics in the API, on the contrary, the client should define the look and feel and the API should only carry the data. This implies that the client has hard-coded coupling of specific resource elements to specific layouts and user interactions.
Is this the end of HATEOAS and thus the end of REST? Yes and no.
Yes, because if we hard-code knowledge about the API into the client, we loose the benefit of HATEOAS: server-side changes may break the client.
No, for two reasons:
Being "RESTful" is a property of the API, not of the client. As long as it is possible, in theory, to build a generic client that offers all capabilities of the API, the API can be called RESTful. The fact that clients don't obey the rules, is not the API's fault. The fact that a generic client would have a lousy user experience is not an issue. Why is it important to know that it is possible to have a generic client, if we don't actually have that generic client? This brings me to the second reason:
A RESTful API offers clients the option to choose how generic they want to be, i.e. how resilient to server-side changes they want to be. Clients which need to provide a great user experience may still be resilient to URI changes, to changes in default values and more. Clients doing batch jobs without user interaction may be resilient to other kinds of changes.
If you are interested in practical examples, checkout my JAREST paper. The last section is about HATEOAS. You will see that with JAREST, even highly interactive and visually attractive clients can be quite resilient to server-side changes, though not 100%.
I think the important point about HATEOAS is not that it is some holy grail client-side, but that it isolates the client from URI changes - it is assumed you are using known (or developer discovered custom) Link Relations that will allow the system to know which link for an object is the editable form. The important point is to use a media type that is hypermedia aware (e.g. HTML, XHTML, etc).
You write:
To make the API very efficient, a lot of link templating for query parameters seems to be needed, which makes out-of-band info creep back in.
If that link template is supplied in the previous request, then there is no out-of-band information. For example a HTML search form uses link templating (/search?q=%#) to generate a URL (/search?q=hateoas), but nothing is known by the client (the web browser) other than how to use HTML forms and GET.