I'm trying to figure out what the latest best practice is when it comes to REST APIs and finding an elegant way to "tell" the client what the response will look like. I'm no web expert. But I just recently joined a new team and I've noticed that in the client code, they have hardcoded URI to APIs... and anytime the schema of the return data changes, they have to patch their client code.
Trying to find a way to make things more dynamic by:
introducing patterns to "discover" API servers.
looking into HATEOAS.
More than anything else though, what I'm trying improve is having to change the client code each time the logic on the server changes as far as the body of a GET response.
I've been reading this:
https://www.programmableweb.com/news/rest-api-design-put-type-content-type/2011/11/18
And in particular, the following comments stood out to me: (under the WRML heading)
this media type communicates, directly to clients, distinct and
complementary bits of information regarding the content of a message.
The Web Resource Modeling Language (WRML, www.wrml.org) provides this
"pluggable" media type to give rich web applications direct access to
structural information and format serialization code. The media type's
self-descriptive and pluggable design reduces the need for information
to be communicated out-of-band and then hard-coded by client
developers
Questions
is WRML still a thing? this book that i'm reading is from 2011... and I'm assuming a lot has changed since then.
Can I cheaply build my own inhouse solution where we use the Content-Type or some other header to provide the clients with schema information?
can you point me to an example / sample code where someone is using custom values in Content-Type or other Headers to accomplish something similar?
Or if you have any other suggestions I'm all ears.
Thank you.
I don't know much about WRML, but I would look into:
HATEOAS formats like HAL/HAL Forms and Siren, which are self-describing.
JSON-Schema to describe responses and requests (and yes they can be linked from HATEOAS responses).
If you don't want to go the hypermedia route, OpenAPI and RAML
I've been developing Ketting for the last 5 years and HATEOAS has been nothing short of magic lately as all the tools have been falling into place.
I'm new to web services world. I have created a rest api in ERP software that creates sales order. How will anyone outside the world know what parameters to send to this and in what format ? From all the videos I have watched and all materials read they talk about no api documentation is needed as it is REST service, People just know what request payload to send. I m not sure if I understand how that is possible. Its like I give somebody the url and tell them go figure. I tried sending orders with different parameter list and it is creating errors. But, if I send it in the way it accepts then it is working fine. Not sure if I understand the concepts well. Should I be creating documentation of this api telling what the request payload should look like ?
I completely disagree with the answer given by Joessel, which just propagates a typical RPC take on it, which is NOT REST at all!
In regards to how a service utilizing the REST architecture style should inform clients about what properties a resource supports and stuff like that, just look at traditional HTML pages. How are Web server able to tell your Browser what input it expects?
HTML is a media type that specifies the syntax to use as well as the semantics of each of the elements and attributes wich are admissible in a HTML document. I.e. HTML Forms enables a server to inform a client on the respective properties a resource support. In addition to that, a server also teaches a client on the respective target URI to send the request to, the HTTP operation to use upon sending the request as well as the media type to use for marshalling the request to a respective representation format. This is why you DON'T NEED any other documentation to interact with Web pages. Most arbitrary Web clients support HTML documents by default and therefore you don't need to reimplement the wheel to process such documents.
For non-HTML resources it is also just a matter of whether your client supports the respective media type or not. I.e. PNG files also follow a certain standard which allows arbitrary clients to show images instead of the actual bytes on your screen.
Most of those so called "REST APIs", which are truely RPC ones, just use custom JSON based message structures. JSON itself just defines the basic syntax but no semantics for any elements, attributes or other properties. It doesn't even add support for links. JSON Hyper-Schema is an extension which at least triest o add support for it, though it already requires to use an other media type than application/json. Though, if such formats are not well-defined and standardized, widespread adoption will not be possible on the long run. Hence creating a common media type is of importance to increase interoperability for such media types. I.e. for JSON based formats, HAL+JSON, HAL Forms, ION, and others provide definitions for basic JSON based message structures, with support for links and other features like form-support and other things.
So, if you take a closer look at the Web, you will find many concepts that you can reuse for a truely RESTful design. After all, REST just takes the ideas used on the Web for decades and attempts to offer the benefits resulting from these concepts to applications rather than humans alone. As such, it is always a good idea to first design the interaction flow as if one would interact with a traditional Web page and then take the concepts used in that design and apply it onto your application domain model.
As you don't need external documentation to interact with Web pages, so you don't need external documentation to interact with well defined message formats that follow a common media type as well. Through content-type negotiation both server and client will communicate with representation formats both support. So, the more (different) media types you support, the more likely you will be in the end to interact with different parties in that environment and if all of the supported media types are standardized you might not need any external documentation at all.
If you follow the academic definition of REST defined by Dr Fielding in his dissertation, it is true that such a service doesn't need any documentation. Hovewer, the chance are high that your service is not RESTful by his definition and that you need a documention to help your consumers. You can find an informative discussion about why a REST api doesn't need a documentation here.
That being said, you will probably need a documentation in most case. Depending on your needs and consumers of your api, a simple text file could be enough; maybe a markdown that you can easily share with other developers. Just remember to add all information needed to understand each route (verb, path, query, body, response...), and you can even throw some example code to make it easy to start.
And if you need something more robust, I can only advise you to follow the open api specification. One of the easiest way to get started is to use the swagger editor . You will even be able to publish your documentation in a nice way using one of the many tools out there (ex: redoc).
Writing a documention is never fun but nobody will use a product that they cannot understand, however great the product is.
Good luck,
Edit:
I rewrote the introduction as it was misleading, thanks to #Roman Vottner for pointing it out. THe previous intro was:
I don't know who told you that a documentation is not needed for a REST Api but I find this highly misleading for newcomers... A documentation is (super) important, especially if you are not the sole consumer of your API, and in that case I would say it's mandatory.
One of the goals of the REST API architecture is decoupling of the client and the server.
One of the questions I have run across in planning a REST API is: "how does the client know what is a valid payload for POST methods?"
Somehow the API needs to communicate to the UI what a valid payload for a given resource’s POST method. Otherwise here we are back at depending on out-of-band knowledge being necessary to work with an API and we are tightly coupled again.
So I’ve had this idea that the API response for a GET on a resource would provide a specification for constructing a valid payload for the POST method on that resource. This would include field names, data type, max length, etc.
This guy has a similar idea.
What's the correct way to handle this? Are most people just relying on out-of-band information? What are people doing in the real world with this problem?
EDIT
Something I have come up with to solve this problem is illustrated in the following sequence diagram:
The client and the api service are separate. The client knows:
Entry point
How to navigate the API via the hypermedia.
Here's what happens:
Someone (user) requests the registration page from the client
The client requests the entry point from the API and receives all hypermedia links with appropriate meta data on how to traverse them legally.
Client constructs the registration form based on the meta data associated with the registration hypermedia POST method.
User fills in the form and submits.
Client POSTs to the API with the correct data and all is well.
No magic /meta resouces, no need to use a method for the meta data. Everything is provided by the API.
Thoughts?
Most people are relying on out-of-band information. This is usually ok, though, because most clients aren't being built dynamically, but statically. They rely on known parts of the API rather than being HATEOAS-driven.
If you are developing or want to support a metadata-driven client, then yes, you're going to need to come up with a schema for providing that information. The implementation you linked to seems reasonable after a quick skim. Note that you've only moved the problem, though. Clients still need to know how to interpret the information in the metadata responses.
Your are right, the client should understand the semantics of the links in the response, and choose the right one from them to achieve its goal. The client is coupled to the semantics the API provides about this and not to the API itself. So for example a client should not retrieve information from the URI structure, since it is tightly coupled to the actual API.
I know of 2 current solution types about this:
by HAL+JSON you use IANA link relations to describe what the link does, and vendor specific MIME types to describe the schema of the fields
by JSON-LD (or any other RDF format) with Hydra vocab you send back RDF metadata according to the operation the link calls. This meta-data can contain the validation details of the fields (xsd vocab) and the semantics of the fields (microdata, microformats, etc...). This information is completely decoupled from the API implementation, so it might be a better option than using vendor specific MIME types, but Hydra is still under development and HAL is much simpler.
However your solution is valid as well, I think you should check both of these, since they are already standard solutions, and the uniform interface / self-descripting message constraint of REST encourages the usage of existing standards instead of custom solutions. But it is up to you if you want to create an own standard.
I think you are asking about, Rest API meta data handling. Unlike SOAP, Rest APIs doesn't use meta data normally, but sometimes it can be pretty useful, once your api size gets bigger.
I think you should look into swagger. It is the most elegant you can find out for rest apis. I have being using it for sometime and with the annotation support it is being rather easy to work with. It also has many examples found on github. Other advantage is, it contains nice configurable ui.
Apart from that you can find other ways of doing it like WADL and WSDL 2.0. Even-though I haven't being using them, you can read more about them here.
With RFC 6861, you can link to your form with create-form and edit-form Link Relations, instead of the client constructing the form by itself. The corresponding form should have the necessary schema to construct the POST request.
During a discussion for designing our REST api's one of the fellow developer told that partial responses are not RESTful.
e.g. While returning response for /forums/{forum_id}/users I am just returning user name and avatar pic.
When somebody wants detailed info about user I send {age, location, points_scored etc. etc.}.
While trying to model this using ember-data we landed up in the problem mentioned at https://github.com/emberjs/data/issues/51
Thats when the developer pointed out that partial responses are against RESTful design. But I could not find any such information in any of the REST books or on Internet. Moreover I found that google actually uses partial responses in their RESTful API's and there are many guys using it.
So wanted to know are partial responses (like the above) really not recommended for RESTful services or its just a design choice issue.
Contrary to a common misconception, REST does not oblige you to read and write the same exact resources.
CouchDB is a really good example for the way to go. It handles:
"objects" (a.k.a. "documents") that can be created, read, updated, and deleted,
and "views", which are computed from documents, that can only be read.
In your case, /forums/01a0/users would be a "view", while /user/99a7 would be an "object".
Apparently, REST is just a set of conventions about how to use HTTP. I wonder which advantage these conventions provide. Does anyone know?
I don't think you will get a good answer to this, partly because nobody really agrees on what REST is. The wikipedia page is heavy on buzzwords and light on explanation. The discussion page is worth a skim just to see how much people disagree on this. As far as I can tell however, REST means this:
Instead of having randomly named setter and getter URLs and using GET for all the getters and POST for all the setters, we try to have the URLs identify resources, and then use the HTTP actions GET, POST, PUT and DELETE to do stuff to them. So instead of
GET /get_article?id=1
POST /delete_article id=1
You would do
GET /articles/1/
DELETE /articles/1/
And then POST and PUT correspond to "create" and "update" operations (but nobody agrees which way round).
I think the caching arguments are wrong, because query strings are generally cached, and besides you don't really need to use them. For example django makes something like this very easy, and I wouldn't say it was REST:
GET /get_article/1/
POST /delete_article/ id=1
Or even just include the verb in the URL:
GET /read/article/1/
POST /delete/article/1/
POST /update/article/1/
POST /create/article/
In that case GET means something without side-effects, and POST means something that changes data on the server. I think this is perhaps a bit clearer and easier, especially as you can avoid the whole PUT-vs-POST thing. Plus you can add more verbs if you want to, so you aren't artificially bound to what HTTP offers. For example:
POST /hide/article/1/
POST /show/article/1/
(Or whatever, it's hard to think of examples until they happen!)
So in conclusion, there are only two advantages I can see:
Your web API may be cleaner and easier to understand / discover.
When synchronising data with a website, it is probably easier to use REST because you can just say synchronize("/articles/1/") or whatever. This depends heavily on your code.
However I think there are some pretty big disadvantages:
Not all actions easily map to CRUD (create, read/retrieve, update, delete). You may not even be dealing with object type resources.
It's extra effort for dubious benefits.
Confusion as to which way round PUT and POST are. In English they mean similar things ("I'm going to put/post a notice on the wall.").
So in conclusion I would say: unless you really want to go to the extra effort, or if your service maps really well to CRUD operations, save REST for the second version of your API.
I just came across another problem with REST: It's not easy to do more than one thing in one request or specify which parts of a compound object you want to get. This is especially important on mobile where round-trip-time can be significant and connections are unreliable. For example, suppose you are getting posts on a facebook timeline. The "pure" REST way would be something like
GET /timeline_posts // Returns a list of post IDs.
GET /timeline_posts/1/ // Returns a list of message IDs in the post.
GET /timeline_posts/2/
GET /timeline_posts/3/
GET /message/10/
GET /message/11/
....
Which is kind of ridiculous. Facebook's API is pretty great IMO, so let's see what they do:
By default, most object properties are returned when you make a query.
You can choose the fields (or connections) you want returned with the
"fields" query parameter. For example, this URL will only return the
id, name, and picture of Ben:
https://graph.facebook.com/bgolub?fields=id,name,picture
I have no idea how you'd do something like that with REST, and if you did whether it would still count as REST. I would certainly ignore anyone who tries to tell you that you shouldn't do that though (especially if the reason is "because it isn't REST")!
Simply put, REST means using HTTP the way it's meant to be.
Have a look at Roy Fielding's dissertation about REST. I think that every person that is doing web development should read it.
As a note, Roy Fielding is one of the key drivers behind the HTTP protocol, as well.
To name some of the advandages:
Simple.
You can make good use of HTTP cache and proxy server to help you handle high load.
It helps you organize even a very complex application into simple resources.
It makes it easy for new clients to use your application, even if you haven't designed it specifically for them (probably, because they weren't around when you created your app).
Simply put: NONE.
Feel free to downvote, but I still think there are no real benefits over non-REST HTTP. All current answers are invalid. Arguments from the currently most voted answer:
Simple.
You can make good use of HTTP cache and proxy server to help you handle high load.
It helps you organize even a very complex application into simple resources.
It makes it easy for new clients to use your application, even if you haven't designed it specifically for them (probably, because they weren't around when you created your app).
1. Simple
With REST you need additional communication layer for your server-side and client-side scripts => it's actually more complicated than use of non-REST HTTP.
2. Caching
Caching can be controlled by HTTP headers sent by server. REST does not add any features missing in non-REST.
3. Organization
REST does not help you organize things. It forces you to use API supported by server-side library you are using. You can organize your application the same way (or better) when you are using non-REST approach. E.g. see Model-View-Controller or MVC routing.
4. Easy to use/implement
Not true at all. It all depends on how well you organize and document your application. REST will not magically make your application better.
IMHO the biggest advantage that REST enables is that of reducing client/server coupling. It is much easier to evolve a REST interface over time without breaking existing clients.
Discoverability
Each resource has references to other resources, either in hierarchy or links, so it's easy to browse around. This is an advantage to the human developing the client, saving he/she from constantly consulting the docs, and offering suggestions. It also means the server can change resource names unilaterally (as long as the client software doesn't hardcode the URLs).
Compatibility with other tools
You can CURL your way into any part of the API or use the web browser to navigate resources. Makes debugging and testing integration much easier.
Standardized Verb Names
Allows you to specify actions without having to hunt the correct wording. Imagine if OOP getters and setters weren't standardized, and some people used retrieve and define instead. You would have to memorize the correct verb for each individual access point. Knowing there's only a handful of verbs available counters that problem.
Standardized Status
If you GET a resource that doesn't exist, you can be sure to get a 404 error in a RESTful API. Contrast it with a non-RESTful API, which may return {error: "Not found"} wrapped in God knows how many layers. If you need the extra space to write a message to the developer on the other side, you can always use the body of the response.
Example
Imagine two APIs with the same functionality, one following REST and the other not. Now imagine the following clients for those APIs:
RESTful:
GET /products/1052/reviews
POST /products/1052/reviews "5 stars"
DELETE /products/1052/reviews/10
GET /products/1052/reviews/10
HTTP:
GET /reviews?product_id=1052
POST /post_review?product_id=1052 "5 stars"
POST /remove_review?product_id=1052&review_id=10
GET /reviews?product_id=1052&review=10
Now think of the following questions:
If the first call of each client worked, how sure can you be the rest will work too?
There was a major update to the API that may or may not have changed those access points. How much of the docs will you have to re-read?
Can you predict the return of the last query?
You have to edit the review posted (before deleting it). Can you do so without checking the docs?
I recommend taking a look at Ryan Tomayko's How I Explained REST to My Wife
Third party edit
Excerpt from the waybackmaschine link:
How about an example. You’re a teacher and want to manage students:
what classes they’re in,
what grades they’re getting,
emergency contacts,
information about the books you teach out of, etc.
If the systems are web-based, then there’s probably a URL for each of the nouns involved here: student, teacher, class, book, room, etc. ... If there were a machine readable representation for each URL, then it would be trivial to latch new tools onto the system because all of that information would be consumable in a standard way. ... you could build a country-wide system that was able to talk to each of the individual school systems to collect testing scores.
Each of the systems would get information from each other using a simple HTTP GET. If one system needs to add something to another system, it would use an HTTP POST. If a system wants to update something in another system, it uses an HTTP PUT. The only thing left to figure out is what the data should look like.
I would suggest everybody, who is looking for an answer to this question, go through this "slideshow".
I couldn't understand what REST is and why it is so cool, its pros and cons, differences from SOAP - but this slideshow was so brilliant and easy to understand, so it is much more clear to me now, than before.
Caching.
There are other more in depth benefits of REST which revolve around evolve-ability via loose coupling and hypertext, but caching mechanisms are the main reason you should care about RESTful HTTP.
It's written down in the Fielding dissertation. But if you don't want to read a lot:
increased scalability (due to stateless, cache and layered system constraints)
decoupled client and server (due to stateless and uniform interface constraints)
reusable clients (client can use general REST browsers and RDF semantics to decide which link to follow and how to display the results)
non breaking clients (clients break only by application specific semantics changes, because they use the semantics instead of some API specific knowledge)
Give every “resource” an ID
Link things together
Use standard methods
Resources with multiple representations
Communicate statelessly
It is possible to do everything just with POST and GET? Yes, is it the best approach? No, why? because we have standards methods. If you think again, it would be possible to do everything using just GET.. so why should we even bother do use POST? Because of the standards!
For example, today thinking about a MVC model, you can limit your application to respond just to specific kinds of verbs like POST, GET, PUT and DELETE. Even if under the hood everything is emulated to POST and GET, don't make sense to have different verbs for different actions?
Discovery is far easier in REST. We have WADL documents (similar to WSDL in traditional webservices) that will help you to advertise your service to the world. You can use UDDI discoveries as well. With traditional HTTP POST and GET people may not know your message request and response schemas to call you.
One advantage is that, we can non-sequentially process XML documents and unmarshal XML data from different sources like InputStream object, a URL, a DOM node...
#Timmmm, about your edit :
GET /timeline_posts // could return the N first posts, with links to fetch the next/previous N posts
This would dramatically reduce the number of calls
And nothing prevents you from designing a server that accepts HTTP parameters to denote the field values your clients may want...
But this is a detail.
Much more important is the fact that you did not mention huge advantages of the REST architectural style (much better scalability, due to server statelessness; much better availability, due to server statelessness also; much better use of the standard services, such as caching for instance, when using a REST architectural style; much lower coupling between client and server, due to the use of a uniform interface; etc. etc.)
As for your remark
"Not all actions easily map to CRUD (create, read/retrieve, update,
delete)."
: an RDBMS uses a CRUD approach, too (SELECT/INSERT/DELETE/UPDATE), and there is always a way to represent and act upon a data model.
Regarding your sentence
"You may not even be dealing with object type resources"
: a RESTful design is, by essence, a simple design - but this does NOT mean that designing it is simple. Do you see the difference ? You'll have to think a lot about the concepts your application will represent and handle, what must be done by it, if you prefer, in order to represent this by means of resources. But if you do so, you will end up with a more simple and efficient design.
Query-strings can be ignored by search engines.