Since REST is an architectural style, not a protocol, it can be applied to most any protocol form - such as websockets. That's exactly what I'd like to do, but I'd like help deciding on an approach.
As I do my research, I'm finding that there are three paradigms I could follow:
Simulated Request-Response. This is what is implemented in the SwaggerSocket library. Each client request has an ID. Each server-push response has the same ID, to allow correlation of request-response.
Notification-Only. The server pushes a resource address, implying that the client should perform a GET on that resource to discover what has changed.
Event Driven. The server-push is designed to look like an HTTP POST, perhaps similar to a webhook request.
I'd like to hear from those who have experience walking these paths, about which they found to be the most effective, and which tools they applied (such as SwaggerSocket mentioned above).
A major concern I have is simplified demuxing and de-serialization. For example, suppose I have a client written in Typescript. I might like to deserialize server-push payload into a declared, typed object. I think the consequences for each paradigm are as follows:
Simulated Request-Response (SRR). The SwaggerSocket message must be de-serialized twice. First to discover the response ID and/or "path", and second to retrieve the actual payload into a "typed" object. A little clumsy, but doable.
Notification-Only. The server-push message can be deserialized into a single pre-defined type, since it contains little more than a REST resource path.
Event Driven. If the "event" has no payload, then this is basically the same thing as the Notification-Only approach. But if there is a payload, then once again a two-step deserialization would likely be necessary.
Other thoughts I have: The SRR might be the most limiting of the three, because every server-push theoretically is instigated by a client request. The other two paradigms don't have that implicit model. The Event Driven approach has the conceptual advantage of being similar-ish to a webhook callback.
To illustrate the Event Driven / webhook idea, I'll give a SignalR example.
//client side.
hubProxy.On<HttpRequest>("EventNotice", request => {
//Take apart the HttpResponse and dispatch through
//my own routing mechanism...to handlers that
//further deserialize the inner payload.
});
//The corresponding server-push would of course be:
_context.Clients.All.EventNotice(myHttpRequest);
The above example is very achievable. In fact, I would not be surprised somebody has example code (please share!) or even a supporting library for this purpose.
Again, of these different paradigms, which would you advise? What supporting tools would you suggest?
Related
I've spent more time than I'd like to admit trying to understand the differences between RPC and restful endpoints. In fact, after going down this rabbit hole, I'm not even sure I fully understand what RPC is anymore. And I'm not alone. I see countless messages repeatedly asking for clarification.
Here's my understanding so far:
RPC and restful endpoints are largely conceptual in nature only. There are no standards.
Both are ways of invoking a function on a remote server.
In the case of restful endpoints, we map HTTP methods to CRUD operations on a resource.
In the case of RPC, we're concerned w/actions on an object.
However, in both cases, we're addressing remote method invocation.
I've read some folks say restful shouldn't be stateful and if it is, it's not restful. But aren't most restful endpoints using state via sessions? I mean, I'm not going to let just anyone delete data w/o being logged in.
And I've also read that functions like domain(.)com/login isn't restful b/c it's an action. This seems to make sense, but since I'm not writing my functions any differently on my server than I would a simple restful function, does it really matter what we call them -- restful vs. RPC.
I truly just don't understand what I'm missing and no posts thus far on StackOverflow appear to address this confusion.
Hoping someone can provide some insight using the above as a guide for understanding. Thanks in advance.
It's an amazing question. There is a huge difference between REST and RPC approaches, I'll try to explain it.
RPC just lets you call some remote computer to make some job. It can be any job, and it can have any inputs and outputs. When you design API, you just define how to name this job operation and which inputs and outputs it will have.
Step 1. Just RPC
For example, you could have operation foo with one input and two outputs like that (I will use JSight API notation because it supports both RPC and REST API types):
JSIGHT 0.3
URL /rpc-api
Protocol json-rpc-2.0
Method foo
Params
{
"input-1": 123 // some integer
}
Result
{
"output-1": 123, // some other integer
"output-2": "some string"
}
When you design your RPC API you can use as many such operations as you want.
So, what's the problem with this? Why do we need REST, if we already have RPC?
The problems are the following:
When you look at the method foo definition you do not have the slightest idea, what it does.
When you design RPC APIs you have absolute freedom, but freedom often can be a bad thing. Because people think in different fashions. You could think, that your API is easy and elegant, but other people would not think so.
Step 2. Standardized RPC
If you are an experienced API designer and you use the RPC style, you will try to follow some pattern to avoid the two faults mentioned above. For example, let's imagine, that your foo method returns a customer entity. Then you will never call it like foo, you will call it getCustomer, like this:
JSIGHT 0.3
URL /rpc-api
Protocol json-rpc-2.0
Method getCustomer
Params
{
"input-1": 123 // some integer
}
Result
{
"output-1": 123, // some other integer
"output-2": "some string"
}
Ok! Now when you just look at this description, you can say, that this method returns a customer; that parameter input-1 is some filter, most possibly, ID; and output-1 and output-2 are some customer properties.
Look, just using the standard way of naming the method, we made our API much more easy-to-understand.
Still, why do we need REST? What's the problem here?
The problem is, that when you have dozens of such RPC methods, your API looks messy. E. g.:
JSIGHT 0.3
URL /rpc-api
Protocol json-rpc-2.0
Method getCustomer
Method updateCustomer
Method getCustomerFriends
Method getCustomerFather
Method askCustomerConfirmation
# etc...
Why it is not easy to understand this bunch of methods? Because it is not the way our brain works. Our brain always groups actions by entity. You naturally think about an entity, and then about actions, you can do with it. E. g. what you can do with a cup? Take, hide, break, wash, turn over. And brain rarely thinks firstly about actions and then about entities. What can I take? Cup, ball, ice cream, ticket, wife. Not very natural.
So, what is REST about?
Step 3. REST
REST is about grouping entities and defining the most standard actions with these entities.
In IT we usually create, read, write, or delete entities. That's why we have POST, GET, PUT, and DELETE methods in HTTP, which is de facto the only implementation of the REST style.
When we group our methods by entities, our API becomes much clearer. Look:
JSIGHT 0.3
GET /customer/{id}
PUT /customer/{id}
GET /customer/{id}/friends
GET /customer/{id}/father
POST /customer/{id}/ask-confirmation
It is much easier to perceive now than in the RPC-style example above.
Conclusion
There are a lot of other issues, concerning differences between RPC and REST. REST has its faults as well as RPC. But I wanted to explain to you the main idea:
REST is a standard, that helps you to organize your API in a user-friendly fashion, that is comfortable for the human brain, so any developer can quickly capture your API structure.
RPC is just a way of calling remote operations.
All the examples above are gathered here: https://editor.jsight.io/r/g6XJwjV/1
I truly just don't understand what I'm missing
Not your fault, the literature sucks.
I think you may find Jim Webber's 2011 guidance enlightening.
Embrace HTTP as an application protocol. HTTP is not a transport protocol which happens to go conveniently through port 80. HTTP is an application protocol whose application domain is the transfer of documents over a network.
HTTP, like remote procedure calls, is a request/response protocol. The client sends a message to the server, the server sends a message back. Big deal.
But in the REST architectural style, we're also following the uniform interface constraint. That means (among other things) that everybody understands requests the same way, and everybody interprets responses the same way.
And that in turn means that general purpose components (browsers, web crawlers, proxies) can understand the request and response messages, and react intelligently to those messages, without needing to know anything at all about the specialization of a particular resource.
In the case of HTTP, we're passing messages about the domain of documents; if I want information from you, I ask for the latest copy of your document. If I want to send information to you, I send you a proposed modification to your document. Maybe you respond to my modification by creating a new document and you tell me to go look at that. And so on.
Useful business activity, then, becomes a side effect of manipulating these documents. Or, expressed another way, the document metaphor is the facade we present to the world.
And I've also read that functions like domain(.)com/login isn't restful b/c it's an action.
See above: the literature sucks.
Consider this resource identifier: https://www.merriam-webster.com/dictionary/login
Does it work like every other resource identifier on the web? Yes. The browser knows how to make a request to GET this resource, and when Merriam-Webster answers that the browser should instead look at https://www.merriam-webster.com/dictionary/log%20on then it fetches that resource as well.
The fact that "login"/"log on" are intransitive English verbs does not matter; it's an identifier, and that's all the browser needs to know.
Alternatively, consider this: URL shorteners work.
Resource identifiers are a lot like variable names in a computer program; so long as the identifier satisfies the production rules of your language, you can use anything you want. The machines don't care.
The key idea is that resources are not actions, they are "information that can be named". https://www.merriam-webster.com/dictionary/log%20on is the name of the document that describes Merriam-Webster's definition of the verb "log on".
Since the machines don't care what names we use, we can use the extra freedom that offers us to choose names that make life easier for people we care about.
I've read some folks say restful shouldn't be stateful and if it is, it's not restful. But aren't most restful endpoints using state via sessions?
Stateless communication is a REST architectural constraint.
each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server.
Cookies are one of the corners where HTTP failed to satisfy the stateless communication constraint.
I'm not going to let just anyone delete data w/o being logged in.
Not a problem: every HTTP request carries its own Authorization metadata; you aren't supposed to "remember" previous messages in the conversation. Instead, you verify the provided credentials every time.
I have a service, which is exposed over HTTP. Most of traffic input gets into it via single HTTP GET endpoint, in which the payload is serialized and encrypted (RSA). The client system have common code, which ensures that the serialization and deserialization will succeed. One of the encoded parameters is the operation type, in my service there is a huge switch (almost 100 cases) that checks which operation is performed and executes the proper code.
case OPERATION_1: {
operation = new Operation1Class(basicRequestData, serviceInjected);
break;
}
case OPERATION_2: {
operation = new Operation2Class(basicRequestData, anotherServiceInjected);
break;
}
The endpoints have a few types, some of them are typical resource endpoints (GET_something, UPDATE_something), some of them are method based (VALIDATE_something, CHECK_something).
I am thinking about refactoring the API of the service so that it is more RESTful, especially in the resource-based part of the system. To do so I would probably split the endpoint into the proper endpoints (e.g. /resource/{id}/subresource) or RPC-like endpoints (/validateSomething). I feel it would be better, however I cannot make up any argument for this.
The question is: what are the advantages of the refactored solution, and what follows: what are the disadvantages of the current solution?
The current solution separates client from server, it's scalable (adding new endpoint requires adding new operation type in the common code) and quite clear, two clients use it in two different programming languages. I know that the API is marked as 0-maturity in the Richardson's model, however I cannot make up a reason why I should change it into level 3 (or at least level 2 - resources and methods).
Most of traffic input gets into it via single HTTP GET endpoint, in which the payload is serialized and encrypted (RSA)
This is potentially a problem here, because the HTTP specification is quite clear that GET requests with a payload are out of bounds.
A payload within a GET request message has no defined semantics; sending a payload body on a GET request might cause some existing implementations to reject the request.
It's probably worth taking some time to review this, because it seems that your existing implementation works, so what's the problem?
The problem here is interop - can processes controlled by other people communicate successfully with the processes that you control? The HTTP standard gives us shared semantics for our "self descriptive messages"; when you violate that standard, you lose interop with things that you don't directly control.
And that in turn means that you can't freely leverage the wide array of solutions that we already have in support of HTTP, because you've introduce this inconsistency in your case.
The appropriate HTTP method to use for what you are currently doing? POST
REST (aka Richardson Level 3) is the architectural style of the world wide web.
Your "everything is a message to a single resource" approach gives up many of the advantages that made the world wide web catastrophically successful.
The most obvious of these is caching. "Web scale" is possible in part because the standardized caching support greatly reduces the number of round trips we need to make. However, the grain of caching in HTTP is the resource -- everything keys off of the target-uri of a request. Thus, by having all information shared via a single target-uri, you lose fine grain caching control.
You also lose safe request semantics - with every message buried in a single method type, general purpose components can't distinguish between "effectively read only" messages and messages that request that the origin server modify its own resources. This in turn means that you lose pre-fetching, and automatic retry of safe requests when the network is unstable.
In all, you've taken a rather intelligent application protocol and crippled it, leaving yourself with a transport protocol.
That's not necessarily the wrong choice for your circumstances - SOAP is a thing, after all, and again, your service does seem to be working as is? which implies that you don't currently need the capabilities that you've given up.
It would make me a little bit suspicious, in the sense that if you don't need these things, why are you using HTTP rather than some messaging protocol?
Until now, I used to think the only difference between GET and POST is the format on data trasmission.
In other examples I have seen that in REST Api, the only command to send information is POST, while for getting information is GET...
But what if I want to use GET also for information sending? I have to send a number of dataset names to be processed (usually a short number...)... I don't understand why GET doesn't fits this operation. Or it's only a "semantic" reason?
EDIT: I don't want answer about the general differences between GET or POST... I need to know if GET should not be used to update info of the server IN ALL CASES, in particular regarding my specific case, or there are some exceptions.
I don't understand why GET doesn't fits this operation. Or it's only a "semantic" reason?
Yes, but semantics are the entire point.
One of the important constraints of the REST architectural style is that of a uniform interface; the idea that all components understand messages the same way. What that gives us is the ability to achieve useful work using general purpose components.
In HTTP (an application built within the constraints of this style), that means that we can use browsers, and caches, and reverse proxies, and server toolkits, and so forth, mixing and matching as we go. It all works, because all of these components understand that a request should be interpreted as described in RFC 7230, and that the request method token is "the primary source of request semantics", and so on.
The semantics of GET have been defined, and we all share in that definition. The wording of the definition has evolved somewhat from its earliest specifications, but the basic semantics have been constant: that GET is effectively read-only, and that the message body (of the request) is of no semantic interest.
If you want to use an HTTP method where the message body is semantically significant, then you need to either relax some of the constraints of GET (for instance, by using POST), choose a different standardized method that better fits your use case (see the IANA HTTP Method Registry), or by defining (and potentially standardizing) your own HTTP method.
The core problem with trying to define a payload for GET - while your bespoke client may know what to do, and your bespoke resource may know what to do, general-purpose intermediates are likely to do the wrong thing (like caching responses without capturing the information in the request body, or discarding the body as unnecessary).
Note that information that is encoded into the target-uri works just fine; HTML forms using the GET method and other variations of URI templates can be used to construct a target-uri from local information that the server can interpret (of course, the defacto limits on target-uri length are much shorter than the defacto limits on payload size; it's not a universal solution).
In my search to understand the HTTP request methods, particularly PUT since I have apparently been using it all wrong, I have stumbled upon a lot of quotes stating “PUT doesn’t belong in REST APIs” and “One should only use GET and POST in modern APIs”.
This just makes me wonder, why would you not use PUT or PATCH or DELETE etc. in REST APIs? Isn’t it what they are there for? To be used because they help with semantics and structure etc.?
Could it have something to do with e.g. the methods receiving the request mainly being methods that directs the data to other methods like the database methods where they are then handled? I’ve used PUT when wanting to update a document, but never has it overwritten a document, even though I only sent part of the data to it.
There is an example of this below, using Express and MongoDB (showing a put method in Express).
app.put('/users/:id', (req, res, next) => {
let id = req.params.id;
userService.getOneUser({_id: ObjectID(id)}).then((user) => {
let updatedData = req.body;
userService.updateUser(user._id, updatedData)
.then((doc) => res.json(doc))
.catch((err) => errorHandler(err, res, next));
}).catch((err) => errorHandler(err, res, next));
})
Basically my question is: In relating to the above statements, how do you use these methods correctly in REST APIs and when do you use them?
EDIT: Example on two references:
PUT doesn't belong in REST API
Only use GET and POST - see third comment on question
I have stumbled upon a lot of quotes stating “PUT doesn’t belong in REST APIs” and “One should only use GET and POST in modern APIs”.
I wouldn't put much stock in those quotes - they imply a lack of understanding about REST, HTTP, and how everything is supposed to fit together.
I'd suggest starting with Jim Webber, who does have a good understanding.
HTTP is an application protocol, whose application domain is the transfer of documents over a network.
PUT, PATCH, DELETE are all perfectly fine methods for describing changes to a document. I GET a document from you, I use my favorite HTTP aware editor/library to make changes to the document, I send you a request describing my changes to the document, you get to figure out what to do on your end.
This just makes me wonder, why would you not use PUT or PATCH or DELETE etc. in REST APIs? Isn’t it what they are there for? To be used because they help with semantics and structure etc.?
One reason that you might not: your media type of choice is HTML -- HTML has native support for links (GET) and forms (GET/POST), but not a lot of direct support in the way of involving the other methods in the flow. You can use code-on-demand, for those clients that support it.
Could it have something to do with e.g. the methods receiving the request mainly being methods that directs the data to other methods like the database methods where they are then handled? I’ve used PUT when wanting to update a document, but never has it overwritten a document, even though I only sent part of the data to it.
An important thing to understand about HTTP methods is that they describe semantics, not implementations. Here's Fielding writing in 2002:
HTTP does not attempt to require the results of a GET to be safe. What it does is require that the semantics of the operation be safe, and therefore it is a fault of the implementation, not the interface or the user of that interface, if anything happens as a result that causes loss of property
In the specific case of PUT, there's an extra hint at the implication of the semantics
A successful PUT of a given representation would suggest that a subsequent GET on that same target resource will result in an equivalent representation being sent in a 200 (OK) response. However, there is no guarantee that such a state change will be observable....
I think Triynko raises a good point:
URIs identified in most modern apps ARE NOT resources to be replaced, updated, etc. They're not documents. They're PROCEDURES being called.
If you are creating a procedure focused API, as opposed to a resource focused API, then there is a decent probability that PUT/PATCH/DELETE aren't actually providing benefits that justify the extra complexity.
One hint that you are procedure focused: how much attention are you paying to cache invalidation? Part of the point of accepting a "uniform interface" constraint is that you want the capabilities that generic components can provide, and in HTTP, caching is a big deal.
Consider a web API method that has no side effects, but which takes binary data as a parameter. An example would be a method that tells the user whether or not their image is photoshopped, but does not permanently store the image or the result on its servers.
Should such a method be a GET or a POST?
GET doesn't seem to have a recommended way of sending data outside of URL parameters, but the behavior of the method implies a GET, which according to the HTTP spec is for safe, stateless responses. This becomes particularly constraining under the semantics of REST, which imply that POST methods create a new object on the server.
This becomes particularly constraining under the semantics of REST, which imply that POST methods create a new object on the server.
While a POST request means that the entity sent will be treated "as a new subordinate of the resource identified by the Request-URI", there is no requirement that this result in the creation of a new permanent object or that any such new object be identified by a URI (so no new object as far as the client knows). An object can be transient, representing the results of e.g. "Providing a block of data, such as the result of submitting a form, to a data-handling process" and not persisting after the entity representing that object has been sent.
While this means that a POST can create a new resource, and is certainly the best way to do so when it is the server that will give that new resource its URI (with PUT being the more appropriate method when the client dictates the new URI) it is can also be used for cases that delete objects (though again if it's a deletion of a single* resource identifiable by a URI then DELETE is far more appropriate), both create and delete objects, change multiple objects, it can mean that your kitchen light turns on but that the response is the same whether that worked or failed because the communication from the webserver to the kitchen light doesn't allow for feedback about success. It really can do anything at all.
But, your instincts are good in wanting this to be a GET: While the looseness of POST means we can make a case for it for just about every request (as done by approaches that use HTTP for an RPC-like protocol, essentially treating HTTP as if it was a transport protocol), this is inelegant in theory, inefficient in practice and clumsy in definition. You have an idempotent function that depends on solely on what the client is interested in, and that maps most obviously the GET in a few ways.
If we could fit everything in a URI then GET would be easy. E.g we can define a simple integer addition with something like http://example.net/addInts?x=1;y=2 representing the addition of 71 and 2 and hence being a permanent immutable resource representing the number 3 (since the results of GET can vary with changes to a resource over time, but this resource never changes) and then use a mechanism like HTML's <form> or javascript to allow the server to inform the client as to how to construct the URIs for other numbers (to maintain the HATEOS and/or COD constraints). Simples!
Your problem here is that you do not have input that can be represented as concisely as the numbers 1 and 2 can above. In theory you could do something like http://example.net/photoshoppedCheck?image=data:image/png;base64,iVBORw0KGgoAAAANSU… and hence create a URI that represents the resource of the results of the check. This URI will though will have 4 characters for every 3 bytes in the image. While there's no absolute limit on URI length both the theory and the practice allow this to fail (in theory HTTP allows for proxies and servers to set a limit on URI length, and in practice they do).
An argument could be made for using GET and sending a request body the same way as you would with a POST, and some webservers will even allow you to do this. However, GET is defined as returning an entity describing the resource identified in the URI with headers restricting how that entity does that describing: Since the request body is not part of that definition it must be ignored by your code! If you were tempted to bend this rule then you must consider that:
Some webservers will refuse the request or strip the body, so you may not be able to.
If your webserver does allow it, the fact that its not specified means you can't be sure an upgrade won't "fix" this and so break your code.
Some proxies will refuse or strip the request.
Some client libraries will most certainly refuse to allow developers to send a request body along with a GET.
So it's a no-no in both theory and practice.
About the only other approach we could do apart from POST is to have a URI that we consider as representing an image that was not photoshopped. Hence if you GET that you get an entity describing the image (obviously it could be the actual image, though it could also be something else if we stretch the concept of content-negotiation) and then PUT will check the image and if its deemed to not be photoshopped it responds with the same image and a 200 or just a 204 while if it is deemed to be photoshopped it responds with a 400 because we've tried to PUT a photoshopped image as a resource that can only be a non-photoshopped image. Because we respond immediately, there's no race-condition with simultaneous requests.
Frankly, this would be darn-right horrible. While I think I've made a case for it by the letter of the specs, it's just nasty: REST is meant to help us design clear APIs, not obtuse APIs we can offer a too-clever-for-its-own-good justification of.
No, in all the way to go here is to POST the image to a fixed URI which then returns a simple entity describing the analysis.
It's perfectly justifiable as REST (the POST creates a transient object based on that image, and then responds with an entity describing that object, and then that object disappears again). It's straight-forward. It's about as efficient as it could be (we can't do any HTTP caching† but most of the network delay is going to be on the upload rather than the download anyway). It also fits into the general use-case of "process something" that POST was first invented for. (Remember that first there was HTTP, then REST described why it worked so well, and then HTTP was refined to better play to those strengths).
In all, while the classic mistake that moves a web application away from REST is to abuse POST into doing absolutely everything when GET, PUT and DELETE (and perhaps the WebDAV methods) would be superior, don't be afraid to use its power when those don't meet the bill, and don't think that the "new subordinate of the resource" has to mean a full long-lived resource.
*Note that a "single" resource here might be composed of several resources that may have their own URIs, so it can be easy to have a single DELETE that deletes multiple objects, but if deleting X deletes A, B & C then it better be obvious that you can't have A, B or C if you don't have X or your API is not going to be understandable. Generally this comes down to what is being modelled, and how obvious it is that one thing depends on another.
†Strictly speaking we can, as we're allowed to send cache headers indicating that sending an identical entity to the same URI will have the same results, but there's no general-purpose web-software that will do this and your custom client can just "remember" the opinion about a given image itself anyway.
It's a difficult one. Like with many other scenarios there is no absolutely correct way of doing it. You have to try to interpret RESTful principles in terms of the limitations of the semantics of HTTP. (Incidentally, I don't think it's right to think of REST having semantics, REST is an architectural style which is commonly used with HTTP services, but can be used for any type of interface.)
I've faced a similar situation in my current project. We chose to use a POST but with the response code being a 200 (OK) rather than the 201 (Resource Created) usually returned by RESTful Web APIs.