I am trying to follow RESTful principles and a little confused on how "Eager" or "Lazy" endpoints would be set up.
For example, a
Shop has many Products
Products have many Ingredients.
Products have many Packging
Of course a "bad" endpoint that would fetch eagerly would be:
api/shop/1
Would return shop id 1's details but also with:
ALL the Products
ALL the Product's Ingredients
ALL the Product's Packging
This of course is a crazy model...so I can only guess RESTful is "always lazy" by default?
But with "lazy be default" say you want to get 10 different products AND their ingredients...
api/shop/1/product/1/ingredients
api/shop/1/product/2/ingredients
api/shop/1/product/3/ingredients
The number of requests is getting a little high...10 seperate HTTP requests for the 10 products.
So lastly, do you instead tend to design the RESTful endpoints based on what the front-end/consumer may want as opposed to modelling the business/database?
api/shop/1/product-details?productId=1,2,3,4,5,6,7,8,9,10
Is the above strictly "RESTful"?
So I guess the real underlying question is sort of:
Is RESTful API design a model of the Data or a model of the Views?
Is RESTful API design a model of the Data or a model of the Views?
Views is closer -- it's a model of resources
Your data model is not your object model is not your resource model is not your affordance model. -- Amundsen
The simplest analogy is to look at java script in an HTML page
we can embed the java script in the HTML page
we can link to the java script from the HTML page.
Both approaches work - they have different trade offs, primarily in how caching works.
Coarse grained resources are somewhat analogous to data transfer objects; exchange a large representation in a single request/response, and then the client can do lots of different things with that one representation.
Fine grained resources give you more control of caching strategies (the different parts can expire at different times), and perhaps respond better to scenarios where we expect the client to be sending back edited representations of those resources.
One issue that fine grained resources have had is the extra burden of round trips. HTTP/2 improves that story, as server push can be used to chain representations of multiple resources onto a single response -- all of the fine grained resources can be sent in a single burst.
But even so, we're talking about identifying resources, not database entities.
https://stackoverflow.com/questions/57420131/restful-syntax-is-it-eager-lazy-or-both
That's an identifier for a web page about a question
https://api.stackexchange.com/2.2/questions/57420131?site=stackoverflow
That's a different resource describing the same question.
REST API's aren't about exposing your data model via HTTP, they are about exchanging documents so that a client can navigate a protocol that gets useful work done. See Webber 2011.
Related
I'm currently redesigning an application where we have a single API (.NET Core 3.1) and two frontend (React) applications. In our current solution we share the same endpoints for both clients, meaning that there is an overhead of data sent from the API because both clients need different information.
I'm looking for a convenient way to split the API so we end up having two different responses, depending on which client is sending the request. I see a few options but I'm quite unfamiliar with pros and cons (besides the obvious ones) of these approaches since I never planned an application for multiple clients.
Single endpoint, different data returned depending on a header/query param
Split the API into two APIs (but where should common endpoints land? creating the third API is an overhead for us)
Split controllers into two when necessary (perhaps using partial classes?) and just implement GET api/resourceX/client1 separately from GET api/resourceX/client2
But maybe there already is a build-in solution for this in .NET Core that I'm not aware of? I tried looking for solutions but ended up getting nowhere near the answer - perhaps my keywords failed me.
This question is somewhat opinion based and you did not give us much details about how different the needs of the two clients are, but I try to answer it.
I have the assumption that you do a CRUD API and you just serve data from database without much transformation or processing in your code. This is ok, but keep in mind that REST should be built around operations and not around data types and CRUD if we use the WSDL terms. This is somewhat related to anemic domain model (defined by Martin Fowler).
If the users of the two clients are the same, then I would add more endpoints to cover the different types of queries. You can use URI templates for this e.g. /stuff?filter=xyz&view=verbose. If you want to support arbitrary queries, then it is better to use a standard solution for URI queries e.g. odata instead of building your own query language. You can use headers too, but as far as I know standard HTTP headers are a lot less flexible than URI queries.
If you have different user groups for different clients based on the privileges e.g. staff and customers, then you can serve different data for the same endpoint based on the privileges. Be aware that if you handle both regular users and admins with a single client, then the code of the client can contain hints about what is accessible for an admin. If this is an issue, then you need to split up the code of the client into multiple files and serve the admin files only for people with admin privileges. Setting the proper cache header is important in this case too. Another possible solution to have different clients for different user groups.
Be aware that in the case of real REST your server gives the URIs or URI templates to the clients just like a webserver gives the links and forms to the browser. This is because the URIs can change in time and it is a lot easier to keep backward compatibility this way. The clients know what a link does based on the metadata you send with the link e.g. IANA link relations, RDF vocabularies, etc. So the only URI you need to hardcode into your clients should be the root URI of the API. Most of the people don't try to meet with this REST constraint, because it requires somewhat different than the usual mindset to do it.
I created a few Rest apis right now and I always preferred a solution, where I created an endpoint for each resource.
For example:
GET .../employees/{id}/account
GET .../supervisors/{id}/account
and the same with the other http methods like put, post and delete. This blows up my api pretty much. My rest apis in general preferred redundancy to reduce complexity but in this cases it always feels a bit cumbersome. So I create another approach where I work with inheritance to keep the "dry" principle.
In this case there is a base class User and via inheritance my employee and supervisor model extends from it. Now I only need one endpoint like
GET .../accounts/{id}
and the server decides which object is returned. Also while this thins out my api, it increases complexity and in my api documentation ( where I use spring rest docs ) I have to document two different Objects for the same endpoint.
Now I am not sure about what is the right way to do it ( or at least the better way ). When I think about Rest, I think in resources. So my employees are a seperate resource as well as my supervisors.
Because I always followed this approach, I tink I might be mentally run in it and maybe lost the objectivity.
It would be great if you can give my any objective advice on how this should be handled.
I built an online service that deals with this too. It's called Wirespec:
https://wirespec.dev
The backend automatically creates the url for users and their endpoints dynamically with very little code. The code for handling the frontend is written in Kotlin while the backend for generating APIs for users is written in Node.js. In both cases, the amount of code is very negligible and self-maintaining, meaning that if the user changes the name of their API, the endpoint automatically updates with the name. Here are some examples:
API: https://wirespec.dev/Wirespec/projects/apis/Stackoverflow/apis/getUserDetails
Endpoint: https://api.wirespec.dev/wirespec/stackoverflow/getuserdetails?id=100
So to answer your question, it really doesn't matter where you place the username in the url.
Try signing in to Wirespec with your Github account and you'll see where your Github username appears in the url.
There is, unfortunately, no wright or wrong answer to this one and it soley depends on how you want to design things.
With that being said, you need to distinguish between client and server. A client shouldn't know the nifty details of your API. It is just an arbitrary consumer of your API that is fed all the information it needs in order to make informed choices. I.e. if you want the client to send some data to the server that follows a certain structure, the best advice is to use from-like representations, such as HAL forms, Ion or even HTML. Forms not only teach a client about the respective properties a resource supports but also about the HTTP operation to use, the target URI to send the request to as well as the representation format to send the data in, which in case of HTML is application/x-www-form-urlencoded most of the time.
In regards to receiving data from the server, a client shouldn't attempt to extract knowledge from URIs directly, as they may change over time and thus break clients that rely on such a methodology, but rely on link relation names. Per URI there might be multiple link relation names attached to that URI. A client not knowing the meaning of one should simply ignore it. Here, either one of the standardized link relation names should be used or an extension mechanism as defined by Web linking. While an arbitrary client might not make sense from this "arbitrary string" out of the box, the link relation name may be considered the predicate in a tripple often used in ontologies where the link relation name "connects" the current resource with the one the link relation was annotated for. For a set of URIs and link relation names you might therefore "learn" a semantic graph over all the resources and how they are connected to each other. I.e. you might annotate an URI pointing to a form resource with prefetch to hint a client that it may load the content of the referenced URI if it is IDLE as the likelihood is high that the client will be interested to load that resource next anyway. The same URI might also be annotated with edit-form to hint a client that the resource will provide an edit form to send some data to the server. It might also contain a Web linking extension such as https://acme.org/ref/orderForm that allows clients, that support such a custom extension, to react to such a resource accordingly.
In your accounts example, it is totally fine to return different data for different resources of the same URI-path. I.e. resource A pointing to an employee account might only contain properties name, age, position, salery while resource B pointing to a supervisor could also contain a list of subordinates or the like. To a generic HTTP client these are two totally different resources even though they used a URI structure like /accounts/{id}. Resources in a REST architecture are untyped, meaning they don't have a type ouf of the box per se. Think of some arbitrary Web page you access through your browser. Your browser is not aware of whether the Web page it renders contains details about a specific car or about the most recent local news. HTML is designed to express a multitude of different data in the same way. Different media types though may provide more concrete hints about the data exchanged. I.e. text/vcard, applciation/vcard+xml or application/vcard+json all may respresent data describing an entity (i.e. human person, jusistic entity, animal, ...) while application/mathml+xml might be used to express certain mathematical formulas and so on. The more general a media type is, the more wiedspread usage it may find. With more narrow media types however you can provide more specific support. With content type negotiation you also have a tool at your hand where a client can express its capabilities to servers and if the server/API is smart enough it can respond with a representation the client is able to handle.
This in essence is all what REST is and if followed correctly allow the decoupling of clients from specific servers. While this might sound confusing and burdensome to implement at first, these techniques are intended if you strive for a long-lasting environment that still is able to operate in decateds to come. Evolution is inherently integrated into this phiolosophy and supported by the decoupled design. If you don't need all of that, REST might not be the thing you want to do actually. Buf if you still want something like REST, you for sure should design the interactions between client and server as if you'd intereact with a typical Web server. After all, REST is just a generalization of the concepts used on the Web quite successfully for the past two decades.
I want to make an API using REST which interacts (stores) data in a database.
While I was reading some design patterns and I came across remote facade, and the book I was reading mentions that the role of this facade is to translate the course grained methods from the remote calls into fine grained local calls, and that it should not have any extra logic. As an explaination, it says that the program should still work without this facade.
Here's an example
Yet I have two questions:
Considering I also have a database, does it make sense to split the general call into specific calls for each attribute? Doesn't it make more sense to just have a general "get data" method that runs one query against the database and converts it into an usable object, to reduce the number of database calls? So instead of splitting the get address to get street, get city, get zip, make on db call for all that info.
With all this in mind, and, in my case using golang, how should the project be structured in terms of files and functions?
I will have the main file with all the endpoints from the REST API, calling the controllers that handle these requests.
I will have a set of files that define those controllers. Are these controllers the remote facade? Should those methods not have logic in that case, and just call the equivalent local methods?
Should the local methods call the database directly, or should they use some sort of helper class that accesses the database?
Assuming all questions are positive, does the following structure make sense?
Main
Controllers
Domain
Database helper
First and foremost, as Mike Amundsen has stated
Your data model is not your object model is not your resource model is not your affordance model
Jim Webber did say something very similar, that by implementing a REST architecture you have an integration model, in the form of the Web, which is governed by HTTP and the other being the domain model. Resources adept and project your domain model to the world, though there is no 1:1 mapping between the data in your database and the representations you send out. A typical REST system does have many more resources than you have DB entries in your domain model.
With that being said, it is hard to give concrete advice on how you should structure your project, especially in terms of a certain framework you want to use. In regards to Robert "Uncle Bob" C. Martin on looking at the code structure, it should tell you something about the intent of the application and not about the framework¹ you use. According to him Architecture is about intent. Though what you usually see is the default-structure imposed by a framework such as Maven, Ruby on Rails, ... For golang you should probably read through certain documentation or blogs which might or might not give you some ideas.
In terms of accessing the database you might either try to follow a micro-service architecture where each service maintains their own database or you attempt something like a distributed monolith that acts as one cohesive system and shares the database among all its parts. In case you scale to the broad and a couple of parallel services consume data, i.e. in case of a message broker, you might need a distributed lock and/or queue to guarantee that the data is not consumed by multiple instances at the same time.
What you should do, however, is design your data layer in a way that it does scale well. What many developers often forget or underestimate is the benefit they can gain from caching. Links are basically used on the Web to reference from one resource to an other and giving the relation some semantic context by the utilization of well-defined link-relation names. Link relations also allow a server to control its own namespace and change URIs as needed. But URIs are not only pointers to a resource a client can invoke but also keys for a cache. Caching can take place on multiple locations. On the server side to avoid costly calculations or look ups on the client side to avoid sending requests out in general or on intermediary hops which allow to take away pressure from heavily requested servers. Fielding made caching even a constraint that needs to be respected.
In regards to what attributes you should create queries for is totally dependent on the use case you attempt to depict. In case of the address example given it does make sense to return the address information all at once as the street or zip code is rarely queried on its own. If the address is part of some user or employee data it is more vague whether to return that information as part of the user or employee data or just as a link that should be queried on its own as part of a further request. What you return may also depend on the capabilities of the media-type client and your service agree upon (content-type negotiation).
If you implement something like a grouping for i.e. some football players and certain categories they belong to, such as their teams and whether they are offense or defense players, you might have a Team A resource that includes all of the players as embedded data. Within the DB you could have either an own table for teams and references to the respective player or the team could just be a column in the player table. We don't know and a client usually doesn't bother as well. From a design perspective you should however be aware of the benefits and consequences of including all the players at the same time in regards to providing links to the respective player or using a mixed approach of presenting some base data and a link to learn further details.
The latter approach is probably the most sensible way as this gives a client enough information to determine whether more detailed data is needed or not. If needed a simple GET request to the provided URI is enough, which might be served by a cache and thus never reach the actual server at all. The first approach has for sure the disadvantage that it doesn't reuse caching optimally and may return way more data then actually needed. The approach to include links only may not provide enough information forcing the client to perform a follow-up request to learn data about the team member. But as mentioned before, you as the service designer decide which URIs or queries are returned to the client and thus can design your system and data model accordingly.
In general what you do in a REST architecture is providing a client with choices. It is good practice to design the overall interaction flow as a state machine which is traversed through receiving requests and returning responses. As REST uses the same interaction model as the Web, it probably feels more natural to design the whole system as if you'd implement it for the Web and then apply the design to your REST system.
Whether controllers should contain business logic or not is primarily an opinionated question. As Jim Webber correctly stated, HTTP, which is the de-facto transport layer of REST, is an
application protocol whose application domain is the transfer of documents over a network. That is what HTTP does. It moves documents around. ... HTTP is an application protocol, but it is NOT YOUR application protocol.
He further points out that you have to narrow HTTP into a domain application protocol and trigger business activities as a side-effect of moving documents around the network. So, it's the side-effect of moving documents over the network that triggers your business logic. There is no straight rule whether to include business logic in your controller or not, but usually you try to keep the business logic in yet their own layer, i.e. as a service that you just invoke from within the controller. That allows to test the business logic without the need of the controller and thus without the need of a real HTTP request.
While this answer can't provide more detailed information, partly due to the broad nature of the question itself, I hope I could shed some light in what areas you should put in some thoughts and that your data model is not necessarily your resource or affordance model.
first of all I am new to this...REST, RDF, LDP etc.
I could able to get an understanding about REST and RDF in a vague manner:
REST is a framework where everything is a resource and complex client side requests are converted to URI based structural requests and
using HTTP methods, we will get the results in RDF resource format:
XML or json format.
RDF is a framework to explain the relational structure or in other words, conceptual model of a web resource.
LDP seems to be same as REST, uses HTTP protocols to interact with RDF resources. What I understand is HTTP protocols are used to communicate with web services and get the result in HTML, jpeg, png or any other format, even XML too. Then what is LDP? - Does it somehow updates the XML using the HTTP methods.
Can't that be done in normal architecture. other than LDP?
LDP, Linked Data Platform, is a W3C specification defining a standard way for servers and clients to interact with resources (primarily RDF resources) over HTTP. In particular, it introduces the notion of Containers, RDFSources, and Non-RDFSources (or binaries).
It may help to think of an RDFSource as a document, kind of like an HTML web page. Only, the content is not HTML, it's a graph (a set of RDF triples) sharing the same subject URI. Together, the triples in this document would typically describe or make up a given entity or object. So, those could be thought of as properties of the object. The document could be expressed in RDF XML, in Turtle, JSON-LD, or possibly other formats. These properties may be literal values or they may be links to other resources.
LDP implements the RESTful architecture, so how you view this RDFSource depends on how you ask for it in your request to the server. If you ask for the resource to be expressed in JSON-LD, you should get back a JSON-LD representation of the resource. If you ask for it as Turtle, you should get back a Turtle representation. This is done by passing certain HTTP headers in the request. Additionally, the RESTful nature of an LDP allows you to use HTTP methods (GET,POST,PUT,DELETE) to interact with the resources in various ways.
A Container is also an RDFSource, but it allows you to manage membership. Containers can have member resources. You could think of a Container kind of like a folder. Only, it doesn't physically contain RDFSources or documents. Instead, it has a set of triples (membership triples) that define or link to its members. You can view a Container as a container or as an RDFSource depending on the preferred interaction model that you specify in a request header.
So, basically, you can think of an LDP as a way of interacting with RDF resources in a way that is similar to a web site with folders and documents. Only everything is RDF, not XHTML. On the back-end, the server may actually manage each resource as a real document (file). Or, as is the case with Carbon LDP, for example, it may put everything in a triplestore (an RDF store / database). Then it just gives you back a set of triples that look like a "document" because they share the same subject URI, which is what you used when making the RESTful request. Since Carbon LDP manages all these "documents" in a triplestore, it can also provide SPARQL query support across everything (though SPARQL support is not part of the LDP spec).
So, in essence, an LDP enables a very "webby" (or RESTful) way of working with RDF data. You make HTTP requests to URI's and they get resolved to resources (Containers or RDFSources), which you can then consume to get at all the triples. And of course you can create resources, update them, list members of a container, etc. In this way, you can build web applications, that use RESTful requests (perhaps async JavaScript or AJAX requests).
One advantage you win is that even though the data you're working with may be very specific to any given application your building on LDP, the REST API you use to work with that data is standard and consistent.
Another advantage is that you're working with RDF, so the properties of your objects, the predicates, can link data across your enterprise or the World Wide Web. This can help you incorporate data and discover things that your app may not have been specifically designed to support. And also, because you're working with the RDF data model, you can use pre-existing vocabularies for your triples, but you don't have near as much hassle with schemas.
In RDF, you can add new triples (new properties or links) without having to update some database schema and the associated code required to interpret it. LDP deals with RDF resources in a very generic way - it doesn't care what the triples that define or make up the resources actually are. When you build an LDP app, you can extend that sort of generic quality into the app in such a way that your data can keep changing and evolving without imposing as heavy costs on the maintenance and evolution of the app itself.
This kind of technology helps you bridge the gap between the current web of hyperlinked documents to a web of linked data, which is easier for computers to understand and interoperate with. For a little more info about RDF and the big difference between a hyperlink and a linked data link, see The Awesome Power of the Link in Linked Data.
You can also find a somewhat technical introduction to LDP in Introduction to: Linked Data Platform, an article I wrote a while back for Dataversity.
I get the basic idea of REST-ful API, but I can see how they're not ideal. I want to know where I'm wrong on the following assumptions.
REST-ful API unnecessarily exposes models
On a REST-ful API, it's usually a set of actions e.g CRUD on an entity. Sometimes performing a business action requires many models to be manipulated, and not just one.
For example, consider a business action of refunding an order, which then decreases a buyer's rating. Several possible models are involved here e.g Order, OrderItem, Payment, Buyer, Refund.
We often end up exposing single 'parent' model with an action that updates itself and its sub-models, or we end up exposing many models that must be updated appropriately to successfully accomplish the business action as a whole.
REST-ful API forces one to think in terms of manipulating models instead of the natural behavior of stating intent
Consider a customer service rating application. A customer can state his / her happiness once a support call ends e.g "I'm satisfied" / "I'm angry" / "I'm neutral".
In a REST-ful API, the customer has to figure out what exact model to manipulate in order to state how he feels. Perhaps a CustomerResponse model, or a Feedback model. Why can't the customer just hit an endpoint, identify himself and simply state whether he's happy or not, without having to know the underlying model that tracks his response?
REST-ful API Update action oversimplifies too much
Sometimes on a model, you want to do more than just an update. An update can be many things.
Consider a Word model. You can reverse the characters, randomize the characters, uppercase / lowercase the characters, split the word and many other actions that actually means a Word model is 'updated' in a certain way.
At this point, exposing just an Update action on Word probably oversimplifies how rich the Word model can be.
I do not believe the points you state above are really drawbacks of a RESTful API. More analytically:
REST-ful API unnecessarily exposes models
No models are exposed. Everything is handled by a single controller. The only thing that is exposed to the user is the route of the appropriate controller.
REST-ful API forces one to think in terms of manipulating models instead of the natural behavior of stating intent
Same as above. A single controller can handle the different customer happiness states. The distinction can be made by passing different post paramenters (ex. { state: "happy"}).
REST-ful API Update action oversimplifies too much
Nothing stops you from manipulating the data that needs to be sent to your model before updating it. You can do whatever you want, however complex it may be, before updating your model.
Finally, I believe that a RESTful API is only as good as its implementation. Furthermore, I believe that if you wanted to find a flaw to the REST technique is the fact that you cannot initiate transactions or push notifications server-side
First, Web service APIs that adhere to the REST architectural constraints are called RESTful APIs. HTTP based RESTful APIs are defined with these aspects:
base URI, such as http://example.com/resources/
an Internet media type for the data. This is often JSON but can be any other valid Internet media type (e.g. XML, Atom, microformats, images, etc.)
standard HTTP methods (e.g., GET, PUT, POST, or DELETE)
hypertext links to reference state
hypertext links to reference related resources
Now with your questions:
REST-ful API unnecessarily exposes models
In your example, if you want to refund someone, you obviously use more than one model.
RESTful doesn't means that you expose only one model, for example, a POST on a /api/refunds is a RESTful way of doing it without expose a single model.
The only thing peole see from you API are routes to your different actions in your different controllers
REST-ful API forces one to think in terms of manipulating models instead of the natural behavior of stating intent
Your RESTful API is called from a front-end (smartphone app, javascript app, etc..) or a back-end (a server), the end user (here, the satisfied/angry/neutral client) is not obligated to see the url called by your front-end, the satisfaction form could be here /support/survey and the server API url could be a POST to /api/support_calls/1/surveys.
REST-ful API Update action oversimplifies too much
An PUT on a RESTful route does NOT means that you should only update one model. You can manipulate params, create a model, update another, and then, update your main model.
Finally
Dont forget that RESTful architecture is a ROUTE and URL architecture created for developpers, you can do anything you want in your controllers, this is just a convention-based way of exposing your URLs to API's consumers
I'll try to address your concerns, but keep in mind that the topic is very mch debatable, so take this as my humble opinion..
1) about the model being unnecessary exposed, well I think it is mostly a matter of modeling your domain entities in a way that allows you to express all the available action and concepts in terms of 'doing something on a resource'. In the refund example, you could for instance model a Refund entity and perform actions on that.
2) here too is a matter of modeling domain entities and services. Your happiness service could be a matter of having a service accepting a user id and an integer expressing the user's satissfaction level.
3) In the case of the Word model, you can simply use the PATCH (or PUT) http verb and provide a service that simply overwrites the resource, which would in turn be manipulated by the client.
Again, REST is a paradigm, so it is (among other things) a way to organize the objects of your domain and the actions that can be performed on those objects. Obviously it has drawbacks and limitations, but I personally think that it is quite 'transparent' (i.e. not imposing artificial constraints on your programming), and that it is quite easy to use, being mostly based on widely accepted conventions and technologies (i.e. http verbs and so on).