We are are going to be creating some RESTful services which are essentially going to be a passthrough to some ye-olde-fashioned SOAP-based webservices. The SOAP services are more generalized and will used across our entire organization (at least that's the plan). The RESTful services are tailored to specific clients. This decision was already made and is unfortunately out of my control to change...
We're struggling with how to structure our RESTful resources in a way that makes sense, follows REST best practices, and call these SOAP services without causing ourselves too much pain.
We have some freedom around the level of granularity for the back-end services, but the general consensus is: keep them coarse grained and don't tailor them to the specific client's needs.
This is leading to some interesting problems. For instance, how to deal with child resources of a parent. The typical example we've been working with is this: a customer with a child address.
We have a back-end SOAP service which updates a customer as a whole entity. However, the REST services client might need to update just the billing address. How do we best handle subsequent updates of the child resource?
Should we do the updates at the "parent" level (the customer), or expose a more fine-grained REST operation that treats the address as a resource and update it that way? The latter seems to be the correct, RESTful way. But, if we do that, we'll essentially be calling a coarse-grained backend service for just one piece of an update. Doesn't seem to make sense as it's a pretty heavyweight call.
We're also struggling a bit with how to correlate RESTful resources with our back-end domain model. We might care about the RESTful resource as a single entity but in our domain on the back-end, it might be many different entities. We've got a relatively simple DB table to handle this now, but I'm not convinced it will scale out as we map more and more resources to domain objects.
These are just a couple of examples of the things we're hitting... I'm wondering if anyone has run into similar issues and has any words of advice or might be able to point me to some articles that might have some best practices.
It seems like it this isn't an unusual problem and will become more relevant as more and more applications use RESTful architectures, but I can't seem to find any other info on it.
Thanks much!
I've found that most domains I model map very easily to REST. The most important thing to do is get into the right state of mind. REST models domains as a set of resources, where SOAP tends to emphasize a set of actions. Another way to look at this is that REST focuses on states and SOAP on state transitions. Even simpler you can think of REST as nouns versus SOAP as verbs. I know this isn't a perfect analogy but it's useful for me.
When you get into this state of mind the mapping should be almost automatic for you. Let's look at your Customer address interaction.
I don't think it's inappropriate to update the whole customer just to update the address, but not being familiar with your domain, maybe it is. If you want to specifically interact with the address just create a sub-resource (or nested resource). You end up with the following mapping of url's and verbs:
GET /customer/72/address # return the address of customer with id 72
PUT /customer/72/address # update the address of customer with id 72
In this particular case it probably doesn't make sense to map the delete, create or list actions.
If your customer has a collection of some entity, let's say widgets, you could map the interactions like this:
GET /customer/72/widgets # return the widgets of customer with id 72
POST /customer/72/widgets # create a new widget for customer with id 72
GET /customer/72/widgets/158 # return widget with id 158 of customer 72
PUT /customer/72/widgets/158 # update widget with id 158 of customer 72
DELETE /customer/72/widgets/158 # delete widget with id 158 of customer 72
Just have the right mindset and keep in mind the mapping won't be one-to-one and you'll be fine.
Don't try and model your domain entities as resources. Model your views/use cases as resources. There is absolutely nothing wrong with building resources for specific clients. REST encourages serendipitous re-use, i.e. unexpected re-use, it does not suggest you try and build resources that will work for every possible scenario.
RESTful frameworks should make resources really cheap and quick to create. If you need five different resources to model all the ways a customer is accessed, then that's fine.
In general I prefer to go with coarse grained resources, just because that is what HTTP is optimized for.
Related
I have been wondering what would be the best approach when adding a resource that is linked to another resource.
I have two objects:
+ Employee: An employee for an IT company
+ Skill: The employee set of technology skills; for example, Java.
Someone can create an employee via REST with the following path:
POST: /employee
Though someone could create a Skill object by itself, similar to creating an Employee object, but it needs to be linked to an Employee, thus...
PATCH: /employee/{employeeId}/skill
This path will create a new Skill to the Employee object, but this is where I am wondering if I am doing something wrong.
Usually when you create a new resource, you use the POST verb, but at the same time I am also updating part of the Employee resource thus it acts like a PATCH verb. Plus, a POST verb should be free from any parameters like {employeeId}.
What would be the best approach / practice when documenting REST API with this scenario?
What would be the best approach / practice when documenting REST API with this scenario?
It may be worth reviewing Jim Webber's 2011 talk.; in short, if you are doing REST, then you are passing around documents -- the useful work performed on your domain model is a side effect of document manipulation.
If you are using PUT/PATCH, then you are fundamentally performing remote authoring -- which is to say, you are asking the server to accept your local copy of a document. And that's fine, even when the server, in accepting your edits, will also be changing its representation of other resources.
You could instead design your application domain protocol so that you are making remote edits to the employee document directly, or even POST-ing edits to that resource, which in turn could have side effects on a skill resource.
Remember, the first application designed using the REST architectural style was the World Wide Web; the most common media-type for defining domain application protocols was HTML, and the only unsafe request method natively supported by HTML was POST -- and this was catastrophically successful.
So POST is fine.
Where things tend to get tricky is when you are trying to support generic components that want to take advantage of caching. Then, you need to be thinking about the rules for invalidating cached data when you design your protocol.
I am trying to follow RESTful principles and a little confused on how "Eager" or "Lazy" endpoints would be set up.
For example, a
Shop has many Products
Products have many Ingredients.
Products have many Packging
Of course a "bad" endpoint that would fetch eagerly would be:
api/shop/1
Would return shop id 1's details but also with:
ALL the Products
ALL the Product's Ingredients
ALL the Product's Packging
This of course is a crazy model...so I can only guess RESTful is "always lazy" by default?
But with "lazy be default" say you want to get 10 different products AND their ingredients...
api/shop/1/product/1/ingredients
api/shop/1/product/2/ingredients
api/shop/1/product/3/ingredients
The number of requests is getting a little high...10 seperate HTTP requests for the 10 products.
So lastly, do you instead tend to design the RESTful endpoints based on what the front-end/consumer may want as opposed to modelling the business/database?
api/shop/1/product-details?productId=1,2,3,4,5,6,7,8,9,10
Is the above strictly "RESTful"?
So I guess the real underlying question is sort of:
Is RESTful API design a model of the Data or a model of the Views?
Is RESTful API design a model of the Data or a model of the Views?
Views is closer -- it's a model of resources
Your data model is not your object model is not your resource model is not your affordance model. -- Amundsen
The simplest analogy is to look at java script in an HTML page
we can embed the java script in the HTML page
we can link to the java script from the HTML page.
Both approaches work - they have different trade offs, primarily in how caching works.
Coarse grained resources are somewhat analogous to data transfer objects; exchange a large representation in a single request/response, and then the client can do lots of different things with that one representation.
Fine grained resources give you more control of caching strategies (the different parts can expire at different times), and perhaps respond better to scenarios where we expect the client to be sending back edited representations of those resources.
One issue that fine grained resources have had is the extra burden of round trips. HTTP/2 improves that story, as server push can be used to chain representations of multiple resources onto a single response -- all of the fine grained resources can be sent in a single burst.
But even so, we're talking about identifying resources, not database entities.
https://stackoverflow.com/questions/57420131/restful-syntax-is-it-eager-lazy-or-both
That's an identifier for a web page about a question
https://api.stackexchange.com/2.2/questions/57420131?site=stackoverflow
That's a different resource describing the same question.
REST API's aren't about exposing your data model via HTTP, they are about exchanging documents so that a client can navigate a protocol that gets useful work done. See Webber 2011.
I want to make an API using REST which interacts (stores) data in a database.
While I was reading some design patterns and I came across remote facade, and the book I was reading mentions that the role of this facade is to translate the course grained methods from the remote calls into fine grained local calls, and that it should not have any extra logic. As an explaination, it says that the program should still work without this facade.
Here's an example
Yet I have two questions:
Considering I also have a database, does it make sense to split the general call into specific calls for each attribute? Doesn't it make more sense to just have a general "get data" method that runs one query against the database and converts it into an usable object, to reduce the number of database calls? So instead of splitting the get address to get street, get city, get zip, make on db call for all that info.
With all this in mind, and, in my case using golang, how should the project be structured in terms of files and functions?
I will have the main file with all the endpoints from the REST API, calling the controllers that handle these requests.
I will have a set of files that define those controllers. Are these controllers the remote facade? Should those methods not have logic in that case, and just call the equivalent local methods?
Should the local methods call the database directly, or should they use some sort of helper class that accesses the database?
Assuming all questions are positive, does the following structure make sense?
Main
Controllers
Domain
Database helper
First and foremost, as Mike Amundsen has stated
Your data model is not your object model is not your resource model is not your affordance model
Jim Webber did say something very similar, that by implementing a REST architecture you have an integration model, in the form of the Web, which is governed by HTTP and the other being the domain model. Resources adept and project your domain model to the world, though there is no 1:1 mapping between the data in your database and the representations you send out. A typical REST system does have many more resources than you have DB entries in your domain model.
With that being said, it is hard to give concrete advice on how you should structure your project, especially in terms of a certain framework you want to use. In regards to Robert "Uncle Bob" C. Martin on looking at the code structure, it should tell you something about the intent of the application and not about the framework¹ you use. According to him Architecture is about intent. Though what you usually see is the default-structure imposed by a framework such as Maven, Ruby on Rails, ... For golang you should probably read through certain documentation or blogs which might or might not give you some ideas.
In terms of accessing the database you might either try to follow a micro-service architecture where each service maintains their own database or you attempt something like a distributed monolith that acts as one cohesive system and shares the database among all its parts. In case you scale to the broad and a couple of parallel services consume data, i.e. in case of a message broker, you might need a distributed lock and/or queue to guarantee that the data is not consumed by multiple instances at the same time.
What you should do, however, is design your data layer in a way that it does scale well. What many developers often forget or underestimate is the benefit they can gain from caching. Links are basically used on the Web to reference from one resource to an other and giving the relation some semantic context by the utilization of well-defined link-relation names. Link relations also allow a server to control its own namespace and change URIs as needed. But URIs are not only pointers to a resource a client can invoke but also keys for a cache. Caching can take place on multiple locations. On the server side to avoid costly calculations or look ups on the client side to avoid sending requests out in general or on intermediary hops which allow to take away pressure from heavily requested servers. Fielding made caching even a constraint that needs to be respected.
In regards to what attributes you should create queries for is totally dependent on the use case you attempt to depict. In case of the address example given it does make sense to return the address information all at once as the street or zip code is rarely queried on its own. If the address is part of some user or employee data it is more vague whether to return that information as part of the user or employee data or just as a link that should be queried on its own as part of a further request. What you return may also depend on the capabilities of the media-type client and your service agree upon (content-type negotiation).
If you implement something like a grouping for i.e. some football players and certain categories they belong to, such as their teams and whether they are offense or defense players, you might have a Team A resource that includes all of the players as embedded data. Within the DB you could have either an own table for teams and references to the respective player or the team could just be a column in the player table. We don't know and a client usually doesn't bother as well. From a design perspective you should however be aware of the benefits and consequences of including all the players at the same time in regards to providing links to the respective player or using a mixed approach of presenting some base data and a link to learn further details.
The latter approach is probably the most sensible way as this gives a client enough information to determine whether more detailed data is needed or not. If needed a simple GET request to the provided URI is enough, which might be served by a cache and thus never reach the actual server at all. The first approach has for sure the disadvantage that it doesn't reuse caching optimally and may return way more data then actually needed. The approach to include links only may not provide enough information forcing the client to perform a follow-up request to learn data about the team member. But as mentioned before, you as the service designer decide which URIs or queries are returned to the client and thus can design your system and data model accordingly.
In general what you do in a REST architecture is providing a client with choices. It is good practice to design the overall interaction flow as a state machine which is traversed through receiving requests and returning responses. As REST uses the same interaction model as the Web, it probably feels more natural to design the whole system as if you'd implement it for the Web and then apply the design to your REST system.
Whether controllers should contain business logic or not is primarily an opinionated question. As Jim Webber correctly stated, HTTP, which is the de-facto transport layer of REST, is an
application protocol whose application domain is the transfer of documents over a network. That is what HTTP does. It moves documents around. ... HTTP is an application protocol, but it is NOT YOUR application protocol.
He further points out that you have to narrow HTTP into a domain application protocol and trigger business activities as a side-effect of moving documents around the network. So, it's the side-effect of moving documents over the network that triggers your business logic. There is no straight rule whether to include business logic in your controller or not, but usually you try to keep the business logic in yet their own layer, i.e. as a service that you just invoke from within the controller. That allows to test the business logic without the need of the controller and thus without the need of a real HTTP request.
While this answer can't provide more detailed information, partly due to the broad nature of the question itself, I hope I could shed some light in what areas you should put in some thoughts and that your data model is not necessarily your resource or affordance model.
I get the basic idea of REST-ful API, but I can see how they're not ideal. I want to know where I'm wrong on the following assumptions.
REST-ful API unnecessarily exposes models
On a REST-ful API, it's usually a set of actions e.g CRUD on an entity. Sometimes performing a business action requires many models to be manipulated, and not just one.
For example, consider a business action of refunding an order, which then decreases a buyer's rating. Several possible models are involved here e.g Order, OrderItem, Payment, Buyer, Refund.
We often end up exposing single 'parent' model with an action that updates itself and its sub-models, or we end up exposing many models that must be updated appropriately to successfully accomplish the business action as a whole.
REST-ful API forces one to think in terms of manipulating models instead of the natural behavior of stating intent
Consider a customer service rating application. A customer can state his / her happiness once a support call ends e.g "I'm satisfied" / "I'm angry" / "I'm neutral".
In a REST-ful API, the customer has to figure out what exact model to manipulate in order to state how he feels. Perhaps a CustomerResponse model, or a Feedback model. Why can't the customer just hit an endpoint, identify himself and simply state whether he's happy or not, without having to know the underlying model that tracks his response?
REST-ful API Update action oversimplifies too much
Sometimes on a model, you want to do more than just an update. An update can be many things.
Consider a Word model. You can reverse the characters, randomize the characters, uppercase / lowercase the characters, split the word and many other actions that actually means a Word model is 'updated' in a certain way.
At this point, exposing just an Update action on Word probably oversimplifies how rich the Word model can be.
I do not believe the points you state above are really drawbacks of a RESTful API. More analytically:
REST-ful API unnecessarily exposes models
No models are exposed. Everything is handled by a single controller. The only thing that is exposed to the user is the route of the appropriate controller.
REST-ful API forces one to think in terms of manipulating models instead of the natural behavior of stating intent
Same as above. A single controller can handle the different customer happiness states. The distinction can be made by passing different post paramenters (ex. { state: "happy"}).
REST-ful API Update action oversimplifies too much
Nothing stops you from manipulating the data that needs to be sent to your model before updating it. You can do whatever you want, however complex it may be, before updating your model.
Finally, I believe that a RESTful API is only as good as its implementation. Furthermore, I believe that if you wanted to find a flaw to the REST technique is the fact that you cannot initiate transactions or push notifications server-side
First, Web service APIs that adhere to the REST architectural constraints are called RESTful APIs. HTTP based RESTful APIs are defined with these aspects:
base URI, such as http://example.com/resources/
an Internet media type for the data. This is often JSON but can be any other valid Internet media type (e.g. XML, Atom, microformats, images, etc.)
standard HTTP methods (e.g., GET, PUT, POST, or DELETE)
hypertext links to reference state
hypertext links to reference related resources
Now with your questions:
REST-ful API unnecessarily exposes models
In your example, if you want to refund someone, you obviously use more than one model.
RESTful doesn't means that you expose only one model, for example, a POST on a /api/refunds is a RESTful way of doing it without expose a single model.
The only thing peole see from you API are routes to your different actions in your different controllers
REST-ful API forces one to think in terms of manipulating models instead of the natural behavior of stating intent
Your RESTful API is called from a front-end (smartphone app, javascript app, etc..) or a back-end (a server), the end user (here, the satisfied/angry/neutral client) is not obligated to see the url called by your front-end, the satisfaction form could be here /support/survey and the server API url could be a POST to /api/support_calls/1/surveys.
REST-ful API Update action oversimplifies too much
An PUT on a RESTful route does NOT means that you should only update one model. You can manipulate params, create a model, update another, and then, update your main model.
Finally
Dont forget that RESTful architecture is a ROUTE and URL architecture created for developpers, you can do anything you want in your controllers, this is just a convention-based way of exposing your URLs to API's consumers
I'll try to address your concerns, but keep in mind that the topic is very mch debatable, so take this as my humble opinion..
1) about the model being unnecessary exposed, well I think it is mostly a matter of modeling your domain entities in a way that allows you to express all the available action and concepts in terms of 'doing something on a resource'. In the refund example, you could for instance model a Refund entity and perform actions on that.
2) here too is a matter of modeling domain entities and services. Your happiness service could be a matter of having a service accepting a user id and an integer expressing the user's satissfaction level.
3) In the case of the Word model, you can simply use the PATCH (or PUT) http verb and provide a service that simply overwrites the resource, which would in turn be manipulated by the client.
Again, REST is a paradigm, so it is (among other things) a way to organize the objects of your domain and the actions that can be performed on those objects. Obviously it has drawbacks and limitations, but I personally think that it is quite 'transparent' (i.e. not imposing artificial constraints on your programming), and that it is quite easy to use, being mostly based on widely accepted conventions and technologies (i.e. http verbs and so on).
I am still in the process of getting comfortable with doing things the REST way.
In my situation, client software will be interacting with a RESTful service. Rarely, the client will upload its entire database of entities (each entity serializes into a roughly 5kb chunk of xml).
Perhaps I'm wrong, but the proper RESTful strategy seems to be to cycle through each entity and individually POST each one. However, there may plausibly be tens of thousands of these entities, and somehow so many rapid-fire POSTs doesn't seem kosher.
In this situation, it feels like packaging all the entities into one big xml representation would violate the RESTful way of doing things, but it would also avoid the need for thousands of POSTs.
Is there some standard-practice for accomplishing this? Thanks in advance!
I don't see why a "Packet of entities" cannot be considered a resource. Transactional writes certainly can consider database transaction to be a resource. I admit I haven't read Fielding's dissertation, but I don't see how wrapping several resources into a single representation would invalidate REST.
Database transactions do something like this. They will wrap smaller resources inside a transaction resource. It's true that usually they do this so that you can post those smaller resources, that can still be large, separately. But since the transaction itself is considered a resource, I don't believe that coming up with a representation for it that you could post as one POST request would make this design any less RESTful.
It's also used to the other direction. When the client GETs search results from the server, the server might wrap these inside a results resource so that the client can just get this one resource instead of several separate ones.
So I'd say that wrapping these small 5kb resources inside a larger collection resource can be considered RESTful and is probably the way you should go for.
There are at least two problems here which prevent you from being RESTful.
Each resource needs to be identified by a URI. Acting on the resource means that you must call the URI using an HTTP call. Consequently, you cannot call multiple actions in multiple resources in just one HTTP call.
The resources are identified by nouns and represent entities. This implies that to insert an Employee and a Car you need to call two different resources for each of the respective entities.
So in summation you cannot take a purely RESTful approach here. However, REST is designed to help by way of conventions, not constrict you. The best solution here is for you to create a custom action which does what you need.
Alternately, you can create a generic wrapper entity with INSERT, UPDATE and other actions which take in blobs of disparate data as XML. However, this will undermine your other end points because now it becomes possible to insert a Car record through the generic wrapper and through the /Car/ URI.
Without knowing much about your actual requirements, I would suggest you don't expose this functionality via REST specifically. Behind the scenes you could still call your INSERT action methods within the various Controllers once you break up the incoming collection if disparate objects.
As long as the big wrapper has a valid media-type then it is fine to treat it as a single resource. Figuring out what that media-type is going to be is the tricky part.
Nothing prevents you from creating more resources upon addition, aka post a resource that is a list of X to a resource that's a list of X using a POST.
You'd then send back a 201 created with the list of URIS of all resources created. Again, it's all perfectly allowable.
What you loose is the visibility to the intermediaries upon PUT, which prevent them from caching or modifying the specific resource at the specific URI. Although a smart intermediary would process the 201 for caching purposes.
And having one doesn't prevent you from having each created resource have its own URI post-creation (after the POST) and enable PUT / DELETE on those resources. Or a combination.