I'm looking for design pattern/library to combine multiple different endpoint to be visible as one.
Let's say I have 3 endpoints, each of them returns the same type of objects, of course objects are different and have unique id. I want to create an endpoint which calls all of them, combine results, filter & sort & page, then return results.
There can be many objects, so it could be good to have caching, to call those three endpoints only when something has changed (let's say I somehow know if I need to refresh cache).
I imagine there's a library which can connect multiple endpoint into one, cache results and deliver filter & sort & page. Some sort of, let's say, Spring repository: however data are not read from database, but from cache, which gathers them from REST endpoints.
I was looking at Gateway design pattern like Spring Gateway or Zuul Proxy, but it seems to be only a wrapper, no possibility process data.
Of course I can do such things manually:
create controller
call three endpoints (if needed to refresh)
fill cache
read data from cache sort, filter, page, and return them
but if I need to do that multiple times, I'm looking for library to do that.
You can implement the GatewayFilterFactory with aggregation and caching in Spring Cloud Gateway. But it looks like the creation of a controller.
Related
I have a question on how should I design my rest api(s) given that I need to return count of different objects in my application. There are multiple approaches that could be thought off
Defining a single api that generally accepts the object identifier in the request body (json) and returns back the count for each of the object identifiers in the response. The drawback is the api is too generic and possibly not restful since there is no resource. The url would look like GET /counts
Define individual api's for each of the resources for which count is needed. Assuming I need to return counts for the fields defined in the software, tables, processes, tasks, jobs etc I would define individual api's for each of these resources. It would look like GET /field/count or GET /table/count. A side effect of this would be there would be many web api's for each of the resources we need the count for. Is that bad?
Kindly share your thoughts on the above approaches and any new ways in which I could design such an API that adheres to the REST standards.
Thanks
It totally depends on the client that is consuming the APIs.
Case 1. If its a WebApp which needs count of many tables on a single page then both will lead to bad design where you will have to make hundreds of calls just for counts data. You can club counts in a single API and send that as a response.
Case 2. If the client are individually using the count then i would recommend the 2nd approach where the resource is clearly defined. For the 2nd approach you are making the client intelligent which is not recommended.
As mentioned in the comments REST is a totally subjective topic so there can be multiple view points to every design.
It seems to me that REST has clean and clear semantics for basic CRUD and for listing resources, but I've seen no discussion of how to handle large lists of resources. It doesn't scale to dump an entire database table over the network in a resource-oriented architecture (imagine a customer table with a million customers!), especially if you only need a few items. So it seems that some semantics should exist to filter, map and reduce a list of resources on the server-side.
So, do you know any tried and true ways to do the following kinds of requests in REST:
1) Retrieve just the count of the resources?
I could imagine doing something like GET /api/customer?result=count
Is that how it's usually done?
I could also imagine modifying the URL (/api/count/customer or /api/customer/count, for example), but that seems to either break the continuity of the resource paths or inflict an ugly hack on the expected ID field.
2) Filter the results on the server-side?
I could imagine using query parameters for this, in a context-specific way (such as GET /api/customer?country=US&state=TX).
It seems tricky to do it in a flexible way, especially if you need to join other tables (for example, get customers who purchased in the last 6 months).
I could imagine using the HTTP OPTIONS method to return a JSON string specifying possible filters and their possible values.
Has anyone tried this sort of thing?
If so, how complex did you get (for example, retrieving the items purchased year-to-date by female customers between 18 and 45 years old in Massachussetts, etc.)?
3) Mapping to just get a limited set of fields or to add fields from joined tables?
4) More complicated reductions than count (such as average, sum, etc.)?
EDIT: To clarify, I'm interested in how the request is formulated rather than how to implement it on the server-side.
I think the answer to your question is OData! OData is a generic protocol for querying and interacting with information. OData is based on REST but extends the semantics to include programatic elemements similar to SQL.
OData is not always URL-based only as it use JSON payloads for some scenarios. But it is a standard (OASIS) so it well structured and supported by many APIs.
A few general links:
https://en.wikipedia.org/wiki/Open_Data_Protocol
http://www.odata.org/
The most common ways of handling large data sets in GET requests are (afaict) :
1) Pagination. The request would be something like GET /api/customer?country=US&state=TX&firstResult=0&maxResults=50. This way the client has the freedom to choose the size of the data chunk he needs (this is often useful for UI-based clients).
2) Exposing a size service, so that the client gets to know how large the data set is before actually requesting it. The service would be something like
GET /api/customer/size?country=US&state=TX
Obviously the two can (and imho should) be used together, so that when/if a client (be it mobile or web or whatever) waints to fill its UI with content, he can choose what's the best data chunk size based also on the size of whole data set (e.g. to avoid creating 100 pages for the user to navigate).
Let's say that there are two microservices representing the resources orders(/orders) and customers(/customers). My requirement is to get all the orders made by a customer.
Had it been a monolithic application, I would have modeled my uri as /customers/{id}/orders. This would have hit the customers resource and made an in-memory service call to get the corresponding orders.
Now, in case of microservices, this isn't possible. So, is the only way to get the orders is to make a remote service call or is there a better way of doing it?
Can we create another resource with the representation /ordersByCustomers/{customerid}?
You can pass some query parameters as filters (this is the most common way I've seen). For example
/orders?customerId=123
I think that's quite clear, that you want to retrieve all customer orders filtered by customer id. In the same way you can add pagination or other filters.
The important thing to remember is that you want the order resource, so the URL should remain the same. I'm mentioning this, because this has been the most difficult thing for me to change... to think about resources rather than remote calls.
In general you should beware of using endpoint that are more or less similar to the one you suggested:
/ordersByCustomers/{customerid}
Why? Because this is not RESTful in general (even in microservices environment) and make the API difficult to understand and you by the consumers. What if you need orderByWhatever? Will you be introducing new endpoint every single time you need a new set of data? Try to avoid so opinionated endpoints.
What #Augutsto suggested is fully correct. If you're afraid of having a complicated logic in GET request this is the situation where you can break REST rules. I mean introducing:
POST /orders/filter/
Where all the filtering logic will be passed in requests body - so it's easier to carry complicated logic as well.
I am working on a REST service and so far all the queries are retrieved using a GET request.
Right now we are using a sort of routing rule like this one:
API/Person/{id} GET
http://api.com/person/1
Now, what if I want to ask to the REST API "Give me a Person with FisrtName = 'Pippo'"
I have a complex DTO that I called PersonQueryDTO that can be sent to the REST method to interview the database using the query criterias.
Is this a good way to do it or should I build complex queries in a different way?
For me it's important to keep the REST principles.
If you want to stick with REST principles, then the way to do something like that is to supply additional parameters in the URL e.g.
GET API/Person?FirstName=SomeName
REST is all about identifying resources, API/Person identifies your collection of Person and the additional parameters are nothing but meta data which the service can use internally to determine what sort of result to return.
Looking for some input on a REST API architectural design. I often find that the desired data is the combination of a view across multiple resources. Would you expect the client to combine them, or provide an API that does the combination for the client?
For example, let's say we are writing a REST API for people to become notified about events. Someone will indicate interest in an event in one of 2 ways:
Join an organization that regularly puts on events that the person has interest in
Search for and then mark a particular event run by an organization I wouldn't normally subscribe to
I can retrieve all of the events for user 100 by doing the following long steps:
GET /user/100/organizations returns 123
GET /organizations/123/events returns [15,16,20]
GET /user/100/savedevents returns [35,36]
GET /events/15,16,20,35,36 returns all of the events
But that seems rather heavy for a client. I almost want a client to be able to say, "give me all of the interesting events for this user":
GET /user/100/events
...and then require the server to understand that it has to go through all of steps 1-4 and return them, or, at the very least, return [15,16,20,35,36] so it becomes 2 steps: get event IDs; get event details.
Does this even make sense, to make a view that cuts across multiple resources that way?
EDIT: To explain further. My hesitation is because I can see how /organizations/123/events is a clean resource; if is identical to saying /events?organizations=123, i.e. "give me resource events where organizations=123". Same for /user/100/organizations.
But /user/100/events is not "give me resource events where organizations=123". It is "give me organizations registrations where user=100, retrieve those organization ids, then give me the events where the organization=123, then give me savedevents where user=100."
Each of the three steps itself is a clean resource mapping. Putting them together seems messy. But so does asking a client (especially a Web client), to figure out all that logic!
I was a bit confused by your question, so I'll try to be as comprehensive as possible and hopefully I'll have hit on an answer you need =P.
I often find that the desired data is the combination of a view across
multiple resources. Would you expect the client to combine them, or
provide an API that does the combination for the client?
In a true RESTful environment all cross-sectional views of data would be done by the server, not by the client.
The primary reason for a RESTful design is allow access to the CRUD model (create, read, update, delete) by way of using standard HTTP verbs (e.g. GET, POST, PUT, DELETE). Storing the returns of these methods in some sort of session or cookie or otherwise external method (e.g. "give me data for bob", "give me data on businesses", "give me data from my first two queries") goes above and beyond the REST methodology.
The way you'll want to leverage RESTful development is to find ways of combining resources in meaningful ways so as to provide a RESTful environment where the method calls are consistent; GET reads data, POST creates data, PUT updates data, DELETE deletes data).
So if you wanted to do something like Steps 1 through 4 I'd recommend something like:
GET /user/{userID}/organizations --> {return all affiliated organizations}
GET /user/{userID}/events --> {return all events associated with userID}
GET /organizations/{organization}/events --> {returns all eventID's assoc. with organization}
GET /user/{userID}/savedevents --> {return all eventID's userID saved to their profile}
GET /events/?eventID=(15,16,20,35,36) --> {return all of the events details for those eventID's}
GET /events/{eventID}--> {return events details for {eventID}}
Whereas you might also have:
GET /events/ --> {return a complete listing of all event ID's}
GET /events/{userID} --> {return all events userID is associated with}
POST /event/ --> {create a new event - ID is assigned by the server}
POST /user/ --> {create a new user - ID is assigned by the server}
PUT /user/{userID} --> {update/modify user information}
Then if you want cross-sectional slices of information, you would have a named resource for the cross section (else pass it as arguments). Be explicit with your resources (Random FYI, name your resources as nouns only - not verbs).
You also asked:
To explain further. My hesitation is because I can see how
/organizations/123/events is a clean resource; if is identical to
saying /events?organizations=123, i.e. "give me resource events where
organizations=123". Same for /user/100/organizations.
Essentially both the named resourced and the resource + argument method can provide the same information. Typically I have seen RESTful design API call for arguments only when an important delineation is required (range requests, date requests, some REALLY small unit of data, etc.). If you have some higher-order grouping of data that CAN BE parsed/introspected further then it's a named resource. In your example, I'd have it both API calls, as the RESTful spec calls for providing data via multiple paths and by way of using the established HTTP methods. However, I'd also expand a bit...
/events?organizations=123 --> {return the eventID's associated with org=123}
/organizations/123/events --> {return event DETAILS for events associated with org=123}
Have a read/go at this, by Apigee
There may be several ways to solve this... however, I think that most of the times (if the service is managed by the same provider) it is better to have the logic on the server-side and make REST calls as independent as possible of each other (i.e., the server performing the multiple operations required - normally read data from DBs that are store the data handled in the API resources).
In the example you talk about this would mean your REST API would expose a "user" resource and a sub-resource "events" (which you call "savedevents") he is interested in. With this in mind you would have something like this:
POST /user/{username}/events stores a new event (or multiple events) the user is interested in
GET /user/{username}/events returns all the events the user is interested in
GET /user/{username}/events/{eventid} returns details of a specific event
To "filter" user events per organization (and other filtering operations) you can use "query parameters":
GET /user/{username}/events?organization=123
So, the server (or API call) would perform the operations you describe from step 1 to step 4 in the GET /user/{username}/events. You can still make the other resources ("organizations" and "events") in your API, however they would be used in other contexts (like store new events or organizations, etc.).
HTH