we are building rest API with spring boot.
We have a scenario where we cannot execute requests for a object in parallel. Actually, in back-end, parallel execution is not possible for same objects.
Example :
Not supported in parallel
Request 1 for action XYZ for object A
Request 2 for action XYZ for object A
Request 3 for action ABC for object A
Supported in parallel
Request 1 for action XYZ for object A
Request 2 for action XYZ for object B
Request 3 for action ABC for object C
So, I am looking best possible ways to achieve this scenario. one major drawback I can see is that it wont keep the rest application stateless.
I can think of doing this by making a set where I can track which object entry already exists. If yes then dont accept request for that object. But with this, I will have to reject the similar request.
Other option where I dont have to reject the request is to maintain a queue where I execute request in a certain object for same object.
But I can sense that this is not the right way to do this. So, if anybody has solved similar situation, please guide.
Related
I want to understand what should be the right request/response structure and API design to solve below problem.
I have 2 entities let say Abc and Xyz. And Xyz has a foreign key of Abc.
So, to create a record for Xyz, there has to be mapped Abc record.
Now for request structure point of view, I need to create one POST request for Abc that is like
POST /Abc
This is pretty much straightforward. But the problem is with Xyz.
The requirement is whenever a user came for to create Xyz, he may also request to update the attached Abc record, too.
For example,
I have created a record for Abc with id 5. Now, whenever I want to create a corresponding Xyz record, I will request to update the Abc record with id 5 and create a new Xyz record for this foreign key.
So,
PATCH /Abc
and
POST /Xyz
But the client requests only once and share whole data on single URI.
So, what is the right way to handle multiple HTTP methods on single URI?
Should I create POST request or PATCH?
I couldn't create 2 requests because the client wants this process as transactional.
Firstly I guess you should think this
I couldn't create 2 requests because the client wants this process as transactional.
in an other way. As I see it the requirement for transactional might just mean - I might be terribly wrong on this so find out the truth - that when there is also an update to Abc the process of creating a new Xyz is transactional with updating Abc. So the Xyz is not created if the update for Abc fails (or vice versa) and some error is returned.
So you might be able to create two endpoints:
one with just POST:ing a new Xyz
another for creating a new Xyz and update Abc at the same time transactionally
So you might be able to create two endpoints. More interesting here is that is the latter POST or PATCH? It seems to somehow be both, yes.
However see - for example - this question & accepted answer, there is about PATCH:
The PATCH method requests that a set of changes described in the request entity be applied to the resource identified by the Request-URI
Now, the following question is that is the changed Abc identified by the Request-URI? If not then it is - as I understand this - a POST.
Which means you should need only one POST endpoint that checks if there is a need and does transactional updates id needed. But maybe it would still be better to have separate endpoints.
Let's say there are two (or more) RESTful microservices serving JSON. Service (A) stores user information (name, login, password, etc) and service (B) stores messages to/from that user (e.g. sender_id, subject, body, rcpt_ids).
Service (A) on /profile/{user_id} may respond with:
{id: 1, name:'Bob'}
{id: 2, name:'Alice'}
{id: 3, name:'Sue'}
and so on
Service (B) responding at /user/{user_id}/messages returns a list of messages destined for that {user_id} like so:
{id: 1, subj:'Hey', body:'Lorem ipsum', sender_id: 2, rcpt_ids: [1,3]},
{id: 2, subj:'Test', body:'blah blah', sender_id: 3, rcpt_ids: [1]}
How does the client application consuming these services handle putting the message listing together such that names are shown instead of sender/rcpt ids?
Method 1: Pull the list of messages, then start pulling profile info for each id listed in sender_id and rcpt_ids? That may require 100's of requests and could take a while. Rather naive and inefficient and may not scale with complex apps???
Method 2: Pull the list of messages, extract all user ids and make bulk request for all relevant users separately... this assumes such service endpoint exists. There is still delay between getting message listing, extracting user ids, sending request for bulk user info, and then awaiting for bulk user info response.
Ideally I want to serve out a complete response set in one go (messages and user info). My research brings me to merging of responses at service layer... a.k.a. Method 3: API Gateway technique.
But how does one even implement this?
I can obtain list of messages, extract user ids, make a call behind the scenes and obtain users data, merge result sets, then serve this final result up... This works ok with 2 services behind the scenes... But what if the message listing depends on more services... What if I needed to query multiple services behind the scenes, further parse responses of these, query more services based on secondary (tertiary?) results, and then finally merge... where does this madness stop? How does this affect response times?
And I've now effectively created another "client" that combines all microservice responses into one mega-response... which is no different that Method 1 above... except at server level.
Is that how it's done in the "real world"? Any insights? Are there any open source projects that are built on such API Gateway architecture I could examine?
The solution which we used for such problem was denormalization of data and events for updating.
Basically, a microservice has a subset of data it requires from other microservices beforehand so that it doesn't have to call them at run time. This data is managed through events. Other microservices when updated, fire an event with id as a context which can be consumed by any microservice which have any interest in it. This way the data remain in sync (of course it requires some form of failure mechanism for events). This seems lots of work but helps us with any future decisions regarding consolidation of data from different microservices. Our microservice will always have all data available locally for it process any request without synchronous dependency on other services
In your case i.e. for showing names with a message, you can keep an extra property for names in Service(B). So whenever a name update in Service(A) it will fire an update event with id for the updated name. The Service(B) then gets consumes the event, fetches relevant data from Service(A) and updates its database. This way even if Service(A) is down Service(B) will function, albeit with some stale data which will eventually be consistent when Service(A) comes up and you will always have some name to be shown on UI.
https://enterprisecraftsmanship.com/2017/07/05/how-to-request-information-from-multiple-microservices/
You might want to perform response aggregation strategies on your API gateway. I've written an article on how to perform this on ASP.net Core and Ocelot, but there should be a counter-part for other API gateway technologies:
https://www.pogsdotnet.com/2018/09/api-gateway-response-aggregation-with.html
You need to write another service called Aggregator which will internally call both services and get the response and merge/filter them and return the desired result. This can be easily achieved in non-blocking using Mono/Flux in Spring Reactive.
An API Gateway often does API composition.
But this is typical engineering problem where you have microservices which is implementing databases per service pattern.
The API Composition and Command Query Responsibility Segregation (CQRS) pattern are useful ways to implement queries .
Ideally I want to serve out a complete response set in one go
(messages and user info).
The problem you've described is what Facebook realized years ago in which they decided to tackle that by creating an open source specification called GraphQL.
But how does one even implement this?
It is already implemented in various popular programming languages and maybe you can give it a try in the programming language of your choice.
I am designing a RESTful API for tests and test runs metadata. I have two resources: Test and TestRun. Under the hood they have a one-to-one relationship in the database.
First I create a Test resource by sending a POST to api/v1/test.
Then I have to start this test. I do this by sending a POST to api/v1/test/{id}/run, which creates a TestRun resource that relates to that Test resource.
Then I can also stop the test by sending a POST to api/v1/test/{id}/finish, which modifies the corresponding TestRun resource (sets some fields, like finish_time, result etc.).
The user of the API will never have GET access to TestRun resources and will only access them through their related Test resources.
While it looks like this design is quite straightforward for the API user, I doubt this is also as straightforward for a developer. Is this design I came up with good enough? Does it violate any REST principles or best practices? I would appreciate any input on this.
An extended design description of the whole API: https://gist.github.com/Ch00k/27724e29ec1bf044ebbfdabef9e842d5
Your API will not allow access to the testrun, so a testrun seems not to be a REST Resource. In fact your URLs are a mix of REST, as far as the test is concerned, and RPC, especially the /run and /finish paths.
RPC is not REST, so I would rework this a little bit:
Use only one type of resource, the test, at api/v1/test/{id}.
A test has state that can be retrievd using a GET request to api/v1/test/{id}:
status: stopped, started, finished, ...
finish_time
result
...
The state of a test is changed using a PATCH request to api/v1/test/{id} with a JSON body containing the new state: {"status": "started"} to start the test, etc. This replaces the RPC calls.
Please keep in mind i have a rudimentary understanding of rest and building services. I am asking this question mostly cause i am trying to decouple a service from invoking a CLI(within the same host) by providing a front to run async jobs in a scalable way.
I want to build a service where you can submit an asynchronous job. The service should be able to tell me status of the job and location of the results.
APIs
1) CreateAsyncJob
Input: JobId,JobFile
Output: 200Ok (if job was submitted successfully)
2) GetAsyncJobStatus
Input: JobId
Output: Status(inProgress/DoesntExist/Completed/Errored)
3)GetAsyncJobOutput
Input: JobId
Output: OutputFile
Question
The second API, GetAsyncJobStatus violates the principles of idempotency.
How is idempotency preserved in such APIs where we need to update the progress of a particular job ?
Is Idempotency a requirement in such situations ?
Based on the link here idempotency is a behaviour demonstrated by an API by producing the same result during it's repeated invocations.
As per my understanding idempotency is at per API method level ( we are more concerned about what would happen if a client calls this API repeatedly). Hence the best way to maintain idempotency would be to segregate read and write operations into separate APIs. This way we can reason more throughly with the idempotent behavior of the individual API methods. Also while this term is gaining traction with RESTful services, the principles hold true even for other API systems.
In the use case you have provided the response to the API call made by the client would differ (depending upon the status of the job).Assuming that this API is read-only and does not perform any write operations on the server, the state on the server would remain the same by invoking only this API - for e.g. if there were 10 jobs in the system in varied states calling this API 100 times for a job id could result in different status every time for the job id (based on it's progress) - however the number of jobs on the server and their corresponding states would be the same.
However if this API were to be implemented in a way that would alter the state of the server in some way - then this call is not necessarily idempotent.
So keep two APIs - getJobStatus(String jobId) and updateJobStatus(String jobId). The getJobStatus is idempotent while updateJobStatus is not.
Hope this helps
I am doing attach and detach operations in my below controller like detaching the relationship from a OU to a ModuleDevice.
What http verbs would those operations mean in a REST API?
ModuleDevicesToOUController
Api/ModuleDevicesToOU/1/OU/1 // Attach for ModuleDevice with Id 1 the OU with Id 1
PUT/Delete/Patch ??? Attach…
Api/ModuleDevicesToOU/1/OU/1// Detach for ModuleDevice with Id 1 the OU with Id 1
PUT/Delete/Patch ??? Detach…
How about using the LINK method? https://datatracker.ietf.org/doc/html/draft-snell-link-method-08
If that's a little experimental for you then just use POST.
If I understand your API correctly, the call is doing a change in the resource at the server, this means that you should use a POST:
Per RFC 2616 , the POST method should be used for any context in
which a request is non-idempotent: that is, it causes a change in
server state each time it is performed, such as submitting a comment
to a blog post or voting in an online poll. In practice, GET is often
reserved, not simply for idempotent actions, but for nullipotent ones,
ones with no side-effects (in contrast to "no side effects on second
or future requests" as with idempotent operations)
http://en.wikipedia.org/wiki/POST_%28HTTP%29
http://en.wikipedia.org/wiki/Http
This is, in my opinion, a very important point between the difference between GET and POST. POST is not only there to allow you to send information in the body, but also to modify the state of the server. I always have it in mind when designing REST interfaces.
I hope it helps.