I have a spring batch job, which needs to fetch details from rest api call and process the particular data on my side. My rest api call will have mainly the below parameters :
StartinIdNumber(offset)
PageSize(limit)
ps: StartinIdNumber serves the same purpose as rownumber or "offset" in this particular API. The API response results are sorted by IdNumber, so by specifying a StartinIdNumber, the API will in turn perform a "where IdNumber >= StartinIdNumber order by IdNumber limit pageSize" in their DB query.
It will return the given number of user details, I need to iterate through all the ids by changing the StartingIdNumber parameter for each request.
I have seen current ItemReader implementations of spring batch framework,which read through database or xml etc. But I didn't come across any reader which helps in my case. Please suggest a way to iterate through the user details as specified above .
Note : If I write my own custom item reader, I have to take care of preserving state (last processed "StartingIdNumer") which is proving challenging to me.
Does implementing ItemStream serves my purpose? Or is there any better way?
Implementing the ItemStream interface and writing my own custom reader served my purpose. It is now state-full as required for me. Thanks.
Related
Spring Cloud Gateway appears to be reordering my query parameters to put duplicate parameters together.
I'm trying to route some requests to one of our end points to a third party system. These requests include some query parameters that need to be in a specific order (including some duplicate parameters), or the third party system returns a 500 error, but upon receiving the initial request with the parameters in the proper order, the Spring Cloud Gateway reorders these parameters to put the duplicates together by the first instance of the parameter.
Example:
http://some-url.com/a/path/here?foo=bar&anotherParam=paramValue2&aThirdParam=paramValue3&foo=bar
Becomes:
http://some-url.com/a/path/here?foo=bar&foo=bar&anotherParam=paramValue2&aThirdParam=paramValue3
Where the last parameter was moved to be by the first parameter because they had the same name.
The actual request output I need is for the query parameters to be passed through without change.
The issue lays in the UriComponentsBuilder which is used in RouteToRequestFilter.
UriComponentsBuilder.fromUri(uri) is going to build up a map of query params. Because this is a LinkedMultiValueMap you see the reordering of the used query params.
Note that RFC3986 contains the following
The query component contains non-hierarchical data that, along with data in the path component (Section 3.3), serves to identify a resource within the scope of the URI’s scheme and naming authority (if any).
Therefor I don’t think there needs to be a fix in Spring Cloud Gateway.
In order to fix this in your gateway, you'll need to add a custom filter which kicks in after the RouteToRequestFilter by setting the order to RouteToRequestUrlFilter.ROUTE_TO_URL_FILTER_ORDER + 1.
Take a look at the RouteToRequestUrlFilter how the exchange is adapted to go to the downstream URI.
Hope that helps! :)
I am trying to create an adapter (WEB API that will act as a pass through)
for invoking the MS Graph APIs for managing my Active Directory.
AD objects like application and users will be customized to meet our application needs (removing some attributes, adding some extension attributes etc.) and a transformation from our application specific object to AD object would happen in our adapter layer before and after calling MS Graph APIs.
MS Graph APIs currently supports OData queries.
Applications and users would be read as page-wise.
If I have to provide the same OData options in my pass thru Web API layer, how can I do that?
i.e.
API A supports OData queries.
API B calls the methods that support OData queries in API A.
API B is exposed to the client. When the client calls the method from API B
with OData $Filter, the result has to be returned.
How can I support the OData options in API B?
Thanks in advance.
Well, I'm not sure I get your question correctly but, from what I understand, you just want to proxy the API calls to MS Graph and make some changes on the fly to the response.
OData queries are just simple query parameters (see the OData tutorial). So, basically, you just have to get those query parameters in your proxy and forward them to the MS Graph. The response you'll get will then be compliant with the original query.
However, depending on how you mangle the data, you may end up not being compliant with the user query. For example:
The user made a $select(Id) query, but your logic add a custom property Foo. The user just wanted Id but you added Foo anyway.
The user made an $orderby Name asc query, but your logic modify the property Name. It may not be ordered after your logic.
The user wants to make $filter query on the Foo property. MS Graph will complain because it doesn't know the Foo property.
Etc.
If you want to handle that cases, well, you'll have to parse the different OData queries and adapt your logic accordingly. $orderby, $top/$skip, $count, $expand and $select should be pretty straight-forward ; $filter and $search would require a bit more work.
Thanks. I was looking for a solution to this.
https://community.apigee.com/questions/8642/how-do-we-fetch-the-query-parameters-odata-standar.html
Instead of parsing the URL to get the OData query parameters, i wanted to understand the standard method to process the OData requests.
Now I am doing the below to extract the OData query parameters and passing them to MSGraph API.
string strODataQuery = String.Join("&", HttpContext.Request.Query.Where(kvp => kvp.Key.StartsWith("$")) .Select(kvp => String.Format("{0}={1}", kvp.Key, Uri.EscapeDataString(kvp.Value))));
And I am performing the conversions after retrieving the results.
Regards
I have to design a REST API in which a search request can take parameters for multiple Queries ( i.e. when the client make a call using this API, he should be able to send parameters to form multiple queries).
We have an existing API where we are using GET and it takes multiple parameters which together forms a single Query and then this API call returns the response for this query.
e.g. currently I can pass firstName, lastName, age etc in the request and then get back the person.
But now I have to enhance this service(or have a separate service) where I should be able to send parameters like firstName1, lastName1, age1 to search person1 ; firstName2, lastName2, age2 to search person2 and so on.
Should I use POST for the new API and then send list of parameters(params for query1, params for query2 and so on)?
Or is there a better approach.
We are using Spring Boot for REST implementation.
Its better to use POST because GET is good for 2,3 parameter but when you have a set of parameter or object then POST is Good.
The best thing to do here will be do POST and then return a JSON object with all the details of the Person in an array.
That way it will be faster and you would not have to deal with long urls for GET.
Also GET has limitations regarding the length of the request whereas there is no such limitation in case of POST.
It is really hard to give a right answer here. In general sending a GET request does have the advantage that you can leverage caching easily on a HTTP level, e.g. by using products like varnish, nginx, etc. But if you already can forsee that your URL including all params you'll have to send a POST request to make it work in all Browsers.
RESTfull architecture should respect the principle of addressability.
Since multiple users can be accessed through a unique request, then ideally this group of user should get an address, which would identify it as a resource.
However I understand that in the real world, URIs have a limited length (maximum length of HTTP GET request?). A POST request would indeed work well, but we lose the benefit of addressability.
Another way would be to expose a new resource : group,.
Lets suppose that your current model is something like this :
.../users/{id}
.../users/search?{arg1}={val1};{arg2}={val2}
You could eventually do something like :
.../users/groups/
.../users/groups/{id}
.../users/search?group={id}
(explanation below)
then you could split your research in two :
first a POST on .../users/groups/ with, as proposed by other response, a JSON description of the search parameters. This request could scan the .../users/groups/ directory, and if this set of parameters exists, return the corresponding address .../users/groups/{id}. (for performance issues you could for instance define {id} with a first part which would give the number of users requested).
Then you could make a request for this group with a GET with something like this : .../users/search?group={id}.
This approach would be a bit more complex to implement, but is more consistent with the resource oriented paradigm.
I am working on a REST service and so far all the queries are retrieved using a GET request.
Right now we are using a sort of routing rule like this one:
API/Person/{id} GET
http://api.com/person/1
Now, what if I want to ask to the REST API "Give me a Person with FisrtName = 'Pippo'"
I have a complex DTO that I called PersonQueryDTO that can be sent to the REST method to interview the database using the query criterias.
Is this a good way to do it or should I build complex queries in a different way?
For me it's important to keep the REST principles.
If you want to stick with REST principles, then the way to do something like that is to supply additional parameters in the URL e.g.
GET API/Person?FirstName=SomeName
REST is all about identifying resources, API/Person identifies your collection of Person and the additional parameters are nothing but meta data which the service can use internally to determine what sort of result to return.
I need to design and implement a REST API where users need to pass many input parameters. Out of those input parameters few are collection of an integer, few of them are date strings etc. After getting all these parameters I need to return unique id in the response. What method type (PUT, POST or GET) I should use in order to implement this API? How can I pass all these parameters to the API? I don't want users to format input parameter list into XML or JSON and post as a request body.
I appreciate if anybody can help on this topic.
POST is for creating new resources.
PUT is for updating existing resources. A PUT call should be idempotent, i.e. issuing the same request twice will end in no side effects.
To get an overall clue on how RESTful services work, read this article.
And yes, if you want your users to submit a complex set of parameters JSON/XML is the best way to go of course.