I develop microservice which communicate within RESTful style.
Is there any ready-to-use frameworks for throttling mechanism implementation?
I use Spring Boot.
Throttling here is a mechanism to reduce frequency of outgoing requests by filtering, for example, duplicate requests.
In my case I think I should use a cache (don't know which one) to filter out duplicate request which was handled already.
And in what period the cache should be cleaned? (daily, hourly, etc)
Please hint me where to dig.
Spring Boot provides the #Cacheable and similar annotations which indicate that and how a method result should be cached. E.g. #CachePut(value="addresses", condition="#Person.name=='Tom'")
public String getAddress(Person person) {...} will cache the result if the parameter person's name was "Tom". This is an example of how filtering could be implemented.
Since you're using Spring Boot you can pull spring-boot-starter-cache in as a dependency and it will provide you with the annotation and the cache.
It's also possible to define how frequently the cache should be evicted, e.g. with Spring's #Scheduled annotation, the specific frequency being dependent on your use case.
This was just a small insight into the following references:
https://www.baeldung.com/spring-cache-tutorial
https://www.baeldung.com/spring-boot-evict-cache
Related
One of the benefits being promoted for API-first design or OpenAPI is that of their use as a single source of truth. To my mind, these schemas only serve as a contract - the actual source of truth for your API lies in your microservices implementation (typically a http endpoint).
How can OpenAPI claim to be a single source of truth when the contract cannot be enforced until the implementation on the API side is complete? I realise there is tooling available to assist with this, such as validation middleware that can be used to match your request and response against your schema, however this is typically only validated at the point that a network request is made, not at compile time.
Of course you could write api tests to validate, but this is very much dependent on good test coverage and not something you get out of the box.
TLDR - OpenAPI markets itself as being a single source of truth for APIs, but this simply isn't true until your API implementation matches the spec. What tools/techniques (if any) can be used to mitigate this?
Did a bit of additional research into available tooling and found a solution that helps mitigate this issue:
open-api-backend (and presumably other such libraries) have capabilities to map your api routes/handlers to a specific openAPI operation or operationID. You can then enforce schema validation such that only routes defined in the spec can be implemented, else a fail-fast error is thrown)
I really like the way Spring cloud function decouples the business logic from the runtime target (local or cloud) and makes it easy to integrate with serverless providers.
I plan to use SCF with AWS Lambda behind an API gateway to design the backend of a system.
However, I am not completely clear on what is the recommended way to handle REST related parameters such as Query params, headers, path etc. inside the Spring cloud functions.
As per our initial analysis, we could derive two possible approaches:
When enabling “Lambda proxy integration” in API Gateway, Query params and other information are available as Message headers inside the SCF.
We can use “Mapping templates” in API Gateway to map all the required information into a JSON body and deserialize as a POJO to take input directly into the SCF.
This way, the SCF does not need to bother about how the required data is passed to the API.
What is the recommended way to achieve this? Are we missing something that enables to do this in a better way?
I don't think you are missing anything featurewise, except perhaps that it might also be convenient to work with composite functions - e.g. marshal|transform, where marshal is a Function<Message<?>, ?> and transform is the business logic. The marshal function could be generic (and convert to some sort of canonical form), and be provided as an autoconfiguration in a shared library (for instance).
There is a Java and CLI-interface to query Yarn RM for node-to-nodelabel (and inverse) mappings. Is there a way to do this via the REST-API as well?
An initial RM-API search revealed only node-label based job submissions as an option.
Sadly that is actually broken in MapR-Hadoop (6.1 as of 6/6/19), so my code has to work around that, by implementing the correct scheduling itself. This works (barely - more broken APIs here as well) using the YarnClient Java API.
But as I want to schedule jobs against different resource managers at the same time, behind firewalls, the REST-API is the most compelling option to achieve this, and the YarnClient API's RPC backend can't be easily transported.
My current worst-case solution would be to parse the YARN-WebUI in some way.
The only solution I found so far:
Request /ws/v1/cluster/nodes - this gets you all nodes.
FlatMap/Distinct on each node's nodeLabels, if you need just the list of node labels. Filter by nodeLabel, if you need all nodes for a specified label.
This does mean, that you always have to query all nodes, then sort/filter/arrange by NodeLabels, which is a lot of client-side magic. But apparently there's no GetNodesToLabel or even GetClusterNodeLabels to help us out.
I assume getLabelsToNodes is just a client-side implementation, as the protocol doesn't define the API, so that's right out the window for REST, unless implemented in the WebService.
Let's say that there are two microservices representing the resources orders(/orders) and customers(/customers). My requirement is to get all the orders made by a customer.
Had it been a monolithic application, I would have modeled my uri as /customers/{id}/orders. This would have hit the customers resource and made an in-memory service call to get the corresponding orders.
Now, in case of microservices, this isn't possible. So, is the only way to get the orders is to make a remote service call or is there a better way of doing it?
Can we create another resource with the representation /ordersByCustomers/{customerid}?
You can pass some query parameters as filters (this is the most common way I've seen). For example
/orders?customerId=123
I think that's quite clear, that you want to retrieve all customer orders filtered by customer id. In the same way you can add pagination or other filters.
The important thing to remember is that you want the order resource, so the URL should remain the same. I'm mentioning this, because this has been the most difficult thing for me to change... to think about resources rather than remote calls.
In general you should beware of using endpoint that are more or less similar to the one you suggested:
/ordersByCustomers/{customerid}
Why? Because this is not RESTful in general (even in microservices environment) and make the API difficult to understand and you by the consumers. What if you need orderByWhatever? Will you be introducing new endpoint every single time you need a new set of data? Try to avoid so opinionated endpoints.
What #Augutsto suggested is fully correct. If you're afraid of having a complicated logic in GET request this is the situation where you can break REST rules. I mean introducing:
POST /orders/filter/
Where all the filtering logic will be passed in requests body - so it's easier to carry complicated logic as well.
We're working on a restservice using Web-API and breaking our heads on what routing strategy to follow.
We got some resources:
Grades
Messages
Homework
(As a side note: We plan on using Hateoas to have linking between resources. )
We're considering Controller[Action][Id] resulting in
API\Grade[?personid] (GET/POST)
API\Grade\{id}[?personid] (GET/PUT/DELETE)
API\Grade\Lastgrades\{days}[?personid] (GET)
Or using a context
API\Student\Grade (GET)
API\Student\Grade\{id} (GET)
API\Student\Grade\Lastgrades\{days} (GET)
AND
API\Parent\Student\{id}\Grade (GET)
API\Parent\Student\{id}\Grade\{id} (GET)
API\Parent\Student\{id}\Grade\Lastgrades\{days} (GET)
AND
API\Teacher\Student\{id}\Grade (GET/POST)
API\Teacher\Student\{id}\Grade\{id} (GET/PUT/DELETE)
API\Teacher\Student\{id}\Grade\Lastgrades\{days} (GET)
Is there a good reason to use one strategy over the other?
In your 1st option, the focus is on grades, which is the resource that the API is designed to manage.
In your 2nd option, by just looking at the URL patterns, there are few more resources involved, like Teachers and Students.
Without really knowing your business use-cases, the answer to your question is more have do with the initial scope of your API .
If it is focusing on Grades but not Teachers nor Students, then you can just provide API to manage the ‘grades’ resources, meaning your option-1.
Later if you need to also manage “Teachers and Students”, you can add the options-2 to your implementation.
They don’t necessary to be mutually exclusive.
Update-1
The role/context should not be part of the URL. It should be handled separately like once each role is logged into the system; there should be already ways to associate with the role in the backend for example via session etc.
The URL design focus should be on the resources, in this case the Grades. Also since logically, Grades should be associated with Students or even associated with a class that a student took, I will suggest designing it like in the following
/api/v1/grades/students/[student-id]/
OR
/api/v1/students/[student-id]/grades/
OR
/api/v1/students/[student-id]/classes/[class-id]/grades/
These are all acceptable solutions and depending on your business use-cases, one may be better than other.
The roles only come to play as what CRUD operations can act on these resources, like Teacher can POST/PUT/GET on
/api/v1/students/[student-id]/grades/
But Students/Parents can only do GET on
/api/v1/students/[student-id]/grades/
You still need to consider the role/context as part of your overall design and perhaps how the URL design may make more sense depending on how you want to support the operations that each role will use the URL to manage the resources. But the 'context/role' should not be part of the URL.