Are there any best practices to minimize the exchanged data between (internal) microservices when calling the API of a service (aka Need-to-Know)?
How to achive something like this:
There are three services:
User
Notification (let's assume just email)
Shipping
When the notification service needs the email address of a user it queries the API of the user service and should get the email (and NOT the full data set).
When the shipping service needs the shipping address of a user it queries the API of the user service and should get the shipping address (and NOT the full data set).
Question:
Should this be handled inside the user service with kind of an ACL (what service "XYZ" is allowed to see)?
Using JWT for authentication, there is a need to exchange keys at all, so during the setup-phase these ACLs could be discussed between the teams.
Should this be handled inside the user service with kind of an ACL
I think this is the best option. You could delegate the actual authorization to a separate service which the User service can call with the identity of the caller and the "claim" the caller is making (eg "I am allowed to see Email Address for User"). The claims can be evaluated on a per call basis.
However, is is arguable whether you actually need to query the user service at all. It would mean a change to your design but imagine for a minute that the Notifications service already knew about the user, for example the user ID and email address, then the notifications service would not need to query anything to be able to do it's job.
In order for the notifications service to have the user data already, it is necessary for that data to have been sent to the notifications service at some point in the past. A good time to do this would be when the user is first created, or any time a user details are changed. The best way to distribute this kind of information is in the form of an event message, although you could have the distribution based on an http POST to the notifications service.
Related
We are developing a portal application where an already existing customer can register their account to see the details of their account (something like you have a credit card and then you register on Bank's portal to see the transactions details - here you are already a customer of the Bank). So when a user is coming very first time for registration then this whole registration flow is not authenticated (as user still doesn't have a username and Password- one will have this after the registration)
We also want that a customer can not have concurrent registrations i.e. if a customer opens multiple tabs (or uses Postman to call our registration API) then only one request should be allowed to register and all other be rejected. For this we have used a registration_session.
So when the first request comes, we find the customer from our master record and generates a GUID/UUID and save it as a registeration_session value against CustomerId as key in Redis (with set expiration). So if any other registration request comes for the same customer then we first search the Redis to see if there is a registeration_session value against the CusrtomerId, and if it exists we will reject this request saying that registration is already in progress.
Now my first question is this: is this behavior is Stateless or not for RESTful APIs? as I am kind of maintaining a request context via registration_session on the server. One may argue that I am not maintaining the application state, true; but if we see it like this: every other registration request has to check the status of any previous registration request - then this means we are no longer having statelessness as per REST principles as now two requests for registrations are no longer independent of each other.
Next requirement is of OTP generation and verify. During registration, we ask the user to identify oneself via an OTP sent to their mobile number (we already have their mobile number from their customer record). A user may request to resend a token multiple times but if use provides wrong input for 3 times, we will put the user's account in locked status. Also we want that once user has verified the OTP check successfully, then any other OTP generation request for the same registration session should not be allowed - as once OTP is verified successfully then to generate OTP again is futile operation (a malicious user may still want to do this via Postman/curl)
Now server has to maintain following information for OTP:
retry count for OTP verification, as the moment it reaches 3, account is to be locked.
verification status for OTP i.e. once it is verified, other request to generate the OTP for same registration session is not allowed
and my second question is this: Is it again violates the REST stateless principal, as it seems we are maintaining the context for requests and every request is dependent on the context of the previous request?
Or is there a gap in my understanding of application state and stored context and above mentioned scenarios do not break Restfulness of an API? OR we can not design Restful APIs for the above mentioned requirements?
Note: I have read enough questions on REST and Context state on SO, but none offered a solution that eliminates my confusion for the specific scenarios that I have asked.
I'm building an SOA architecture which consists of a simple NGINX-based API gateway which forwards calls from browser clients to an appropriate backend API based on their prefix, for example:
/auth/login will route the call to the login endpoint on the Authentication service
/users/update/widget-1 will route the call to the update endpoint on the Users service
etc.
Each service has its own datastore and follows SOLID design principles. I use events on a queue to keep services informed about interesting things that happen to data that they both know about. For example, both the Users service and the Authentication service need to store the user's email address as it's used for authentication and emailing. So when a user's email is changed I queue a 'user email change' event onto a User Events queue. The Authentication service subscribes to this queue and uses the event to keep itself up to date.
For simple events, I can include enough details in the event to avoid needing more information. But, thinking ahead, what if a lot of changes have happened to the user and I have a datawarehouse that subscribes to every event type. I don't want to start having huge events - I would rather just include enough information for the interested service to use the event to trigger a call to ask for more details.
So the sequence in this example would be:
Client synchronously calls user update with JWT bearer token
User update service validates JWT and uses it to carry out the update
User update service generates a 'user updated' event to the queue, containing the User ID
Datawarehouse picks up the event and calls a 'get user details' endpoint on the User service to get full details of the update.
How do I authenticate the 'internal service call'? I can't use the original JWT as the internal request is happening asynchronously and the calling service doesn't have the JWT. It might not even be valid any more by the time the Datawarehouse requests the user details. It feels like I need some 'internal' JWT - for example, in this case, would the answer be for the Datawarehouse service to have the ability to generate its own JWT with its own private key then the User service checks the signature using the Datawarehouse service's public key? In which case, doesn't this mean each service would have to know about all the other services that could call it?
If it helps, my current implementation uses Lumen for the services with the jwt-auth package to check the JWT at the API level.
Any advice is appreciated, thanks.
Here is my scenario. Imagine there is a Yoga studio that uses a professional booking and reservation system that exposes an API. Through this API an application can make a reservation for a client. The API takes the client's userid and password to make the reservation. The booking API doesn't use OAuth or any social media sign-ins.
My desire is to create an Assistant Action that would retrieve the list of classes and allow the client to make a booking.
My puzzle is what design/architecture to look towards to supply the userid/password pair required by the booking API.
How have others solved this puzzle?
Should I store the userid/password as "user state" associated with the action?
First, you should have a conversation with the API provider about why they don't provide an OAuth-based solution. This is a security vulnerability waiting to happen, if it hasn't already.
Second, you need to think very carefully about your own risk profile in this case:
Google does not allow you to collect credential information (ie - passwords) through your Action.
Because of this, you must use Account Linking to authenticate them.
This means that you will need something (ie - a database or data store) to manage their account on your side.
This database would be a good place to keep the username/password you need to use for them for the API...
...but it now means that you need to take extreme care about protecting this database.
You don't really say how this API allows for accounts to be created and managed. If these accounts are just used for you (ie - the user doesn't necessarily see them), then you can mitigate some of that risk by treating the username/password as an opaque token that you manage and generate and that the user never sees.
If this is something that the user is aware of, then you'll need to approach the account linking in one of two ways:
Have them log into your service via an app or webapp using this credential info that you will need to save (ack!) and then link to the Assistant using OAuth.
Have them log into your service via an app or webapp using Google Sign-In, which will carry over to your Action. Then have them provide the credential info for the API, which you will need to save (ack!).
I'm in the process of implementing a user management Microservice (MS) and wanted to find out whether what I'm doing is ok. Users are created from the UI, which interacts with an API. The API makes an RPC call to the user management MS, and publishes a CreateUserCommand to an InMem-bus. The consumer then handles the command by then creating a user in the DB, but then I need this user also registered within Auth0 - would the way to go about this be to send a different command to a persistent queue, for a subscriber to pick it up and register that user with Auth0 (persistent queue in case can't reach Auth0). Once that completes successfully, I could then publish a UserCreatedEvent?
Any help with this would be much appreciated.
You have two Bounded Contexts: User management and Authentication.
User management BC deals with the life-cicle of a user (creation, mutating and deletion).
Authentication BC deals with how the users identify themselves in the system.
So, it is a valid assumption that a user can exists even if it has (yet) no possibility to identify himself in the system.
That being said, you should emit the AUserWasCreatedEvent immediately after the User management BC processes the CreateUserCommand because in that moment the user is born. It has an ID, let's name it UserID, so it exists.
Then, this user needs a mean to identify himself and a Saga (or Process manager or whatever you want to call it) catch the event and create a CreateAuth0UserCommand that it is sent to the Authentication BC by calling the Auth0 API. The API respond with some data, possibly including a token; that token is handled by the Authentication BC and it is associated with the UserID.
Context
I m having two microservices :
User that manages users (crud operations)
Billing that manages billing informations, with a reference to a user
Actually, for me (tell if I m wrong) it's a good idea to store user informations into billing data using hateoas. So we can "walk through it" with an hyperlink in the response of the API right ?
We could obtain something like :
billing:{
// some informations
_links:{
owner:"http://80.80.80.80:7000/users/123456789"
}
}
Questions
How should I do, to create a new billing ? In fact, when somebody post a new billing on the microservice, he sends the user too. Does it mean that I need to have a UserEntity in my Billing service AND my User Service ? So the billing service will be able to marshall the request, meaning code duplication between the two services ? Or should I do something else ?
Then, is this the role of the front end (API consumer) to make 2 requests (one for billing, and one for the user related to the billing) to get the ressource ? Or should the BillingService get the User before responding to the front ?
I have seen in an article, that it's a good thing to use amqp / bus when dealing with microservice, to know if a ressource exists, or to check if it exists. Now we need a service container/registry to dynamically discover other services. In my case I use Zookeeper. But how can I do to tell Zookeeper "give me the location of the service(s) related to the ressource with hateoas links : http://80.80.80.80:7000/users/123456789" ? Am I missing an important information in my hateoas schema ?
How should I do, to create a new billing ? In fact, when somebody post
a new billing on the microservice, he sends the user too. Does it mean
that I need to have a UserEntity in my Billing service AND my User
Service ? So the billing service will be able to marshall the request,
meaning code duplication between the two services ? Or should I do
something else ?
The user that the billing service need is not the same one in the user service. Usually, the user's identity is all the consumer need to post a new billing. If the billing service need more information of the user, it may query from the user service. There may be some code duplicates here, but the code plays different roles in each service which means they can evolve without disrupting each other. Some questions may explain further here: Bounded contexts sharing a same aggregate, Handling duplication of domain logic using DDD and CQRS
Then, is this the role of the front end (API consumer) to make 2
requests (one for billing, and one for the user related to the
billing) to get the ressource ? Or should the BillingService get the
User before responding to the front ?
I think it brings the most flexibility to let API consumer navigate the links. What if the consumer is not interested in the owner detail?
I have seen in an article, that it's a good thing to use amqp / bus
when dealing with microservice, to know if a ressource exists, or to
check if it exists. Now we need a service container/registry to
dynamically discover other services. In my case I use Zookeeper. But
how can I do to tell Zookeeper "give me the location of the service(s)
related to the ressource with hateoas links :
http://80.80.80.80:7000/users/123456789" ? Am I missing an important
information in my hateoas schema ?
Not quite understand this one. If the consumer has the link like "http://80.80.80.80:7000/users/123456789" already, it can access the resource directly. Why should it ask the zookeeper? I think the zookeeper helps the billing service assemble the URI for owner. For example, the billing service tell the zookeeper "Give me the location of the service related to user resource".
Another solution would be that you store all the necessary information in both services. E.g., if you need data of the user within a billing, then just store all the data in the billings datastorage as well. The sync between both services you would do through a queue ( subscribe / publish ). This comes with pros and cons but in the end you end up having one synchronous http call, if you want receive the data for a specific billing.