HTTP REST Response in CQRS - rest

I am building HTTP REST API and implementing CQRS, DDD, ES, and microservices concept for the API.
For example, I want to build API for ordering system. The order contains order status, customer data, employee data, cart items, shipping data. The order command service and order query service are using different code, different databases. and different ip / port.
Customer data consists of customer id, first name, last name, contact numbers, etc. Employee data like sales and staff consists of employee id, email, name, and role. Shipping data consists of shipping address, city, zipcode, shipping note, shipping method, and shipping cost. Cart items contains product id and qty. Customer data, product data, employee data, and shipping method data are produced from other services.
I have two questions when I want to developing it.
When I build the POST method to create new order from sales / staff, what should be in a request payload? Is it possible to only send customer id and the employee id as request payload? Or should I send the customer name and contact numbers as payload?
When I build the POST / PUT method, should I return response code 200? What should be in the response? For example, I only send the customer id and employee id when a staff create new order to the order command service. Should the response body shows the full order data that contains other services result (like name and contact numbers from customer id, and so on)? Not only that, the denormalizer system is rather slow. The denormalizer is needed to get the customer and employee detail. It depends on network latency and other services responses. But, user want to get the order data instantly.
What should I do to develop HTTP REST API in a correct way?
Thank you.

I am building HTTP REST API and implementing CQRS, DDD, ES, and microservices concept for the API.
Cool. Keep firmly in your mind that the REST API part is part of your integration component; its purpose is to ensure that old clients can continue to take advantage of your services, even as the implementation of those services evolves.
Jim Webber, Domain-Driven Design for RESTful Systems
The web is not your domain, it's a document management system. All the HTTP verbs apply to the document management domain. URIs do NOT map onto domain objects - that violates encapsulation. Work (ex: issuing commands to the domain model) is a side effect of managing resources. In other words, the resources are part of the anti-corruption layer. You should expect to have many many more resources in your integration domain than you do business objects in your business domain.
When I build the POST method to create new order from sales / staff, what should be in a request payload?
Note the point above about "many more resources" -- you can provide as many different ways (protocols) to create a new order as you are willing to support.
From the point of view of microservices, placing an order probably touches on a number of different business capabilities. See Udi Dahan's writings on UI Composition Techniques, the same concerns have play in machine to machine interfaces.
So from the point of view of the web endpoint that ends up communicating the order with the service(s), the information that you need is going to depend on what the exposed endpoints need.
From the point of view of the client: part of the point of providing a REST API is that the integration resources can guide the client to construct the final submission. Think online shopping experience - Amazon doesn't ask you to type in a bunch of SKUs, or customer ids. You just submit a sequence of forms until you get to a screen that lets you make one last check of the data, and then you submit it if it looks right.
So one of the screens along the way could be something that helps the client figure out what the employee id should be (here's a search form, here's a list to choose from, etc).
When I build the POST / PUT method, should I return response code 200?
Generally speaking, 200 is fine. Unless you are anticipating clients that react cleverly to different forms of success, that will cover most of your cases.
202 Accepted is a finer grained status that's often used when processing is not going to be completed immediately. "Hi, we got your order. Here's a link you can use to check on progress."
For instance, if you were using an architecture where the REST api was going to simply write the order request into a queue for the microservice consumers to pick up, then 202 is conceptually accurate "we wrote it down, and the services will decide for themselves what to do about it".
user want to get the order data instantly.
Instantly isn't a latency that you are ever likely to meet, so you'll want an SLA that's a bit more forgiving.
I wouldn't expect that kind of a problem; after all, if your design is CQRS, you can have a denormalized representation of a lot of the data available in a hot cache. If that representation is independently addressable, and cacheable, you may be able to re-use a copy that's stored on the client.
But, how the backend API know that user wants to buy that product to an order? We receive the SKU as payload, don't we?
Not necessarily -- we receive something the REST API can use to calculate the SKU. It could be something else. But a SKU is fine: the main point from the perspective of the client is that the message formats didn't change; the main point from the perspective of the shopper is that she didn't have to care what was in the payload, because she had stable semantic cues to work from (ie: she was looking at the product name, but what the form submitted under the covers was the SKU).
What is the ideal way to map the HTTP verbs, URI, and domain objects?
Other way around: what's the right way to make your backend service look like a web site?
Ever use your web browser to configure a wireless router? How does that work -- a routers internals are nothing like a web site. There's no reason that the app that you use to ask questions on stack overflow should work as a router configuration tool. Yet it does, because the API on the router knows how to duck like a web site while at the same time sending the right instructions to the back end.
The REST client is only looking at the messages it receives; it doesn't know anything about domain objects; it only knows about representations of resources (think documents describing things).
The ideal way to map the HTTP verbs, and so on, it to pretend to be a dumb document store.

This is a bit of a late response, yet it might be helpful for someone who is looking how to implement CQRS architectural pattern via HTTP.
During my discovery process, I decided to share my findings and ended up writing a series of articles on how to do CQRS via HTTP.
CQRS: Intro
CQRS: Commanding via HTTP
CQRS: Querying via HTTP
I particularly want to point out that RESTful architecture is conceptually different from CQRS, thus not a good fit for a purpose. I've tried to give an exhaustive comparison of two architectures in the Commanding via HTTP article.
Apart from the theoretical aspect, during the writing process, I managed to build two ready-to-use, independent middleware that are built to help you adopt CQRS in your ASP NET Core project. HTTP Commanding and HTTP Querying.
Hope it helps, if you have any questions, don't hesitate and shoot me a message.

Related

Are REST Webservices stateful?

We are talking about REST Webservices in my university right now.
There came the terms "Application" and "Interaction" up.
I totally understand Interaction, but not quite Application.
Does anybody know what Application means in the context of REST Webservices?
There came the thesis "REST Webservices" are stateful up.
This was explained with self describing messages.
The argument was, that in the self describing messages the required data format is described.
I do not find any proof of that on the Internet, rather the opposite.
Can anybody explain both terms and explain to me why REST Webservices should be stateful?
Cheers,
Henrik
Heuristic: anytime you are thinking about REST, consider the web as an example. How does it work with web pages and browsers?
State:
Resources (web pages) are stateful. The representations (bytes) can change over time.
Messages (http requests and responses) are stateless - you don't need to know anything about "session" or "context" to understand the message. Alternative spelling - the meaning of a request is completely independent of any previous request.
Fielding discusses applications and application state when discussing the REST data view
... looking-up a word in an on-line dictionary is one application, as is touring through a virtual museum, or reviewing a set of class notes to study for an exam. Each application defines goals for the underlying system, against which the system's performance can be measured....
An application's state is therefore defined by its pending requests, the topology of connected components (some of which may be filtering buffered data), the active requests on those connectors, the data flow of representations in response to those requests, and the processing of those representations as they are received by the user agent.
An application reaches a steady-state whenever it has no outstanding requests; i.e., it has no pending requests and all of the responses to its current set of requests have been completely received or received to the point where they can be treated as a representation data stream. For a browser application, this state corresponds to a "web page," including the primary representation and ancillary representations, such as in-line images, embedded applets, and style sheets. Fielding, 2000

RESTful API design with CQRS

I'm aware of this Q/A. The answer doesn't help my situation much.
I'm designing a CQRS-based RESTful service that provides the current reputation of existing accounts. The reputation calculation can take up to 30 seconds.
Due to the long calculation time, and due to our desire to use CQRS on this for other reasons, the entire process is done in two (or more) requests:
FIRST REQUEST
A client tells the RESTful service to begin the calculation for the account.
The RESTful service basically responds "OK" and has another service start the calculation.
SUBSEQUENT REQUESTS
The client asks the RESTful service for the calculated reputation score, if available.
The RESTful service responds with either the reputation score, or a note that it's still calculating.
QUESTION #1
How should I structure the first request URI? For account number 12345, should it be something like one of the following?
PUT /accounts/12345 payload: {}
I'm concerned about this approach because I read PUT should be idempotent.
Another option:
POST /accounts/12345 payload: {} // ...but shouldn't POST contain the whole entity in the payload?
...or, maybe change the entity from an account to a command...
POST /command/reputation-calculation/12345 payload: {} // ...feels like we're getting off-course here
...or something else?
QUESTION #2
For the second request, that seems a bit more straightforward. Should the URI be something like this?
GET /accounts/12345/reputations
I appreciate your suggestions. Thanks.
I may have found an answer. It involves moving the CQRS away from the client, and into the control of the RESTful service, which may employ it optionally.
For each client request, the URI could be:
GET /accounts/12345/reputations
Upon receipt, the RESTful service could check to see if a recently-calculated reputation is available. If a recent reputation is available, the RESTful service replies with a 200 OK status and delivers the response payload containing the reputation.
If no recent reputation is available (nor is it in-process according to the calculating service), the RESTful service kicks into CQRS mode. It tells the calculating service to begin calculating the reputation.
Then, whether it initiated the calculation, or whether it found one already in process, it returns to the client a 202 Accepted with follow-up link.
This kind of async situation seems to be what was intended with the 202 Accepted response, per the docs.
REST is not a great fit for CQRS command-oriented systems. So if the client is command-oriented then just use HTTP as the transport.
Alternatively, the client could create a reputation-enquiry resource that has a status of calculating or ready. The back-end decides if the result of a recent calculation can be reused, in which case the status will be immediately ready. A score is supplied when the resource is ready.
How should I structure the first request URI?
The spelling of the URI doesn't matter -- the machines don't care very much. If you think about how generic clients cache documents, you might choose to use a target-uri specifically to invalidate one specific resource in the cache.
Using POST is a natural fit with the way HTTP caching works. Also, it's how we would do it in HTML, so you know it has a track record.
So if you do GET /accounts/12345 to view the current reputation score, then POST /accounts/12345 is a perfectly reasonable way to kick off the upgrade process.
If you were using hypermedia (which is one of the REST constraints) then the response you get from the POST would include in it the URI for the status monitor. So that URI could be anything you want (since the client just goes where you tell it to). The server will probably want to get a hint back to know which update you are asking about; so it could be GET /reputationUpdate/67890.
There might be advantages to having the status monitor in the same hierarchy as the account, so GET /accounts/12345/reputationUpdate/67890 would also be fine.
Spellings that are consistent with the web linking specification often make implementing the client or server easier, because you can grab a library off the shelf that understands templates.

Adding Data from UI to different microservices

Imagine you have a user registration form, where you fill in the form:
First Name, Last Name, Age, Address, Prefered way of communication: Sms, Email (radiobuttons). You have 2 microservices:
UserManagement service
Communication service
When user is registered we should create 2 aggregates in 2 services: User in UserManagementContext and UserCommunicationSettings in Communication. There are three ways I could think of achieving this:
Perform 2 different requests from UI. What if one of them fails?
Put all that data in User and then raise integration event with all that data, catch it in CommunicationContext. Fat events, integration events shouldn't contain domain data, just the ids of aggreagates.
Put the message in the queue, so both contexts would have the adapters to take the needed info.
What is the best option to split this data and save the information?
Perform 2 different requests from UI. What if one of them fails?
I don't think this would do. You are going to end up in inconsistent state.
I'm for approach 3#:
User is persisted (created) in your user store.
UserRegistered event is sent around, containing the ID of the user.
All interested parties handle UserRegistered event.
I'd opt for slim events, because your services may need different user data and its better to let them to get this data on their own rather than putting all the things into the event.
As you mentioned to store communication settings, assuming that communication data are supposedly not part of the bounded context of UserManagement Service.
I see a couple of problems with approach #3 proposed in other answer. Though I think approach #3 in original answer is quite similar to my answer.
What if more communication modes are added? Naturally, it should only cause Communication Service to change, not UserManagement Service. SRP. Communication settings MS should store all communication settings related data in its own datastore.
What if user updates his communication settings preference only? Why user management service should take burden of handling that? Communication settings change should just trigger changes in its corresponding micro-service that is Communication Service in our case.
I just find it better to use some natural keys to identify & correlate entities across micro-services rather than internal IDs generated by DB. Consider that tomorrow you decide to use completely different strategy to create "ids" of user for UserManagement service e.g. non-numeric IDs, different id generation algorithm etc. I would want to keep other micro-services unaffected of any such decisions.
Proposed approach:
Include API Gateway in architecture. Frontend always talks to API Gateway.
API Gateway sends commands to message queue like RegisterUser to be consumed by interested micro-services.
If you wish to keep architecture simple, you may go with publishing a single message with all data that can be consumed by any interested micro-services. If you strictly want individual micro-services to see only its relevant data, create a message queue per unique data structure expected by consuming services.

RESTful web requests and user activity tracking websites

Someone asked me this question a couple of days ago and I didn't have an answer:
As HTTP is a stateless protocol. When we open www.google.com, can it
be called a REST call?
What I think:
When we make a search on google.com all info is passed through cookie and URL parameters. It looks like a stateless request.
But the search results are not independent of user's past request. Search results are specific to user interest and behavior. Now, it doesn't look like a stateless request.
I know this is an old question and I have read many SO answers like Why HTTP is a stateless protocol? but I am still not able to understand what happens when user activity is tracked like on google or Amazon(recommendations based on past purchases) or any other user activity based recommendation websites.
Is it RESTful or is it RESTless?
What if I want to create a web app in which I use REST architecture and still provide user-specific responses?
HTTP is stateless, however the Google Application Layer is not. The specific Cookies and their meaning is part of the Application Layer.
Consider the same with TCP/IP. IP is a stateless protocol, but TCP isn't. The existence of state in TCP embedded in IP packets does not mean that IP protocol itself has a state.
So does that make it a REST call? No.
Although HTTP is stateless & I would suspect that www.google.com when requested with cookies disabled, the results would be the same for each request, making it almost stateless (Google still probably tracks IP to limit request frequency).
But the Application Layer is not stateless. One of the principles of REST is that the system does not retain state data about about the client between requests for the purpose of modifying the responses. In the case of Google, that clearly is not happening.
It seems that the meaning of "stateless" is being (hypothetically) taken beyond its practical expression.
Consider a web system with no DB at all. You call a (RESTful) API, you always get the exactly the same results. This is perfectly stateless... But this is perfectly not a real system, either.
A real system, in practically every implementation, holds data. Moreover, that data is the "resources" that RESTful API allows us to access. Of course, data changes, due to API calls as well. So, if you get a resource's value, change its value, and then get its value again, you will get a different value than the first read; however, this clearly does not say that the reads themselves were not stateless. They are stateless in the sense that they represent the very same action (or, more exact, resource) for each call. Change has to be manually done, using another RESTful API, to change the resource value, that will then be reflected in the next call.
However, what will be the case if we have a resource that changes without a manual, standard API verb?
For example, suppose that we have a resource that counts the number of times some other resource was accessed. Or some other resource that is being populated from some other third party data. Clearly, this is still a stateless protocol.
Moreover, in some sense, almost any system -- say, any system that includes an authentication mechanism -- responds differently for the same API calls, depending, for example, on the user's privileges. And yet, clearly, RESTful systems are not forbidden to authenticate their users...
In short, stateless systems are stateless for the sake of that protocol. If Google tracks the calls so that if I call the same resource in the same session I will get different answers, then it breaks the stateless requirement. But so long as the returned response is different due to application level data, and are not session related, this requirement is not broken.
AFAIK, what Google does is not necessarily related to sessions. If the same user will run the same search under completely identical conditions (e.g., IP, geographical location, OS, browser, etc.), they will get the very same response. If a new identical search will produce different results due to what Google have "learnt" in the last call, it is still stateless, because -- again -- that second call would have produced the very same result if it was done in another session but under identical conditions.
You should probably start from Fielding's comments on cookies in his thesis, and then review Fielding's further thoughts, published on rest-discuss.
My interpretation of Fielding's thoughts, applied to this question: no, it's not REST. The search results change depending on the state of the cookie header in the request, which is to say that the representation of the resource changes depending on the cookie, which is to say that part of the resource's identifier is captured in the cookie header.
Most of the problems with cookies are due to breaking visibility,
which impacts caching and the hypertext application engine -- Fielding, 2003
As it happens, caching doesn't seem to be a big priority for Google; the representation returned to be included a cache control private header, which restricts the participation by intermediate components.

Is an Email Service an appropriate "service" in a SOA?

Does it make sense to create a service whose only responsibility is to send emails for other services?
Let me try to express my doubts more clearly and give a little bit of context. BTW, you can ignore the term "SOA" if you like. My intent in including it was to communicate that I am talking about a distributed system that is partitioned by function.
The reasons why I am uncertain as to whether an "Email Service" is appropriate or not are:
It provides a technical function
rather than an organization
function. It doesn't compose the
email messages, it just processes
them. Would it make sense to have
the Email Service compose the
messages by responding to domain
events? Would this be beneficial or
harmful?
It seems to introduce dependencies
into all other services which
utilize it. Particularly, I can't
see how one could avoid RPCish
interactions between the client and
the Email Service. Even if you use
messaging, the messages would be of
the command style (telling the Email
Service to send an email) which as I
understand it are inappropriate for
communications between services
since they increase coupling due to
the knowledge the client has to have
about the service it is consuming in
order for it to tell it what to do.
Unless of course the Email Service
composes the messages in response to
domain events from other services
(see point 1).
It is questionable how much
"service" it provides. In other
words, isn't the SMTP server already
the "Email Service"? Of course, the
custom "Email Service" might provide
things like queuing and parsing of
delivery reports. How much and what
should be in the Email Service for
it to really be necessary?
The alternative would be to have each service within the organization be responsible for sending out it's own email messages. However, this would mean that each service would have to be dependent on the SMTP server, but is that any different from being dependent on a custom "Email Service"? It would also mean that each service would be responsible for queuing and delivery management. Is this beneficial or harmful?
In addition, the email messages are considered domain entities, meaning that the organization is interested in the messages themselves in addition to the events that initiated them and the information that they carried. This means that users will be interested in viewing the messages that were sent out within context. For example you might look at a customer's account and ask to view the messages that were sent to that customer (these messages might include: account created confirmation, order placed confirmation, order shipped notification, experience feedback request, etc.).
I apologize if my question makes certain assumptions or is unclear, but based on what I've written, can anyone suggest an approach or discuss an approach that they have taken and how it worked out? I've already looked around SO for similar questions and googled on the topic but did not find anything that really applied, but if anyone can point out any resources I would greatly appreciate it. I would also be interested in answers that point out things that I might be overlooking or misunderstanding. Any sort of discussion on the matter seems valuable to me.
It depends on the context. What is a service? Is it a technical resource or a business resource?
If you were working at a technical level, partitioning a large technical soluiton into smaller parts (separation of concerns, etc) then I'd agree that an email service might well fit into this.
If the services are business services (e.g: "customer credit check") then an email service wouldn't fit into this.
And of course there's no reason why you can have both: a "top" layer of business services, implemented by a (technical) solution (that is composed of various sub-systems and layers) which includes a collection of technical services.