I started with reading about CQRS and I'm little confused.
Is it allowed to call the read side within the write side for getting additional informations?
http://cqrs.nu/Faq/command-handlers here they say it is not allowed, but in the cqrs journey code I found that they call a service 'IPricingService' which internally uses a DAO service class.
So what I must do to get additional informations inside my aggregation root?
CQRS Journey should not be seen as a manual. This is just a story of some team fighting their way to CQRS and having all limitations of using only Microsoft stack. Per se you should not use your read model in the command handlers or domain logic. But you can query your read model from the client to fetch the data you need in for your command and to validate the command.
Since I got some downvotes on this answer, I need to point, that what I wrote is the established practice within the pattern. Neither read side accesses the write side, not write side gets data from the read side.
However, the definition of "client" could be a subject of discussion. For example, I would not trust a public facing JS browser application to be a proper "client". Instead, I would use my REST API layer to be the "client" in CQRS and the web application would be just a UI layer for this client. In this case, the REST API service call processing will be a legitimate read side reader since it needs to validate all what UI layer send to prevent forgery and validate some business rules. When this work is done, the command is formed and sent over to the write side. The validations and everything else is synchronous and command handling is then asynchronous.
UPDATE: In the light of some disagreements below, I would like to point to Udi's article from 2009 talking about CQRS in general, commands and validation in particular.
The CQRS FAQ (http://cqrs.nu/Faq) suggests:
"How can I communicate between bounded contexts?
Exclusively in terms of their public API. This could involve subscribing to events coming from another bounded context. Or one bounded context could act like a regular client of another, sending commands and queries."
So although within one BC its not possible to use read-side from write-side and vice-versa, another bounded context or service could. In essence, this would be acting like a human using the user interface.
Yes assuming you have accepted the eventual consistency of the read side. Now the question is where. Although there is no hard rule on this, it is preferable to pass the data to command handler as opposed to retrieving it inside. From my observation there are two ways:
Do it on domain service
Basically create a layer where you execute necessary queries to build the data. This is as straightforward as doing API calls. However if you have your microservices running on Lambda/Serverless it's probably not a great fit as we tend to avoid a situation where lambda is calling another lambda.
Do it on the client side
Have the client query the data then pass it to you. To prevent tampering, encrypt it. You can implement the decryption in the same place you validate and transform the DTO to a command. To me this is a better alternative as it requires fewer moving parts.
I think it depends.
If in your architecture the "command side" updates the projections on real-time (synchronously) you could do that calling the query api. (although that seems strange)
But, if your projections (query side) is updated asyncronously would a bad idea to do it. Would be a posibility to get a "unreal" data.
Maybe this situation suggests a design problem that you should solve.
For instance: If from one context/domain you think you need information from another, could a domain definition problem.
I assume this, because read data from itself (same domain) during a command operation doesn't make much sense. (in this case could be a API design problem)
Related
POST , PUT, PATCH and GET are all different. Idempotent and safety being the key difference makers.
While writing RESTFul APIs , I encountered guidelines on when and where to use one of the HTTP methods. Since I am using Java for the back-end implementation, I can control the behavior of the HTTP methods on the persistent data.
For example , GET v1/book/{id} can be replaced with POST v1/book (with "id" in body) now with that id I can perform a query on db , fetching that particular book. (assuming book with that id already exists).
Similarly , I can achieve the workings of PATCH and PUT with POST itself.
Now, coming to the question , why don't we just use POST instead of GET , PUT and PATCH almost every time, ALMOST, when we can control the idempotent and safety behavior in the back-end?
Or , Is it just a guideline mentioned in RESTFul docs somewhere or stated by Roy fielding and we all are blindly following? Even if the guidelines are so what is the major idea behind them?
https://restfulapi.net/rest-put-vs-post/
https://restful-api-design.readthedocs.io/en/latest/methods.html
https://www.keycdn.com/support/put-vs-post
Above resources just mention either what does all the methods do or their differences. Articles mention the workings as if they were some guidelines , none of the docs online speak about the reason behind them.
None of them says , what if I used POST instead of PUT, PATCH and GET, what would be the side-effects? (as I can control their behaviors in the back-end)
Http methods are designed in the way that each method holds some responsibility. I will say that REST are the standards which are conventions and not the obligation. The convention doesn't stress us to follow the rules but they are designed for our code betterment. You can tweak the things and can use them in your way but that would be a bad idea. Like in this case if you are performing all the three actions with one method it would create great confusion in code (As the simple definition of POST is to create an object and that is what understood by everyone) and also degrade our coding standards.
I strongly discourage to replace three methods with one.
If you do that, you can't say you are "writing RESTFul APIs".
Whoever knows the RESTFul standard, will be confused about the behaviour of your apis.
If you fit the standard, then you will have an easier life.
After all, you have no real benefit in your approach.
HTTP is a transport protocol which as its name suggest is responsible for transfering data such as files or db entries across the wire to or from a remote system. In version 0.9 you basically only had the GET operation at your disposal while in HTTP 1.0 almost all of the current operations were added to the spec.
Each of these methods fulfills its own purpose. POST i.e. does process the payload according to the server's own semantics, whatever they will be. In theory it could be used therefore for retrieving, updating or removing content. Though, to a client it is basically unclear what a server actually does with the payload. There is no guarantee whether invoking a URI with that method is safe (the remote resource being altered) or not. Think of a crawler that is just invoking any URIs it finds and one of the links is an order link or a link where you perform a payment process. Do you really want a crawler to trigger one of your processes? The spec is rather clear that if something like that happens, the client must not made accountable for that. So, if a crawler ordered 10k products as one of your links, did trigger such a process, and the products are created in that process, you can't claim refund from the crawler's maintainer.
In additon to that, a response from a GET operation is cacheable by default. So if you invoke the same resource twice in a certain amount of time, chances are that the resource does not need to be fetched again a second (third, ...) time as it can be reused from the cache. This can reduce the load on the server quite significantly if used propperly.
As you've mentioned Fielding and REST. REST is an architectural style which you should use if you have plenty of different clients connecting to your services that are furthermore not under your control. Plenty of so-called REST APIs aren't adhering to REST as they follow a more simple and pragmatic RPC approach with external documentations such as Swagger and similar. RESTs main focus is on the decoupling of clients from servers which allow the latters to evolve freely without having to fear breaking clients. Clients on the other hand get more robust to changes.
Fielding only added few constraints a REST architecture has to adhere to. One of them is support for caching. Though Fielding later on wrote a well-cited blog-post where he explains what API designers have to consider before calling their API REST. True decoupling can only occur if all of the constraints are followed strictly. If only one clients violates these premises it won't benefit from REST at all.
The main premise in REST is (and should always be): Server teaches clients what they need and clients only use what they are served with. In the browsable Web, the big cousin of REST, a server will teach a client i.e. on what data the server expects via Web Forms through HTML and links are annotated with link-relation names to give the browser some hints on when to invoke that URI. On a Web page a trash bin icon may indicate a delition while a pencil icon may indicate an edit link. Such visual hints are also called affordacne. Such visual hints may not be applicable in machine to machine communication though such affordances may hint on other things they may provide. Think of a stylesheet that is annotated with preload. In HTTP 2 i.e. such a resource could be attempted to be pushed by the server or in HTTP 1.1 the browser could alread load that stylesheet while the page is still parsed to speed things up. In order to gain whitespread knowledge of those meanings, such values should be standardized at IANA. Through custom extensions or certain microformats such as dublin core or the like you may add new relation names that are too specific for common cases but are common to the domain itself.
The same holds true for media-types client and server negotiate about. A general applicable media-type will probably reach wider acceptance than a tailor-made one that is only usable by a single company. The main aim here is to reach a point where the same media-type can be reused for various areas and APIs. REST vision is to have a minimal amount of clients that are able to interact with a plethora of servers or APIs, similar to a browser that is able to interact with almost all Web sites.
Your ultimate goal in REST is that a user is following an interaction protocol you've set up, which could be something similar to following an order process or playing a text game or what not. By giving a client choices it will progress through a certain process which can easily be depicted as state machine. It is following a kind of application-driven protocol by following URIs that caught the clients attention and by returning data that was taught through a form like representation. As, more or less, only standardized representation formats should be used, there is no need for out-of-band information on how to interact with the API necessary.
In reality though, plenty of enterprises don't really care about long-lasting APIs that are free to evolve over the years but in short-term success stories. They usually also don't care that much whether they use the propper HTTP operations at all or stay in bounds with the HTTP spec (i.e. sending payloads with HTTP GET requesst). Their primary intent is to get the job done. Therefore pragmatism usually wins over design and as such plenty of developers follow the way of short success and have to adept their work later on, which is often cumbersome as the API is now the driving factor of their business and therefore they can't change it easily without having to revampt the whole design.
... why don't we just use POST instead of GET , PUT and PATCH almost every time, ALMOST, when we can control the idempotent and safety behavior in the back-end?
A server may know that a request is idempotent, but the client does not. Properties such as safe and idempotency are promisses to the client. Whether the server satisfies these or not is a different story. How should a client know whether a sent payment request reached the server and the response just got lost or the initial request didn't make it to the server at all in case of a temporary connection issue? A PUT requests does guarante idempotency. I.e. you don't want to order the same things twice if you resubmit the same request again in case of a network issue. While the same request could also be sent via POST and the server being smart enough to not process it again, the client doesn't know the server's behavior unless it is externally documented somehwere, which violates REST principles again also somehow. So, to state it differently, such properties are more or less promisses to the client, less to the server.
I want to make an API using REST which interacts (stores) data in a database.
While I was reading some design patterns and I came across remote facade, and the book I was reading mentions that the role of this facade is to translate the course grained methods from the remote calls into fine grained local calls, and that it should not have any extra logic. As an explaination, it says that the program should still work without this facade.
Here's an example
Yet I have two questions:
Considering I also have a database, does it make sense to split the general call into specific calls for each attribute? Doesn't it make more sense to just have a general "get data" method that runs one query against the database and converts it into an usable object, to reduce the number of database calls? So instead of splitting the get address to get street, get city, get zip, make on db call for all that info.
With all this in mind, and, in my case using golang, how should the project be structured in terms of files and functions?
I will have the main file with all the endpoints from the REST API, calling the controllers that handle these requests.
I will have a set of files that define those controllers. Are these controllers the remote facade? Should those methods not have logic in that case, and just call the equivalent local methods?
Should the local methods call the database directly, or should they use some sort of helper class that accesses the database?
Assuming all questions are positive, does the following structure make sense?
Main
Controllers
Domain
Database helper
First and foremost, as Mike Amundsen has stated
Your data model is not your object model is not your resource model is not your affordance model
Jim Webber did say something very similar, that by implementing a REST architecture you have an integration model, in the form of the Web, which is governed by HTTP and the other being the domain model. Resources adept and project your domain model to the world, though there is no 1:1 mapping between the data in your database and the representations you send out. A typical REST system does have many more resources than you have DB entries in your domain model.
With that being said, it is hard to give concrete advice on how you should structure your project, especially in terms of a certain framework you want to use. In regards to Robert "Uncle Bob" C. Martin on looking at the code structure, it should tell you something about the intent of the application and not about the framework¹ you use. According to him Architecture is about intent. Though what you usually see is the default-structure imposed by a framework such as Maven, Ruby on Rails, ... For golang you should probably read through certain documentation or blogs which might or might not give you some ideas.
In terms of accessing the database you might either try to follow a micro-service architecture where each service maintains their own database or you attempt something like a distributed monolith that acts as one cohesive system and shares the database among all its parts. In case you scale to the broad and a couple of parallel services consume data, i.e. in case of a message broker, you might need a distributed lock and/or queue to guarantee that the data is not consumed by multiple instances at the same time.
What you should do, however, is design your data layer in a way that it does scale well. What many developers often forget or underestimate is the benefit they can gain from caching. Links are basically used on the Web to reference from one resource to an other and giving the relation some semantic context by the utilization of well-defined link-relation names. Link relations also allow a server to control its own namespace and change URIs as needed. But URIs are not only pointers to a resource a client can invoke but also keys for a cache. Caching can take place on multiple locations. On the server side to avoid costly calculations or look ups on the client side to avoid sending requests out in general or on intermediary hops which allow to take away pressure from heavily requested servers. Fielding made caching even a constraint that needs to be respected.
In regards to what attributes you should create queries for is totally dependent on the use case you attempt to depict. In case of the address example given it does make sense to return the address information all at once as the street or zip code is rarely queried on its own. If the address is part of some user or employee data it is more vague whether to return that information as part of the user or employee data or just as a link that should be queried on its own as part of a further request. What you return may also depend on the capabilities of the media-type client and your service agree upon (content-type negotiation).
If you implement something like a grouping for i.e. some football players and certain categories they belong to, such as their teams and whether they are offense or defense players, you might have a Team A resource that includes all of the players as embedded data. Within the DB you could have either an own table for teams and references to the respective player or the team could just be a column in the player table. We don't know and a client usually doesn't bother as well. From a design perspective you should however be aware of the benefits and consequences of including all the players at the same time in regards to providing links to the respective player or using a mixed approach of presenting some base data and a link to learn further details.
The latter approach is probably the most sensible way as this gives a client enough information to determine whether more detailed data is needed or not. If needed a simple GET request to the provided URI is enough, which might be served by a cache and thus never reach the actual server at all. The first approach has for sure the disadvantage that it doesn't reuse caching optimally and may return way more data then actually needed. The approach to include links only may not provide enough information forcing the client to perform a follow-up request to learn data about the team member. But as mentioned before, you as the service designer decide which URIs or queries are returned to the client and thus can design your system and data model accordingly.
In general what you do in a REST architecture is providing a client with choices. It is good practice to design the overall interaction flow as a state machine which is traversed through receiving requests and returning responses. As REST uses the same interaction model as the Web, it probably feels more natural to design the whole system as if you'd implement it for the Web and then apply the design to your REST system.
Whether controllers should contain business logic or not is primarily an opinionated question. As Jim Webber correctly stated, HTTP, which is the de-facto transport layer of REST, is an
application protocol whose application domain is the transfer of documents over a network. That is what HTTP does. It moves documents around. ... HTTP is an application protocol, but it is NOT YOUR application protocol.
He further points out that you have to narrow HTTP into a domain application protocol and trigger business activities as a side-effect of moving documents around the network. So, it's the side-effect of moving documents over the network that triggers your business logic. There is no straight rule whether to include business logic in your controller or not, but usually you try to keep the business logic in yet their own layer, i.e. as a service that you just invoke from within the controller. That allows to test the business logic without the need of the controller and thus without the need of a real HTTP request.
While this answer can't provide more detailed information, partly due to the broad nature of the question itself, I hope I could shed some light in what areas you should put in some thoughts and that your data model is not necessarily your resource or affordance model.
I am creating the high level design for a new service. The complexity of the service warrants using DDD (I think). So I did the conventional thing and created domain services, aggregates, repositories, etc. My repositories encapsulate the data source. So a query can look for an object in the cache, failing that look in the db, failing that make a REST call to an external service to fetch the required information. This is fairly standard. Now the argument put forward by my colleagues is that abstracting the data source this way is dangerous because the developer using the repository will not be aware of the time required to execute the api and consequently not be able to calculate the execution time for any apis he writes above it. May be he would want to set up his component's behaviour differently if he knew that his call would result in a REST call. They are suggesting I move the REST call outside of the repository and maybe even the caching strategy along with it. I can see their point but the whole idea behind the repository pattern is precisely to hide this kind of information and not have each component deal with caching strategies and data access. My question is, is there a pattern or model which addresses this concern?
They are suggesting I move the REST call outside of the repository
Then you won't have a repository. The repository means we don't know persistence details, not that we don't know there is persistence. Every time we're using a repository, regardless of its implementation (from a in memory list to a REST call) we expect 'slowness' because it's common knowledge that persistence usually is the bottleneck.
Someone who will use a certain repository implementation (like REST based) will know it will deal with latency and transient errors. A service having just a IRepository dependency still knows it deals with persistence.
About caching strategies, you can have some service level (more generic) caching and repository level (persistence specific) caching. These probably should be implementation details.
Now the argument put forward by my colleagues is that abstracting the data source this way is dangerous because the developer using the repository will not be aware of the time required to execute the api and consequently not be able to calculate the execution time for any apis he writes above it. May be he would want to set up his component's behaviour differently if he knew that his call would result in a REST call.
This is wasting time trying to complicate your life. The whole point of an abstraction is to hide the dirty details. What they suggest is basically: let's make the user aware of some implementation detail, so that the user can couple its code to that.
The point is, a developer should be aware of the api they're using. If a component is using an external service (db, web service), this should be known. Once you know there's data to be fetched, you know you'll have to wait for it.
If you go the DDD route then you have bounded contexts (BC). Making a model dependent on another BC is a very bad idea . Each BC should publish domain events and each interested BC should subscribe and keep their very own model based on those events. This means the queries will be 'local' but you'll still be hitting a db.
Repository pattern aim to reduce the coupling with persistence layer. In my opinion I wouldn't risk to make a repository so full of responsibility.
You could use an Anti Corruption Layer against changes in external service and a Proxy to hide the caching related issues.
Then in the application layer I will code the fallback strategy.
I think it all depends where you think the fetching/fallback strategy belongs, in the Service layer or in the Infrastructure layer (latter sounds more legit to me).
It could also be a mix of the two -- the Service is passed an ordered series of Repositories to use one after the other in case of failure. Construction of the series of Repos could be placed in the Infrastructure layer or somewhere else. Fallback logic in one place, fallback configuration in another.
As a side note, asynchrony seems like a good way to signal the users that something is potentially slow and would be blocking if you waited for it. Better than hiding everything behind a vanilla, inconspicuous Repository name and better than adding some big threatening "this could be slow" prefix to your type, IMO.
I have a question about REST in general.
Imagine I have a WCF webservice that is used to add an operation performed on a bank card.
The problem is that there are about 30 different parameters to pass on the WS.
On WCF that's pretty easy to do, calling a RPC with all those parameters.
The problem is that I wanted to switch this WCF WS to a REST API with ServiceStack.
The problem I encountered is that if i try to create the operation using REST and passing parameters through 'querystring', I have a string that is AWFUL for reading and VERY VERY LONG (?amount=1234&operationID=12& etc.).
I know this way of doing is not good as it's not resource oriented, but does that i mean i should split the creation of that item into SEVERAL steps (I mean, first create using POST then adding new infos/fields using several post ?).
In this situation I can't see clearly the gain with REST.
If you are passing these parameters in a query string I assume you are performing an HTTP GET. In a REST API GET's are generally reserved for getting data back and the only parameters you pass in are to filter your results. If you are performing an operation that changes the state of the system you want to perform a POST or PUT and pass the data in the body of the message as either XML or JSON, not in the query string.
The gain with REST is if you are opening this API up to other as it makes it much more portable to heterogeneous systems and there are some performance benefits. It also opens your API up to being used by clients such as web browsers. But if this API is just for internal use with .NET application that is not run in a browser then you may want to stick with WCF. REST is not the answer for every problem.
I am not sure to understand your question... REST doesn't mean "no payload". On the contrary, REST means "representational state transfer", so the body of HTTP requests (aka "representational state") is essential.
For a lot of reasons, in the case of a bank, resources are usually bank operations. CouchDB's guide has a very nice scenario about that.
In other words, your "parameters" would be the attributes of the resource representation (in JSON, XML or what you want) you would GET, POST, PUT or DELETE.
Is exposing CRUD operations through SOAP web services a bad idea? My instinct tells me that it is not least of which because the overhead of doing database calls overhead could be huge. I'm struggling to find documentation for/against this (anti)pattern so I was wondering if anyone could point me to some documentation or has an opinion on the matter.
Also, if anyone knows of best practises (and/or documentation to that effect) when designing soap services, that would be great.
Here's an example of how the web service would look:
Create
Delete
Execute
Fetch
Update
And here's what the implementation would look like:
[WebMethod]
public byte[] Fetch(byte[] requestData)
{
SelectRequest request = (SelectRequest)Deserialize(requestData);
DbManager crudManager = new DbManager();
object result = crudManager.Select(request.ObjectType, request.Criteria);
return Serialize(result);
}
If you want to use SOAP in a RESTful manner then there is a interesting standard for this, WS-Transfer; which provides loosely coupled CRUD endpoints; from which you inspect the message and act upon your entities accordingly.
Then you can layer whatever else you want on top, WS-Secure, WS-Reliable messaging and so on.
I think publishing a SOAP service that exposes CRUD operations to anonymous, public "users" would be a particularly bad idea. If, however, you can restrict one or both of these caveats, then I see nothing wrong with it (moreover I've implemented such services many times).
You can require, in addition to whatever method parameters your require to perform the operation, username & password parameters that in effect authenticates the originator prior to processing the request: a failure to authenticate can be signalled with the return of a SOAP exception. If you were especially paranoid, you could optionally run the service over SSL
You can have the server solution that deals with sending and receiving the requests filter based on IP, onyl allowing requests from a list of approved addresses.
Yes, there are overheads to running requests over SOAP (as opposed to exposing direct database access) - namely the processing time to wrap a request into a HTTP request, open a socket & send it (and the reverse at the receiving end and the again for the response) - but, it does have advantages.
Java (though the NetBeans IDE) and .Net (through VS), both support consumption of Web Services into projects / solutions - the biggest benefit of this is that objects / structures on the remote service are automatically translated into native objects in the consuming application, which is exceptionally handy.
If all you want to do is CRUD over the web, I'd look at some different technologies for doing REST instead of using WS*. SQL Data Services (formerly Project Astoria) might actually be a good alternative.
There is nothing wrong with exposing the CRUD operations via SOAP web-services per se.
You will obviously find quite a lot of examples for such services.
However depending on your particular requirements you might find that for you using SOAP is too much overhead or that you could be better off using use JSON/AJAX etc.
So I believe that unless you will provide additional details about your particular details there is no good answer for your question.