When integrating across Bounded Contexts in DDD which of the following is considered better practice?
1) Publish events when an entity changes within a source BC, listen to those events in a consuming BC, shape that data into the entity required and store it within the consuming BC.
or
2) Make an API call synchronously to the BC that owns an entity when that information is required by another BC.
or is there another option that's considered better practice than the above?
If you are interested in autonomy, then you don't want to have services that require other services to be available.
So you should probably be thinking the other way around -- how does the consumer work when the remote data provider is unavailable is your primary use case, and then consider whether there are any enhancements to add when the data provider is live.
This typically means that the each service caches a copy of the data that it will need.
Having the consumers pull data that they need is commonly simpler than trying to push the data to them -- see Greg Young's talk on Polyglot Data.
I think that the question shouldn't be «API vs event», but «sync vs async», and it doesn't have to be with best or worst practices. It depends on your requirements about how you can integrate your BCs. It depends on your domain.
You can implement async integration with API instead of events, doing calls to the remote API every certain period, polling requests.
Related
I have an aggregate root with the business logic in a c# project. Also in the solution is a REST web.api project that passes commands / requests to the aggregate root to do work and handle queries. This is my microservice. Now I want some of my events / commands / request to come of a message queue. I'm considering this:
Put a console app in the solution to listen for messages from a message queue. Then reference the aggregate root project in the console app
Is it a bad pattern to share "microservice business logic" between two services? Because now I have two "services" an api and a console app doing the work. I would have to ensure that when the business logic changes both services are deployed.
Personally I think it is fine to do what I suggest, a good CI/CD pipeline should mitigate that. But are there any other cons I might have missed?
For some background I would suggest watching DDD & Microservices: At Last, Some Boundaries! by Eric Evans.
A bounded context is the micro service. How you surface it is another matter. What you describe seems to be what I actually do quite frequently. I have an Identity & Access open source project that I'm working on (so depending on when you read this it may be in a different state) that demonstrates this structure.
Internal to an organization one may access the BC either via a service bus or via the web-api. External parties would utilize only the web-api as messaging should not be exposed.
The web-api either returns data from the query layer or sends commands via the service bus (messaging) to the BC functional endpoint. Depending on the complexity of the system I may introduce an orchestration concern that interacts with multiple BCs. It is probably a BC in its own right much along the lines of a reporting BC.
I know that messaging system is non blocking and scalable and should be used in microservices environment.
The use case that i am questioning is:
Imagine that there's an admin dashboard client responsible for sending API request to create an Item object. There is a microservice that provides API endpoint which uses a MySQL database where the Item should be stored. There is another microservice which uses elastic search for text searching purposes.
Should this admin dashboard client :
A. Send 2 API Calls; 1 Call to MySQL service and another elasticsearch service
or
B. Send message to topic to be consumed by both MySQL service and elasticsearch service?
What are the pros and cons when considering A or B?
I'm thinking that it's a little overkill when only 2 microservices are consuming this topic. Also, the frequency of which the admin is creating Item object is very small.
Like many things in software architecture, it depends. Your requirements, SLAs and business needs should make it clearer.
As you noted, messaging system is not blocking and much more scalable, but, API communication got it pluses as well.
In general, REST APIs are best suited to request/response interactions where the client application sends a request to the API backend over HTTP.
Message streaming is best suited for notifications when new data or events occur that you may want to take action upon.
In you specific case, I would go with a messaging system with is much more scalable and non-blocking.
Your A approach is coupling the "routing" logic into your application. Pretend you need to perform an API call to audit your requests, then you will need to change the code and add another call to your application logic. As you said, the approach is synchronous and unless you're not providing threading logic, your calls will be lined up and won't scale, ie, call mysql --> wait response, then call elastic search --> wait response, ...
In any case you can prefer this approach if you need immediate consistency, ie, the result call of one action feeding the second action.
The B approach is decoupling that routing logic, so, any other service interested in the event can subscribe to the topic and perform the action expected. Totally asynchronous and scalable. Here you will have eventual consistency and you have to recover any possible failure.
Here is the background:
We have a cluster (of 3) different services deployed on various containers (like Tomcat, TomEE, JBoss) etc. Each of the services does one thing. Like one service manages a common DB and provides REST services to CRUD the db. One service puts some data into a JMS Queue, Another service reads from the Queue and updates the DB. There is a client app that makes a REST service call to one of the service that sets off creating a row in the db, pushing that row into a queue etc.
Question: We need to implement the client app so that we know at any given point in time where the processing is. How do I implement this in RcJava 2/Java 9?
First, you need to determine what functionality in RxJava 2 will benefit you.
Coordination between asynchronous sources. Since you have a) event-driven requests from one side, and b) network queries on the other sides, this is a good fit so far.
Managing a stream of data, transforming and combining from one or more sources. You have given no indication that this is required.
Second, you need to determine what RxJava 2 does not provide:
Network connections. This is provided by your existing libraries.
Database management. Again, this is provided in your existing solutions.
Now, you have to decide whether the firstlies add up to something you can benefit from, given the up-front costs of learning a new library.
I am developing micro-services, I am using Event Sourcing with CQRS pattern, in my case, If a user is deleted/ updated from one service I want it to publish an event and other service to subscribe it and delete the entries regarding that user from its db as well.
I wanted to ask how can I use pub/sub pattern in Event Sourcing, Which Event store can be used for it as currently I have seen some people using Azure Tables but how can it be used as pub/sub?
Which Event store can be used for it ...?
If you have the luxury of choosing the technology to use, then I would suggest you start out by looking into Greg Young's Event Store
Yes, that's the same guy that introduced CQRS to the world.
(You may also want to review his talk on polyglot data, which includes discussion of pull vs push based models).
how can I use pub/sub pattern in Event Sourcing
This use case naturally lays down on eventsourcing and if accurately to realize it, then the question about notifications will disappear by itself.
It is the best of all to realize interaction by means of the common bus. Each microservice realizing your aggregates or projections is connected in the uniform logical bus, and signed on all events, and also can send any events there.
Of course, when if the system is under a heavy load, it is necessary to do some to optimization, for example, to enter name spaces for events and to specify to the broker of the bus what events and to what microservice it is necessary to deliver. Also if some information is private for microservice, then it makes a sense to make private channel in the bus, however it isn't provided by the theory of eventsourcing, exactly the same as validation between aggregates.
Also thanks to the concept of the common bus, you also receive "as a gift" reactivity for clients of system, for example, of browsers. However you shan't subscribe for projections or statuses of aggregates, only for events. If server events aren't equal client, you can enter the intermediate entity on their broadcasting, however it is not responsibility of storage of events any more.
I have a few different RESTful services that are hosted on different servers which use different DBs. I have a few RESTful services which call multiple such services above in what is supposed to be a transactional unit. We end up with data consistency issues if any of these RESTful services fail. Is there a neat architectural way of orchestrating a rollback? Or is having transaction managers the way to go?
As a simplistic example, RESTful service 1 has a POST request which reduces item count of thingamajig by 1.
RESTful service 2 POSTs a payment. If service 2 fails, how can we cleanly implement a rollback on service 1, without having a new RESTful refund service (it is ok if this has to be the way to go). I am looking for an architectural answer to above issue, which is in keeping with REST principles.
Your answer: https://stackoverflow.com/a/1390393/607033 You cannot use transactions because by REST the client maintains the client state and the server maintains the resource state. So if you want the resource state to be maintained by the client then it is not REST, because it would violate the stateless constraint. Violating the stateless constraint usually causes bad scalability. In this case it will cause bad horizontal scalability because you have to sync ongoing transactions between the instances. So please, don't try to build multi-phase commits on top of REST services.
Possible solutions:
You can stick with immediate consistency and use only
a single webservice instead of two. By resources like database, filesystem, etc. the multi phase commit is a necessity. When you break up a bigger REST service and move the usage of these resources into multiple smaller REST services, then problems can occur if you do this splitting wrongly. This is because one of the REST services will require a resource, which it does not have access to, so it has to use another REST service to access that resource. This will force the multi phase commit code to move to a higher abstraction level, to the level of REST services. You can fix this by merging these 2 REST services and move the code to the lower abstraction level where it belongs.
Another workaround to use REST with eventual consistency so you can respond with 202 accepted immediately and you can process the accepted request later. If you choose this solution then you must be aware by developing your application that the REST services are not always in sync. Ofc. this approach works only by inner REST services by which you are sure that the client retry if a REST service is not available, so if you write and run the client code.
Another, probably the ugliest workaround to store every transaction
as a resource, so you could POST commits and rollbacks. I think this possible solution is not viable, because it would violate the uniform interface constraint. We would use POST /transactions/ {resource: "/forums/12/messages/45", method: "PUT", data: "..."} and POST /transactions/1/commit instead of for example PUT /forums/12/messages/45
Distributed transactions are complex and require each participating system to support a notion of rollback. In the case of your services, they would each have to support some form of rollback. Co-ordinating something like this in a distributed system may not be practical or advisable in a synchronous way. In a situation like this, you would want to asynchronously roll back and the system would eventually reach consistency at a certain point in the future. There are obviously many other details (timeouts, error handling, retries, etc.).
For more details on eventual consistency, check out the wikipedia entry here.
Basically the problem is that you need to use transactions in an environment (HTTP) that by default is not transactional - in DB sense (because in HTTP a transaction is a successful request - response cycle).
The content of #leeor response is fully correct, what I'd like to add is how I'd solve the problem from the design site.
So you need a single endpoint, may be /transactions. Via POST method you add a new transaction (with all the necessary details) that is immutable - after creation you can only ask for it's data/status via GET method. What can update the transaction status/data is the server itself only.
Under the hood (during transaction creation) a snapshot (that could be reversed later on) for each resource taking part in the transaction should be created. Then execution for all the operations should begin and in case of any fail all the snapshots should be reversed. You do not mention any technologies so it's really hard to advise something reasonable. What you'd need for sure is comprehensive logging.