I am trying to wrap my head around Saga's using Jonathan Olivers EventStore and CommonDomain. I understand how Aggregates are working with the CommonDomain/EventStore but I am stuck on grasping Saga usage. I have read both of Jonathan's Saga's with Event Sourcing Part I & II but sill lost in actual implementation
1) More of observation, when persisting the saga the EventStore is utilizing the Headers to persist the Saga and Commands that need to be sent out and it looks like the Payload is storing the Event that triggered the Saga to "wake up". Wondering reasons for this. Would we never want to store individual commands vs having them all in the header?
1) It seems like the Event that triggered the Saga gets replayed multiple times since the "Transition" method in SagaBase always re-adds the event to uncommitted collection. (Unlike ARs that have an internal Apply method vs public Domain method). Maybe I am not using the Transition method properly
2) Typically the bus that you use with the EventStore will publish Events (I implemented IPublishMessages). If I need my Saga to publish a command there does not seem to be a Send option. Do I need to parse the Headers to grab the commands myself?
I am thinking I am using the CommonDomain / EventStore incorrectly as working with Aggregates was easy but Saga's seem "incomplete" to me. I am assuming its because I am not doing it correctly. Still very new to CQRS. Does anyone have a working example of Saga's using J Olivers Common Domain / Event Store? I think that would clear things up considerably.
[EDIT]
I think I figured it out but would like some input. Saga's really should not be publishing events. They send out commands. Thus on the publish side of things for the EventStore (IPublishMessages) I should first be checking the type of message (AggregateType vs SagaType) For AggregateTypes I can publish Events but for SagaTypes only publish the commands (found in Header). This eliminates the same event (say OrderSubmittedEvent) that triggers the creation of the Saga to not publish it again when persisting the saga.
The commands are put in the headers to be sent on the bus. The EventStore is concerned with the storage of events so the events that caused saga transitions are persisted. Later, when the saga is loaded from the event stream, the events will be passed to the saga's transition method to bring it to the current state.
The transition method serves a dual purpose in the saga implementation. Transition is called to handle incoming messages and to load the saga from peristence. In SagaEventStoreRepository.BuildSaga ClearUncomittedEvents and ClearUndispatchedMessages are called on the saga after the current state is built up thus avoiding duplicate event and command processing.
I haven't personally done this but I would use a separate EventStore instance for my sagas. This would allow for the usage of a separate IPublishMessages implementation to take the commands from the event headers and send them.
Related
I'm trying to implement an event sourced system with dddd. Currently I'm struggling how and where my events are crossing the boundaries of the bounded contexts.
Imagine there are two bounded contexts:
Product Management
Logistics System
Product Management has all the knowledge about the products. For simplification it is just "Name". The logistics system also has products, but has no knowledge about their meta data. For them it is mostly only a physical box with an Id. But when somebody scans this product, they want to show the name either. So the ProductManagement BC should inform the Logistics BC, that a product is registered and a name has changed. So I will end up with the events in ProductManagement, raised from inside the ProductAggregate:
ProductManagement.Events.ProductRegistered
ProductManagement.Events.ProductNameChanged
When I got it correctly these are the events which I will save into the event store. And these are also the events which will be published into the message bus. So at the logistics side I will subscribe to these events. So far so good.
The problem now is:
How will I work with this event on the Logistics side? Vaughn Vernon said in a talk, that it is best practice to have an event handler there, which is in the application layer, so it will basically be an application service. He also said, that it would be best to transform it to one or several commands. Do I save all received events on the logistics side again? Do I also save the commands? How can I reproduce my current state if something went wrong? Or how will I know, that it is not the fault of the processing in the receiving Bounded Context, but rather a wrong event. What will I do if my transformed commands getting rejected?
I know that there are no calculations or changes in aggregates on logistics side. But I think this doesn't really matter for my questions.
Couple of things here.
First, you do not have to import the Logistics BC about name changes. You can get this information from the PM BC when needed, from the client. This is usually done by some sort of composite UI. The UI composition can be done on the client or on the (web) server. You may want to check the article The secret of better UI composition by Mauro Servienti, describing this.
But in general, this usually works like this:
domain event -> pub/sub -> message consumer -> command -> domain command handler
So,
you publish your domain event to the bus, from the PM BC
there is an event handler for this event in the Logistics
the event handler may do some checks, and send the RegisterProduct command to the same BC
the command is handled as usual and new Product aggregate is created in the Logistics
It works like this not only in event-sourced system but in any system with multiple services, using event-driven architecture.
For the use case you describe, you just need some properties of the product to be used by the logistics system. The logistics system therefor could keep a local cache of the product information it needs by subscribing to the events you describe - this could be a simple in-memory cache. They don't need converting into commands or anything like that as you're dealing with the read model I.e. A view. Just have a simple event handler handle the event and update some state somewhere - no need to event source it on the read side. When the logistics system needs the name of the product, it just gets it from its local cache. You haven't broken the autonomy of the two contexts as Product Management is still the source of truth.
If you ever need to rebuild the state you can just wipe the cache and replay all events through your handlers. But remember the Product Management context owns these events so they should only be saved there, not in the Logistics context - you would need a way of republishing them if you ever wanted to rebuild state
By the way this series of blog posts describes this exact use case:
https://www.tigerteam.dk/2014/micro-services-its-not-only-the-size-that-matters-its-also-how-you-use-them-part-1/
(At part 5 if I recall correctly)
Alternatively you can do some sort of UI composition where the name is taken from the Product Management context and the other details from the Logistics context (also discussed in the above blog posts)
[...] He also said, that it would be best to transform it to one or several commands. Do I save all received events on the logistics side again? Do I also save the commands?
First, you have to ask the domain experts if that event will cause a side effect that impact the LS context. Only in this case, you have to subscribe to this event and send the related command to the LS Aggregate that will change and commit its state or, if you choose to event-source this aggregate too, another event.
How can I reproduce my current state if something went wrong? Or how will I know, that it is not the fault of the processing in the receiving Bounded Context, but rather a wrong event? What will I do if my transformed commands getting rejected?
An event is a representation of something happened, so it can't be "wrong". Anyway, commands triggered by an event can fail. Which type of failure are you talking about? Technical or domain specific? In the first case, the source event will stay in the bus for a future retry (maybe after some bug fix). In the second case, if the PM aggregate needs to be informed about the result, the LS Aggregate should emit an appropriate event which, in turn, will be handled by the PM aggregate.
In are app, we are quite heavily using Akka Event Stream to handle logic that is not related to the main business flow. Things like: send emails, sync records, etc... All of these events are currently fired and forgotten.
system.eventStream.publish(<event>)
And they are handled by listeners, asynchronously, in most cases.
However, I am now investigating an option of extending class functionality through events system and that, sometimes, requires a return value from an event?
Is it even possible to get some result back from an event? I could not find anything specific in this regard.
Thanks,
It's a feature of event based systems: you know who triggered the event, you don't know who will handle it, nor when (can be asynchronous).
So, the most idiomatic solution, if your listener should generate an answer, is to do it in an asynchronous way. And as such, the listener should send an event with it's answer.
Another way to do it is to bypass the event bus and register you handlers as listeners of your event source (you should handle your self the registe/unregister functions through akka messages and a simple list). And when your event source triggers the listeners, it can do it using the ask pattern.
I want to use elements of CQRS pattern in my project. I wonder if i do it right with Command and Events.
The thing that I'm not sure is if event can invoke command. To better show what i want to do I will use diagram and example.
This is an example:
User invoke TripCreateCommand. TripCreateCommandHandler do his job and after success publish TripCreatedEvent.
Now we have two listener to TripCreatedEvent (the order of listener execution does not matter)
First listener (can be execute after the second listener):
for each user in trip.author.friends invoke two Command (the order of commands is important)
PublishTripOnUserWallCommand
SendNewTripEmailNotificationCommand
SendNewTripPlatformNotification
Second listener (can be execute before the first listener):
PublishTripOnUserSocials
And this is sample diagram:
Is this a good way ? Can EventListener invoke Command, or maybe I should do it in some other way ?
Your question is about Mesage Driven Architecture which works together with but otherwise unrelated to CQRS.
Anyway, your diagram is almost correct. The event subscriber/handler (I prefer this terminology) can send new Commands via the service bus, but it's not a rule that you should always do this. I implement quite a lot of functionality directly in the event handler, although probalby would be more clean and reliable to send a new command. It really depends on what I want to do.
Note that the message handlers (commands or events) should not know about other handlers. They should know about the bus and the bus takes care of handling. This means that in your app, the event handlers would take the bus as dependency, create the command and send it via the bus. The event handler itself doesn't know what command handler generated the event and can 'reply' to it.
Usually the commands would be handled independently and you can't guarantee the order (unless they're handled synchronously) so maybe you want the second command to be issued as a result of the first command's handling. Indeed, it can be the case for a Saga.
AFAIK you are talking only about doing things synchronously, so your approach works in this case but it's probably not scalable. Moving to async handling will break this execution flow. However your application can be fine with it, not everyhting needs to be twitter.
A message driven architecture is not that straightforward and for some cases (like you want an immediate response from the backend) it's quite complicated to implement, at least more complicated than with the 'standard' approach. So maybe for those particular cases you might want to do it the 'old' way.
If you're worried about decoupling and testing, you can still design the services as they were message handlers but use them directly, instead of a service bus.
Not sure why you would need Commands for performing the updating the information on the user's wall. Why would you choose not to use a View Model Updater for that task.
Sending an email can be considered a Command but could also easily be viewed as just another View Model update.
Not clear on what the purpose of the SendNewTripPlatformNotification is, so I cannot give any suggestions there...
Some of this could also be a candidate for a Saga. Secondly I'm missing your Domain in the diagram, that is what should be responsible for publishing any events, or do you consider the CommandHandler to be the Domain?
According to CQRS à la Greg Young, event handlers (and the downstream event denormalizers) react on incoming events that were published before by the event publisher.
Now lets suppose that at runtime we want to add a new event denormalizer: Basically, this is easy, but it needs to get to its data to the current state.
What is the best way to do this?
Should I send an out-of-order request to the event store and ask for all previously emitted events?
Or is there a better way to do this?
You can fetch and replay all (required) events against the new handler. This can be done in a separate process since what you essentially want is to get the persisted view models into the proper state.
Have a look at Rinat Abdullin's Lokad.CQRS sample project for a production example. Especially the SaaS.Engine.StartupProjectionRebuilder might be an interesting source even though it's rather complex.
One can also build the projections so that they remember what event they saw last. Then on any startup, they ask for this event and all forward. Re-starting an old projection and building a new one then become roughly the same thing.
If you embrace bounded context complex integration you may need to drop the entire read model and rebuild it.
I am attempting to learn and apply the CQRS design approach (pattern and architecture) to a new project but seem to be missing a key piece.
My client application executes a query and retrieves a list of light-weight, read-only DTOs from the read model. The user selects an item and clicks a button to initiate some action. The action is performed by creating and sending the corresponding command object to the write model (where the command handler carries out the action, updates the data store, etc.) At some point, however, I need to update the UI to reflect changes to the state of the application resulting from the action.
How does the UI know when it is time to refresh the original list?
Additional Info
I have noticed that most articles/blogs discussing CQRS use MVC client apps in their examples. I am working on a Silverlight client right now and am beginning to wonder if the pattern simply doesn't work in that case.
Follow-Up Question
After thinking more about Bartlomiej's response and subsequent discussion, I am wondering about error handling in CQRS. Given that commands are basically fire-and-forget asynchronous operations, how do we report an error condition to the UI?
I see 'refreshing the UI' to take one of two forms:
The operation succeeds, data has changed and the UI should be updated to reflect these changes
The operation fails, data has not changed but the user should be notified of the failure and potential corrective actions.
Even with a Post-Redirect-Get pattern in an MVC, you can't really Redirect until you know the outcome of the operation. None of the examples I've seen thus far address these real-world concerns.
I've been struggling with similar issues for a WPF client. The re-query trigger for any data is dependent on the data your updating, commands tend to fall into categories:
The command is a true fire and forget method, it informs the back-end of a state change but this change does not need to be reflected in the UI, or the change simply isn't important to the UI.
The command will alter the result of a single query
The command will alter the result of multiple queries, usually (in my domain at least) in a cascading fashion, that is, changing the state of a single "high level" piece of data will likely affect many "low level" caches.
My first trigger is the page load, very few items are exempt from this as most pages must assume data has been updated since it was last visited. Though some systems may be able to escape with only updating financial and other critical data in this way.
For short commands I also update data when 'success' is returned from a command. Though this is mostly laziness as IMHO all CQRS commands should be fired asynchronously. It's still an option I couldn't live without but one you may have to if your implementation expects high latency between command and query.
One pattern I'm starting to make use of is the mediator (most MVVM frameworks come with one). When I fire a command, I also fire a message to the mediator specifying which command was launched. Each Cache (A view model property Retriever<T>) listens for commands which affect it and then updates appropriately. I try to minimise the number of messages while still minimising the number of caches that update unnecessary from a single message so I'll (hopefully) eventually end up with a shortlist of update reasons, with each 'reason' updating a list of caches.
Another approach is simple honesty, I find that by exposing graphically how the system updates itself makes users more willing to be patient with it. On firing a command show some UI indicating you're waiting for the successful response, on error you could offer to retry / show the error, on success you start the update of the relevant fields. Baring in mind that this command could have been fired from another terminal (of which you have no knowledge) so data will need to timeout eventually to avoid missing state changes invoked by other machines also.
Noting the irony that the only efficient method of updating cache's and values on a client is to un-separate the commands and queries again, be it through hardcoding or something like a hashmap.
My two cents.
I think MVVM actually fits into CQRS quite well. The ViewModel simply becomes an observable ReadModel.
1 - You initialize your ViewModel state via a query on the ReadModel.
2 - Changes on your ViewModel are automatically reflected on any Views that are bound to it.
3 - Certain changes on your ViewModel trigger a command to propegate to a message queue, an object responsible for sending those commands to the server takes those messages off the queue and sends them to the WriteModel.
4 - Clients should be well formed, meaning the ViewModel should have performed appropriate validation before it ever triggered the command. Once the command has been triggered, any event notifications can be published onto an event bus for the client to communicate changes to other ViewModels or components in the system interested in those changes. These events should carry the relevant information necessary. Typically, this means that other view models usually don't have to re-query the read model as a result of the change unless they are dependent on other data that needs to be retrieved.
5 - There is an object that connects to the message bus on the server for real-time push notifications when other clients make changes that this client is interested in knowing about, falling back to long-polling if necessary. It propagates those to the internal message bus that ties the components on the client together.
6 - The last part to handle is the fact that clients can be occasionally connected, which should be the only reason a command fails (they don't have internet access at the moment), which is when the client should be notified of problems.
In my ASP.NET MVC 3 I use 2 techniques depending on use case:
already well-known Post-Redirect-Get pattern which fits nicely with CQRS. Your MVC action that triggers the command returns a redirection to action that performs a query.
in some cases, like real-time updates of other clients, I rely on domain events/messages. I create an event handler that uses singlarR to push changes to all connected and interested clients.
There are two major ways you can take as far as I know :
1) design your UI , so that the user does not see its changes right away. Like for instance a message to tell him his action is a success, and offering him different choices to continue his work. this should buy you enough time to have updated your readmodel.
2) more complex, but you might keep the information you have send to the server and shows them in the interface.
The most important I guess, educate your user if you can so that they know why the data is not here... yet!
I am thinking about it only now, but these are for sync command handling, not async, in async things go really harder on the brain...the client interface becomes an event eater too..