According to CQRS à la Greg Young, event handlers (and the downstream event denormalizers) react on incoming events that were published before by the event publisher.
Now lets suppose that at runtime we want to add a new event denormalizer: Basically, this is easy, but it needs to get to its data to the current state.
What is the best way to do this?
Should I send an out-of-order request to the event store and ask for all previously emitted events?
Or is there a better way to do this?
You can fetch and replay all (required) events against the new handler. This can be done in a separate process since what you essentially want is to get the persisted view models into the proper state.
Have a look at Rinat Abdullin's Lokad.CQRS sample project for a production example. Especially the SaaS.Engine.StartupProjectionRebuilder might be an interesting source even though it's rather complex.
One can also build the projections so that they remember what event they saw last. Then on any startup, they ask for this event and all forward. Re-starting an old projection and building a new one then become roughly the same thing.
If you embrace bounded context complex integration you may need to drop the entire read model and rebuild it.
Related
Using Axon framework- I was able to replay the entire event store and re-create the view model. But is it possible to replay event for a particular type or for a particular Id.
Let's say, I have a customer event and I want to replay all the event of a customer with Id= 100. Is it make sense to do a replay for a particular customer or it make more sense to replay for the entire event store always?
Thanks in advance
It is OK to do whatever makes sense for you, for this particular ReadModel.
One reason to re-process only one customer is the speed. If it's a lot faster than a complete rebuild (i.e because you have a lot of customers) and the outcome is the same then do it.
As Constantin points out, this request to replay a specific view makes total sense.
The provided replay process in Axon Framework at this point only provides to trigger a replay for a specific Processing Group allowing you to set the point in time from when you want to replay it.
There are ideas to provide a more fine grained solution to replaying, I'd however be hard pressed to tell you when that'll happen.
Thus replaying just a single view model for, for example speed, will require some custom code.
Let me know if you'd be interested in some pointers on how to do that.
Update
I'd like to state that with more recent versions of Axon Framework is is possible to tell a TrackingEventProcessor to reset itself, thus replay a set of events.
The API for this is the TrackingEventProcessor#resetTokens(TrackingToken), which allows you to reset a Tracking Event Processor from a given point in time.
This still doesn't give you the option to replay a given instance of a Read Model. This would still require some handy work from your part.
I'm trying to implement an event sourced system with dddd. Currently I'm struggling how and where my events are crossing the boundaries of the bounded contexts.
Imagine there are two bounded contexts:
Product Management
Logistics System
Product Management has all the knowledge about the products. For simplification it is just "Name". The logistics system also has products, but has no knowledge about their meta data. For them it is mostly only a physical box with an Id. But when somebody scans this product, they want to show the name either. So the ProductManagement BC should inform the Logistics BC, that a product is registered and a name has changed. So I will end up with the events in ProductManagement, raised from inside the ProductAggregate:
ProductManagement.Events.ProductRegistered
ProductManagement.Events.ProductNameChanged
When I got it correctly these are the events which I will save into the event store. And these are also the events which will be published into the message bus. So at the logistics side I will subscribe to these events. So far so good.
The problem now is:
How will I work with this event on the Logistics side? Vaughn Vernon said in a talk, that it is best practice to have an event handler there, which is in the application layer, so it will basically be an application service. He also said, that it would be best to transform it to one or several commands. Do I save all received events on the logistics side again? Do I also save the commands? How can I reproduce my current state if something went wrong? Or how will I know, that it is not the fault of the processing in the receiving Bounded Context, but rather a wrong event. What will I do if my transformed commands getting rejected?
I know that there are no calculations or changes in aggregates on logistics side. But I think this doesn't really matter for my questions.
Couple of things here.
First, you do not have to import the Logistics BC about name changes. You can get this information from the PM BC when needed, from the client. This is usually done by some sort of composite UI. The UI composition can be done on the client or on the (web) server. You may want to check the article The secret of better UI composition by Mauro Servienti, describing this.
But in general, this usually works like this:
domain event -> pub/sub -> message consumer -> command -> domain command handler
So,
you publish your domain event to the bus, from the PM BC
there is an event handler for this event in the Logistics
the event handler may do some checks, and send the RegisterProduct command to the same BC
the command is handled as usual and new Product aggregate is created in the Logistics
It works like this not only in event-sourced system but in any system with multiple services, using event-driven architecture.
For the use case you describe, you just need some properties of the product to be used by the logistics system. The logistics system therefor could keep a local cache of the product information it needs by subscribing to the events you describe - this could be a simple in-memory cache. They don't need converting into commands or anything like that as you're dealing with the read model I.e. A view. Just have a simple event handler handle the event and update some state somewhere - no need to event source it on the read side. When the logistics system needs the name of the product, it just gets it from its local cache. You haven't broken the autonomy of the two contexts as Product Management is still the source of truth.
If you ever need to rebuild the state you can just wipe the cache and replay all events through your handlers. But remember the Product Management context owns these events so they should only be saved there, not in the Logistics context - you would need a way of republishing them if you ever wanted to rebuild state
By the way this series of blog posts describes this exact use case:
https://www.tigerteam.dk/2014/micro-services-its-not-only-the-size-that-matters-its-also-how-you-use-them-part-1/
(At part 5 if I recall correctly)
Alternatively you can do some sort of UI composition where the name is taken from the Product Management context and the other details from the Logistics context (also discussed in the above blog posts)
[...] He also said, that it would be best to transform it to one or several commands. Do I save all received events on the logistics side again? Do I also save the commands?
First, you have to ask the domain experts if that event will cause a side effect that impact the LS context. Only in this case, you have to subscribe to this event and send the related command to the LS Aggregate that will change and commit its state or, if you choose to event-source this aggregate too, another event.
How can I reproduce my current state if something went wrong? Or how will I know, that it is not the fault of the processing in the receiving Bounded Context, but rather a wrong event? What will I do if my transformed commands getting rejected?
An event is a representation of something happened, so it can't be "wrong". Anyway, commands triggered by an event can fail. Which type of failure are you talking about? Technical or domain specific? In the first case, the source event will stay in the bus for a future retry (maybe after some bug fix). In the second case, if the PM aggregate needs to be informed about the result, the LS Aggregate should emit an appropriate event which, in turn, will be handled by the PM aggregate.
I'm building a simple REST API for generating some objects that must be created and sent periodically out of the API. The nature of the objects doesn't matter, neither the framework supporting the REST interface (Spray, Play Framework, whatever else). My question is, what would be a good scalable actor design for this system using Akka? Suppose the service crashes or it's migrated or whatever that causes to stop it. In order to recover the description of the tasks about what objects must be sent and when, is akka-persistence a good way to go here? or it's better to persist such things in a traditional DB?
Thanks.
NOTE: also I would like to know, supposing there's some actor which is not stateful himself, but creates many children actors, if it's a good practice to use akka-persistence in order to replay the messages which causes this actor to create his children again (the children being also non-stateful).
In a traditional DB you would most likely end up modeling this with timestamps and events, and with event sourcing this is already the native model.
Akka-persistence would be a natural fit for this scenario since it will persist every event about what objects must be created and sent periodically out. The snapshot support will also help with speed of recovery when the number of events gets very large.
In the case of crashes or migration, the recovery process will handle this just fine.
Regarding your note, if the actor is truly stateless then there is no need to persist the events that cause the children to be created since they can be recreated on demand. If the existence of the children does need to be recovered, then the actor is not stateless. In that case then it may indeed make sense to persist those events.
I am currently writing my first saga and I am a bit puzzled with the read model. Let's explain it with an example :
I have three bounded context : programming, contractor and control. Each of them has its specific read model.
worflow :
programming send an event "JobScheduled"
Saga receives this event and tell contractor to "schedule the Work".
When done the Contractor send an event "JobDone".
The Saga receives this event and tell Control to "Start Control Period".
Everything turns out to be fine here. We are on the write side so we are passing vital information for the process to go on.
My question comes with unnecessary information. Let's say that the event "JobScheduled"
has a note field : "test note", and before this job is done this field is changed to "test note important". This change is of no importance to the workflow as described, but it is important that the contractor might see the change in the field when looking at the read model of the contractor bounded context.
Am I to give to the saga the event NoteChanged and process it or should I create a projection which directly listens to this event in my contractor bounded context?
To give it to the saga looks to me like unnecessary work because I am only updating readmodel here there is no domain involved in the change.
On the other hand, making a direct coupling between the two bounded context removes one of the assets of sagas which is the possibilty of modifying the interactions the bounded context have between each other in the workflow.
Thanks for your reading,
If changing the note is important, it should be modelled explicitly. This can be accomplished by introducing an Event just like you already did.
If said Event has any relevancy to the process, it can be handled by a Saga. If it only needs to be represented in different Read Models, then just handling it in their respective projections should be fine.
One context may very well listen to and handle the events of another one, even across application boundaries. At least this is how cross-context integration should work in an event centric architecture.
Personally I would send a command "Change note", because I see it like an information that have to be saved in the event stream of your aggregate. Than if your saga don't "feel" to tell anyone about this command, or simply give the information to an handler that quietly update your read model I guess is fine.
I am trying to wrap my head around Saga's using Jonathan Olivers EventStore and CommonDomain. I understand how Aggregates are working with the CommonDomain/EventStore but I am stuck on grasping Saga usage. I have read both of Jonathan's Saga's with Event Sourcing Part I & II but sill lost in actual implementation
1) More of observation, when persisting the saga the EventStore is utilizing the Headers to persist the Saga and Commands that need to be sent out and it looks like the Payload is storing the Event that triggered the Saga to "wake up". Wondering reasons for this. Would we never want to store individual commands vs having them all in the header?
1) It seems like the Event that triggered the Saga gets replayed multiple times since the "Transition" method in SagaBase always re-adds the event to uncommitted collection. (Unlike ARs that have an internal Apply method vs public Domain method). Maybe I am not using the Transition method properly
2) Typically the bus that you use with the EventStore will publish Events (I implemented IPublishMessages). If I need my Saga to publish a command there does not seem to be a Send option. Do I need to parse the Headers to grab the commands myself?
I am thinking I am using the CommonDomain / EventStore incorrectly as working with Aggregates was easy but Saga's seem "incomplete" to me. I am assuming its because I am not doing it correctly. Still very new to CQRS. Does anyone have a working example of Saga's using J Olivers Common Domain / Event Store? I think that would clear things up considerably.
[EDIT]
I think I figured it out but would like some input. Saga's really should not be publishing events. They send out commands. Thus on the publish side of things for the EventStore (IPublishMessages) I should first be checking the type of message (AggregateType vs SagaType) For AggregateTypes I can publish Events but for SagaTypes only publish the commands (found in Header). This eliminates the same event (say OrderSubmittedEvent) that triggers the creation of the Saga to not publish it again when persisting the saga.
The commands are put in the headers to be sent on the bus. The EventStore is concerned with the storage of events so the events that caused saga transitions are persisted. Later, when the saga is loaded from the event stream, the events will be passed to the saga's transition method to bring it to the current state.
The transition method serves a dual purpose in the saga implementation. Transition is called to handle incoming messages and to load the saga from peristence. In SagaEventStoreRepository.BuildSaga ClearUncomittedEvents and ClearUndispatchedMessages are called on the saga after the current state is built up thus avoiding duplicate event and command processing.
I haven't personally done this but I would use a separate EventStore instance for my sagas. This would allow for the usage of a separate IPublishMessages implementation to take the commands from the event headers and send them.