I am very new to scala and akka so forgive if this question is very lame.
I have build a reactive application and it creates events and stores perfectly in cassandra which I am using as journal.
There can be a scenario that my application might have some bug and I would have stored a wrong event in the journal.How would I deal with it? If I will release a new fixed version of application events after that will be right but I still be having a wrong event in my journal and during a restore it will play them wrong.
Related
We are using axon framework version 3.4.2 and found a bug inside our code. The bug relates to a missing event which was not published. The solution is to fix the code but this would not fix the event store and views.
My question is how would one fix this? We thought of appending the events to the event store (we use a JDBC event store), but without the correct data, the new events would not be processed. The best would be to do it in the application by publishing the event in axon and let axon handle all the details, but this is a once-off, correcting action.
Is there any way of "injecting" a once-off event into axon?
The comment which Matt shared is conceptually what you should do.
Thus, to resolve the issue you unintendedly introduced, you should produce a compensation action, aka a command. This command will be handled in your command model, will validate the model's state and publish the desired event.
Added, I am assuming this event of your should originate from an Aggregate, correct?
In Axon terms, that means you want to publish a domain event rather than a regular event.
Although you can publish events on the EventBus or store in the EventStore directly, it is rather complicated to make those domain events through that process.
Thus, as I started off with and what Matt Freeman commented on your question, a compensating action would be the way to go, with or without Axon.
Last note, know that Axon 4.2 is already out for some time now. Although Axon 3 will still under go bug fixes, none of these have occurred in the last year. Simply put, there is no active development on Axon 3. Migrating to a more recent version would thus be beneficial for your project.
I am using Axon 3.1.1 and wanted to know,
How can I get a list of eventprocessor in my configuration file,
I went through the springAmQPmessageSource file but still not sure how to exactly do it.
So that I can pass my event to appropriate eventhandler on Query side.
List<Consumer<List<? extends EventMessage<?>>>> eventProcessors = new CopyOnWriteArrayList<>();
Updated
I was retrieving message from kafka topic and wanted to wire them to specific eventhandler but since I am not able to get evenprocessors, I am not able to do that.
Can you please tell me how to do it, if I am using Axon 3.0.5
If you're using the SpringAmqpMessageSource, you will not need to retrieve the list of eventProcessors you've shared, as Axon will automatically subscribe all the event handling components to it for you.
Subsequently, the events the Message Source receives will automatically be pushed to all the listeners in your query side.
As this is all covered as Axon infrastructure under the hood, there is no one-off way to pull them out of it for your own use (other than potentially wiring them yourself).
Hence, you shouldn't have to do this yourself.
But, maybe I'm missing an obvious point here.
Could you elaborate a little more why you need the list of handlers in the first place?
I am using Kafka queue 0.8.2 and have implemented standard poll and push calls. Now wanna go to pollByIndex methods that require implementation of Simple Consumer.
Does somebody knows some custom library which already deals with methods like this since implementing Simple Consumer can be a lot of work :)
Upgrading to 0.9 to use ConsumerAPI not option yet for me.
Ok. So creating SimpleConsumer is not really easy. So I went for solution of implementing another command that creates different group each time I queue for results which is stored in zookeeper and then assign offset I need on topic. That way default streaming queue will not be destroyed, it create small load on zookeeper but nothing significant.
Also after every querying I take care to remove group from zookeeper to keep it clean.
1 day of coding more less.
With new versions 0.9 and newer we will have ConsumerAPI which has its own calls also for this situation. Until then more work from dev side needed.
I am trying to understand how state management in Spark Streaming works in general. If I run this example program twice will the second run see state from the first run?
https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/streaming/StatefulNetworkWordCount.scala
Is there a way how to achieve this? I am thinking about redeploying an application an I would like not to loose the current state.
tl;dr It depends on what you need the other instance to see. Checkpointing is usually a solution.
ssc.checkpoint(".") (at the line 50 in StatefulNetworkWordCount) enables checkpointing that (quoting the official documentation):
Spark Streaming needs to checkpoint enough information to a fault-tolerant storage system such that it can recover from failures.
A failure can be considered a form of redeployment. It is described in the official documentation under Upgrading Application Code that lists two cases:
Two instances run in parallel
One is gracefully brought down, and the other reads state from checkpoint directory.
I'm building a simple REST API for generating some objects that must be created and sent periodically out of the API. The nature of the objects doesn't matter, neither the framework supporting the REST interface (Spray, Play Framework, whatever else). My question is, what would be a good scalable actor design for this system using Akka? Suppose the service crashes or it's migrated or whatever that causes to stop it. In order to recover the description of the tasks about what objects must be sent and when, is akka-persistence a good way to go here? or it's better to persist such things in a traditional DB?
Thanks.
NOTE: also I would like to know, supposing there's some actor which is not stateful himself, but creates many children actors, if it's a good practice to use akka-persistence in order to replay the messages which causes this actor to create his children again (the children being also non-stateful).
In a traditional DB you would most likely end up modeling this with timestamps and events, and with event sourcing this is already the native model.
Akka-persistence would be a natural fit for this scenario since it will persist every event about what objects must be created and sent periodically out. The snapshot support will also help with speed of recovery when the number of events gets very large.
In the case of crashes or migration, the recovery process will handle this just fine.
Regarding your note, if the actor is truly stateless then there is no need to persist the events that cause the children to be created since they can be recreated on demand. If the existence of the children does need to be recovered, then the actor is not stateless. In that case then it may indeed make sense to persist those events.