Axon Framework: Handle only events published by the same JVM instance? - cqrs

Hi Axon Framework community,
I'd like to have your opinion on how to solve the following problem properly.
My Axon Test Setup
Two instances of the same Spring Boot application (using axon-spring-boot-starter 4.4 without Axon Server)
Every instance publishes the same events on a regular interval
Both instances are connected to the same EventSource (single SQL Server instance using JpaEventStorageEngine)
Every instance is configured to use TrackingEventProcessors
Every instances has the same event handlers registered
What I want to achieve
I'd like that events published by one instance are only handled by the very same instance
If instance1 publishes eventX then only instance1 should handle eventX
What I've tried so far
I can achieve the above scenario using SubscribingEventProcessor. Unfortunately this is not an option in my case, since we'd like to have the option to replay events for rebuilding / adding new query models.
I could assign the event handlers of every instance to differed processing groups. Unfortunately this didn't worked. Maybe because every TrackingEventProcessors instance processes the same EventStream ? - not so sure about this though.
I could implement a MessageHandlerInterceptor which only proceeds in case the event origin is from the same instance. This is what I implemented so far and which works properly:
MessageHandlerInterceptor
class StackEventInterceptor(private val stackProperties: StackProperties) : MessageHandlerInterceptor<EventMessage<*>> {
override fun handle(unitOfWork: UnitOfWork<out EventMessage<*>>?, interceptorChain: InterceptorChain?): Any? {
val stackId = (unitOfWork?.message?.payload as SomeEvent).stackId
if(stackId == stackProperties.id){
interceptorChain?.proceed()
}
return null
}
}
#Configuration
class AxonConfiguration {
#Autowired
fun configure(eventProcessingConfigurer: EventProcessingConfigurer, stackProperties: StackProperties) {
val processingGroup = "processing-group-stack-${stackProperties.id}"
eventProcessingConfigurer.byDefaultAssignTo(processingGroup)
eventProcessingConfigurer.registerHandlerInterceptor(processingGroup) { StackEventInterceptor(stackProperties) }
}
}
Is there a better solution ?
I have the impression that my current solution is properly not the best one, since ideally I'd like that only the event handlers which belongs to a certain instance are triggered by the TrackingEventProcessor instances.
How would you solve that ?

Interesting scenario you're having here #thowimmer.
My first hunch would be to say "use the SubscribingEventProcessor instead".
However, you pointed out that that's not an option in your setup.
I'd argue it's very valuable for others who're in the same scenario to know why that's not an option. So, maybe you can elaborate on that (to be honest, I am curious about that too).
Now for your problem case to ensure events are only handled within the same JVM.
Adding the origin to the events is definitely a step you can take, as this allows for a logical way to filter. "Does this event originate from my.origin()?" If not, you'd just ignore the event and be done with it, simple as that. There is another way to achieve this though, to which I'll come to in a bit.
The place to filter is however what you're looking for mostly I think. But first, I'd like to specify why you need to filter in the first place. As you've noticed, the TrackingEventProcessor (TEP) streams events from a so called StreamableMessageSource. The EventStore is an implementation of such a StreamableMessageSource. As you are storing all events in the same store, well, it'll just stream everything to your TEPs. As your events are part of a single Event Stream, you are required to filter them at some stage. Using a MessageHandlerInterceptor would work, you could even go and write a HandlerEnhacnerDefinition allowing you to add additional behaviour to your Event Handling functions. However you put it though, with the current setup, filtering needs to be done somewhere. The MessageHandlerInterceptor is arguably the simplest place to do this at.
However, there is a different way of dealing with this. Why not segregate your Event Store, into two distinct instances for both applications? Apparently they do not have the need to read from one another, so why share the same Event Store at all? Without knowing further background of your domain, I'd guess you are essentially dealing with applications residing in distinct bounded contexts. Very shortly put, there is zero interest to share everything with both applications/contexts, you just share specifics portions of your domain language very consciously with one another.
Note that support for multiple contexts, using a single communication hub in the middle, is exactly what Axon Server can achieve for you. I am not here to say you cant configure this yourself though, I have done this in the past. But leaving that work to somebody or something else, freeing you from the need to configure infrastructure, that would be a massive timesaver.
Hope this helps you set the context a little of my thoughts on the matter #thowimmer.

Sumup:
Using the same EventStore for both instances is probably no an ideal setup in case we want to use the capabilities of the TrackingEventProcessor.
Options to solve it:
Dedicated (not mirrored) DB instance for each application instance.
Using multiple contexts using AxonServer.
If we decide to solve the problem on application level filtering using MessageHandlerInterceptor is the most simplest solution.
Thanks #Steven for exchanging ideas.
EDIT:
Solution on application level using CorrelationDataProvider & MessageHandlerInterceptor by filtering out events not originated in same process.
AxonConfiguration.kt
const val METADATA_KEY_PROCESS_ID = "pid"
const val PROCESSING_GROUP_PREFIX = "processing-group-pid"
#Configuration
class AxonConfiguration {
#Bean
fun processIdCorrelationDataProvider() = ProcessIdCorrelationDataProvider()
#Autowired
fun configureProcessIdEventHandlerInterceptor(eventProcessingConfigurer: EventProcessingConfigurer) {
val processingGroup = "$PROCESSING_GROUP_PREFIX-${ApplicationPid()}"
eventProcessingConfigurer.byDefaultAssignTo(processingGroup)
eventProcessingConfigurer.registerHandlerInterceptor(processingGroup) { ProcessIdEventHandlerInterceptor() }
}
}
class ProcessIdCorrelationDataProvider() : CorrelationDataProvider {
override fun correlationDataFor(message: Message<*>?): MutableMap<String, *> {
return mutableMapOf(METADATA_KEY_PROCESS_ID to ApplicationPid().toString())
}
}
class ProcessIdEventHandlerInterceptor : MessageHandlerInterceptor<EventMessage<*>> {
override fun handle(unitOfWork: UnitOfWork<out EventMessage<*>>?, interceptorChain: InterceptorChain?) {
val currentPid = ApplicationPid().toString()
val originPid = unitOfWork?.message?.metaData?.get(METADATA_KEY_PROCESS_ID)
if(currentPid == originPid){
interceptorChain?.proceed()
}
}
}
See full demo project on GitHub

Related

Should a RESTful API return data which is reshaped for the UI

Lets assume ONE team uses the API to return data to a SPA in the browser.
Is it RESTful to return data which is specially prepared for the UI of the SPA?
Instead the client could prepare the data specially with JS what many want to avoid.
WHAT means reshaped? =>
public async Task<IEnumerable<SchoolclassCodeDTO>> GetSchoolclassCodesAsync(int schoolyearId)
{
var schoolclassCodes = await schoolclassCodeRepository.GetSchoolclassCodesAsync(schoolyearId);
var allPupils = schoolclassCodes.SelectMany(s => s.Pupils).Distinct<Pupil>(new PupilDistinctComparer());
var allPupilsDTOs = allPupils.Select(p => p.ToPupilDTO());
var schoolclassCodeDTOs = schoolclassCodes.Select(s => s.ToSchoolclassDTO());
// Prepare data specially for UI DataGrid with checkboxes
foreach (var s in schoolclassCodeDTOs)
{
foreach (var p in allPupilsDTOs)
{
var targetPupil = s.Pupils.SingleOrDefault(pupil => pupil.Id == p.Id);
if(targetPupil == null)
{
p.IsSelected = false;
s.Pupils.Add(p);
}
else
{
targetPupil.IsSelected = true;
}
}
}
return schoolclassCodeDTOs;
}
This is a good question. Problem is, it's likely that only you have the answer.
tl;dr Good-old "it's application-specific".
You probably need to look at this as a continuum, not a binary decision. On one extreme you have the server generate HTML views and is thus responsible for a lot of UI concerns, on the other you have the server expose a data model with very generic CRUD functionality, and is thus not responsible for any UI concerns. Somewhere in the middle, you can also find a design where a server interface is specific to an application but does not necessarily expose HTML (although with the magic of conn-neg everything falls into place) for possibly very obvious reasons (when it comes to so-called SPAs).
So, what should you choose?
I expect some people to advise you to decouple client and server as much as possible, but I personally believe there is no such ultimate "good practice". In fact, it might be premature. Design your logical components first, and then decide on where and how they should run.
Remember:
A good architecture allows major decisions to be deferred and maximizes the number of decisions not made.
(Uncle Bob)
Why? Well, because these (domain logic and execution environment) are truly separate concerns: they might evolve independently. For example, you might decide to create a thinner client for mobile and a thicker client for desktop if the application is compute-intensive (e.g. to save battery power). Or, you might do the exact opposite if the application is network-intensive (e.g. to save roundtrips when connectivity is bad, also consider "offline-first"). Maybe you'll provide all variants and let the user choose, or maybe choose automatically based on available resources -- whatever the requirements warrant and however they might change.
I think these are more appropriate questions and architectural decisions to make (but again: only after you've already designed the boundaries of your logical components). These clearer requirements will help you decide which components of your application should run where. They will drive the way you represent your boundaries (whether they be internal or remote APIs, private or public) but not how you shape them (that's already done). Your RESTful API (if you decide you need one and that a REST-style architecture is appropriate) is just a representation for an arbitrary boundary.
And that's how you will eventually answer your own question in the context of your scenario -- which should hopefully have become very intuitive by then.
End note: While having the domain logic strictly shape your boundaries is nice and pure, it's inevitable that some concerns pertaining to the execution environment (like who controls certain network hosts, where the data should reside, etc) will feed back into the domain design. I don't see it as a contradiction; your application does influence whatever activity you're modelling, so its own concerns must be modelled too. Tools also influence the way you think, so if HTTP is a tool and you're very good at using it, you might start using it everywhere. This is not necessarily bad (e.g. the jury is still out on "micro-services"), though one should be aware that knowing too few tools often (not always) push developers to awkward corners. How could I not finish with: "use the right tool for th--" ah, it's getting old, isn't it ;).

Does vert.x have centralized filtering?

I am new to Vert.X.
Does Vert.x have a built in facility for centralized filters? What I mean are the kind of filters that you would use on a J2EE application.
For instance, all pages have to go through the auth filter, or something like that.
Is there a standardized way to achieve this in Vert.x?
I know this question is quite old, but for those still looking for filter in Vertx 3, the solution is to use subRouter as a filter:
// Your regular routes
router.route("/").handler((ctx) -> {
ctx.response().end("...");
});
// Have more routes here...
Router filterRouter = Router.router(vertx);
filterRouter.get().handler((ctx)->{
// Do something smart
// Forward to your actual router
ctx.next();
});
filterRouter.mountSubRouter("/", router);
Filtering is an implementation of the chain of responsibility in the servlet container. Vert.x does not have this kind of concept but with yoke (or apex in the new release) you are able to easily reproduce this behavior.
Give a look in the routing section: https://github.com/vert-x3/vertx-web/blob/master/vertx-web/src/main/asciidoc/index.adoc
HTH,
Carlo
Vert.x is unopinionated about how many things should be handled. But generally speaking, these types of features are typically implemented as "bus mods" (i.e. modules/verticles which receive input and produce output over the event bus) in Vert.x 2. In fact, the auth manager module may help you get a better understanding of how this is done:
https://github.com/vert-x/mod-auth-mgr
In Vert.x 3 the module system will be/is gone, but the pattern will remain the same. It's possible that some higher level framework built on Vert.x could support these types of filters, but Vert.x core will not.
If also recommend you poke around in Vert.x Apex if you're getting started building web applications on Vert.x:
https://github.com/vert-x3/vertx-apex
Vert.x is more similar to node.js than any other java based frameworks.
Vert.x depends on middlewares. You can define them and attach them to a route. Depending on the order they are defined in they will get called.
For example lets say you have a user application where you would like to run logging and request verification before the controller is called.
You can do something like follows:
Router router = Router.router(vertx);
router.route("/*").handler(BodyHandler.create()); // Allows Body to available in post calls
router.route().handler(new Handler<RoutingContext>() {
#Override
public void handle(RoutingContext event) {
//Handle logs
}
})
router.route("/user").handler(new Handler<RoutingContext>() {
#Override
public void handle(RoutingContext event) {
// handle verification for all apis starting with /user
}
});
Here depending on the route set of middleware will get called.
From my POV, this is exactly the opposite to what vert.x tries to achieve. A verticle being the core building block of the framework is supposed to keep the functions distributed, rather than centralized.
For the multithreaded (cluster) async environment that makes sense, because as soon as you start introducing something "centralized" (and usually synchronous), you would lose the async ability.
One of the options is to implement auth in your case would be to exchange the messages with respecive auth-verticle over the event bus. In this case you would have to handle the async aspect of such a request.

Form-related problems

I am new to Lift and I am thinking whether I should investigate it more closely and start using it as my main platform for the web development. However I have few "fears" which I would be happy to be dispelled first.
Security
Assume that I have the following snippet that generates a form. There are several fields and the user is allowed to edit just some of them.
def form(in : NodeSeq): NodeSeq = {
val data = Data.get(...)
<lift:children>
Element 1: { textIf(data.el1, data.el1(_), isEditable("el1")) }<br />
Element 2: { textIf(data.el2, data.el2(_), isEditable("el2")) }<br />
Element 3: { textIf(data.el3, data.el3(_), isEditable("el3")) }<br />
{ button("Save", () => data.save) }
</lift:children>
}
def textIf(label: String, handler: String => Any, editable: Boolean): NodeSeq =
if (editable) text(label, handler) else Text(label)
Am I right that there is no vulnerability that would allow a user to change a value of some field even though the isEditable method assigned to that field evaluates to false?
Performance
What is the best approach to form processing in Lift? I really like the way of defining anonymous functions as handlers for every field - however how does it scale? I guess that for every handler a function is added to the session with its closure and it stays there until the form is posted back. Doesn't it introduce some potential performance issue when it comes to a service under high loads (let's say 200 requests per second)? And when do these handlers get freed (if the form isn't resubmitted and the user either closes the browser or navigate to another page)?
Thank you!
With regards to security, you are correct. When an input is created, a handler function is generated and stored server-side using a GUID identifier. The function is session specific, and closed over by your code - so it is not accessible by other users and would be hard to replay. In the case of your example, since no input is ever displayed - no function is ever generated, and therefore it would not be possible to change the value if isEditable is false.
As for performance, on a single machine, Lift performs incredibly well. It does however require session-aware load balancing to scale horizontally, since the handler functions do not easily serialize across machines. One thing to remember is that Lift is incredibly flexible, and you can also create stateless form processing if you need to (albeit, it will not be as secure). I have never seen too much of a memory hit with the applications we have created and deployed. I don't have too many hard stats available, but in this thread, David Pollak mentioned that demo.liftweb.net at the time had 214 open sessions consuming about 100MB of ram (500K/session).
Also, here is a link to the Lift book's chapter on Scalability, which also has some more info on security.
The closure and all the stuff is surely cleaned at sessionShutdown. Earlier -- I don't know. Anyway, it's not really a theoretical question -- it highly depends on how users use web forms in practice. So, for a broader answer, I'd ask the question on the main channel of liftweb -- https://groups.google.com/forum/#!forum/liftweb
Also, you can use a "statical" form if you want to. But AFAIK there are no problems with memory and everybody is using the main approach to forms.
If you don't create the handler xml/html -- the user won't be able to change the data, that's for sure. In your code, if I understood it correctly (I'm not sure), you don't create "text(label,handler)" when it's not needed, so everything's secure.

What is the best way to implement a request/response protocol using akka and scala?

I've been studying about how I could develop a distributed architecture that implements the protocol request/response using the concept of concurrency through actors.
I concluded that the best way to do this is by creating a response system with synchronous handling of Futures/Promises, and soon after the response, leaving an open channel to receive notifications.
Thus an architecture that would work exactly like a inbox message.
It has some problems.
Thus I would have to maintain two endpoints (actors in the two layers)?
The Problem:
The view module requests that a particular element is processed. She sends this command to be processed via RemoteActor on the application server. This server should immediately return the promise that it will notify you when the element is processed. After this, the view module will be waiting the notification of completion of processing.
How do you see this problem?
I am using Scala, Akka and Google Guice.
I believe it is a generic problem that everyone can make use of their solutions. Excuse me if I'm hurting the terms of stackoverflow site.
Thanks in advance
I don't want to distract from any good answers you may get on Akka, because I unfortunately don't know much about Akka and it's distributed actors features, but I'd like to ask if you've considered any alternatives.
It seems that you basically need an asynchronous RPC library. There are two powerful libraries written in Scala that I know of that may satisfy your requirements - http://sna-projects.com/norbert/ and http://twitter.github.com/finagle/. Finagle gives some really powerful combinators for expressing asynchronous computation dependencies and registering listeners on futures. I'm currently maintaining Norbert, which we use at LinkedIn for some of our distributed systems, such as search.
//On "server" node
class SomeActor extends Actor {
def receive = {
case messageToRemoteActor => self reply_? TypeOfResponseMessage()
}
}
Actor.remote.register("myRemoteActor", actorOf[SomeActor])
//On "client" node
val remoteActor = Actor.remote.actorFor("myRemoteActor", "hostnameOrIpOfNodeContainingTheActorAbove", port)
val f: Future[TypeOfResponseMessage] = remoteActor !!! messageToRemoteActor
f.onComplete( _.value.get match {
case Left(exception) => handle(exception)
case Right(result) => handle(result)
})
Why not just use one of 0MQ's REQ-REP sockets?
https://github.com/zcox/akka-zeromq-java
That way you solve your immediate problem and at the same time, begin learn an architecture that will take you a long way and supports communications with clients written in many languages.
For an example of where this might lead look at http://blog.getintheloop.eu/2009/05/22/lift-amqp-with-rabbitmq-and-scala-tutorial-and-screencast/
Note that I am NOT suggesting that you use AMQP today since a RabbitMQ broker would be overkill for your immediate problem. Rather, I am suggesting that you invest your time in using an architecture (message queueing) that will pay you dividends in the long run.

Rhino Mocks Calling instead of Recording in NUnit

I am trying to write unit tests for a bit of code involving Events. Since I need to raise an event at will, I've decided to rely upon RhinoMocks to do so for me, and then make sure that the results of the events being raised are as expected (when they click a button, values should change in a predictable manner, in this example, the height of the object should decrease)
So, I do a bit of research and realize I need an Event Raiser for the event in question. Then it's as simple as calling eventraiser.Raise(); and we're good.
The code for obtaining an event raiser I've written as is follows (written in C#) (more or less copied straight off the net)
using (mocks.Record())
{
MyControl testing = mocks.DynamicMock<MyControl>();
testing.Controls.Find("MainLabel",false)[0].Click += null;
LastCall.IgnoreArguments();
LastCall.Constraints(Rhino.Mocks.Constraints.Is.NotNull());
Raiser1 = LastCall.GetEventRaiser();
}
I then test it as In playback mode.
using (mocks.Playback())
{
MyControl thingy = new MyControl();
int temp=thingy.Size.Height;
Raiser1.Raise();
Assert.Greater(temp, thingy.Size.Height);
}
The problem is, when I run these tests through NUnit, it fails. It throws an exception at the line testing.Controls.Find("MainLabel",false)[0].Click += null; which complains about trying to add null to the event listener. Specifically, "System.NullReferenceException: Object Reference not set to an instance of the Object"
Now, I was under the understanding that any code under the Mocks.Record heading wouldn't actually be called, it would instead create expectations for code calls in the playback. However, this is the second instance where I've had a problem like this (the first problem involved classes/cases that where a lot more complicated) Where it appears in NUnit that the code is actually being called normally instead of creating expectations. I am curious if anyone can point out what I am doing wrong. Or an alternative way to solve the core issue.
I'm not sure, but you might get that behaviour if you haven't made the event virtual in MyControl. If methods, events, or properties aren't virtual, then I don't think DynamicMock can replace their behaviour with recording and playback versions.
Personally, I like to define interfaces for the classes I'm going to mock out and then mock the interface. That way, I'm sure to avoid this kind of problem.