I am new in the topic reactive programming and therefore have some questions.
I am developing a small software.
I would like to take the opportunity to get to know reactive programming better.
So I looked at Spring's project-reactor.
I also use R2DBC to reactively access the database.
I would like to know if there is any way that database responds to changes.
Or rather: If a user saves an entry in the database, then servers (for example, RestController) should be notified.
How could I go about doing that?
Enresponding controllers, configuration, entities, etc. I have already implemented.
Sorry for spelling mistakes.
Complement: The updates to the frontend are then made by Server Sent Events.
Basically, what Nick Tsitlakidis mentioned. Let me add a couple of things here.
The typical database query pattern is to query for a number of records. Databases respond with their results and indicate that the query is complete once a server has sent all records to your application. If new records arrive while the query is active or after the query is complete, you do not see these changes immediately because the of isolation and in case the query is complete, then you no longer have a reference to the query.
The feature you're asking is event-driven consumption of data. Databases call this feature continuous queries. Some stores (such as MongoDB with Tailable cursors or Postgres Logical Decoding) come with features that allow keeping a cursor/query open and your client is able to receive continuous updates.
Kafka and JMS also follow the idea of sending (messages) that are consumed typically by listeners or even through a reactive stream.
So it all boils down to the technology that you're using.
My understanding is that reactor can't solve this problem for you on its own. If you want your application to respond (react) on some database change, then you need to identify who's making this change and implement some kind of integration there.
Example, if you have Service1 updating the database, and Service2 needs to respond then Service1 can either call Service2, or, you can emit an event from Service1 and listen for the event from Service2.
The first approach is simpler and easier to implement but it has the disantvantage that is couples the two services. The second is trickier to implement but services are decoupled.
Reactor can help you in both cases :
For events, reactor can give you a way to listen to the events. For example using the reactor-rabbitmq module or the reactor-kafka.
For service-to-service calls, reactor can help you if you use Spring Webflux.
Perhaps you can tell us more about your case so we can provide a more specific solution?
Related
Context
Greetings,
One day I randomly found RethinkDB and I was really fascinated by the whole real-time changes thing. In order to learn how to use this tool I quickly spinned up a container running RethinkDB and i started making a small project. I wanted to make something very simple therefore i thought about creating a service in which speakers can create room and the audience can ask questions. Other users can upvote questions in order to let the speaker know which one are the best. Obviously this project has a lot of realtime needs that i believe are best satisfied by using RethinkDB.
Design
I wanted to use a vary specific set of tools for this. The backend would be made in Laravel Lumen, the frontend in Vue.JS and the database of course would be RethinkDB.
The problem
RethinkDB as it seems is not designed to be exposed to the end user directly despite the fact that no security concern exists.
Assuming that the user only needs to see the questions and the upvoted in real time, no write permissions are needed and if a user changed the room ID nothing bad will happen since the rooms are all publicly accessible.
Therefore something is needed in order to await data updates and push it through a socket to the client (socket.io for example or pusher).
Given the fact that the backend is written in PHP i cannot tell Lumen to stay awake and wait for data updates. From what i have seen from the online tutorials a secondary system should be used that should listen for changes and then push them. (lets say a node.js service for example)
This is understandable however i strongly believe that this way of transferring the data to the user is inefficient and it defeats the purpose of RethinkDB.
If I have to send the action from the client's computer (user asks a question), save it to the database, have a script that listens for changes, then push the changes to socket.io and finally have the client (vue.js) act when a new event arrives, what is the point of having a real-time database in the first place?
I could avoid all this headache simply by having the Lumen app push the event directly to socket.io and user any other database system instead.
I really cant understand the point of all this. I am not experienced with no-sql databases by any means but i really want to experiment with them.
Thank you.
This is understandable however i strongly believe that this way of transferring the data to the user is inefficient and it defeats the purpose of RethinkDB.
RethinkDB has no built in mechanism to transfer data to end-users. It has no access control (in the conventional sense) as well. The common way, like you said, is to spin up one / multiple node instance(s) running socket.io. On each instance you can listen on your RethinkDB change streams and use socket.io's broadcast functionality. This would be a common way, but as RethinkDB's streams are pretty optimized, you could also open a change stream for every incoming socket.io connection.
Usually when talk implementing CQRS it is supposed that commands are sorces for events. But can queries made by user be source of created events in event store? Or such actions (when we need an event that reflects query) should be implemented using command still?
But can queries made by user be source of created events in event store?
Go not to the elves for counsel, for they will answer both no and yes.
So, the "no" part: queries are distinguished by the fact that they don't change the domain model. In a CQRS implementation, the queries are being served by the read model, which may not even have access to your event store at all.
when we need an event that reflects query
The yes part: there's no law that says you can't assemble a history of queries, and stick that in your event store.
But I'm stumped, in that I don't see a clear case where the domain needs an event that reflects a query. That's really weird. My guess would be that needing an event that reflects a query is a hint that your model is broken.
You may be able to make some progress with this by exploring the source of the requirement.
If the requirement is coming from operations, analytics, reporting, usability... then the domain model probably isn't the right place for that information.
If the requirement is coming from your domain experts ("we need to capture these queries so that the model supports the right changes later"), then you should be looking to identify what entity is responsible for tracking that the query happened, and sending an appropriate command to that entity.
I recently read a lot about CQRS and for me it seems like it's closely tied up to Event Sourcing.
But like this answer said https://stackoverflow.com/a/9217461/277067
For me Event SOurcing seems a bit too complicated/scary for a beginners like me ("what ? my object current state is nto stored anywhere ??").
So i'd like to know if indeed they are tied up or if there is any tools/famework that would help for doing cqrs (event observer, command handler) without the complicated part of event sourcing.
Thanks
Short answer: No, CQRS and event-sourcing are not tied to each other.
Long answer: No, CQRS and event-sourcing are not tied to each other, and they aren't tied as well to domain-driven design (DDD).
If you want to define what CQRS, event-sourcing and DDD are in a few words, you may come up with explanations as the following ones (yes of course, they are over-simplified, but that's exactly the point here):
CQRS is a design pattern that separates writing state from reading state (commands vs queries).
Event-sourcing is a way to store data in a database, where the deltas are stored rather than the actual state.
DDD is a method to make communication on the domain easier within interdisciplinary teams.
Each of them works without the others very well. E.g., you can model a domain using DDD, and then implement it without CQRS or event-sourcing. You may also do event-sourcing without ever needing DDD or CQRS. And so on…
But: The three concepts play very well together, which is why they are often called together within a single sentence. So, no they aren't tied to each other, but they make a lot of sense in combination with each other.
The following picture shows how they may interact with each other:
(The image is taken from the documentation of wolkenkit, a CQRS and event-sourcing framework for JavaScript and Node.js.)
CQRS describes that you send commands to the write model, and that you receive events and subscribe to queries from the read model.
Event-sourcing is used with the write model to store the events that are published as result of the commands the client sends.
DDD is used within the write model to turn commands into events and to run the appropriate logic.
You can use CQRS without Event Sourcing. In command handler you are using some Repository to get or save last state of aggregate root. Just implement simple Repository, wich will save and load state straight from database.
No they are not tied up IMO, you can find my rationale to a related question here
in a short answer, I should say: we can have CQRS without event sourcing. but we can not have event sourcing without CQRS. in general, we have 3 type of CQRS: standard, event sourcing, and eventual consistency.
CQRS and EventSourcing are independent of each other. But based on requirement, we can combine them achieve great result.
Lets take some examples:
CQRS (with out event sourcing):
Ecommerce load balancer: Mostly all requests to ecommerce website are get requests, where user will browse through the available products. And there are sellers who will update these products and related information, but will do less frequently but in bulk. This seller requests can be served from one server and user requests can be served from other servers. Both these servers are fetching/updating data from same DB(single point of source).
Here there is no event sourcing. But we are able to split the read & write at load balancer level.
Database master/slave: Some times we can use slave to handle read requests if database throughput is high. Here again we are able to split read & write logic without event sourcing.
EventSourcing(with out CQRS):
Ecommerce callbacks: Lets say you want to send a mail/notification to customer regarding order confirmation or cancellation after an order state change. Here we can create an event after order state change and all the subscribers listening to that event will consume these events. In our example mail/notification class will listen to this event and will immediately send mail or notif. Here there is no CQRS involved.
Does anyone use ElasticSearch for building read model in CQRS approach? I have some questions related to such solution:
Where do you store your domain
events? In JDBC database? In
ElasticSearch?
Do you build indexes by event handlers that processes domain events or using ElasticSearch River functionality?
How do you handle complete rebuild of view model - for example in case when view is corrupted? Do you process all events to rebuid view?
Where the authoritative repository for your domain event is located is an implementation detail. For example, you could store serialized versions on S3 or CouchDB or any other number of storage implementations. The easiest if you're just getting started is a relational database.
Typically people use event handlers that understand the business intent behind each message and can then properly denormalize the message into the read model structure appropriate for the needs of your views.
If the view model is ever corrupted or perhaps you have a bug in a view model handler, there are a few simple steps to follow after fixing the bug:
Temporarily enqueue the flow of events arriving from the domain--these are the typical messages that are being published as your domain is doing work. We still want these messages, but not just yet. This could be done by turning off any message bus or not connecting to your queuing infrastructure if you use one.
Read all events from event storage. As each event is received (this can be done through a simple DB query), run each message through the appropriate message handler. Make sure that you keep track of the last 10,000 (or so) identifiers for all messages processed.
Now reconnect to your queues and start processing normally. If the identifier for the message has been seen, drop the message. Otherwise, process it normally.
The reason for tracking identifiers is to avoid a race condition where you're getting all events from the event store but the same message is coming across through the message queue.
Another technique that's highly related, but involves keeping track of all message identifiers can be found here: http://blog.jonathanoliver.com/2011/03/removing-2pc-two-phase-commit.html
My question is regarding the consuming event services which subscribes to the event published by commands in CQRS.
Say I have a document generation service which will generate some documents based on certain events, does the document generation service load the data from the domain via an aggregate root? If so, wouldn't the document generation service load data which may have been updated subsequently of the event being received by the generation service? How would you stop that from happening?
I guess I am assuming that the event will only pass the information received by command DTO and passing the whole domain model data to the event feels very wrong.
You really should build your read model from your events, unless you consider your documents a part of the domain (and you would have a CreateDocumentX command)
All i can say is that when you are speaking in cqrs you should describe the issue more deeply to solve or provide help to it properly.
However from what I've read is that you can have persistent storage on your write side, but ensure that you are not reaching out of your aggregate context.
Related issue reading-data-from-database-on-write-side-in-cqrs