Event-Sourcing how to change business rules - cqrs

My application use cqrs and event sourcing. It's already in production.
Now i must add a business rules. My business rules are in my aggregate root UserAggregate.
My commands :
public class CallUserForMarketingPlanCommand
{
public Guid UserId {get;set;}
public DateTime CallDate {get;set;}
public Guid PlanId {get;set;}
}
public class AcceptMarketingPlanCommand
{
public Guid UserId {get;set;}
public Date AswerDate {get;set;}
public Guid PlanId {get;set;}
}
... the same thing for RefuseMarketingPlanCommand
these commands are applied on my aggregate root which generate events stored in event store
Now if 50 days after the call, the user do not give answer, the user must be recalled by operator. To do this, i think generate event UserDoNotRepliedInDelayEvent and use it to project to a read model with recall informations.
My solution is to create a deferred command (from UserCalledForMarketingPlanEvent handler) CheckUserAnswerCommand which check the call date and generate UserDoNotRepliedInDelayEvent if necessary across the aggregate. Ok.
My problem is how to deffered this command on users already in my event store (before this change) ?
EDIT :
Without considering deferred message, how to change business rules (or a business rules parameter) affecting the state of an aggregate. Simple example :
Disable account if two payments are not permformed.
this rule come with the first deployement. Ok, now there are 1000 accounts disabled. The boss change the rule because the business is impacted, and want disable account if 5 payments are not performed.
How to enable account having less than 5 payments not performed ?
Thanks for your help.

Now if 50 days after the call, the user do not give answer, the user must be recalled by operator. To do this, i think generate event UserDoNotRepliedInDelayEvent and use it to project to a read model with recall informations.
If I undestood your question correctly, the main point here, is that the user "not replying" in time is not an action (command) of your domain, quite the contrary, it is the absence of an action. So in this scenario, I don't think you need an event at all.
You simply need a read model which will register all sent invitations and their statuses (whether they're replied, their reply dates and how long did they stand unanswered). Then, you can check this read model for unanswered invitations that exceed your deadline of 50 days (which should be simple enough at this point).
So, up to this point, no new events are generated in your "Invitations" event store. You're simply interpreting the store into a specific read model that will answer you a question you have (which invitations were not answered).
From this point, it depends on your architecture.
You might want a recurring process to check this read model for invitations that exceed your deadline, having those specific invitations trigger a "InvitationExpiredEvent" or something to notify the interested parties (those who will resend them, for instance)
Or you simply might want a more passive approach, not needing an extra event, simply reading this Read Model when appropriate (on the GUI, maybe) and listing the expired invitations.
This will then fix itself... since you can generate the read model retroactively (finding users from any given point in the that never answered their invitations) and put them through the re-invitation pipeline.
Without considering deferred message, how to change business rules (or a business rules parameter) affecting the state of an aggregate. Simple example :
Disable account if two payments are not permformed.
this rule come with the first deployement. Ok, now there are 1000 accounts disabled. The boss change the rule because the business is impacted, and want disable account if 5 payments are not performed.
How to enable account having less than 5 payments not performed ?
This part of your question is more confusing. From what I understood, you once had a rule that stated "Accounts with two or more expired payments should be inactivated" and you want to change this rule to "Accounts with five or more expired payments should be inactivated". If that's the case, you have to deal with this on multiple levels...
First, you must first implement the new rule on your command model, the same way it always have been but with the updated parameter.
Second, you cannot retroactively reactivated accounts with 2,3,4 expired payments by ignoring their "deactivation events". From your event store point of view, this happened and you must abide by the rules that an event store is a "push only" storage. So, you must use compensating events to reactivate them after the rule change.
So, if you took care of the first topic (and your domain is up and running with the new rule) and since you can't take a shortcut because of the second topic, one of your easier options is to simply develop a one-shot operation that will find accounts with 2,3,4 expired payments that are currently disabled and append to their event stores a reactivation event. At this point you will have to regenerate any affected read models if your architecture doesn't do this automatically.
That way, the next time commands are executed against these accounts, their event stores will reflect the fact that they have been reactivated and thus are currently active.
From an event store point of view... each of these accounts will have something like this on their event streams:
... > Payment Expired > Account Disabled > (maybe other stuff happened) > Account Re-Enabled
So your event store will be a pretty accurate representation of your business scenario... once you chose to disable accounts with only 2 expired payments, so a certain account was disabled by that... later you changed your mind, and even without paying their debts, these accounts were re-enabled.
EDIT:
In fact, i think the problem can be summarized by "how to integrate retroactive rules in event sourced system"
If that's the case, than the answer will be more focused on the lines of "there shouldn't be retroactive actions in an event-sourced domain".
As I said in my original answer, an event stream should be a "push-only" storage and that's mainly because only the exact order of events, as they happened, can guarantee the integrity of your rules as they were when those events happened. In that sense, an event storage is less flexible than a traditional one as it will be way more sensitive to external interference and that will sometimes be a pain (were used to meddling with the data sources directly to fix stuff).
However, we should really try to keep the rule and acknowledge that whatever happened, happened and can't be changed. What you can do, is add, to the end of event stream "compensation events" that is, new events that will register a change of state at a given time to reflect your rules-changing. And then you will need a one-shot process to go through your entities and decide which of them are eligible for such a compensating event.
Now, of course, rules are meant to be broken when needed and with enough consideration, you can go wild into the event store. Just know the risks. If you choose to go "full time machine mode" into the event store, the main risks you will face (and should guard against) are:
Entities going into invalid states in their lifetime. It doesn't matter the entity "ends" the event stream in a valid state. You must validate it never enters an invalid state as that is a prerequisite of event streams. So, for each entity affected by your editing, you will need to evaluate its validity step by step through the new event stream.
Mismatches between source code and event stream. This is a little trickier. But one of the maneuvers you can pull with an event sourced system is rollback your source code repository to a given date and "discard" events from that date forward. That way, you can re-execute actions as they would have happened in the past. If you edit past events though, you might face situations where the recorded events don't match what would have happened in the past based on source code. That might be critical and extremely misleading in the future. You should monitor that.
If your architecture integrates different contexts/domains/microservices, that might also need further evaluation. Say context-A issued a cross-boundary message to context-B because of a given state of an entity. Moving forward, you change the entity state by meddling with the event stream. Now there's a chance these contexts might be left inconsistent between them as context-B believes that entity had a state it no longer has. This might be very relevant in your scenario.

you could also use a Saga that keeps track of the process and than create a command like "recallneeded" when the time is up. it also keeps track of events that tells the Saga to complete if there was a call within the 50 days. (Keep in mind that a Saga is part of your Domain logic and acts as an AR if doing DDD)

Related

Is it correct performing GET requests and checks inside a POST handler?

I'm designing a ticket booking API. Right now booking a ticket resolves into POST /users/{id}/tickets but each /events/{id} has a maximum of available tickets. How do I properly design a check?
I've come up with two ways:
1) having an availibleTickets: field into the /events/{id} that gets checked and possibly updated each time I POST a new ticket.
2) having a maxTickets: field into /events/{id} and check the length of GET /events/{id}/tickets array, compare it to maxTickets
Anyway I have to perform a GET request inside the POST handler but it doesn't look right to me, do you have any suggestions?
How would you desing a ticketing system for a Web page? The same steps you apply to a Web page also apply to REST as it is just a generalization of the same interaction flow used on the Web.
Usually, on the Web you have a link you can see an event you can order tickets for. On this page you have a link to order tickets for that particular show. Depending on the system you use, you might see a layout of the event venue in the form of buttons or images to click if there is a certain seat order where available seats are marked as green and ones that are already booked as red or whatever color scheme you use. A click on a seat will trigger some reservation logic on the server that returns almost the same page as before but this time with the seat marked as orange to indicate a reservation. Next you click the available seat next to that seat to reserve a further seat. This story continues until you either have enough seats marked as reserved or no available seats are available and you have no options left as to either cancel the reservation, proceed to the order step or unreserve seats you marked as reserved beforehand. Once you are satisfied with your choice, you will find an order or submit button or link where you turn your reservation into a booking. This might involve some further steps like entering your contact and/or billing information. Though this is in principle how I'd design such a system for the Web.
As you might see, this turns out into some kind of state machine where the server tells you all of the options you have available at this current state of the process. This is exactly what Asbjørn Ulsberg mentiones when talking about affordance and state machines. From the blueprint of the venue and the respective seats on that blueprint, which are actually buttons or images you might click, you knew what these widges are for and you somehow know what will happen when you click on one of the seats. This is what affordance is all about. By seeing it you know what you can do with it.
The interaction concept outlined above should be taken and translated to REST. As a client you don't need to know the structure of the URI, all you need to know is what seats are available and what happens when you click certain links. This is usually done in REST through link relation names that give the mentioned link some semantical context to the current state of the resource the client just fetched. Such link-relations may seem like a-priori knowledge needed by the client, which is a bit anti-REST, as REST tries to decouple clients from servers to allow the latter one to evolve freely without risking clients to break, though as link-relations should be standardized, or should be based on extensions, such as dublin-core or other microformats. Buidling up on standards will either lead to broad acceptance and support by different clients or on mechanisms to plug-in such knowledge into a client later on. This in general avoids so-called out-of-band information or process flows that force you to lookup up the manual on how to use that system.
The approach outlined above would utilize an own reservation resource that is uniquely created on "entering" the reservation, which is kept till the order ticket step is invoked. This reservation resource keeps track of the reserved seats the user has chosen so far. Whether the system considers reserved seats by other users as taken or not is an implementation detail. It is ok to either use a first-come system or a more polite one that guarantees the reserver his seats until some grace-period has passed and the user didn't order them. This gives you a good impression that such resources can be volatile and just be part of a certain process.
In regards whether to use GET, POST or other HTTP methods, a Web page that sends you to a reservation page will show you a form containing all of the seats of the venue. As HTML does only support GET or POST, the latter one is the most appropriate thing. In a REST or HTTP API you might use PUT though. A server might already have assigned you a certain, unique "reservation" link that you can just invoke with PUT. If the reservation resource does not exist yet, it will be created for you, if it did, the whole content will just be updated. Especially when you dealing with reservations and money flows you want to use idempotent methods such as PUT.
I hope I could give you some ideas on how you might design your reservation system by letting a server teach a client everything it needs to know to proceed through its task.
It's inside the post method (server-side) that you must check if tickets are available before book the event.
you can create a specific route to know how many tickets is available if needed. the client could call it before book an event. Or give the availibleTickets in the get /events/{id}
Imagine 10 client trying to buy the last ticket at the same time, if the security is not in the post method, you'll book 9 imaginary tickets

Client Interaction With Event Sourcing

I have been recently looking into event sourcing and have some questions about the interactions with clients.
So event-sourcing sounds great. decoupling all your microservices, keeping your information in immutable events and formulating a stored states off of that to fit your needs is really handy. Having event propagate through your system/services and reacting to events in their own way is all fine.
The issue i am having lies with understanding the client interaction.
So you want clients to interact with the system, but they need to do this now by events. They can not longer submit a state to mutate your existing one.
So the question is how do clients fire off specific event and interact with (not only an event based system) but a system based on event sourcing.
My understanding is that you no longer use the rest api as resources (which you can get, update, delete, etc.. handling them as a resource), but you instead post to an endpoint as an event.
So how do these endpoint work?
my second question is how does the user get responses back?
for instance lets say we have an event to place an order.
your going to fire off an event an its going to do its thing. Again my understanding is that you dont now validate the request, e.g. checking if the user ordering the order has enough money, but instead fire it to be place and it will be handled in the system.
e.g. it will not be
- order placed
- this will be picked up by the pricing service and it will either fire an reserved money or money exceeded event based on if the user can afford it.
- The order service will then listen for those and then mark the order as denied or not enough credit.
So because this is a async process and the user has fired and forgotten, how do you then show the user it has either failed or succeeded? do you show them an order confirmation page with the order status as it is (even if its pending)
or do you poll it until it changes (web sockets or something).
I'm sorry if a lot of this is all nonsense, I am still learning about this architecture and am very much in the mindset of a monolith with REST responses.
Any help would be appreciated.
The issue i am having lies with understanding the client interaction.
Some of the issue may be understanding, but I promise you a fair share of the issue is that the literature sucks.
In particular, the word "Event" gets re-used a lot of different ways. If you aren't paying very careful attention to which meaning is being used, you are going to get knotted.
Event Sourcing is really about persistence - how does a micro-server store its private copy of state for later re-use? Instead of destructively overwriting our previous state, we write new information that links back to the previous state. If you imagine each microservice storing each change of state as a commit in its own git repository, you are in the right ballpark.
That's a different animal from using Event Messages to communicate information between one microservice and another.
There's some obvious overlap, of course, because the one message that you are likely to share with other microservices is "I just changed state".
So how do these endpoint work?
The same way that web forms do. I send you a representation of a form, the client displays the form to you. You fill in your data and submit the form, the client processes the contents of the form, and sends back to me an HTTP request with a "FormSubmitted" event in the message body.
You can achieve similar results by sending new representations of the state, but its a bit error prone to strip away the semantic intent and then try to guess it again on the server. So you are more likely to instead see task based user interfaces, or protocols that clearly identify the semantics of the change.
When the outside world is the authority for some piece of data (a shopper's shipping address, for example), you are more likely to see the more traditional "just edit the existing representation" approach.
So because this is a async process and the user has fired and forgotten, how do you then show the user it has either failed or succeeded?
Fire and forget really doesn't work for a distributed protocol on an unreliable network. In most cases, at-least-once delivery is important, so Fire until verified is the more common option. The initial acknowledgement of the message might be something like 202 Accepted -- "We received your message, we wrote it down, here's our current progress, here are some links you can fetch for progress reports".
It doesnt seem to me that event-sourcing fits with the traditional REST model where you CRUD a resource.
Jim Webber's 2011 talk may help to prune away the noise. A REST API is a disguise that your domain model wears; you exchange messages about manipulating resources, and as a side effect your domain model does useful work.
One way you could do this that would look more "traditional" is to work with representations of the event stream. I do a GET /08ff2ec9-a9ad-4be2-9793-18e232dbe615 and it returns me a representation of a list of events. I append a new event onto the end of that list, and PUT /08ff2ec9-a9ad-4be2-9793-18e232dbe615, and interesting side effects happen. Or perhaps I instead create a patch document that describes my change, and PATCH /08ff2ec9-a9ad-4be2-9793-18e232dbe615.
But more likely, I would do something else -- instead of GET /08ff2ec9-a9ad-4be2-9793-18e232dbe615 to fetch a representation of the list of events, I'd probably GET /08ff2ec9-a9ad-4be2-9793-18e232dbe615 to fetch a representation of available protocols - which is to say, a document filled with hyper links. From there, I might GET /08ff2ec9-a9ad-4be2-9793-18e232dbe615/603766ac-92af-47f3-8265-16f003ce5a09 to obtain a representation of the data collection form. I fill in the details of my event, submit the form, and POST /08ff2ec9-a9ad-4be2-9793-18e232dbe615 the form data to the server.
You can, of course, use any spelling you like for the URI.
In the first case, we need something like an HTTP capable document editor; the second case uses something more like a web browser.
If there were lots of different kinds of events, then the second case might well have lots of different form resources, all submitting POST /08ff2ec9-a9ad-4be2-9793-18e232dbe615 requests.
(You don't have to have all of the forms submitting to the same URI, but there are advantages to consider).
In a non event sourcing pattern I guess that would be first put into the database, then the event gets risen.
Even when you aren't event sourcing, there may still be some advantages to committing events to your durable store before emitting them. See Pat Helland: Data on the Outside versus Data on the Inside.
So you want clients to interact with the system, but they need to do this now by events.
Clients don't have to. Client may even not be aware of the underlying event store.
There are a number of trade-offs to consider and decisions to take when implementing an event-sourced system. To start with you can try to name a few pre computer era examples of event-sourced systems and look at their non-functional characteristics.
So the question is how do clients fire off specific event
Clients don't send events. They rather should express an intent (a command). Then it is the responsibility of the event-sourced system to validate the intent and either reject it or accept and store the corresponding event. It would mean that an intent to change the system's state was accepted and the stored event confirms the change.
My understanding is that you no longer use the rest api as resources
REST is one of the options. You just consider different things as resources. A command can be a REST resource. An event-sourced entity can be a resource, to which you POST a command. If you like it async - you can later GET the command to check its status. You can GET an entity to know its current state. You cant GET events from a class of entities as a means of subscription.
If we are talking about an end user, then most likely it doesn't deal with the event store directly. There is some third tier in between, which does CQRS. From a user client perspective it can be provided with REST, GraphQL, SOAP, gRPC or event e-mail. Whatever transport solution you find suitable. Command-processing part from CQRS is what specifically domain-driven. It decides which intent to accept and which to reject.
Event store itself is responsible for the data consistency. I.e. it should not allow two concurrent event leading to invalid state be published. This is what pre-computer event-sourced systems are good at. You usually have some physical object as an entity, so you lock for update by just getting hand of it.
Then an end-user client usually reads from some prepared read model. The responsibility of a read (R in CQRS) component is to prepare read-optimised data for clients. This data may come from multiple event-sourced of the same or different classes. Again, client may interact with a read model with whatever transport is suitable.
While an event-store is consistent and consistent immediately, a read model is eventually consistent. But it's up to you to tune this eventuality.
Just try to throw REST out of the architecture for a while. Consider it a one of available transport options - that may help to look at the root.

Creating an UNDO flow for transacted fields

I've been thinking about the applications for goangular. In the need for immediate storage/database updates, such as a chat application or stocks application etc., I can see how goangular can be extremely useful in the sense of SignalR methodologies. But could it be applied to the traditional form with ten fields and a save button on it? All I could think of, was the traditional form, with ten fields on it -less the save button. If all ten fields are on the scope of the controller, than there would be no need for a save button. Every change of a field would be commemorated to the goinstant storage. Now having said that, how would one UNDO lets say any changes to those ten modified fields? Control+Z ten times? Not so robust. Any ideas on a UNDO all Changes button for such a form? (desperately trying to expand the bonds of real time database transactions)
I'll attempt to answer what I believe to be the spirit of your question first.
Most of the time, when using GoAngular, we're focused on synchronizing application state. Aka: Active clients sharing session data. Inevitably we drift into the territory of long-term persistence. At this point, rigorous validation / sanitization become a necessity, which we can't discuss without some context.
Let's say our user is completing their profile. This profile will be used to create a User model, which we will persist. Now that we have context, it becomes clear that we shouldn't persist a partially complete form, because it wouldn't represent a valid User model. We persist the form once it is complete, and valid.
Implementing this is as simple as creating a custom $scope.onSubmit method and validating the form input before calling $save on our new $scope.user model.
Undo would be easy to implement too, if you use $scope.users.$add, a key will be generated and returned, you could use this key to remove the new user. If you wanted to roll-back a change, you'd need to implement some system for versions, and roll back to the previous version of that User.
Hope I've answered your question in here somewhere :)

New/Read Flags in CQRS

I am currently drafting a concept for a (mostly) HTML-based collaboration suite which I plan to implement using CQRS. This software will contain messages that can be sent to the user (which can either be read or unread, obviously) and other elements which shall be marked "new" if they were created after the last user login.
Hardly something new, but I am not quite sure how that would be correctly implemented using CQRS. As I understand it, Change of any kind should, without exception, only be possible via Commands. But creating commands for every single (new) element that is being accessed seems a bit too much, not to mention the overhead.
I don't know if I need it, but what would be the best way to implement a Last-Accessed Timestamp on elements. Basically the same problem like the above, with the difference that the change happens EVERY time the element is accessed, not only the first time for each user.
CQRS seems to be an awesome concept but it really needs more learning material. Can't wait till a book is released :)
Regards
[Edit] No one? Wouldn't have thought that this is such a complicated issue..
I assume you're using event-sourcing in which case once you allow your query-service/event-handlers to raise appropriate events then this becomes fairly easy to solve.
For your messages/elements; when handling the specific creation events of your elements either add to existing or create additional event-handlers, to store to a messages read-model with a status of new and appropriate information about the element.
As part of you're user login I don't see why you can't raise a user-logged-in event (from the security/query service depending on how your implementing authentication) to say the user has logged in. An event-handler could capture this and write the last-login timestamp to a specific user-last-login read-model.
In addition the user-logged-in event-handler would need to update all the new messages (for that user) to an unread status. Seeing as we're changing the status of the messages as the user logs in do you still need to store the last-login timestamp?
For your last-accessed timestamp, perhaps you could just work this into your query service as queries for your different elements complete. Raise a query-completed event with element id/type information.

Transactions in REST?

I'm wondering how you'd implement the following use-case in REST. Is it even possible to do without compromising the conceptual model?
Read or update multiple resources within the scope of a single transaction. For example, transfer $100 from Bob's bank account into John's account.
As far as I can tell, the only way to implement this is by cheating. You could POST to the resource associated with either John or Bob and carry out the entire operation using a single transaction. As far as I'm concerned this breaks the REST architecture because you're essentially tunneling an RPC call through POST instead of really operating on individual resources.
Consider a RESTful shopping basket scenario. The shopping basket is conceptually your transaction wrapper. In the same way that you can add multiple items to a shopping basket and then submit that basket to process the order, you can add Bob's account entry to the transaction wrapper and then Bill's account entry to the wrapper. When all the pieces are in place then you can POST/PUT the transaction wrapper with all the component pieces.
There are a few important cases that aren't answered by this question, which I think is too bad, because it has a high ranking on Google for the search terms :-)
Specifically, a nice propertly would be: If you POST twice (because some cache hiccupped in the intermediate) you should not transfer the amount twice.
To get to this, you create a transaction as an object. This could contain all the data you know already, and put the transaction in a pending state.
POST /transfer/txn
{"source":"john's account", "destination":"bob's account", "amount":10}
{"id":"/transfer/txn/12345", "state":"pending", "source":...}
Once you have this transaction, you can commit it, something like:
PUT /transfer/txn/12345
{"id":"/transfer/txn/12345", "state":"committed", ...}
{"id":"/transfer/txn/12345", "state":"committed", ...}
Note that multiple puts don't matter at this point; even a GET on the txn would return the current state. Specifically, the second PUT would detect that the first was already in the appropriate state, and just return it -- or, if you try to put it into the "rolledback" state after it's already in "committed" state, you would get an error, and the actual committed transaction back.
As long as you talk to a single database, or a database with an integrated transaction monitor, this mechanism will actually work just fine. You might additionally introduce time-outs for transactions, which you could even express using Expires headers if you wanted to.
In REST terms, resources are nouns that can be acted on with CRUD (create/read/update/delete) verbs. Since there is no "transfer money" verb, we need to define a "transaction" resource that can be acted upon with CRUD. Here's an example in HTTP+POX. First step is to CREATE (HTTP POST method) a new empty transaction:
POST /transaction
This returns a transaction ID, e.g. "1234" and according URL "/transaction/1234". Note that firing this POST multiple times will not create the same transaction with multiple IDs and also avoids introduction of a "pending" state. Also, POST can't always be idempotent (a REST requirement), so it's generally good practice to minimize data in POSTs.
You could leave the generation of a transaction ID up to the client. In this case, you would POST /transaction/1234 to create transaction "1234" and the server would return an error if it already existed. In the error response, the server could return a currently unused ID with an appropriate URL. It's not a good idea to query the server for a new ID with a GET method, since GET should never alter server state, and creating/reserving a new ID would alter server state.
Next up, we UPDATE (PUT HTTP method) the transaction with all data, implicitly committing it:
PUT /transaction/1234
<transaction>
<from>/account/john</from>
<to>/account/bob</to>
<amount>100</amount>
</transaction>
If a transaction with ID "1234" has been PUT before, the server gives an error response, otherwise an OK response and a URL to view the completed transaction.
NB: in /account/john , "john" should really be John's unique account number.
Great question, REST is mostly explained with database-like examples, where something is stored, updated, retrieved, deleted. There are few examples like this one, where the server is supposed to process the data in some way. I don't think Roy Fielding included any in his thesis, which was based on http after all.
But he does talk about "representational state transfer" as a state machine, with links moving to the next state. In this way, the documents (the representations) keep track of the client state, instead of the server having to do it. In this way, there is no client state, only state in terms of which link you are on.
I've been thinking about this, and it seems to me reasonable that to get the server to process something for you, when you upload, the server would automatically create related resources, and give you the links to them (in fact, it wouldn't need to automatically create them: it could just tell you the links, and it only create them when and if you follow them - lazy creation). And to also give you links to create new related resources - a related resource has the same URI but is longer (adds a suffix). For example:
You upload (POST) the representation of the concept of a transaction with all the information. This looks just like a RPC call, but it's really creating the "proposed transaction resource". e.g URI: /transaction
Glitches will cause multiple such resources to be created, each with a different URI.
The server's response states the created resource's URI, its representation - this includes the link (URI) to create the related resource of a new "committed transaction resource". Other related resources are the link to delete the proposed transaction. These are states in the state-machine, which the client can follow. Logically, these are part of the resource that has been created on the server, beyond the information the client supplied. e.g URIs: /transaction/1234/proposed, /transaction/1234/committed
You POST to the link to create the "committed transaction resource", which creates that resource, changing the state of the server (the balances of the two accounts)**. By its nature, this resource can only be created once, and can't be updated. Therefore, glitches committing many transactions can't occur.
You can GET those two resources, to see what their state is. Assuming that a POST can change other resources, the proposal would now be flagged as "committed" (or perhaps, not available at all).
This is similar to how webpages operate, with the final webpage saying "are you sure you want to do this?" That final webpage is itself a representation of the state of the transaction, which includes a link to go to the next state. Not just financial transactions; also (eg) preview then commit on wikipedia. I guess the distinction in REST is that each stage in the sequence of states has an explicit name (its URI).
In real-life transactions/sales, there are often different physical documents for different stages of a transaction (proposal, purchase order, receipt etc). Even more for buying a house, with settlement etc.
OTOH This feels like playing with semantics to me; I'm uncomfortable with the nominalization of converting verbs into nouns to make it RESTful, "because it uses nouns (URIs) instead of verbs (RPC calls)". i.e. the noun "committed transaction resource" instead of the verb "commit this transaction". I guess one advantage of nominalization is you can refer to the resource by name, instead of needing to specify it in some other way (such as maintaining session state, so you know what "this" transaction is...)
But the important question is: What are the benefits of this approach? i.e. In what way is this REST-style better than RPC-style? Is a technique that's great for webpages also helpful for processing information, beyond store/retrieve/update/delete? I think that the key benefit of REST is scalability; one aspect of that is not needing to maintain client state explicitly (but making it implicit in the URI of the resource, and the next states as links in its representation). In that sense it helps. Perhaps this helps in layering/pipelining too? OTOH only the one user will look at their specific transaction, so there's no advantage in caching it so others can read it, the big win for http.
I've drifted away from this topic for 10 years. Coming back, I can't believe the religion masquerading as science that you wade into when you google rest+reliable. The confusion is mythic.
I would divide this broad question into three:
Downstream services. Any web service you develop will have downstream services that you use, and whose transaction syntax you have no choice but to follow. You should try and hide all this from users of your service, and make sure all parts of your operation succeed or fail as a group, then return this result to your users.
Your services. Clients want unambiguous outcomes to web-service calls, and the usual REST pattern of making POST, PUT or DELETE requests directly on substantive resources strikes me as a poor, and easily improved, way of providing this certainty. If you care about reliability, you need to identify action requests. This id can be a guid created on the client, or a seed value from a relational DB on the server, it doesn't matter. For server generated ID's, use a 'preflight' request-response to exchange the id of the action. If this request fails or half succeeds, no problem, the client just repeats the request. Unused ids do no harm.This is important because it lets all subsequent requests be fully idempotent, in the sense that if they are repeated n times they return the same result and cause nothing further to happen. The server stores all responses against the action id, and if it sees the same request, it replays the same response. A fuller treatment of the pattern is in this google doc. The doc suggests an implementation that, I believe(!), broadly follows REST principals. Experts will surely tell me how it violates others. This pattern can be usefully employed for any unsafe call to your web-service, whether or not there are downstream transactions involved.
Integration of your service into "transactions" controlled by upstream services. In the context of web-services, full ACID transactions are considered as usually not worth the effort, but you can greatly help consumers of your service by providing cancel and/or confirm links in your confirmation response, and thus achieve transactions by compensation.
Your requirement is a fundamental one. Don't let people tell you your solution is not kosher. Judge their architectures in the light of how well, and how simply, they address your problem.
If you stand back to summarize the discussion here, it's pretty clear that REST is not appropriate for many APIs, particularly when the client-server interaction is inherently stateful, as it is with non-trivial transactions. Why jump through all the hoops suggested, for client and server both, in order to pedantically follow some principle that doesn't fit the problem? A better principle is to give the client the easiest, most natural, productive way to compose with the application.
In summary, if you're really doing a lot of transactions (types, not instances) in your application, you really shouldn't be creating a RESTful API.
You'd have to roll your own "transaction id" type of tx management. So it would be 4 calls:
http://service/transaction (some sort of tx request)
http://service/bankaccount/bob (give tx id)
http://service/bankaccount/john (give tx id)
http://service/transaction (request to commit)
You'd have to handle the storing of the actions in a DB (if load balanced) or in memory or such, then handling commit, rollback, timeout.
Not really a RESTful day in the park.
First of all transferring money is nothing that you can not do in a single resource call. The action you want to do is sending money. So you add a money transfer resource to the account of the sender.
POST: accounts/alice, new Transfer {target:"BOB", abmount:100, currency:"CHF"}.
Done. You do not need to know that this is a transaction that must be atomic etc. You just transfer money aka. send money from A to B.
But for the rare cases here a general solution:
If you want to do something very complex involving many resources in a defined context with a lot of restrictions that actually cross the what vs. why barrier (business vs. implementation knowledge) you need to transfer state. Since REST should be stateless you as a client need to transfer the state around.
If you transfer state you need to hide the information inside from the client. The client should not know internal information only needed by the implementation but does not carry information relevant in terms of business. If those information have no business value the state should be encrypted and a metaphor like token, pass or something need to be used.
This way one can pass internal state around and using encryption and signing the system can be still be secure and sound. Finding the right abstraction for the client why he passes around state information is something that is up to the design and architecture.
The real solution:
Remember REST is talking HTTP and HTTP comes with the concept of using cookies. Those cookies are often forgotten when people talk about REST API and workflows and interactions spanning multiple resources or requests.
Remember what is written in the Wikipedia about HTTP cookies:
Cookies were designed to be a reliable mechanism for websites to remember stateful information (such as items in a shopping cart) or to record the user's browsing activity (including clicking particular buttons, logging in, or recording which pages were visited by the user as far back as months or years ago).
So basically if you need to pass on state, use a cookie. It is designed for exactly the very same reason, it is HTTP and therefore it is compatible to REST by design :).
The better solution:
If you talk about a client performing a workflow involving multiple requests you usually talk about protocol. Every form of protocol comes with a set of preconditions for each potential step like perform step A before you can do B.
This is natural but exposing protocol to clients makes everything more complex. In order to avoid it just think what we do when we have to do complex interactions and things in the real world... . We use an Agent.
Using the Agent metaphor you can provide a resource that can perform all necessary steps for you and store the actual assignment / instructions it is acting upon in its list (so we can use POST on the agent or an 'agency').
A complex example:
Buying a house:
You need to prove your credibility (like providing your police record entries), you need to ensure financial details, you need to buy the actual house using a lawyer and a trusted third party storing the funds, verify that the house now belongs to you and add the buying stuff to your tax records etc. (just as an example, some steps may be wrong or whatever).
These steps might take several days to be completed, some can be done in parallel etc.
In order to do this, you just give the agent the task buy house like:
POST: agency.com/ { task: "buy house", target:"link:toHouse", credibilities:"IamMe"}.
Done. The agency sends you back a reference to you that you can use to see and track the status of this job and the rest is done automatically by the agents of the agency.
Think about a bug tracker for instance. Basically you report the bug and can use the bug id to check whats going on. You can even use a service to listen to changes of this resource. Mission Done.
You must not use server side transactions in REST.
One of the REST contraints:
Stateless
The client–server communication is further constrained by no client context being stored on the server between requests. Each request from any client contains all of the information necessary to service the request, and any session state is held in the client.
The only RESTful way is to create a transaction redo log and put it into the client state. With the requests the client sends the redo log and the server redoes the transaction and
rolls the transaction back but provides a new transaction redo log (one step further)
or finally complete the transaction.
But maybe it's simpler to use a server session based technology which supports server side transactions.
I think that in this case it is totally acceptable to break the pure theory of REST in this situation. In any case, I don't think there is anything actually in REST that says you can't touch dependent objects in business cases that require it.
I really think it's not worth the extra hoops you would jump through to create a custom transaction manager, when you could just leverage the database to do it.
In the simple case (without distributed resources), you could consider the transaction as a resource, where the act of creating it attains the end objective.
So, to transfer between <url-base>/account/a and <url-base>/account/b, you could post the following to <url-base>/transfer.
<transfer>
<from><url-base>/account/a</from>
<to><url-base>/account/b</to>
<amount>50</amount>
</transfer>
This would create a new transfer resource and return the new url of the transfer - for example <url-base>/transfer/256.
At the moment of successful post, then, the 'real' transaction is carried out on the server, and the amount removed from one account and added to another.
This, however, doesn't cover a distributed transaction (if, say 'a' is held at one bank behind one service, and 'b' is held at another bank behind another service) - other than to say "try to phrase all operations in ways that don't require distributed transactions".
I believe that would be the case of using a unique identifier generated on the client to ensure that the connection hiccup not imply in an duplicity saved by the API.
I think using a client generated GUID field along with the transfer object and ensuring that the same GUID was not reinserted again would be a simpler solution to the bank transfer matter.
Do not know about more complex scenarios, such as multiple airline ticket booking or micro architectures.
I found a paper about the subject, relating the experiences of dealing with the transaction atomicity in RESTful services.
I guess you could include the TAN in the URL/resource:
PUT /transaction to get the ID (e.g. "1")
[PUT, GET, POST, whatever] /1/account/bob
[PUT, GET, POST, whatever] /1/account/bill
DELETE /transaction with ID 1
Just an idea.