I have a set of resources represented by the /ticket/ endpoint. Users can POST a request to this endpoint in order to book a ticket, but this may fail due to various unpredictable situations (e.g. tickets have sold out).
I would like to be able to check before they start the booking process whether they will be able to complete the booking. For example, I make some status request and the server replies with "Nope, those are already sold out" and I don't bother filling in the booking form.
I feel like ideally this would be represented by some method on the same /ticket/ endpoint, but I don't know which one would be most appropriate. I guess I could create a new endpoint /ticket/availability or something but that doesn't feel as tidy.
I found this discussion, but it's about GET rather than POST: https://stackoverflow.com/a/10259779/1590962
Here's what I would do. First, I'd make a POST to a /reservation which in the backend will put my tickets into a "reserved" state. Then, I'd make a second POST to the /booking resource in order to complete the transaction.
If the first POST to reservation fails (tickets are sold out), then the second POST (the booking process, filling forms, etc.) would be cancelled.
Feel free to change the URL resource names as you see fit.
The problem with your suggestion (to check the availability first) is that it could respond with an "OK, tickets available", you start filling forms, another user starts the same process and finishes the booking process faster than you and takes the tickets. By the time you finish filling your forms, the tickets have been given to someone else.
In my case, if another user tries to make a reservation, it would fail because it would check only tickets that are not in a "reserved" state.
Related
I'm currently developing an online course system where students can choose any course and enroll to it. The course will be held one-to-one principle so student will choose a certain date for the course and on that date, there will be an online video meeting with the instructor. Only one instructor and one student for each course session.
Typical use-case flow is:
Student press "Enroll" button and proceeds to the next page.
On this page, student chooses the course date from the calendar (only from available dates) and proceeds to checkout page.
On checkout page, student enters his/her card details and certain amount charged from student.
A race condition may occur in this scenario (let's say there are only 2 users and 1 instructor):
User1 chooses date from the calendar and proceeds to the checkout page.
Meanwhile User2 also chooses exactly same date and proceeds to checkout page.
User2 enters card details faster that User1 and reserves that date.
User1 enters card details and system charges both students and BINGO (there are two students for the same date).
I don't want to check date availability before payment so I think It'll give a bad user experience so the User must go to the previous step again and choose another date. Even this could happen infinitely :)
Any ideas will be welcomed.
Also, I can change the current enrollment flow to protect security.
The reference you want to review is Pat Helland 2007: Memories, Guesses and Apologies
You've got a distributed system, and remote clients are looking at local copies of your data that may be out of date. So your protocol needs to recognize that you will be receiving messages about decisions based on stale data, and have explicit handling for the contingency that the desired outcome of the decision is not currently available.
The REST part is "just" providing the correct affordances for your protocol.
One possible change to your protocol that may help is to introduce the idea of a provisional hold; Alice has a provisional hold on the time slot, and therefore when Bob asks the slot is unavailable, but it might become available later if Alice declines to exercise the option.
(This doesn't eliminate the race condition, of course, it just moves it around).
A common protocol solution here is overbooking - you accept both claims on the time slot, and then clean up the mess later.
Commercial airlines do this sort of thing all the time; they want to maximize their profit per flight, which means selling more tickets than there are seats on the plane. They can do this, because enough travelers change their plans later that there is an effective surplus.
But sometimes, too many paying customers show up for the same flight, and then the contingency plans come out -- standby passengers are deferred, ticketed customers are offered compensation packages for changing their plans, and so on.
You probably need contingency protocols anyway (what happens if the instructor has to cancel the appointment, for example because of illness); the race condition during booking is just one more contingency protocol to add to the run book.
Having established what the contingency protocol should be, you then have a second question to explore: what parts of that protocol should be automated. If conflicts are rare, it may make sense to escalate the problem to a human being to solve, rather than doing so in code. Sometimes the right answer is for the machine to stay out of the way.
I want to implement double-check mechanism so:
When User1 proceeds to calendar page it will create a persistent connection with the server (SSE or WebSocket). And available dates will be shown on real-time. So when User1 selects any date and proceeds to checkout an event will be published and that date will be marked as BLOCKED until the payment done.
When User1 enters his card details and clicks Pay button the system will check again if that date is really reserved by User1.
If payment is successful this date will be updated from BLOCKED to RESERVED.
BUT taking into account that this is a REST API with React client, all of the endpoints will be visible to anyone. So an attacker could make a simple brute-force to BLOCK all available dates for the course.
I hope you are well, i am not a developer and i wanted to draw from the massive pool of expertise in here. I have an odd ish question that i can not accept the answer that i have been given as it does not add up from a security perspective.
the situation is that our API is passing a token with reference number for payment to a card payment provider which is Payment Card Industry Data Security Standard compliant, we do not want that responsibility hence we contract them. the customer enters all the details (name, card number etc etc) on the contractors site. They have a secure reporting portal that we do the reporting of daily transactions, refunds etc, so there is no need for us to have any data other than a reference number to marry it up with the token sent from us. It transpired earlier today that their API returns not only the token with the unique reference we need but the name, last 4 digits of card, address and other identifiable information, which we do not need or we want to have sight off.
The Contractor's reply was to and i quote "just ignore the data that it is return through the API and you do not need". I asked them what happens to that data a number of times and they did not provide a direct reply they just said other organisations use it that way with no issues...which as you expect have drove me absolutely bazurke.
i have found this 5 year old answer that says that disappears to the ether. I cant accept that data just disappears, insert GDPR concerns here.
What happens to unused function return values?
Apologies for the rant
TLDR: we sent token with unidentifiable personal information to card payment provider through API, Card Providers API returns name, card, address and other identifiable data. Card Providers response just ignore the information returned from the API you do not need.
thank you in advance for all your help.
So since you use a website to contact this API I will try to break down what is occurring.
You enter in a number on your website, which in turn becomes the key reference for the API call to the payment processor. The processor receives the ref number and grabs info pertaining to that number from their database. They then send this data as a response to your API call and the data is returned to the website. Now I am just speculating here but I am guessing your website does not do anything with this data, except display it. If this is the case the data is sitting in volatile memory, on the server the website is running on. Volatile memory (RAM) is memory that is not long lived, once space is needed it will be overwritten or if the system is turned off, it will be wiped immediately. Even when this data is in volatile memory it is only used in the context of your session on the website. Once you leave the page, there is no real way(easy anyway) to get that data back. It may still exist in RAM, but it is not accessible to anyone anymore and will be destroyed or overwritten once the server realizes it is not being used anymore.
There is a chance however that your website does save the API responses you get back in your own database. It sounds like this is not the case but I cannot be sure. But to ultimately answer the question, you can ignore this data and it is not very vulnerable or accessible to the outside world, so you don't have to worry about it getting into the wrong hands in this case. I hope this helps you some! Let me know if I can clarify anything for you further!
If no one uses the data, or looks at it, or stores it -- if it's just ignored -- then, yes, it disappears.
More specifically, in the computer that receives it, it's probably written into some space in volatile memory, and then the space is reused and overwritten the next time a response come in. Conceptually, at least.
It's possible that the receiving application has some kind of logs that are writing out data that is received, regardless of whether the app uses it or not, but other than that, without knowing what the app is doing, it's impossible to guess further.
I am developing a small REST API. As I got into analyzing all the possible failure scenarios, which I have to handle to create a reliable and stable system, I went into thinking about how to make my APIs atomic.
If we take a simple case of creating a contact through the POST API.
The server gets the POST request for the new contact.
Creates the contact in the DB.
Creates a response to send back to the client.
The server crashes before sending the response.
The client gets a timeout error (or connection refused?)
The client is bound to think that the contact creation has failed, though, in fact, the contact was in the DB.
Is this a rare case we can ignore? How do big companies deal with such an issue?
To handle this, you should make your write APIs idempotent i.e. If the same operation is executed multiple times, the result should be same as the operation was done only once.
To achieve this in your current example, you need to be able to identify a contact uniquely based on some parameter, say emailAddress. So, if the createContact is called again with the same emailAddress, check in the DB if a contact already exists with the emailAddress. If so, return the existing contact. Else, create a new contact with the emailAddress and return it.
Hope this helps.
If the request times out, the client should not make any assumption about whether it failed or succeeded.
If it is just a user making a request from a web form, then the timeout should just be exposed to the user, and they can hit the back button and check whether the operation succeeded or not, and if not they submit the request again. (This is fine as long as you always keep a consistent state. If your operation has multiple steps and fails mid way, you need to roll back.)
However if reliable messaging is important to your application you will have to use a library or build your own reliable messaging layer. This could work by having the client assign a unique ID to every request, and having another request that lets you check the result of that request ID later. Then you can do automated retries but only where necessary.
I have a scenario for my app which is similar to sending friend request in Facebook.
When user A sends friend request to user B, internally a new friend request document is created. At a later time when user B also wants to send friend request to A, system would find out that a friend request document existed and so they should be friend of each other, no new friend request document would be created.
I'm trying to figure out the case when user A and user B both simultaneously sends friend request to each other which will then create 2 friend request documents and leading to undetermined behaviour...
Thanks for your suggestions.. Really appreciated!
Edit:
A few had suggested to use a request queue to solve this; however,
I'm confused about using queue because i thought it would make my rest api endpoint process requests sequentially. Wouldn't I lose all the benefit of multi-threading by using queue? I can't help but imagine how bad it would be if my service has millions of requests queued and waiting to be executed one by one just due to this issue. Has anyone seen something along similar problems seen in production?
I had similar situation with my client which has concurrent writes in the database, What I have implemented is a Queue service.
Create a request in the queue rather than writing in the database, a separate reader will
read one message from the queue at a time and check if it is valid to write it to
database, write only if there is no previous request.
You can implement your own queue or you can use service like AWS-SQS, rabbitmq, MSMQ etc.
// Specific to your case
In mongodb write operations on a single document are atomic.
mongodb has a feature of unique index.
Hence if you insert the document with an _id(or any other unique index) with person names A and B by creating unique index for both (such as "A_B" by lexicographically sorting the names) before doing insertion. You will inherently be able to insert only one instance of that document.
// General
What essentially we would like to have are transactions but since mongodb doesn't support such, as of now. There are a few tricks to achieve this:
2 phase commits :
https://docs.mongodb.org/v3.0/tutorial/perform-two-phase-commits/
Using an external source to maintain a flag, for example using memcache which supports insertion in transactional manner/Compare and Swap.
Here if you use system calls method in frontend then you should fire one request to frontend from Database when some user like, I send you request then within a sec database send you one system call and your frontend code immediate correct the button text like
"Add a friend" to "incoming request"
or else.
if you are only setting up database then just make a system call which send it to UI when friend request arrives or as you say Document created, the further process will be handled by UI Developer.
Thank you.
if you don't like the answer then I m apologize for that but don't downvote me because I M new in Stack Overflow Community.
I would like to capture the following parameters:
lastAccessedTime - The time when the user visited the site the last time (usually shown during the login process)
isOnline - A boolean to represent if a user is online or not.
a. Would it make sense to have these variables as part of the User table itself or should this be handled via a separate user audit table?
b. If certain SOAP / REST API's expose the functionality via API calls, how do you track the above parameters (e.g. Would you modify the lastAccessedTime in such cases - this might confuse the user if he logs into the portal, isOnline bit also will not make sense if the user does API calls).
I would create a session table that links back to the user. Instead of an isOnline field, I would just run a query for sessions that have been active within the last x amount of time. I would also update that session field with each request, even if that request is coming through an API.
This does create some overhead in pruning the session table, but you also don't clutter up your user table with non-user information, which can't be pruned.
Make the lastTimeActive a field in the user table, and update it with each page access. Your "Users Online" list is all users whose lastTimeActive is within 5 minutes.
I would create another table (userid, lastTimeActive), and frequently update & check the table.
// update
update onlineusers set lastTimeActive = getdate() where userid=1234
// check
delete from onlineusers where lastTimeActive < dateadd(minute,-5,getdate())
The biggest problem with tracking user presence (onine/offline) over HTTP is how to determine when the user has gone offline.
It's easy to determine when the user has come online - the mere presence of an authenticated request assumes that the user is active. However, since HTTP is stateless, the lack of a subsequent request can mean either that the user is gone offline, or that the user is online, but just hasn't done anything specific with your app recently.
Thus the best guess you can make is to have a timeout and if the user has not made a request during that timeout, to switch to offline state.
The simplest implementation would be to have a lastTimeActive, as Jonathan Sampson suggested. However, this won't give you the length of the user session, only an approximation of who's online at this moment.
More complex approach would be to have lastTimeActive and lastTimeLoggedIn. LastTimeLoggedIn is set at the time of first auth request that is more than 5 minutes from a previous auth request. A user is considered online, if there was an authenticated request in the last five minutes. The session length for the user is the time difference between lastTimeActive and lastTimeLoggedIn.
If your app also offers the choice of logging out to the user, you chouls consider that action also as going offline. However, unless your app is a banking app, chances are the users will just close their browser.
Also, avoid any background threads for updating the offline/online status of your users. You should be running the logic above only when there's an explicit request about the status of particular user and you should be updating only the users you were asked for.