I have an application in which there are users, issuers, certificates and issues. users are the basic account, and issuers are the upgraded accounts who are able to distribute certificates to users. And distributing of certificates are called issues. The app uses postgres 14.4.
There is something called 'passive issue' in the application, which refers to issue of a certificate to a user that is not yet registered. Passive issue executes a transaction as follows:
check if user exists, if it does ignore it, if it does not, continue
create a passive issue which has awaiting register status
and then when that user actually registers, a trigger fires and updates the issue status.
The problem is that, after checking if user exists, and it does not, but right at that moment the user creates the account, and we try to passive issue, it will throw an error saying user already exists.
There are 2 possible approaches to solve this you might have thought so far:
just catch and check the error and redirect it to normal issuing path rather than passive issue within the api itself
return the error and let client retry the request by sending a request to normal issue path
The problem with above solutions:
It can not be done. Simply because this application uses blockchain, and normal issue endpoint requires the signature of the issuer. Passive issue is specifically implemented to be sent without a signature, and signature is generated on the client-side. And sending the private key to api is NOT even a possibility due to security concerns.
This is possible. This way the client will be noticed that this account registered, and it can prepare the signature and directly send a request to normal issue api. But it requires a longer and more complex implementation with retry logic etc.
And what I think would be the most neat solution in my case is such a scenario of passive issue:
check if user exists, and if it doesn't, lock the insert into the user table for that specific email ONLY
create passive issues for those accounts with await registration status (tx committed at this point and lock is released)
now even if the user tried to register in the middle, it will wait until issues are created and then user will be registered, and then trigger will fire, and since there are passive issues on this user now, it will update them.
So... Long story short: is there a way to put a lock on insertion of non-existing rows? And if there is, is it more feasible than the 2nd or any other possible solution?
Related
So, for the past week I was trying to do something with this kerberos issue.
Long story short, we have a server and it identifies incoming user do to some work. Recently, we needed to upload some of the results, so user is now delegated so it can authorize in another server. And the problem is that, that on the next day, after the logon, server can't delegate the same user. It fails with krb5_cc_notfound error while forming AP_REQ message.
If I try to iterate over the cache it fails at krb5_cc_start_seq_get with the same error.
If I try to get some other ticket (on day after successful logon) it fail at krb5_get_credentials, and on futher attempts (I guess cache becomes invalid? If it even was valid at this point) it could not resolve the default principal.
Cache type is MSLSA.
Every time after all the fails, in the event log I can see a kerberos warrning followed by an error. First says something like 'TGT was expired, an attempt to renew was made and failed' and the other one is KRB_AP_ERR_TKT_EXPIRED.
We have so few experience with kerberos in our room, so if you could share some of it - it would be cool.
It was a bug in a third party library. It basically stored the windows logon handle for eternity.
In our design we have something of a paradox. We have a database of projects. Each project has a status. We have a REST api to change a project from “Ready” status to “Cleanup” status. Two things must happen.
update the status in the database
send out an email to the approvers
Currently RESTful api does 1, and if that is successful, do 2.
But sometimes the email fails to send. But since (1) is already committed, it is not possible to rollback.
I don't want to send the email prior to commit, because I want to make sure the commit is successful before sending the email.
I thought about undoing step 1, but that is very hard. The status change involves adding new records to the history table, so I need to delete them. And if another person make other changes concurrently, the undo might get messed up.
So what can I do? If (2) fails, should I return “200 OK” to the client?
Seems like the best option is to return “500 Server Error” with error message that says “The project status was changed. However, sending the email to the approvers failed. Please take appropriate action.”
Perhaps I should not try to do 1 + 2 in a single operation? But that just puts the burden on the client, which is worse!
Just some random thoughts:
You can have a notification sent status flag along with a datetime of submission. When an email is successful then it flips, if not then it stays. When changes are submitted then your code iterates through ALL unsent notifications and tries to send. No idea what backend db you are suing but I believe many have the functionality to send emails as well. You could have a scheduled Job (SQL Server Agent for MSSQL) that runs hourly and tries to send if the datetime of the submission is lapsed a certain amount or starts setting off alarms if it fails as well.
If ti is that insanely important then maybe you could integrate a third party service such as sendgrid to run as a backup sending mech. That of course would be more $$ though...
Traditionally I've always separated functions like this into a backend worker process that handles this kind of administrative tasking stuff across many different applications. Some notifications get sent out every morning. Some get sent out every 15 minutes. Some are weekly summaries. If I run into a crash and burn then I light up the event log and we are (lucky/unlucky) enough to have server monitoring tools that alert us on specified application events.
I am developing a small REST API. As I got into analyzing all the possible failure scenarios, which I have to handle to create a reliable and stable system, I went into thinking about how to make my APIs atomic.
If we take a simple case of creating a contact through the POST API.
The server gets the POST request for the new contact.
Creates the contact in the DB.
Creates a response to send back to the client.
The server crashes before sending the response.
The client gets a timeout error (or connection refused?)
The client is bound to think that the contact creation has failed, though, in fact, the contact was in the DB.
Is this a rare case we can ignore? How do big companies deal with such an issue?
To handle this, you should make your write APIs idempotent i.e. If the same operation is executed multiple times, the result should be same as the operation was done only once.
To achieve this in your current example, you need to be able to identify a contact uniquely based on some parameter, say emailAddress. So, if the createContact is called again with the same emailAddress, check in the DB if a contact already exists with the emailAddress. If so, return the existing contact. Else, create a new contact with the emailAddress and return it.
Hope this helps.
If the request times out, the client should not make any assumption about whether it failed or succeeded.
If it is just a user making a request from a web form, then the timeout should just be exposed to the user, and they can hit the back button and check whether the operation succeeded or not, and if not they submit the request again. (This is fine as long as you always keep a consistent state. If your operation has multiple steps and fails mid way, you need to roll back.)
However if reliable messaging is important to your application you will have to use a library or build your own reliable messaging layer. This could work by having the client assign a unique ID to every request, and having another request that lets you check the result of that request ID later. Then you can do automated retries but only where necessary.
I've developed an app that allows users to upload some photos and share them on Facebook/Dropbox/Twitter etc. Recently it went live in the app store.
However, I'm having a problem now: a bot is creating accounts and uploading many photos on my server. I've temporarily disabled the app, but now I'm looking for an efficient way to prevent this bot from doing this.
The bot's ip address is changing very often so it's impossible to block the ip. He creates accounts with a very realistic name and email address so it's hard to find out which users are real and which are created by the bot.
I was thinking of using a captcha, but I'm not sure if my app will be rejected by Apple if I implement this. I'm preferably looking for a way so I can prevent him from doing his work and so I don't have to resend the app to Apple again.
Could anyone give me some advice on what I could possibly do?
Thanks!
This is how I solved a similar problem:
I implemented a token-generator, which generates a one-time token for every single data transfer with the server, so even one for login-data, sending a file etc. This token is generated by a secret algorithm and can be verified server side, since you know how you generate one.
After one token is used, put it in a temporary list for the next X minutes/hours/days (depending on how many data transfers your server can handle). When a user tries to send data with a used token (i.e. the token matches one in the "banned" list), you can be sure that someone's trying to spam you -> mark the account as "spammer" and decide what you wish to do.
The algorithm must produce a different token each time (the best way would be a one-way hash), but you have to assure specific "properties", with which you can proof its authenticity.
So one very simple example:
Your algorithm in the client is generating a number between 1000000000000000000000 and 99999999999999999999999, this number is then multiplied with 12456564 and incremented by 20349.
The server becomes a specific command and data, and the generated token. Now it checks, whether (number - 20349)%12456564 is 0. If it's 0, it was likely generated by your "secret" algorithm.
It's a very basic example but you get the idea…
I would like to capture the following parameters:
lastAccessedTime - The time when the user visited the site the last time (usually shown during the login process)
isOnline - A boolean to represent if a user is online or not.
a. Would it make sense to have these variables as part of the User table itself or should this be handled via a separate user audit table?
b. If certain SOAP / REST API's expose the functionality via API calls, how do you track the above parameters (e.g. Would you modify the lastAccessedTime in such cases - this might confuse the user if he logs into the portal, isOnline bit also will not make sense if the user does API calls).
I would create a session table that links back to the user. Instead of an isOnline field, I would just run a query for sessions that have been active within the last x amount of time. I would also update that session field with each request, even if that request is coming through an API.
This does create some overhead in pruning the session table, but you also don't clutter up your user table with non-user information, which can't be pruned.
Make the lastTimeActive a field in the user table, and update it with each page access. Your "Users Online" list is all users whose lastTimeActive is within 5 minutes.
I would create another table (userid, lastTimeActive), and frequently update & check the table.
// update
update onlineusers set lastTimeActive = getdate() where userid=1234
// check
delete from onlineusers where lastTimeActive < dateadd(minute,-5,getdate())
The biggest problem with tracking user presence (onine/offline) over HTTP is how to determine when the user has gone offline.
It's easy to determine when the user has come online - the mere presence of an authenticated request assumes that the user is active. However, since HTTP is stateless, the lack of a subsequent request can mean either that the user is gone offline, or that the user is online, but just hasn't done anything specific with your app recently.
Thus the best guess you can make is to have a timeout and if the user has not made a request during that timeout, to switch to offline state.
The simplest implementation would be to have a lastTimeActive, as Jonathan Sampson suggested. However, this won't give you the length of the user session, only an approximation of who's online at this moment.
More complex approach would be to have lastTimeActive and lastTimeLoggedIn. LastTimeLoggedIn is set at the time of first auth request that is more than 5 minutes from a previous auth request. A user is considered online, if there was an authenticated request in the last five minutes. The session length for the user is the time difference between lastTimeActive and lastTimeLoggedIn.
If your app also offers the choice of logging out to the user, you chouls consider that action also as going offline. However, unless your app is a banking app, chances are the users will just close their browser.
Also, avoid any background threads for updating the offline/online status of your users. You should be running the logic above only when there's an explicit request about the status of particular user and you should be updating only the users you were asked for.