How to properly account for system failure when deducting a users balance and calling a 3rd party service? - postgresql

Question

Presume a user would like to withdraw $5.00 balance on a website for PayPal balance. So they hit a withdraw endpoint which makes sure they have enough balance, calls PayPal pay api, and deducts the users on-site balance in a single serializable transaction.
What would happen if the database server drops and the transaction fails to commit after the PayPal pay request is executed successfully and the users gets their on-site balance back?
Is there a way I can encapsulate all of these calls in one atomic transaction?

Answer 1

I assume you are looking for two-phase commit
https://www.postgresql.org/docs/current/static/sql-prepare-transaction.html
PREPARE TRANSACTION prepares the current transaction for two-phase
commit. After this command, the transaction is no longer associated
with the current session; instead, its state is fully stored on disk,
and there is a very high probability that it can be committed
successfully, even if a database crash occurs before the commit is
requested.
Once prepared, a transaction can later be committed or rolled back
with COMMIT PREPARED or ROLLBACK PREPARED, respectively.
(empasis mine)

Answer 2

You shouldn't try to do this atomically. The nature of internet APIs makes this impossible.
You should probably do something resembling this pseudocode:
payment_id = random_payment_id()
try:
db:
insert into payments (payment_id, order_id, payment_amount, status, created)
values (:payment_id, :order_id, :payment_amount, 'pending', now());
commit;
remote.create_payment(payment_id, payment_amount);
except remote.error:
throw payment_error
On payment confirmation:
try:
remote.execute_payment(payment_id);
db:
update payments set status='completed' where payment_id=:payment_id;
commit;
except remote.error:
throw payment_error
And periodically you have to check a status of 'pending' payments, as you can't be sure that you'll receive all payment confirmations:
db:
select payment_id from payments
where status='pending' and created<now()-'10 minutes';
for payment_id in db.result:
if remote.payment_status(payment_id) == 'approved':
remote.execute_payment(payment_id);
db:
update payments set status='completed' where payment_id=:payment_id;
commit;
You should also periodically clean expired unconfirmed payments:
db:
select payment_id from payments
where status='pending' and created<now()-'10 days';
for payment_id in db.result:
remote.cancel_payment(payment_id);
db:
update payments set status='failed' where payment_id=:payment_id;
commit;