In Java application I frequently see such errors:
org.springframework.r2dbc.UncategorizedR2dbcException: executeMany; SQL [select * from table1 where id=:id and column1 <> :a for update]; could not serialize access due to concurrent update; nested exception is io.r2dbc.postgresql.ExceptionFactory$PostgresqlTransientException: [40001] could not serialize access due to concurrent update
Transaction with query select * from table1 where id=:id and column1 <> :a for update was rollbacked.
Transaction isolation level - REPEATABLE READ.
How can I see what has locked this row? Lock is very short (milliseconds).
I see no helpful information in Postgres log and application log.
The problem here is not a concurrent lock, but a concurrent data modification. It is perfectly normal to get that error, and if you get it, you should simply repeat the failed transaction.
There is no way to find out which concurrent transaction updated the row unless you log all DML statements.
If you get a lot of these errors, you might consider switching to pessimistic locking using SELECT ... FOR NO KEY UPDATE.
Related
Can someone please explain to me the following situation:
I am using Postgresql 12 as main rdbms in my project, there are several background jobs accessing and writing to the database in parallel, also there are some user interactions (which of course produce updates and inserts to the database from the front of application)
Periodically i am getting exceptions like this one:
SQLSTATE[40P01]: Deadlock detected: 7 ERROR: deadlock detected
DETAIL: Process 18046 waits for ShareLock on transaction 212488; blocked by process 31036.
Process 31036 waits for ShareLock on transaction 212489; blocked by process 18046.
HINT: See server log for query details.
CONTEXT: while updating tuple (1637,16) in relation "my_table"
Inside my application i don't lock manually any rows or tables during my transactions. But i have 'large' transactions that can modify a lot of rows in single operation frequently. So the questions are:
Does ordinary transactions produce table-wide locks, or row-wide locks? (I assume yes, unless this whole situation is magic)
Shouldn't the rdbms resolve automatically this kind of problems when two queries are trying to modify the same resource, if they are wrapped inside transaction?
If answer to the second question is "no" then how i should handle that kind of situations?
re 1) DML statements only lock the rows that are modified. There is no lock escalation in Postgres where the whole table is locked for writes. There is a "table lock" but that is only there to prevent concurrent DDL - a SELECT will also acquire that. Those share locks don't prevent DML on the table.
re 2) no, the DBMS can not resolve this because a deadlock means tx1 is waiting for a lock to be released from tx2 and tx2 is waiting for a lock to be released by tx1. How would the DBMS know what to do? The only way the DBMS can solve this is by choosing one of the two sessions as a victim and kill the transaction (which is the error you see).
re 3) the usual approach to avoiding deadlocks is to always update rows in the same order. Which usually turns the deadlock into a simple "lock wait" for the second transaction.
Assume the following UPDATE sequence
tx1 tx2
-------------------------------
update id = 1 |
| update id = 2
update id = 2 |
(tx1 waits) |
| update id = 1
(now we have a deadlock)
If you always update the rows in e.g. ascending order this changes to:
tx1 tx2
-------------------------------
update id = 1 |
| update id = 1
| (waits)
update id = 2 |
|
|
commit; |
(locks released) |
|
| update id = 2
| commit;
So you don't get a deadlock, just a wait for the second transaction.
All SQL statements that affect tables will take a lock on the table (albeit not necessarily a strong one). But that doesn't seem to be your problem here.
All SQL statements that modify a row (or SELECT ... FOR UPDATE) will lock the affected rows. Your two transactions probably blocked on a row-level lock.
Yes; that is what the error message shows. PostgreSQL has resolved the deadlock by killing one of the involved transactions.
If transactoin 1 holds a lock that transaction 2 is waiting for and vice versa, there is no other way to resolve the situation. The only way to release a lock is to end the transaction that holds it.
You should catch the error on your application code and retry the database transaction. A deadlock is a transient error.
If you get a lot of deadlocks, you should try to reduce them. What helps is to keep your transactions short and small. If that is not an option, make sure that all transactions that lock several rows lock them in the same order.
I'm pretty new to PostgreSQL and I'm sure I'm missing something here.
The scenario is with version 11, executing a big drop table and insert transaction on a given table with the nodejs driver, which may take 30 minutes.
While doing that, if I try to query with select on that table using the jdbc driver, the query execution waits for the transaction to finish. If I close the transaction (by finishing it or by forcing it to exit), the jdbc query becomes responsive.
I thought I can read a table with one connection while performing a transaction with another one.
What am I missing here?
Should I keep the table (without dropping it at the beginning of the transaction) ?
DROP TABLE takes an ACCESS EXCLUSIVE lock on the table, which is there precisely to prevent it from taking place concurrently with any other operation on the table. After all, DROP TABLE physically removes the table.
Since all locks are held until the end of the database transaction, all access to the dropped table is blocked until the transaction ends.
Of course the files are only removed when the transaction commits, so you might wonder why PostgreSQL doesn't let concurrent transactions read in the mean time. But that would mean that COMMIT may be blocked by a concurrent reader, or a SELECT might cause a system error in the middle of reading, both of which don't sound appealing.
Imagine there is an open, ongoing transaction in Postgresql inserting a record and doing something else as well.
BEGIN
INSERT INTO films(id, name) VALUES(10, 'A comedy')
# We are at this moment in time
# The transaction is not yet committed
# ...
COMMIT
Is there any non-blocking way to discover from outside of this transaction that there is an ongoing transaction inserting record with ID=10 right now?
The only way I could think about was:
BEGIN
SET statement_timeout to 100
INSERT INTO films(id, name) VALUES(10, '') ON CONFLICT (id) DO NOTHING
ROLLBACK
if I get a timeout from INSERT than it means that there is another ongoing transaction
if I inserted nothing then there was a transaction which is now finished and conflict on unique ID occured
if the INSERT succeeded and thenI rolled-back than it means there is no transaction right now trying to insert a row with ID=10
However this is less than ideal as:
It is not non-blocking, I am waiting 100ms here
I am doing an INSERT operation, whereas I would prefer a read-only solution
As far as I understand I am actively triggering a conflict but I cannot in any easy way enforce that the 2nd transaction gets deadlock and that I won't ever interrupt the work of the 1st transaction
I am effectively trying to workaround the lack of READ UNCOMMITTED transaction isolation level in Postgresql.
I am in charge of both parts of the code so I can change them in any way necessary to allow it.
I am facing a deadlock problem from a PL/pgSQL function in my PostgreSQL database. Please find the SQL statement in the code block (just example):
BEGIN
UPDATE accounts SET balance = 0 WHERE acct_name like 'A%';
UPDATE accounts SET balance = balance + 100 WHERE acct_name like '%A';
EXCEPTION WHEN OTHERS THEN RAISE NOTICE SQLERRM;
END;
I've found that the deadlock occurred during this statement was running. But I'm not sure that there were other statements trying to update this table in the same time (because I didn't find any in my logging system).
So, is it possible that the deadlock occurred within this statement? As far as I know, if we blocked whole statement with BEGIN/END. There will be the same transaction and should not be locked by itself.
There is definitely some other process competing for the same resource. That is the nature of a deadlock. A function like you display can never deadlock itself. See comment by #kgrittn below, who is an expert on concurrency in PostgreSQL.
Your version of PostgreSQL is missing. Modern versions raise a detailed error message. Both processes that compete for resources are listed in detail with standard logging settings. Check your db logs.
The fact that you catch the error may prevent Postgres from giving you the full details. Remove the EXCEPTION block from your PL/pgSQL function, if you don't get the information in the db log and try again.
To alleviate deadlocks, you can do a number of things. If all your clients access resources in a synchronized order, deadlocks cannot occur. The manual provides the basic strategy to solve most cases in the chapter about deadlocks.
As for version 8.3: consider upgrading to a more recent version. In particular this improvement in version 8.4 should be interesting for you (quoting the release notes):
When reporting a deadlock, report the text of all queries involved in
the deadlock to the server log (Itagaki Takahiro)
Also, version 8.3 will meet its end of life in February 2013. You should start to consider upgrading.
A deadlock situation involving VACUUM should have been fixed in 8.3.1.
You would not get deadlock problem, if you add commit, to release exclusive locks.
BEGIN
UPDATE accounts SET balance = 0 WHERE acct_name like 'A%';
COMMIT;
UPDATE accounts SET balance = balance + 100 WHERE acct_name like '%A';
EXCEPTION WHEN OTHERS THEN RAISE NOTICE SQLERRM;
END;
In PostgreSQL, begin means that you start batch transaction.
Your first update will lock rows for accounts WHERE acct_name like 'A%';
Those rows are exclusively locked after first update.
Second update tries to open up exactly same rows as first update , to update
fail, because first update has NOT yet committed yet.
Thus second update hit deadlock was rollback.
In SQLServer, you can use syntax "(nolock)" to ensure the query doesn't lock the table or isn't blocked by other queries locking the same table.
e.g.
SELECT * FROM mytable (nolock) WHERE id = blah
What's the equivalent syntax in Postgres? I found some documentation on table locking in PG (http://www.postgresql.org/docs/8.1/interactive/sql-lock.html), but it all seems geared at how to lock a table, not ensure it's not locked.
A SELECT doesn't lock any table in PostgreSQL, unless you want a lock:
SELECT * FROM tablename FOR UPDATE;
PostgreSQL uses MVCC to minimize lock contention in order to allow for reasonable performance in multiuser environments. Readers do not conflict with writers nor other readers.
I've done some research and it appears that the NOLOCK hint in SQL Server is roughly the same as READ UNCOMMITTED transaction isolation level. In PostgreSQL, you can set READ UNCOMMITTED, but it silently upgrades the level to READ COMMITTED. READ UNCOMMITTED is not supported.
PostgreSQL 8.4 documentation for Transaction Isolation: http://www.postgresql.org/docs/8.4/static/transaction-iso.html
This is an old question, but I think the actual question has not been answer.
A SELECT query (that does not contain an for update clause) will never lock any rows (or the table) nor will it block concurrent access to the table. Concurrent DML (INSERT, UPDATE, DELETE) will also not block a SELECT statement.
Simply put: there is no need for (nolock) in Postgres.
Readers never block writers and writers never block readers
The purpose of the nolock or readpast is to see if the record is currenlty locked. The user can use this in an update to see if the record identified was changed (rowsaffected); if the record was not locked, then therowsaffected would be 1; if o, then the record is locked
Based upon that outcome, then the user can use a select for update to lock it for their own use.
Every SQL statement is an implicit transaction. The NOLOCK hint corresponds to READ UNCOMMITTED (DIRTY READ) transaction isolation level.
BEGIN TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
SELECT COUNT(1) FROM my_table;
END;
Actually, this code do the same that BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED but guarantees the expected behavior further.
Also avoid to use COUNT(*) except you really need it