I have the following schema
ID (PK)| REF_ID | ACTIVE | STATUS
ID - Primary Key
I am using following query to select and update
BEGIN;
select * from table where ref_id = $1 and is_active is true for update;
UPDATE table set status = $1 where id =$2;
END;
Explanation for above
1) Select query result will be used to lock all the rows with provided ref ID and that result is used for some business logic
2) Update query to update the STATUS of a row which is part of same ref ID
ISSUE
postgres#machine ERROR: deadlock detected
postgres#machine DETAIL: Process 28297 waits for ShareLock on transaction 4809510; blocked by process 28296.
Process 28296 waits for ShareLock on transaction 4809502; blocked by process 28297.
Process 28297: select * from jobs where ref_id ='a840a8bd-b8a7-45b2-a474-47e2f68e702d' and is_active is true for update
Process 28296: select * from jobs where ref_id ='a840a8bd-b8a7-45b2-a474-47e2f68e702d' and is_active is true for update
postgres#machine ERROR: deadlock detected
postgres#machine DETAIL: Process 28454 waits for ShareLock on transaction 4810111; blocked by process 28384.
Process 28384 waits for ShareLock on transaction 4810092; blocked by process 28297.
Process 28297 waits for AccessExclusiveLock on tuple (113628,5) of relation 16817 of database 16384; blocked by process 28454.
Process 28454: select * from jobs where ref_id ='a840a8bd-b8a7-45b2-a474-47e2f68e702d' and is_active is true for update
Process 28384: select * from jobs where ref_id ='a840a8bd-b8a7-45b2-a474-47e2f68e702d' and is_active is true for update
Process 28297: select * from jobs where ref_id ='a840a8bd-b8a7-45b2-a474-47e2f68e702d' and is_active is true for update
This table is used in highly concurrent and distributed application (100's in parallel with same ref_id) and thats why i wanted to avoid distributed lock by having select and then update in same transaction.But i am facing with this deadlock error I don't know why explicit locking is not working.
Expected behaviour is that any other job with same reference ID must wait if any one else with same reference ID has acquired the lock
Help me figure out what I am missing or another workaround for this. I am still not clear even after explicit locking and being within transaction why is deadlock occurring.
As Laurenz said, in this simple case you should be able to eliminate the possibility of deadlock with an ORDER BY in your locking query.
A deadlock arises when, for example:
Process A acquires a lock on row 1
Process B acquires a lock on row 2
Process A requests a lock on row 2 (and waits for B to release it)
Process B requests a lock on row 1 (and waits for A to release it)
...And at this point, the processes will be waiting on each other forever (or rather, until the server notices, and kills one of them off).
But if both processes had agreed ahead of time to lock row 1 and then row 2, then this wouldn't have happened; one process would still be waiting on the other, but the other is free to proceed.
More generally, as long as all processes agree to follow the same ordering when acquiring locks, it's guaranteed that at least one of them is always making progress; if you only ever try to acquire locks which are "higher" than the ones you already hold, then whoever holds the "highest" lock will never be waiting on anyone.
The ordering needs to be unambiguous, and stable over time, so a generated primary key is ideal (i.e. you should ORDER BY id).
Related
I wrote a function which returns me a record ID which is in the PENDING state (the column state). And it should limit active downloads (records that are in states STARTED or COMPLETED). So, it helps me to avoid resource limits problems. The functions is:
CREATE FUNCTION start_download(max_run INTEGER) RETURNS INTEGER
LANGUAGE PLPGSQL AS
$$
DECLARE
started_id INTEGER;
BEGIN
UPDATE downloads SET state='STARTED' WHERE id IN (
SELECT id FROM downloads WHERE state='PENDING' AND (
SELECT max_run >= COUNT(id) FROM downloads WHERE state::TEXT IN ('STARTED','COMPLETED'))
LIMIT 1 FOR UPDATE)
RETURNING id INTO started_id;
RETURN started_id;
COMMIT;
END;
$$;
Is it safe in the meaning of race-conditions? I mean that the resources limit will not be hit because of race condition (2 or more threads will get ID of some PENDING record or even of the same record and the limit of active, i.e. STARTED/COMPLETED downloads will be reached).
In short this function should work as test-and-set procedure and to return the available ID (switched from PENDING to STARTED, but it's irrelevant details). Does it have such property - free from race conditions? Or maybe I have to use some locks...
PS. 1) downloads table has columns id, state (an enum with values like STARTED, PENDING, ERROR, COMPLETED, PROCESSED) and others that don't matter for the question's context. 2) max_run is the limit of active downloads.
Yes, there is a race condition in the innermost subquery.
If two concurrent sessions run your function at the same time, they both could find that max_run is equal to the count of started or completed jobs, and they would both start a job, thereby pushing the number of started or running jobs over the limit.
That is not easy to avoid, unless you lock all the started or completed jobs (very bad for concurrency) or use a higher transaction isolation level (SERIALIZABLE).
I use a Postgres 12 with autocommit = off and try to avoid deadlocks by using explict locking. When I execute:
ROLLBACK;
SELECT * FROM account WHERE id = 12345 FOR UPDATE;
where id is the primary key of account. Sometimes I get a deadlock on the second statement. I would have expected the execution to wait until all other locks on the line were released instead.
The server log usually shows me several (> 1) conflicting transactions for this. But all transactions working with a user should also lock the row as above.
How can such a deadlock as above occur and how can I avoid these deadlocks?
Edit: Amazingly, the locks of the other processes that the server log shows me are also in completely different tables, e.g.:
HINT: See server log for query details.
CONTEXT: while locking tuple (2892,8) in relation "account"
LOCATION: DeadLockReport, deadlock.c:1146
STATEMENT: SELECT * FROM account WHERE id = 197375 FOR UPDATE
LOG: 00000: process 17583 detected deadlock while waiting for ShareLock on transaction 1091990904 after 1000.057 ms
DETAIL: Process holding the lock: 17438. Wait queue: .
CONTEXT: while updating tuple (4588,22) in relation "subscription"
Edit 2: I found a second deadlock in the logs which is interesting:
Conflicting process A:
SQL statement "SELECT 1 FROM ONLY "public"."account" x WHERE "id" OPERATOR(pg_catalog.=) $1 FOR KEY SHARE OF x"
SQL statement "WITH inserted_rows AS (
INSERT INTO payment_token (
account_id, ..., blocking_time
)
VALUES (
account_id, ..., the_blocking_time
)
RETURNING *
)
SELECT * FROM inserted_rows"
The first statement comes not from my code directly, while the second Is part of a stored function.
Conflicting process B:
UPDATE account SET address_id = $2, update_time = CURRENT_TIMESTAMP WHERE id = $1
Recently we have started getting lot of deadlock errors in logs. (Postgres server 9.6.5)
Our table consist of two columns one is an auto-increment primary key , while other is a json object.
two attributes from json object are defined as unique .
Now in logs we keep on getting errors that two simple insert queries on different rows are blocking each other.
============
process 65325 detected deadlock while waiting for ShareLock on transaction 2934224126 after 1000.050 ms
DETAIL: Process holding the lock: 35530. Wait queue: .
CONTEXT: while inserting index tuple (128,10) in relation "A"
STATEMENT: INSERT INTO A AS t (info) VALUES('{"x":"y",....)
ERROR: deadlock detected
DETAIL: Process 65325 waits for ShareLock on transaction 2934224126; blocked by process 35530.
Process 35530 waits for ShareLock on transaction 2934224125; blocked by process 65325.
Process 65325: INSERT INTO A AS t (info) VALUES({"x":"y",....)
Process 35530: INSERT INTO A AS t (info) VALUES({"x":"z",....)
====================
So basically two different rows are in deadlock condition.
Is there any suggestion on what conditions such deadlocks may occur?
Rows can never be in deadlock. It is not two different rows, but two different transactions, that are in deadlock. Your log is showing you the most recent insertion attempt by each transaction. Presumably, there were previous inserts as well in each transaction. But those won't show up in the log, unless they show up for some other reason (like log_statement=all).
So if T1 successfully (and invisibly, looking at your log file) inserted "x":"y", the T2 successfully and invisibly inserted "x":"z", and now T1 tries to insert "x":"z" and T2 tries "x":"y", there will be a deadlock. Assuming the unique index is on info->>'x'
This would be the same issue if you were not using JSON.
Mitigations would be, don't insert more than one row per transaction. Or if you do, always insert them in a specified order (for example, "y" before "z" due to the latin alphabet ordering), although in this case you just replace the deadlock error with a unique key violation. Or, just be prepared to catch the deadlock and try again.
I would like to lock a table for writing during a period of time, while leaving it available for reading.
Is that possible ?
Ideally I would like to lock the table with a predicate (for example prevent writing rows "where country = france").
If you really want to lock against such inserts, i.e. the query should hang and only continue when you allow it, you would have to place a SHARE lock on the table and keep the transaction open.
This is usually not a good idea.
If you want to prevent any such inserts, i.e. throw an error when such an insert is attempted, create a BEFORE INSERT trigger that throws an exception if the NEW row satisfies the condition.
You can use FOR SHARE lock, which blocks other transactions from performing like UPDATE and DELETE, while allowing SELECT FOR SHARE. (Read the docs for details: https://www.postgresql.org/docs/9.4/explicit-locking.html [13.3.2])
For example, there are 2 processes accessing table user_table, in the following sequence:
Process A: BEGIN;
Process A: SELECT username FROM user_table WHERE country = france FOR SHARE;
Process B: SELECT * FROM user_table FOR SHARE; (In here, process B can still read all the rows of the table)
Process B: UPDATE user_table SET username = 'test' WHERE country = france; (In here, process B is blocked and is waiting for process A to finish its transaction)
This is what the sp does:
SET TRANSACTION ISOLATION LEVEL Serializable
begin transaction
IF EXISTS( SELECT 1 FROM dbo.Portfolio WHERE RawMessageID = #RawMessageID)
begin
----do some cleaning job like delete the portfolio, or something
end
insert into Portfolio
select #a,#b,#c,#d
commit
Randomly, I see deadlock happen and this deadlock graph showing the detail.
So it has to be that one instance of the call hold the shared lock from the select statement and asking for an null resource lock. Another instance holds the null resource lock and asks for shared lock.
Is my guess right? I was trying to write an simplified version to demo this, but never can trigger this deadlock.
Can someone help?