Note: I'm using DBeaver 21.1.3 as my PostgreSQL development tool.
For my testing, I have created a table:
CREATE TABLE test_sk1(n numeric(2));
then I have disabled Auto-Commit on my DBEaver to verify whether I can see the blocking query for my other transaction.
I have then executed an insert on my table:
INSERT INTO test_sk1(n) values(10);
Now this insert transaction is un-committed, which will block the table.
Then I have taken another new SQL window and tried alter command on the table.
ALTER TABLE test_sk1 ADD COLUMN v VARCHAR(2);
Now I see the alter transaction got blocked.
But when I verified in the locks, I see that this Alter transaction got blocked by "Show search_path;' transaction, where I'm expecting "INSERT..." transaction as blocking query.
I used below query to fetch the lock:
SELECT p1.pid,
p1.query as blocked_query,
p2.pid as blocked_by_pid,
p2.query AS blocking_query
FROM pg_stat_activity p1, pg_stat_activity p2
WHERE p2.pid IN (SELECT UNNEST(pg_blocking_pids(p1.pid)))
AND cardinality(pg_blocking_pids(p1.pid)) > 0;
Why is this happening on our databases?
Try using psql for such experiments. DBeaver and many other tools will execute many SQL statements that you didn't intend to run.
The query that you see in pg_stat_activity is just the latest query done by that process, not necessarily the one locking any resource.
Related
According to official documentation, select query only need sharelock, but I found my select query acquired Exclusive lock. How did it happen? Here is my select query:
select gc.id
from group_access_strategy ga
left outer join person_group pg on gp.person_group_id=pg.id
where gp.id=3
what is different from official documentation is that I added left join.
Most likely you ran another command like ALTER TABLE person_group ... (Access Exclusive) or an UPDATE/INSERT/DELETE (Row exclusive) in the same transaction. Locks will persist until a transaction is completed or aborted.
So if you ran:
BEGIN; --BEGIN starts the transaction
UPDATE group_access_strategy SET column = 'some data' where id = 1;
SELECT
gc.id,
FROM
group_access_strategy ga
LEFT OUTER JOIN person_group pg ON (gp.person_group_id = pg.id)
WHERE
pg.id = 3
The UPDATE statement would have created a Row Exclusive Lock that will not be released until you end the transaction by:
Saving all of the changes made since BEGIN:
COMMIT;
OR
nullifying any of the effects of statements since BEGIN with
ROLLBACK;
If you're new to Postgres and typically run your queries in an IDE like PG Admin or DataGrip, the BEGIN / COMMIT ROLLBACK commands are issued behind the scenes for you when you click the corresponding UI buttons.
I'm using Npgsql 3.2 and it seems that the driver is now preparing statements automatically.
I see the following in the PostgreSQL logs:
LOG: execute <unnamed>:SELECT * FROM database.update_item($1, $2, $3 , $4)
I may be wrong but this is not activated on the connection string and I see no previous 'prepare' call in the logs.
What am I missing ?
Also I'm using Dapper 1.50.2 on top of Npgsql to query the database.
This is not yet implemented at this level but I see that there are some talks about such a feature on GitHub.
I'm using a READ COMMITTED transaction to update a row in the database. The row is updated twice with 2 disctinct statements.
When playing the statements one by one in a pgadmin query window it works fine.
When the statements are played by the driver through execute, the first statement seems to put locks on the record, and the second statement hang.
Here is the queries ran in the query window (runs to completion) :
BEGIN;
SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
SELECT * FROM database.update_item(params...);
SELECT * from database.update_item(params...);
ROLLBACK;
The corresponding PG logs:
LOG: statement: BEGIN;
LOG: statement: SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
LOG: statement: SELECT * FROM database.update_item(params...);
LOG: statement: SELECT * from database.update_item(params...);
LOG: statement: ROLLBACK;
The PostgreSQL logs when played by Npgsql:
LOG: statement: BEGIN
LOG: statement: SET TRANSACTION ISOLATION LEVEL READ COMMITTED
LOG: execute <unnamed>: select * from database.update_item(params...)
LOG: execute <unnamed>: select * from database.update_item(params...) <-- hangs
Any help appreciated.
EDIT 1:
Additional information: The update_item() plpgsql function is updating the row with an EXECUTE statement.
EDIT 2:
Added PG logs for query window.
First, Npgsql's automatic preparation feature is opt-in - it's not activated without the connection string. Even then, the same SQL needs to be executed several times (5 by default) before Npgsql prepares it. See the documentation for more details.
Regarding your deadlock, are you running your code concurrently? In other words, do you have multiple transactions at the same time, updating the same rows? If so, then this is probably the expected PostgreSQL behavior and has nothing to do with Npgsql. When you update rows in a transaction, these rows are locked until the transaction is committed. The fix in general is to update rows in the same order. See the PostgreSQL documentation for more details.
I'm reading this document on Explicit Locks and when they are automatically used by PostgreSQL.
What happens when one lock conflicts with another? Does the second transaction just wait until the first finishes? Does it abort?
So say some transaction opens up an ACCESS SHARE lock on table called apples. Then say another transaction tries to add a column issuing an ALTER TABLE query which is an ACCESS EXCLUSIVE lock. What happens to the second query? Does it hang? Abort?
The second query ALTER TABLE waits for the first transaction to complete. You can see that in pg_locks.
Select * from pg_locks;
To simulate this scenario:-
1) Open three separate SQL Editors in pgAdmin
2) SQL Editor 1: Execute the below statements
BEGIN;
Select * FROM table_name;
3) SQL Editor 2: Check the locks using pg_locks. There should be AccessShareLock
4) SQL Editor 3:
BEGIN;
alter table table_name ADD COLUMN new_column varchar(30);
The window should show Query is running.
5) SQL Editor 2: Check the locks using pg_locks. There should be AccessExclusiveLock
6) SQL Editor 1: Execute END; (i.e. Ending the select transaction)
7) SQL Editor 3: Query (i.e. alter table) should be executed successfully
8) SQL Editor 2: Check the locks using pg_locks. There shouldn't be any 'AccessShare' lock
9) SQL Editor 3: Execute END; (i.e. Ending the alter table transaction)
10) SQL Editor 2: Check the locks using pg_locks. There shouldn't be any 'AccessExclusive' lock.
Could you tell me why this query works in pgAdmin, but doesn't with software using ODBC:
CREATE TEMP TABLE temp296 WITH (OIDS) ON COMMIT DROP AS
SELECT age_group AS a,male AS m,mode AS t,AVG(speed) AS speed
FROM person JOIN info ON person.ppid=info.ppid
WHERE info.mode=2
GROUP BY age_group,male,mode;
SELECT age_group,male,mode,
CASE
WHEN age_group=1 AND male=0 THEN (info_dist_km/(SELECT avg_speed FROM temp296 WHERE a=1 AND m=0))*60
ELSE 0
END AS info_durn_min
FROM person JOIN info ON person.ppid=info.ppid
WHERE info.mode IN (7) AND info.info_dist_km>2;
I got "42P01: ERROR: relation "temp296" does not exist".
I also have tried with "BEGIN; [...] COMMIT;" - "HY010:The cursor is open".
PostgreSQL 9.0.10, compiled by Visual C++ build 1500, 64-bit
psqlODBC 09.01.0200
Windows 7 x64
I think that the reason why it did not work for you because by default ODBC works in autocommit mode. If you executed your statements serially, the very first statement
CREATE TEMP TABLE temp296 ON COMMIT DROP ... ;
must have autocommitted after finishing, and thus dropped your temp table.
Unfortunately, ODBC does not support directly using statements like BEGIN TRANSACTION; ... COMMIT; to handle transactions.
Instead, you can disable auto-commit using SQLSetConnectAttr function like this:
SQLSetConnectAttr(hdbc, SQL_ATTR_AUTOCOMMIT, SQL_AUTOCOMMIT_OFF, 0);
But, after you do that, you must remember to commit any change by using SQLEndTran like this:
SQLEndTran(SQL_HANDLE_DBC, hdbc, SQL_COMMIT);
While WITH approach has worked for you as a workaround, it is worth noting that using transactions appropriately is faster than running in auto-commit mode.
For example, if you need to insert many rows into the table (thousands or millions), using transactions can be hundreds and thousand times faster than autocommit.
It is not uncommon for temporary tables to not be available via SQLPrepare/SQLExecute in ODBC i.e., on prepared statements e.g., MS SQL Server is like this. The solution is usually to use SQLExecDirect.
In pgAdmin, if I execute an insert query, I don't see any way to either commit or rollback the statement I just ran (I know it auto commits). I'm used to Oracle and SQL developer, where I could run a statement, and then rollback the last statement I ran with a press of a button. How would I achieve the same thing here?
Use transaction in the SQL window:
BEGIN;
DROP TABLE foo;
ROLLBACK; -- or COMMIT;
-- edit --
Another example:
BEGIN;
INSERT INTO foo(bar) VALUES ('baz') RETURNING bar; -- the results will be returned
SELECT * FROM other_table; -- some more result
UPDATE other_table SET var = 'bla' WHERE id = 1 RETURNING *; -- the results will be returned
-- and when you're done with all statements and have seen the results:
ROLLBACK; -- or COMMIT
I also DEARLY prefer the Oracle way of putting everything in a transaction automatically, to help avoid catastrophic manual mistakes.
Having auto-commit enabled by default in an Enterprise product, IMO, is beyond vicious, and nothing but a COMPLETELY, UTTERLY INSANE design-choice :(
Anyways --- working with Postgres, one always needs to remember
BEGIN;
at the start of manual work or sql-scripts.
As a practical habit: then, when you would say: COMMIT;
in Oracle, I use the line
END; BEGIN;
in Postgres which does the same thing, i.e commits the current transaction and immediately starts a new one.
When using JDBC or similar, to create a connection, always use some method, e.g. getPGConnection(), that includes:
...
Connection dbConn = DriverManager.getConnection(dbUrl, dbUser, dbPassword);
dbConn.setAutoCommit(false);
...
to make sure every connection has auto-commit disabled.
If you are using pgAdmin4, you can turn the auto commit and/or auto rollback on and off.
Go to the File drop down menu and select Preferences option. In the SQL editor tab -> Options you can see the options to turn auto commit/rollback on and off.
Auto commit/rollback option