ideally, Queries wait only for ExclusiveLock on a table but we saw a weird behaviour in our prod infra. select queries were waiting for accessShareLock on a view in postgres, any reason why it can be possible? Here is the output of lock monitoring query, pg_locks table of postgres
https://docs.google.com/spreadsheets/d/1xt0sfYicrDiPEdd3QdVVEm--cdHI3QGHSKb2jP2ofjI/edit#gid=1512744594
The only lock that conflicts with an AccessShareLock is an AccessExclusiveLock, which is tanken by statements like TRUNCATE, DROP TABLE or ALTER TABLE. So either one of these statements is currently running or is waiting for a transaction to finish (locks are queued).
Avoid long database transactions, and the problem will go away.
Related
I would like to change schema of few tables in my Postgres DB. Problem is that all the time there are long running queries and as I understand schema change needs exclusive lock.
Question is how can I do it? Of course I can kill all existing queries and try to do schema rename (move table to different schema) but there is a huge chance that in the meantime new queries will appear.
Thanks for help!
run SELECT pg_backend_pid() before running the ALTER TABLE
start the ALTER TABLE statement
in a second database session, run SELECT pg_blocking_pids(12345), where 12345 is the result from the first query
cancel all the blocking transactions found with the previous query with SELECT pg_cancel_backend(23456)
I have a table in Aurora Postgres 9.6 that is just:
create table myTable
(
id uuid default extensions.uuid_generate_v4() not null
blobs jsonb not null
);
The blobs can get rather large at time, but are usually a few MBs. And end up getting stored in the toast tables.
Under increased load, I started to see tables locks in Aurora replica (lock_relation) or AccessExclusiveLock in Postgres equivalent. Looking at pg_locks tables, it seems that the cause for the table lock is a system process.
We are not able to select any rows from that table. What I found is that once we kill the vacuum, the table locks is released and we can fetch rows.
Locks
id,locktype,database,relation,page,tuple,virtualxid,transactionid,classid,objid,objsubid,virtualtransaction,pid,mode,granted,fastpath
1,relation,16394,142767200,,,,,,,,1/0,33244,AccessExclusiveLock,true,false
Relation Translation
142767200 -> pg_toast.pg_toast_142767196
pg_toast.pg_toast_142767196 - mySchema.myTable
Vacuum Run
0 years 0 mons 0 days 1 hours 37 mins 47.688907 secs rdsadmin autovacuum: VACUUM pg_toast.pg_toast_142767196 active
Screenshots of locks
Questions:
My understanding is that an auto vacuum shouldn't cause table locks that interfere with other operations, why do I see those?
Is there any other system processes that is connected to auto vacuum that once killed, unlocks the table (pids don't match up directly)
I am used to do this in MySQL:
INSERT INTO ... SELECT ...
which would lock the table I SELECT from.
Now, I am trying to do something similar in PostgreSQL, where I select a set of rows in a table, and then I insert some stuff in other tables based on those rows values. I want to prevent having outdated data, so I am wondering how can I lock a SELECT in PostgresSQL.
There is no need to explicitly lock anything. A SELECT statement will always see a consistent snapshot of the table, no matter how long it runs.
The result will be no different if you lock the table against concurrent modifications before starting the SELECT, but you will harm concurrency unnecessarily.
If you need several queries to see a consistent state of the database, start a transaction with the REPEATABLE READ isolation level. Then all statements in the transaction will see the same state of the database.
We have thousands of tables. Out of these tables we have few tables. Which are busy some times. If I execute any ALTER statement or creating trigger on those tables I am unable to do it. How to check whether table is busy or free before running the ALTER or creating TRIGGER on that table in postgresql database.
The easiest way would be to run
LOCK TABLE mytable NOWAIT;
If you get no error, the ALTER TABLE statement can proceed without waiting.
Query below returns locked objects in a database.
select t.relname, l.locktype, page, virtualtransaction, pid, mode, granted
from pg_locks l, pg_stat_all_tables t
where l.relation=t.relid
order by relation asc;
I'm trying to perform ALTER TABLE on a huge table without full access locking.
According to ALTER TABLE documentation:
An ACCESS EXCLUSIVE lock is held unless explicitly noted
Hence I do the following:
BEGIN;
LOCK TABLE MyTable IN SHARE MODE;
ALTER TABLE MyTable ALTER COLUMN size SET DATA TYPE BIGINT;
COMMIT;
I expect that it gives me a possibility to perform SELECT queries during the upgrade.
But in fact it doesn't. Looking at pg_locks during the transaction I've found that:
SELECT c.relname, l.mode, l.granted, l.pid
FROM pg_locks as l JOIN pg_class as c on c.oid = l.relation
WHERE relname='MyTable' AND granted IS TRUE;
relname | mode | granted | pid
----------+---------------------+---------+------
MyTable | ShareLock | t | 2277
MyTable | AccessExclusiveLock | t | 2277
So AccessExclusiveLock unexpectedly was taken as well and it explains why my SELECTs are hanging until the end of the transaction
I use PostgreSQL 9.4
You seem to misinterpret the unless explicitly noted bit.
It means that, given that there are different actions grouped under ALTER TABLE, for some of them it's possible that a lock weaker than ACCESS EXCLUSIVE might be sufficient, and when it's in the case, it's explicity noted in the documentation.
For instance (from
https://www.postgresql.org/docs/9.5/static/sql-altertable.html ):
SET STATISTICS acquires a SHARE UPDATE EXCLUSIVE lock.
...
Changing per-attribute options acquires a SHARE UPDATE EXCLUSIVE lock.
...
Validation acquires only a SHARE UPDATE EXCLUSIVE lock on the table being altered. If the constraint is a foreign key then a ROW SHARE lock is also required on the table referenced by the constraint
It doesn't mean that the actions that require an ACCESS EXCLUSIVE lock (such as changing the type of a column) could be influenced by a previous explicit weaker lock grabbed on the table in the same transaction. They are going to need an ACCESS EXCLUSIVE lock no matter what.