The below query is getting locked by executing through perl
ALTER TABLE base_table
DETACH PARTITION part_table;
By using this query
select pid,state,
usename,
pg_blocking_pids(pid) as blocked_by,
query as blocked_query
from pg_stat_activity
where cardinality(pg_blocking_pids(pid)) > 0;
pid | state | usename | blocked_by | blocked_query
-------+--------+----------+------------+------------------------------------
20521 | active | parttest | {20306} | ALTER TABLE BASE +
| | | | DETACH PARTITION part_table
| | | |
(1 row)
Can you please suggest me how to solve this issue?
Related
I am currently using Postgres 11. For some reason whenever I check running autovacuums, I would see several multiple autovacuum: ANALYZE or autovacuum: VACUUM on the same table.
postgres=# SELECT backend_type,
pid,
now() - xact_start AS duration,
query,
state
FROM pg_stat_activity
WHERE query LIKE 'autovacuum%'
ORDER BY duration DESC LIMIT 64;
backend_type | pid | duration | query | state
-------------------+------+-----------------+-------------------------------------------------------------------+--------
autovacuum worker | 5900 | 00:48:29.175656 | autovacuum: ANALYZE someschema.foo | active
autovacuum worker | 3552 | 00:48:18.72438 | autovacuum: ANALYZE someschema.foo | active
autovacuum worker | 6122 | 00:47:29.907655 | autovacuum: VACUUM someschema.foo | active
autovacuum worker | 5897 | 00:42:45.380395 | autovacuum: VACUUM someschema.bar | active
autovacuum worker | 5583 | 00:38:45.406516 | autovacuum: VACUUM someschema.bar | active
The DDL looks like the following:
Table "someschema.foo"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------------+------+-----------+----------+---------+----------+--------------+-------------
foo_id | text | | not null | | extended | |
baz_id | text | | | | extended | |
Indexes:
"foo_pkey" PRIMARY KEY, btree (foo_id)
"baz_reference_id_idx" UNIQUE, btree (foo_id, baz_id)
Options: autovacuum_vacuum_cost_limit=2000, autovacuum_vacuum_cost_delay=10
I am unsure what is going on, and I cannot find my answers from the official documentations:
https://www.postgresql.org/docs/11/routine-vacuuming.html#AUTOVACUUM
https://www.postgresql.org/docs/11/sql-analyze.html
max_worker_processes = 15
autovacuum_max_workers = 64
I have a parent/child relationship between table T and T_CHILD. When I have an Active transaction SELECTING from the parent - T, and I try to drop the child table (T_CHILD), the DROP statement just hangs, as this session tries to get an Access Exclusivelock on both the parent and child tables.
1. Is this expected behavior?
2. Why should Postgres prevent a child table from being dropped, eventhough dropping the child table has no impact on the SELECT on the parent table? Can you please explain this?
dev_sporuran=# \d+ t
Table "public.t"
Column | Type | Modifiers | Storage | Stats target | Description
--------+---------+-----------+---------+--------------+-------------
id | integer | not null | plain | |
Indexes:
"t_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "t_child" CONSTRAINT "t_child_fk" FOREIGN KEY (t_id) REFERENCES t(id)
dev_sporuran=# \d+ t_child
Table "public.t_child"
Column | Type | Modifiers | Storage | Stats target | Description
--------+-----------------------+-----------+----------+--------------+-------------
id | integer | | plain | |
t_id | integer | | plain | |
name | character varying(10) | | extended | |
Foreign-key constraints:
"t_child_fk" FOREIGN KEY (t_id) REFERENCES t(id)
***********26727 session starts a SELECT ********
dev_sporuran=# begin;
BEGIN
dev_sporuran=# select * from t;
id
----
(0 rows)
*****26727 gets a shared lock for the SELECT on the table and the PK******
dev_sporuran=# select locktype,database,relation::regclass,transactionid,pid,mode,granted
from pg_locks where pid in (26727,26780) order by pid;
locktype | database | relation | transactionid | pid | mode | granted
------------+----------+----------+---------------+-------+-----------------+---------
relation | 19752133 | pg_locks | | 26727 | AccessShareLock | t
relation | 19752133 | t_pkey | | 26727 | AccessShareLock | t
relation | 19752133 | t | | 26727 | AccessShareLock | t
virtualxid | | | | 26727 | ExclusiveLock | t
(4 rows)
**************26827 tries to DROP the t_child table ************
dev_sporuran=# select pg_backend_pid();
pg_backend_pid
----------------
26867
(1 row)
dev_sporuran=# drop table t_child;
********* This just hangs;
Looking at pg_locks - 26867 requires not only an Access Exclusive lock on t_child, it also needs an Access Exclusive Lock on the parent table - T.
But it cannot get it, because 26727 has a Share lock on it thru the SELECT statement. So it has to wait*****
dev_sporuran=# select locktype,virtualxid,relation::regclass,transactionid,pid,mode,granted
from pg_locks where pid in (26727,26867) and relation::regclass in ('t','t_pkey','t_child') order by pid;
locktype | virtualxid | relation | transactionid | pid | mode | granted
----------+------------+----------+---------------+-------+---------------------+---------
relation | | t_pkey | | 26727 | AccessShareLock | t
relation | | t | | 26727 | AccessShareLock | t
relation | | t_child | | 26867 | AccessExclusiveLock | t
relation | | t | | 26867 | AccessExclusiveLock | f
(4 rows)
Thanks
Murali
Seems like something you could raise for discussion on pgsql-hackers. At a glance I don't see a good reason it shouldn't be ExclusiveLock not AccessExclusiveLock, so it'd still permit SELECT. But most table structure alterations require A.E.L.
Take a look at the ALTER TABLE lock strengths reduction patch for how complex this sort of thing can be.
After a system crash my Postgresql database does have a lock on a row.
The pg_locks table contains a lot of rows without a pid. i.e.
select locktype,database,relation,virtualtransaction, pid,mode,granted from pg_locks p1;
locktype | database | relation | virtualtransaction | pid | mode | granted
---------------+----------+----------+--------------------+-------+------------------+---------
relation | 16408 | 31459 | -1/40059 | | AccessShareLock | t
relation | 16408 | 31459 | -1/40059 | | RowExclusiveLock | t
relation | 16408 | 31022 | -1/40060 | | AccessShareLock | t
transactionid | | | -1/40060 | | ExclusiveLock | t
relation | 16408 | 31485 | -1/40060 | | AccessShareLock | t
How do I get the transaction 40060 killed and the locks removed?
Ok, solution found by myself:
Find the gid to the transaction (i.e. 40060 in the case above) by select * from pg_prepared_xacts where transaction = 40060;
Find an awful long gid.
ROLLBACK PREPARED gid;
This will clear the locks.
Im trying to create index on large table:
datid | datname | pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_start | xact_start | query_start | state_change | waiting | state | backend_xid | backend_xmin | query
-------+----------------------+-------+----------+-------------------------+--- --------------------------+--------------+-----------------+-------------+------ -------------------------+-------------------------------+---------------------- ---------+-------------------------------+---------+--------+-------------+----- ---------+---------------------------------------------------------------------- ---------------------
25439 | messengerdb | 30692 | 25438 | messengerdb_rw | pgAdmin III - Przegl??darka | 10.167.12.52 | | 50593 | 2016-08-11 05:27:12.101452+02 | 2016-08-11 05:28:01.535943+02 | 2016-08-11 05:28:01.535943+02 | 2016-08-11 05:28:01.535958+02 | t | active | | 1173740991 | CREATE INDEX CONCURRENTLY user_time_idx +
| | | | | | | | | | | | | | | | | ON core.conversations (user_id ASC NULLS LAST, last_message_timestamp ASC NULLS LAST);+
Is this query working? I'm worried about 'waiting' column === 't' does it mean that it is waiting for lock or sth?
Creating an index concurrently may take a long time since it does not lock the table from writes and it waits until other transactions are finished. However, it may wait forever if you have connections that stay idle in transactions (for example when a client or application keeps an open connection without rollback/commit).
Check if there are some idle connections in transactions (you should be able to see them in the processes list). You can also check PostgreSQL logs.
Section about creating index concurrently in PostgreSQL documentation can be helpful. There is also a nice article about concurrent indexes under this link.
I am trying to run a CREATE INDEX CONCURRENTLY command against a Postgres 9.2 database. I implemented a MigrationResolver as shown in issue 655. When this migration step is run via mvn flyway:migrate or similar, the command starts but hangs in waiting mode.
I verified that the command is executing via the pg_stat_activity table:
test_2015_04_13_110536=# select * from pg_stat_activity;
datid | datname | pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_start | xact_start | query_start | state_change | waiting | state | query
-------+------------------------+-------+----------+----------+------------------+-------------+-----------------+-------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------+---------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
21095 | test_2015_04_13_110536 | 56695 | 16385 | postgres | psql | | | -1 | 2015-04-13 11:10:01.127768-06 | 2015-04-13 11:13:08.936651-06 | 2015-04-13 11:13:08.936651-06 | 2015-04-13 11:13:08.936655-06 | f | active | select * from pg_stat_activity;
21095 | test_2015_04_13_110536 | 56824 | 16385 | postgres | | 127.0.0.1 | | 52437 | 2015-04-13 11:12:55.438927-06 | 2015-04-13 11:12:55.476442-06 | 2015-04-13 11:12:55.487139-06 | 2015-04-13 11:12:55.487175-06 | f | idle in transaction | SELECT "version_rank","installed_rank","version","description","type","script","checksum","installed_on","installed_by","execution_time","success" FROM "public"."schema_version" ORDER BY "version_rank"
21095 | test_2015_04_13_110536 | 56825 | 16385 | postgres | | 127.0.0.1 | | 52438 | 2015-04-13 11:12:55.443687-06 | 2015-04-13 11:12:55.49024-06 | 2015-04-13 11:12:55.49024-06 | 2015-04-13 11:12:55.490241-06 | t | active | CREATE UNIQUE INDEX CONCURRENTLY person_restrict_duplicates_2_idx ON person(name, person_month, person_year)
(3 rows)
An example project that replicates this problem can be found in my github: chrisphelps/flyway-experiment
My suspicion is that the flyway query against schema version which is idle in transaction is preventing postgres from proceeding with the index creation.
How can I resolve the conflict so that postgres will proceed with the migration? Has anyone been able to apply this sort of migration to postgres via flyway?
In the meantime, there is a Resolver included in flyway which looks for some magic in the filename.
Just add the prefix 'NT' (for No-Transaction) to your migration file, i. e.
V01__usual_migration_1.sql
V02__another_migration.sql
NTV03__migration_that_does_not_run_in_transaction.sql
V04__classical_migration_4.sql
etc.