I was trying to figure out why a query was taking so long to run. When I ran EXPLAIN on the query it doesn't return (after waiting 5+ minutes). The query is
SELECT *
FROM Foo JOIN Bar USING (FileNum, LineNum)
The tuple (FileNum, LineNum) is the primary key of both tables. In addition, (FileNum, LineNum) is a foreign key in Bar that references Foo.
I'm having performance issues with the database but this seems to be a new phenomenon.
I've run ANALYZE on the database and have not made any changes since then.
Does anyone have any idea why the query planner doesn't return a result?
The simplest explanation is that another session has grabbed an AccessExclusive lock on one of the tables involved in the query and won't release it.
This happens if, for instance, the other session does an ALTER TABLE in a transaction and never commits.
You might try the queries in
https://wiki.postgresql.org/wiki/Lock_Monitoring to check if there is such a blocking transaction in progress, then possibly terminate it with pg_cancel_backend() or pg_terminate_backend().
Note that if the blocking transaction is a prepared transaction:
the locks would survive to a server restart.
the transaction would not have a corresponding session in pg_stat_activity, so any query that joins through pid would miss it.
Prepared transactions can be seen through pg_prepared_xacts.
Related
I have a table with 500k elements. Now I want to add a new column
(type boolean, nullable = false) without a default value.
The query to do so is running like for ever.
I'm using PostgreSQL 12.1, compiled by Visual C++ build 1914, 64-bit on my Windows 2012 Server
In pgAdmin I can see the query is blocked by PID 0. But when I execute this query, I can't see the query with pid = 0
SELECT *
FROM pg_stat_activity
Can someone help me here? Why is the query blocked and how can I fix this to add a new column to my table.
UPDATE attempt:
SELECT *
FROM pg_prepared_xacts
Update
It works after rollback all prepared transaction.
ROLLBACK PREPARED 'gid goes here';
You have got stale prepared transactions. I say that as in "you have got the measles", because it is a disease for a database.
Such prepared transactions keep holding locks and block autovacuum progress, so they will bring your database to its knees if you don't take action. In addition, such transactions are persisted, so even a restart of the database won't get rid of them.
Remove them with
ROLLBACK PREPARED 'gid goes here'; /* use the transaction names shown in the view */
If you use prepared transactions, you need a distributed transaction manager. That is a piece of software that keeps track of all prepared transactions and their state and persists that information, so that no distributed transaction can become stale. Even if there is a crash, the distributed transaction manager will resolve in-doubt transactions in all involved databases.
If you don't have that, don't use prepared transactions. You now know why. Best is to set max_prepared_transactions to 0 in that case.
I'm pretty new to PostgreSQL and I'm sure I'm missing something here.
The scenario is with version 11, executing a big drop table and insert transaction on a given table with the nodejs driver, which may take 30 minutes.
While doing that, if I try to query with select on that table using the jdbc driver, the query execution waits for the transaction to finish. If I close the transaction (by finishing it or by forcing it to exit), the jdbc query becomes responsive.
I thought I can read a table with one connection while performing a transaction with another one.
What am I missing here?
Should I keep the table (without dropping it at the beginning of the transaction) ?
DROP TABLE takes an ACCESS EXCLUSIVE lock on the table, which is there precisely to prevent it from taking place concurrently with any other operation on the table. After all, DROP TABLE physically removes the table.
Since all locks are held until the end of the database transaction, all access to the dropped table is blocked until the transaction ends.
Of course the files are only removed when the transaction commits, so you might wonder why PostgreSQL doesn't let concurrent transactions read in the mean time. But that would mean that COMMIT may be blocked by a concurrent reader, or a SELECT might cause a system error in the middle of reading, both of which don't sound appealing.
Before I try to insert a row into a PostgreSQL table, should I query whether the insert would violate a constraint?
I do check when the insert would cause unwanted side-effects (e.g., auto-increment) upon an error.
But, if there are no possible side effects, is it OK to just blindly try to insert into a table? Or, is it better practice to prevent errors by anticipating them when possible (as advised in Objective-C)?
Also, when performing the insert inside an SQL function, will other queries (e.g., CTEs) inside the function get rolled back if the insert fails?
In general testing before hand is not a good idea because it requires you to explicitly lock tables to prevent other clients from changing or inserting data between your test and inserts. Explicit locking is bad for concurrency.
Serials getting auto incremented by failed inserts is in general not a problem. Just don't assume the values inserted into the database are consecutive.
A database and obj-c are two completely different things. Let the database check for problems, it is much easier to add the appropriate constraints to your schema then it is to check everything in your client program.
The default is to rollback to the start of the transaction. But you can control it with savepoints and rollback to savepoint. However a CTE is part of the query and queries are always rolled back completely when part of them fails. However you might be able to work around that by splitting the CTE of into a full query that creates a temp table. Then you can use the temp table instead of the cte.
I'm using PostgreSQL 9.2 in a Windows environment.
I'm in a 2PC (2 phase commit) environment using MSDTC.
I have a client application, that starts a transaction at the SERIALIZABLE isolation level, inserts a new row of data in a table for a specific foreign key value (there is an index on the column), and vote for completion of the transaction (The transaction is PREPARED). The transaction will be COMMITED by the Transaction Coordinator.
Immediatly after that, outside of a transaction, the same client requests all the rows for this same specific foreign key value.
Because there may be a delay before the previous transaction is really commited, the SELECT clause may return a previous snapshot of the data. In fact, it does happen sometimes, and this is problematic. Of course the application may be redesigned but until then, I'm looking for a lock solution. Advisory Lock ?
I already solved the problem while performing UPDATE on specific rows, then using SELECT...FOR SHARE, and it works well. The SELECT waits until the transaction commits and return old and new rows.
Now I'm trying to solve it for INSERT.
SELECT...FOR SHARE does not block and return immediatley.
There is no concurrency issue here as only one client deals with a specific set of rows. I already know about MVCC.
Any help appreciated.
To wait for a not-yet-committed INSERT you'd need to take a predicate lock. There's limited predicate locking in PostgreSQL for the serializable support, but it's not exposed directly to the user.
Simple SERIALIZABLE isolation won't help you here, because SERIALIZABLE only requires that there be an order in which the transactions could've occurred to produce a consistent result. In your case this ordering is SELECT followed by INSERT.
The only option I can think of is to take an ACCESS EXCLUSIVE lock on the table before INSERTing. This will only get released at COMMIT PREPARED or ROLLBACK PREPARED time, and in the mean time any other queries will wait for the lock. You can enforce this via a BEFORE trigger to avoid the need to change the app. You'll probably get the odd deadlock and rollback if you do it that way, though, because INSERT will take a lower lock then you'll attempt lock promotion in the trigger. If possible it's better to run the LOCK TABLE ... IN ACCESS EXCLUSIVE MODE command before the INSERT.
As you've alluded to, this is mostly an application mis-design problem. Expecting to see not-yet-committed rows doesn't really make any sense.
Is there any safe way of acquiring an advisory lock before executing a particular statement without using two separate queries? E.g., I assume that if I do something like the following, there is no guarantee that the lock will be acquired before the insert:
WITH x AS (SELECT pg_advisory_lock(1,2)) INSERT ...
But is there some similar way of getting the desired effect?
I'm pretty sure that SQL standards require implementations to behave as if the very first thing they do is to effectively materialize the common table expressions in the WITH clause. PostgreSQL complies with this requirement.
Common table expressions behave (mostly) as named objects. Multiple CTEs are materialized in the order they're declared. Backward references by name work as you'd expect, and forward references by name raise an error.
So I'm pretty sure that, in the general case, the CTE will have to materialize before the INSERT statement will run. But in your case, using PostgreSQL, I'm not dead certain, and here's why.
PostgreSQL's implementation [of common table expressions]
evaluates only as many rows of a WITH query as are actually
fetched by the parent query.
I'm not sure an INSERT statement fetches a row in this sense.
You haven't really told us enough about your use case to be sure, but in general for explicit locks to be useful in PostgreSQL they need to be acquired before the transaction acquires its snapshot. Advisory locks can be acquired before you start the transaction, and most locks with transactional scope should be acquired right after your begin your transaction; before anything which will need a transaction ID.
If you really don't need to acquire the lock before you have your transaction ID assigned and your snapshot set, and it is important to you that you issue one statement to acquire the lock and perform the insert, create a function which does both.