I am currently trying to debug "idle in transaction" scenarios in my application.I can find out the process id and transaction start time for a query with state 'idle in transaction' by looking at pg_state_activity.
select pid,query from pg_stat_activity where state='idle in transaction' OR state='idle'
Is there any way to identify list of all queries executed within a transaction corresponding to the query with 'idle in transaction'
Are you attempting to get a list of all previous statements run by the transaction that is now showing up as idle in transaction?
If so, there is no easy and safe way to do so at the SQL level. You should use CSV logging mode to analyze your query history and group queries up into transactions. Handily, you can do this with SQL, by COPYing the CSV into a PostgreSQL table for easier analysis.
Alternately, use ordinary text logs, and set a log_line_prefix that includes the transaction ID and process ID.
(I could've sworn I saw an extension for debugging that collected a query trace, but cannot find it now, and I'm not sure it's that useful as you must run a command on the problem connection to extract the data it's collected).
Related
I have a table with 500k elements. Now I want to add a new column
(type boolean, nullable = false) without a default value.
The query to do so is running like for ever.
I'm using PostgreSQL 12.1, compiled by Visual C++ build 1914, 64-bit on my Windows 2012 Server
In pgAdmin I can see the query is blocked by PID 0. But when I execute this query, I can't see the query with pid = 0
SELECT *
FROM pg_stat_activity
Can someone help me here? Why is the query blocked and how can I fix this to add a new column to my table.
UPDATE attempt:
SELECT *
FROM pg_prepared_xacts
Update
It works after rollback all prepared transaction.
ROLLBACK PREPARED 'gid goes here';
You have got stale prepared transactions. I say that as in "you have got the measles", because it is a disease for a database.
Such prepared transactions keep holding locks and block autovacuum progress, so they will bring your database to its knees if you don't take action. In addition, such transactions are persisted, so even a restart of the database won't get rid of them.
Remove them with
ROLLBACK PREPARED 'gid goes here'; /* use the transaction names shown in the view */
If you use prepared transactions, you need a distributed transaction manager. That is a piece of software that keeps track of all prepared transactions and their state and persists that information, so that no distributed transaction can become stale. Even if there is a crash, the distributed transaction manager will resolve in-doubt transactions in all involved databases.
If you don't have that, don't use prepared transactions. You now know why. Best is to set max_prepared_transactions to 0 in that case.
I am not very familiar with databases or SQL and wanted to make sure that I don't mess anything up. I did:
SELECT pid, state, usename, query FROM pg_stat_activity;
To check if I had any queries and there were several that had the state active. Do I just cancel them by doing:
select pg_cancel_backend(PID);
And this won't affect anything except the my queries, correct? I also wanted to figure out why those queries were still in the state active. I have a python file where I read in my sql file, but I stopped running the python file in the middle of reading my sql file. Is that possibly why it happened and why the states are still active?
Yes, this is what pg_cancel_backend(pid) is for. Why exactly the query is still running depends on a few things - could be waiting to grab a lock, or the query could just take a long time - but given the python processes that started the queries have exited, the connection is technically already closed, the PG backend process just hasn't noticed yet. It won't notice until the query completes and it tries to return the query status to the client, at which point it'll rollback the transaction when it sees the connection is no longer present.
The only effect pg_cancel_backend on the PIDs of those backends should have is to cause PG to notice the connection is closed immediately, rather than whenever the query completes.
Is it possible to find which records on which table are locked by a specific statement, transaction or attachment on FirebirdSQL 2.5?
I'm getting some Lock conflict on no wait transaction error messages on some situations.
I'm trying to trace the problem looking for ACTIVE transactions and statements using MON$STATEMENTS, MON$TRANSACTIONS and MON%ATTACHMENTS. I know the exact sql statement from MON$STATEMENTS but on some cases it is a stored procedure execution that updates many tables. So, finding the actual locked record or table would make analysis easier.
I have a huge process(program with activerecord), which lock different tables for an amount of time.
Now I want to check all my locks during the process. So which tables are locked and for how long. I could use the activity monitor, but I need more information.
Is there a tool like the SQL Server Profiler, which list all locks during a process? Or is somewhere a logtable, which I can check?
Further Information:
There is a process in our program which use half of the tables from our database. Create new rows, update existing rows, select informations... The process runs only during the night. Now they want to run this process during the day and I have to evaluate to possibility of that request. I already checked the sourcecode, but I also want to check the database for longer locks, tablelocks and such stuff, just to be sure. The idea is, to start that process in our test environment and collect all lock informations. But I don't see all locks in the activity monitor and I can't look for an hour over the activity monitor.
There are many DMVS which will help you out to gather lock stats.Run this query based on your frequency through a SQL job and log this to table for later analysis..
--This shows all the locks involved in each session
SELECT resource_type, resource_associated_entity_id,
request_status, request_mode,request_session_id,
resource_description
FROM sys.dm_tran_locks lck
WHERE resource_database_id = db_id()
--You also can use SYS.DM_EXEC_Requests DMV to gather blockings,wait_types to understand more
select status,wait_type,last_wait_type,txt.text from sys.dm_exec_requests ec
cross apply
sys.dm_exec_sql_text(ec.sql_handle) txt
I am investigating a connection leak in PostgreSQL and would like to gather some diagnostics.
In particular, I would like to associate some information with each connection I make and then be able to query it directly.
If I was designing this kind of thing in MS-SQL I would change my connection factory to execute an upsert against a diagnostics table after each connection is made, containing a ##spid, backtrace and other diag information.
Then to diagnose what is happening I could query sysprocesses joining to my diagnostics table on spid. This would give me clean application backtraces associated with each connection.
How can I achieve something similar on PostgreSQL?
PostgreSQL forks a new process to handle each connection. You can get the PID of this process easily:
SELECT pg_backend_pid();
This corresponds to the process ID visible in standard administration tools (top, ps, etc.). You can terminate connections using standard tools too (kill) or, with the appropriate permissions, by saying SELECT pg_terminate_backend(pid).
A list of current sessions is also accessible in the database:
SELECT * FROM pg_stat_activity;
One last note: PIDs are guaranteed to be unique as of the time the query is run, but will be re-used by the operating system eventually. You can ensure uniqueness by pairing the PID with the backend_start column of pg_stat_activity. For that matter, you might as well lump that in with your logging:
INSERT INTO log_table (pid, backend_start, message, backtrace)
SELECT procpid, backend_start, 'my message', 'my backtrace'
FROM pg_stat_activity
WHERE procpid=pg_backend_pid();