This question already has answers here:
user postgres launches process that takes all CPUs 100% usage
(2 answers)
Closed 2 years ago.
I installed timescaledb extension (v.5.0) for PostgreSQL (v9.6) on 16.04 Linux box and observes that Postgres process occupies 100% CPU:
here is result of top command:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19885 postgres 20 0 192684 3916 1420 S 98.3 0.1 5689:04 x3606027128
I run query SELECT pid, datname, usename, query FROM pg_stat_activity and find two weird queries
pid datname username query
19882 postgres postgres select Fun013301 ('./x3606027128 &')
19901 postgres postgres select Fun013301 ('./ps3597605779 &')
function Fun013301 is not existed in postgres database. I cannot figure out what ./x3606027128 command is!?
I had a similar issue. It was due to - some transactions was getting stuck and running for long time. Thus, CPU utilization was at 100% all the time. Following command helped to find out the connections running for the longest time:
SELECT max(now() - xact_start) FROM pg_stat_activity
WHERE state IN ('idle in transaction', 'active');
This command shows the time since when a connection is running. This time should not be greater than an hour. So killing the connection which was running from long time or stuck at any point, worked for me. I followed this post for monitoring and solving my issue. Post includes lots of useful commands to monitor this situation.
Related
I have installed postgre on Ubuntu 16.04 with postgres user, and after running my application for a couple of days, I am observing the CPU usage from watchbog command is 100% constantly forever untill i restart the server.
I have checked running queries in pg_stat_activity there are no queries running.
Following is from top
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
9964 postgres 20 0 619320 14368 4028 S 399.0 0.1 102527:09 watchbog
You've been hacked. Watchbog is a name known to be used by malware, and is unused by PostgreSQL itself. They may have come in through your database server and leveraged that to run commands in the OS as 'postgres', or maybe they just hacked into the OS account named 'postgres' directly.
I am looking for a way to cancel queries that are currently running from cli.
I have found these links:
https://www.postgresql.org/docs/11/libpq-cancel.html
https://www.postgresql.org/docs/11/contrib-dblink-cancel-query.html
but seems that it is not what I am looking for.
Given the pid of the session running the query (the process ID of the corresponding backend process, which you can find in pg_stat_activity, or from ps, top, etc.), you can use:
psql -c "SELECT pg_cancel_backend(<your_pid>)"
If you're trying to kill all queries meeting some criteria (e.g. those which have been running/blocking/idle for some period of time, or those running against a particular database), something like this is often useful:
psql -c "SELECT pg_cancel_backend(pid) FROM pg_stat_activity WHERE <your_conditions>"
You can also disconnect them using pg_terminate_backend(pid).
To cancel the most recently started query:
SELECT pg_cancel_backend(pid)
FROM pg_stat_activity
WHERE pid <> pg_backend_pid()
ORDER BY query_start DESC
LIMIT 1;
pg_backend_pid() is the connection you're using to run the command; without this filter, the "latest query" would be the one you're currently executing.
I'm having Postgres 9.6.4 in Ubuntu 17. After every few hours (say 6 hours or so), the OS CPU utilization becomes very high (upto 98% or more).
I did the following:
# ps aux | grep postgres
Whenever the process/CPU is high, the above command shows postgres processes like this:
31928 1 ./x3665600000 0.2 99.8
The process name with x3665600000 always consumes near to 100% CPU.
When I checked the pg_stat_activity table, it shows SQL query like this:
select Fun310280 ('./x3665600000 &')
select Fun310280 ('./ps3657178651 &')
What is this function causing Postgres to use very high CPU?
I keep running into this issue where my postgresql database hangs because I didn't finish a transaction while debugging in PyCharm.
The log has several of these messages:
[16:30:40 PDT] unexpected EOF on client connection with an open transaction
Now the database is hung and I don't know how to recover from it other than shutting down the database (vagrant halt; vagrant up)
Is there any way to clear those stuck transactions so I don't have to go through stopping and restarting the database?
Thanks for any info
I found this solution here:
SELECT * FROM pg_stat_activity ORDER BY client_addr ASC, query_start ASC;
will list all your hung/idle transactions, then you can run
SELECT pg_terminate_backend(3592)
using the pid listed in the table.
and it is much faster than rebooting vagrant or postgresql
How can one tell if the autovacuum daemon in Postgres 9.x is running and maintaining the database cluster?
PostgreSQL 9.3
Determine if Autovacuum is Running
This is specific to Postgres 9.3 on UNIX.
For Windows, see this question.
Query Postgres System Table
SELECT
schemaname, relname,
last_vacuum, last_autovacuum,
vacuum_count, autovacuum_count -- not available on 9.0 and earlier
FROM pg_stat_user_tables;
Grep System Process Status
$ ps -axww | grep autovacuum
24352 ?? Ss 1:05.33 postgres: autovacuum launcher process (postgres)
Grep Postgres Log
# grep autovacuum /var/log/postgresql
LOG: autovacuum launcher started
LOG: autovacuum launcher shutting down
If you want to know more about the autovacuum activity, set log_min_messages to DEBUG1..DEBUG5. The SQL command VACUUM VERBOSE will output information at log level INFO.
Regarding the Autovacuum Daemon, the Posgres docs state:
In the default configuration, autovacuuming is enabled and the related configuration parameters are appropriately set.
See Also:
http://www.postgresql.org/docs/current/static/routine-vacuuming.html
http://www.postgresql.org/docs/current/static/runtime-config-autovacuum.html
I'm using:
select count(*) from pg_stat_activity where query like 'autovacuum:%';
in collectd to know how many autovacuum are running concurrently.
You may need to create a security function like this:
CREATE OR REPLACE FUNCTION public.pg_autovacuum_count() RETURNS bigint
AS 'select count(*) from pg_stat_activity where query like ''autovacuum:%'';'
LANGUAGE SQL
STABLE
SECURITY DEFINER;
and call that from collectd.
In earlier Postgres, "query" was "current_query" so change it according to what works.
You can also run pg_activity to see the currently running queries on your database. I generally leave a terminal open with this running most of the time anyway as it's very useful.
set log_autovacuum_min_duration to the time length that you desire and the autovacuum execution exceeds the time length will be logged.