I have configured my PostgreSQL 9.4 install to log statements that take over 500 ms using log_min_duration_statement
Now, I have a query that is expected to take in the order of 2 seconds. So it gets recorded every time is run.
I would like to whitelist this query so it doesn't show up in the logs or, even better, to configure log_min_duration_statement to only log that query is it takes over 5000 ms without affecting the rest.
There is no direct way to do that, but there are some workarounds. You can set log_min_duration_statement from your client code before you run that one query. Or you could run that query under a different user and have that user configured differently.
Related
When checking log files, there is a lot of internal/system queries (runned internally by pgadmin tool for example) and not directly executed by user.
I there any way to distinguish which queries run by user directly, and which is internal queries?
(btw, config params I use for logging: log_statement=all and log_min_duration_statement=0)
I am not sure what you mean by "internal/system queries". Unless you use auto_explain with auto_explain.log_nested_statements = on, PostgreSQL will only log the queries that get sent by the client.
It may well be that some of these queries are internal queries of your database driver or application server, but PostgreSQL doesn't know that. For that, you'd have to enable logging at a component that has that information.
PostgreSQL 9.5.
A very small update SQL uses very high CPU for a long time like a hang.
My Windows console application uses a simple UPDATE statement to update the latest time as follows.
UPDATE META_TABLE SET latest_time = current_timestamp WHERE host = 'MY_HOST'
There are just 2 console applications which issue above SQL.
No index on META_TABLE.
Only 1 row.
When it is hanging, no lock information.
UNLOGGED Table.
IDLE status in pg_stat_activity
Commit after UPDATE.
During that the hang time, I can INSERT or DELETE data with the table above.
After starting the application, about in 20 minutes, this issue happens.
I think it is not a SQL statement or table structure issue, probably something wrong in a database side.
Can you guess anything to resolve this issue?
Update
There are 2 db connections in the console application. 1 for Select & 1 for DML.
I tried to close DML DB connection every 2 minutes. Then, I haven't seen the issue!! However, the hang issue happened on SELECT statement (also very simple SELECT).
It seems that there is a some limit per the session.
Now, I am also closing the Select db connection as well per 3 minutes and monitoring.
How to time how long it took for Postgres to actually execute in query using Knex? Is there a way to do this?
Ideally, I'd like to see a breakdown of timings like: time to acquire socket from pool, time for PG to execute query, time to receive bytes.
Run your app with environment variable DEBUG=knex:* and it will print lots of info about how long certain things took.
Also you can add event handlers to query events http://knexjs.org/#Interfaces-Events and do your own timers there. Those events can be also set globally like knex.on('query', ...).
I have the following setting in my postgresql.conf
log_statement='all'
log_duration=on
When I run psql, and run any query like select/create user/ etc, it logs the statement as well as the duration. If I give the query to connect to a different database
\c <database_name>
it logs neither the duration nor the statement. I thought it might not log duration and statement for meta-commands that starts with a backslash. But it does log the statement and duration for some of them, like \dt \l.
I think there might be a list of queries/commands, for which log will be generated or not. is there any such list?
Having said that, the original issue, that led me to dig this is --
My application (golang-react application) interacts with postgresql and logs statement and its duration. But after that it logs two more durations, whose statement is not printed. (I have commented log_min_duration_statement), so I'm not sure what queries, this duration is for.
Also if I do SELECT query,xact_start,query_start FROM pg_stat_activity; at that time to see the running queries,it does not show any other query than the one which I see in the logs.
How can I know which queries is this duration for?
But after that it logs two more durations, whose statement is not printed.
These are probably the parse, bind and execute durations for the three phases of statement execution, if you have log_duration = on. It's only done in three phases via client applications - psql runs in a single go.
If you just want the total, use log_min_duration_statement = 0 instead.
Yes, that's confusing. Maybe it's worth writing it up and submitting it as a usability problem report to pgsql-general.
I am trying to get my postgresql 9.3 server to log all sql that runs longer than 1 second. I have set:
log_min_duration=1s
log_statement='mod'
log_duration=off
for most queries, the logging is working correclty, but some statements, such as "CREATE TABLE AS" or "INSERT" are not logging the statement. The log the duration, but not the complete sql.
Has anyone else seen this type of issue before, or know why postgres may not be logging the sql? (possibly a transaction block?) Any help is greatly appreciated, thanks.
The correct parameter is log_min_duration_statement and not log_min_duration.
And this note, from docs,
Note: When using this option together with log_statement, the text of
statements that are logged because of log_statement will not be
repeated in the duration log message. If you are not using syslog, it
is recommended that you log the PID or session ID using
log_line_prefix so that you can link the statement message to the
later duration message using the process ID or session ID.
consider is a good explanation for problem.