I'm working with a PostgreSQL server version 10.5 (can't upgrade it at the moment) running in Docker. XL Deploy is connected to it and I upload a new archive that is 232MB. I get the error FATAL: terminating connection due to idle-in-transaction timeout.
In the config, I saw that idle_in_transaction_session_timeout was set to 11000ms. I updated the value to 600000ms and reload the configuration. I see the message LOG: parameter "idle_in_transaction_session_timeout" changed to "600000" in the logs, so I know the setting has taken.
However, when I upload the archive again, I still get the same FATAL timeout message. It's as if the setting is having no effect.
As it turns out, the issue was in Postgres, but not in the config file. It seems you can update the value of idle_in_transaction_session_timeout at various levels. In my case, it was at the ROLE level. Issuing this SQL statement fixed the timeout.
ALTER ROLE role_abc SET idle_in_transaction_session_timeout = '10min';
We have a PostgreSQL DB on Heroku, without warning a few days ago it started logging all data changes as if the log_statement setting was set to mod which is having a massive impact on performance when we do our nightly batch updates.
I've checked the settings in the DB via the pg_settings view and using the Heroku CLI and it very clearly says the the log_statement setting is set to ddl.
We haven't made any changes to the DB or its settings. Does anyone know why this might be happening and how we can reset the log level?
There is a Query that keeps timing out exactly after 30 seconds. I have seen a few posts on Stackoverflow which are similar to the issue i'm facing, the only thing that makes my issue different is that, i'm not using a DSN-LESS Connection, I'm using ODBC ADMINISTRATOR.
The following Error is the error i'm receiving:
16:33:16.06 System.Data.Odbc.OdbcException (0x80131937): ERROR [57014] ERROR: canceling statement due to statement timeout;
I guess my question is, how can i go about increasing the connection timeout using ODBC Administrator ?
Any help would be appreciated
You can try to run in each database session something similar to:
postgres=# set statement_timeout='300s';
SET
postgres=# show statement_timeout;
statement_timeout
-------------------
5min
(1 row)
postgres=#
To avoid doing this in each database session it would be easier to change this parameter in postgresql.conf: the new setting would be enabled for all database sessions.
I want to set a default statement_timeout for my access to a postgres database. After configuring my environment variables, I now have it where psql logs me on my preferred database and table. However, while I'm exploring several of tables in it, I'd like to have a statement timeout of around a minute. This can be done simply by typing SET statement_timeout TO '1min'; at the beginning of each session, but this is obnoxious to type every time. I don't have access to the server configuration nor would I want to change it. Ideally I could do something to the effect of alias psql='psql -c "SET statement_timeout TO '1min';"' except the-c` flag of psql doesn't allow interactive input. Are there any nice solutions to this problem, or am I always doomed to set the timeout manually for each interactive session?
You could use your .psqlrc file (if you don't have one in your home directory, create it; if you're on Windows instead of *nix, the file is %APPDATA%\postgresql\psqlrc.conf instead) and set the following command:
set statement_timeout to 60000; commit;
That setting is in milliseconds, so that'll set the timeout to 1 minute. .psqlrc isn't used with -c nor -X invocations of psql, so that should allow you to get your interactive-mode timeout to 1 minute.
You can then execute the following in psql to verify that the configuration has taken effect:
show statement_timeout;
Postgres allows you to set configuration parameters such as statement_timeout on a per-role (user) level.
ALTER ROLE <your-username> SET statement_timeout = '60s';
This change will apply to all new sessions for that user, starting on the next login.
Source: Postgres docs
Some clients connect to our postgresql database but leave the connections opened.
Is it possible to tell Postgresql to close those connection after a certain amount of inactivity ?
TL;DR
IF you're using a Postgresql version >= 9.2
THEN use the solution I came up with
IF you don't want to write any code
THEN use arqnid's solution
IF you don't want to write any code
AND you're using a Postgresql version >= 14
THEN use Laurenz Albe's solution
For those who are interested, here is the solution I came up with, inspired from Craig Ringer's comment:
(...) use a cron job to look at when the connection was last active (see pg_stat_activity) and use pg_terminate_backend to kill old ones.(...)
The chosen solution comes down like this:
First, we upgrade to Postgresql 9.2.
Then, we schedule a thread to run every second.
When the thread runs, it looks for any old inactive connections.
A connection is considered inactive if its state is either idle, idle in transaction, idle in transaction (aborted) or disabled.
A connection is considered old if its state stayed the same during more than 5 minutes.
There are additional threads that do the same as above. However, those threads connect to the database with different user.
We leave at least one connection open for any application connected to our database. (rank() function)
This is the SQL query run by the thread:
WITH inactive_connections AS (
SELECT
pid,
rank() over (partition by client_addr order by backend_start ASC) as rank
FROM
pg_stat_activity
WHERE
-- Exclude the thread owned connection (ie no auto-kill)
pid <> pg_backend_pid( )
AND
-- Exclude known applications connections
application_name !~ '(?:psql)|(?:pgAdmin.+)'
AND
-- Include connections to the same database the thread is connected to
datname = current_database()
AND
-- Include connections using the same thread username connection
usename = current_user
AND
-- Include inactive connections only
state in ('idle', 'idle in transaction', 'idle in transaction (aborted)', 'disabled')
AND
-- Include old connections (found with the state_change field)
current_timestamp - state_change > interval '5 minutes'
)
SELECT
pg_terminate_backend(pid)
FROM
inactive_connections
WHERE
rank > 1 -- Leave one connection for each application connected to the database
If you are using PostgreSQL >= 9.6 there is an even easier solution. Let's suppose you want to delete all idle connections every 5 minutes, just run the following:
alter system set idle_in_transaction_session_timeout='5min';
In case you don't have access as superuser (example on Azure cloud), try:
SET SESSION idle_in_transaction_session_timeout = '5min';
But this latter will work only for the current session, that most likely is not what you want.
To disable the feature,
alter system set idle_in_transaction_session_timeout=0;
or
SET SESSION idle_in_transaction_session_timeout = 0;
(by the way, 0 is the default value).
If you use alter system, you must reload configuration to start the change and the change is persistent, you won't have to re-run the query anymore if, for example, you will restart the server.
To check the feature status:
show idle_in_transaction_session_timeout;
Connect through a proxy like PgBouncer which will close connections after server_idle_timeout seconds.
From PostgreSQL v14 on, you can set the idle_session_timeout parameter to automatically disconnect client sessions that are idle.
If you use AWS with PostgreSQL >= 9.6, you have to do the following:
Create custom parameter group
go to RDS > Parameter groups > Create parameter group
Select the version of PSQL that you use, name it 'customParameters' or whatever and add description 'handle idle connections'.
Change the idle_in_transaction_session_timeout value
Fortunately it will create a copy of the default AWS group so you only have to tweak the things that you deem not suitable for your use-case.
Now click on the newly created parameter group and search 'idle'.
The default value for 'idle_in_transaction_session_timeout' is set to 24 hours (86400000 milliseconds). Divide this number by 24 to have hours (3600000) and then you have to again divide 3600000 by 4, 6 or 12 depending on whether you want the timeout to be respectively 15, 10 or 5 minutes (or equivalently multiply the number of minutes x 60000, so value 300 000 for 5 minutes).
Assign the group
Last, but not least, change the group:
go to RDS, select your DB and click on 'Modify'.
Now under 'Database options' you will find 'DB parameter group', change it to the newly created group.
You can then decide if you want to apply the modifications immediately (beware of downtime).
I have the problem of denied connections as there are too much clients connected on Postgresql 12 server (but not on similar projects using earlier 9.6 and 10 versions) and Ubuntu 18.
I wonder if those settings
tcp_keepalives_idle
tcp_keepalives_interval
could be more relevant than
idle_in_transaction_session_timeout
idle_in_transaction_session_timeout indeed closes only the idle connections from failed transactions, not the inactive connections whose statements terminate correctly...
the documentation reads that these socket-level settings have no impact with Unix-domain sockets but it could work on Ubuntu.
Up to PostgreSQL 13, you can use my extension pg_timeout.