I'm sort of an "accidental dba" so apologies for a real noob question here. I'm using pgbouncer in pool_mode = transaction mode. Yesterday I started getting errors in my php log:
no more connections allowed (max_client_conn)
I had max_client_conn = 150 to match max_connections in my postgresql.conf.
So my first question is, should pgbouncer max_client_conn be set equal to postgresql max_connections, or am I totally misunderstanding that relationship?
I have 20 databases on a single postgres instance behind pgbouncer with the default default_pool_size = 20. So should max_client_conn be 400? (pool_size * number_of_databases)?
Thanks
https://pgbouncer.github.io/config.html
max_client_conn Maximum number of client connections allowed.
default_pool_size How many server connections to allow per user/database pair.
so max_client_conn should be way larger then postgres max_connections, otherwise why you use connection pooler at all?..
If you have 20 databases and set default_pool_size to 20, you will allow pgbouncer to open 400 connections to db, so you need to adjust posgtres.conf max_connections to 400 and set pgbouncer max_client_conn to smth like 4000 (to have average 10 connections in pool for each actual db connection)
This answer is only meant to provide an example for understanding the settings, not as a statement to literally follow. (eg I just saw a config with:
max_client_conn = 10000
default_pool_size = 100
max_db_connections = 100
max_user_connections = 100
for cluster with two databases and max_connections set to 100). Here the logic is different, also mind max_db_connections is set and in fact connection limits are set individually per database in pgbouncer [database] section.
So - play with small settings to get the idea of how config influence each other - this is "how to determine max_client_conn for pgbouncer" the best
Like almost everyone, then you are setting your pool size way to high. Don't let your postgresql server do the connection pooling. If you do then it severely hurts your performance.
The optimal setting for how many concurrent connection to postgresql is
connections = ((core_count * 2) + effective_spindle_count)
That means that if you are running your database on a 2 core server, then your total pool size from pgbouncer should be no more than 5. Pgbouncer is a lot better at handling pooling than postgresql, so let it do that.
So, leave the max_connections in my postgresql.conf to it's default 100 (no reason to change as it is a max. Also this should always be higher than what your application needs as some logging, admin and backup processes needs connections as well)
And in your pgbouncer.ini file set
max_db_connections=5
default_pool_size=5
max_client_conn=400
For more information https://www.percona.com/blog/2018/06/27/scaling-postgresql-with-pgbouncer-you-may-need-a-connection-pooler-sooner-than-you-expect/
Related
I have several clients (FreeRadius servers) that connect to a single central Pgbouncer.
When I utilise one of these FreeRadius servers I can see that several database connections are created by Pgbouncer.
select *
from pg_stat_activity
where datname = 'master_db';
When I utilise the same freeradius server again, pgbouncer isn't re-using the existing open connections but keeps creating new ones. Once I reach over 30 connections the whole database comes to an halt.
PGbouncer.ini
server_idle_timeout = 10
max_client_conn = 500
default_pool_size = 30
postgresql.conf: (Postgres 13)
max_connections = 150
Based on my research Pgbouncer is supposed to allocate a single database connection to a client (from the default_pool_size) and then create as many internal connections the client needs (up to max_client_conn).
But what I observe here is the opposite. What am I missing, please?
UPDATE:
The solution Laurenz suggested works but throws this error, when using asyncpg behind the scenes:
NOTE: pgbouncer with pool_mode set to "transaction" or "statement" does not support prepared statements properly. You have two options: * if you are using pgbouncer for connection pooling to a single server, switch to the connection pool functionality provided by asyncpg, it is a much better option for this purpose; * if you have no option of avoiding the use of pgbouncer, then you can set statement_cache_size to 0 when creating the asyncpg connection object.
You will need to use the session option for POOL_MODE so that it is able to maintain connection sessions opened by asyncpg because of the asynchronous nature
You should the following in your
if using pgbouncer.ini file
[pgbouncer]
pool_mode = session
...
...
...
or if using env variable
POOL_MODE=session
extra source: https://magicstack.github.io/asyncpg/current/faq.html#why-am-i-getting-prepared-statement-errors
If you are getting intermittent prepared statement
"asyncpg_stmt_xx" does not exist or prepared statement
“asyncpg_stmt_xx” already exists errors, you are most likely not
connecting to the PostgreSQL server directly, but via pgbouncer.
pgbouncer, when in the "transaction" or "statement" pooling mode, does
not support prepared statements. You have several options:
if you are using pgbouncer only to reduce the cost of new connections
(as opposed to using pgbouncer for connection pooling from a large
number of clients in the interest of better scalability), switch to
the connection pool functionality provided by asyncpg, it is a much
better option for this purpose;
disable automatic use of prepared statements by passing
statement_cache_size=0 to asyncpg.connect() and asyncpg.create_pool()
(and, obviously, avoid the use of Connection.prepare());
switch pgbouncer’s pool_mode to session.
I have set in pgBouncer this limits
max_client_conn = 2000
default_pool_size = 40
When i execute this SQL in phpPgAdmin, only 2 or 4 connections appears:
SELECT datname, usename, pid, query, query_start
FROM pg_catalog.pg_stat_activity
WHERE datname='example'
ORDER BY usename, pid
This is normal or pgBouncer don't loaded the .ini when started?
The amount of connections in pg_stat_activity depends on the actual load. Also it depend more on pool_mode - if you have pool_mode = session, you will see more sessions just because the are released less often and slower.
Regarding your options, check out the docs (allowed - is a key word):
default_pool_size
How many server connections to allow per user/database pair. Can be overridden in the per-database
configuration.
Default: 20
and
max_client_conn
Maximum number of client connections allowed. When increased then the file descriptor limits should also be increased.
Note that actual number of file descriptors used is more than
max_client_conn.
Emphasis mine.
Yesterday I have test pgpool with pgbench :
pgbench -c 30 -T 20 -r pgbench -p9999 -h192.168.8.28
Concurrent connections is 30, pgpool default num_init_children is 32.
So, when I set -c 33 ,test will blocked unless I break out.
My question is :
If my concurrent connections online is 10000, should I set num_init_children=10000?
It is terrible that num_init_children=10000 means pgpool start with 10000 process.
Is there something wrong ?
How can I config pgpool with 10000 concurrent connections?
One pgpool child process can handle exactly one client connection at any time. So value of num_init_children is directly proportional to the number of expected maximum concurrent connections. If you want 10,000 concurrent connections through pgpool there is no other way than setting num_init_children to 10,000.
PostgreSQL also spawns a dedicated process to handle each client connection so if at any instance the PostgreSQL sever has 10,000 connected clients, it would also have 10,000 child processes. The difference in pgpool and PostgreSQL in this respect is that pgpool prespawns the num_init_children number of connections and PostgreSQL do it on demand.
Yes you have to mention num_init_children=10000 and max_pool=1. You can also write like num_init_children=1000 and max_pool=10
I am using Postgres DB for my product. While doing the batch insert using slick 3, I am getting an error message:
org.postgresql.util.PSQLException: FATAL: sorry, too many clients already.
My batch insert operation will be more than thousands of records.
Max connection for my postgres is 100.
How to increase the max connections?
Just increasing max_connections is bad idea. You need to increase shared_buffers and kernel.shmmax as well.
Considerations
max_connections determines the maximum number of concurrent connections to the database server. The default is typically 100 connections.
Before increasing your connection count you might need to scale up your deployment. But before that, you should consider whether you really need an increased connection limit.
Each PostgreSQL connection consumes RAM for managing the connection or the client using it. The more connections you have, the more RAM you will be using that could instead be used to run the database.
A well-written app typically doesn't need a large number of connections. If you have an app that does need a large number of connections then consider using a tool such as pg_bouncer which can pool connections for you. As each connection consumes RAM, you should be looking to minimize their use.
How to increase max connections
1. Increase max_connection and shared_buffers
in /var/lib/pgsql/{version_number}/data/postgresql.conf
change
max_connections = 100
shared_buffers = 24MB
to
max_connections = 300
shared_buffers = 80MB
The shared_buffers configuration parameter determines how much memory is dedicated to PostgreSQL to use for caching data.
If you have a system with 1GB or more of RAM, a reasonable starting
value for shared_buffers is 1/4 of the memory in your system.
it's unlikely you'll find using more than 40% of RAM to work better
than a smaller amount (like 25%)
Be aware that if your system or PostgreSQL build is 32-bit, it might
not be practical to set shared_buffers above 2 ~ 2.5GB.
Note that on Windows, large values for shared_buffers aren't as
effective, and you may find better results keeping it relatively low
and using the OS cache more instead. On Windows the useful range is
64MB to 512MB.
2. Change kernel.shmmax
You would need to increase kernel max segment size to be slightly larger
than the shared_buffers.
In file /etc/sysctl.conf set the parameter as shown below. It will take effect when postgresql reboots (The following line makes the kernel max to 96Mb)
kernel.shmmax=100663296
References
Postgres Max Connections And Shared Buffers
Tuning Your PostgreSQL Server
Adding to Winnie's great answer,
If anyone is not able to find the postgresql.conf file location in your setup, you can always ask the postgres itself.
SHOW config_file;
For me changing the max_connections alone made the trick.
EDIT: From #gies0r: In Ubuntu 18.04 it is at
/etc/postgresql/11/main/postgresql.conf
If your postgres instance is hosted by Amazon RDS, Amazon configures the max connections for you based on the amount of memory available.
Their documentation says you get 112 connections per 1 GB of memory (with a limit of 5000 connections no matter how much memory you have), but we found we started getting error messages closer to 80 connections in an instance with only 1 GB of memory. Increasing to 2 GB let us use 110 connections without a problem (and probably more, but that's the most we've tried so far.) We were able to increase the memory of an existing instance from 1 GB to 2 GB in just a few minutes pretty easily.
Here's the link to the relevant Amazon documentation:
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Limits.html#RDS_Limits.MaxConnections
change max_connections variable
in postgresql.conf file located in
/var/lib/pgsql/data or /usr/local/pgsql/data/
Locate postgresql.conf file by below command
locate postgresql.conf
Edit postgresql.conf file by below command
sudo nano /etc/postgresql/14/main/postgresql.conf
Change
max_connections = 100
shared_buffers = 24MB
to
max_connections = 300
shared_buffers = 80MB
I am using PostgreSQL and Flas-SQLAlchemy extension for Flask.
# app.py
app = Flask(__name__)
app.config['SQLALCHEMY_POOL_SIZE'] = 20
db = SQLAlchemy(app)
# views.py
user = User(***)
db.session.add(user)
db.session.commit()
Note that I am not closing the connection as suggested by documentation:
You have to commit the session, but you don’t have to remove it at the end of the request, Flask-SQLAlchemy does that for you.
However, when I run the following PostgreSQL query I can see some IDLE connections:
SELECT * FROM pg_stat_activity;
Does it mean that I have a problem with Flask-SQLAlchemy not closing connections? I am worried about it because recently I got remaining connection slots are reserved for non-replication superuser connections error.
SQLAlchemy sets up a pool of connections that will remain opened for performance reasons. The PostgreSQL has a max_connections config option. If you are exceeding that value, you need to either lower your pool count or raise the max connection count. Given that the default max is 100, and you've set your pool to 20, what's more likely is that there are other applications with open connections to the same database. max_connections is a global setting, so it must account for all applications connecting to the database server.