The maxIdle property of Apache DBCP is described as :
"The maximum number of connections that can remain idle in the pool, without extra ones being released, or negative for no limit. "
can someone explain what does "without extra ones being released" mean.
thanks
Here "extra ones" means the connections that are NOT active
Related
As written in HikariCP docs the formula for counting connection pool size is connections = ((core_count * 2) + effective_spindle_count). But which core count is this: my app server or database server?
For example: my is app running on 2 CPUs, but database is running on 16 CPUs.
This is Kevin's formula for connection pool size, where cores and spindles (you can tell is is an old formula) are the database server's.
This assumes that the connections are kept fairly busy. If you have transactions with longer idle times, you might need to make the pool bigger.
In the end, only trial and error can find the ideal pool size.
The quote is from PostgreSQL wiki which is related to database cores/server
database server only has so many resources, and if you don't have enough connections active to use all of them, your throughput will generally improve by using more connections.
Notice that this formula may be outdated (comment by #mustaccio)
That wiki page was last updated nearly 5 years ago, and the advice in question is even older. I/O queue depth might be more relevant today than the number of spindles, even if the latter are actually present
We are running a PostgreSQL database in the Google Cloud. On top of this we have an app. In the app we can configure runtime connection pooling settings for the database.
Our Google SQL server has 30GB ram so the default max_connections is 500 as I read in the Google docs. In our app we have set the following connection pooling (Apache commons pooling) settings:
MaxActive: 200
MaxIdle: 200
MinIdle: 50
We are experiencing issues with these settings. First of all, we often run into the MaxActive limit. I can see a flatline in the connections graph at 200 connections a couple times a day. At those moments our logs are flooded with SQL connection errors.
The server is using around 28GB ram on peak moments (with 200 active connections). So we are close to the RAM limit as well.
Instead of blindly increasing the RAM and MaxActive I was hoping to get some insights on what would be a best practice in our situation. I see 2 solutions to our problem:
Increase RAM, increase MaxActive and increase MaxIdle (not very cost efficient)
Increase MaxActive, keep MaxIdle the same (or even lower) and keep MinIdle the same (or even lower)
Option 1 would be more cost expensive so I am wondering about option 2. Because I lower the Idle connections I would take up less RAM. However, will this have a noticeable impact on performance? I was thought to keep MaxIdle as close to MaxActive as possible, to ensure least overhead in creating new connections.
I have done quite some research, but I came to the conclusion that tuning these settings are very situation specific and there is not really a general best practice on these settings. I could not find a definitive answer to the performance impact of option 1 vs option 2.
Ps. we are also experiencing some slow queries in our app, so of course we can optimize things or change the design of our app to decrease the amount of concurrent connections.
I really hope someone can give some helpful insights / advice / best practices. Thanks a lot in advance!
Say I have set max_connections=10 in my postgresql.conf and make 11 concurrent connections. What happens with the 11-nth connection?
Will it be refused with an error or will it wait until some connection slots free up?
If exceeding connections wait in queue, is there a timeout limit for them? Where can that be set? Is it a fair queue?
Is there any documentation on this? I can't find this in the official docs
--------- edit ----------
found a source on this (no connection pooling): https://wiki.postgresql.org/wiki/Number_Of_Database_Connections
The decision not to include a connection pooler inside the PostgreSQL server itself has been taken deliberately and with good reason:
From documentation
max_connections (integer)
Determines the maximum number of concurrent
connections to the database server. The default is typically 100
connections, but might be less if your kernel settings will not
support it (as determined during initdb). This parameter can only be
set at server start.
In combination with the name of the variable it should be clear what happens: If you try to open up more than the number of set connections, you will get an error. In this case you will get a very prominent error that the number of connections is exhausted.
Postgres itself is not including any connection pooling or similar so it's either "Yes, you got in" or "No, you are not".
I followed this tutorial and there is configuration connections per host.
What is this?
connectionsPerHost are the amount of physical connections a single Mongo client instance (it's singleton so you usually have one per application) can establish to a mongod/mongos process. At time of writing the java driver will establish this amount of connections eventually even if the actual query throughput is low (in order words you will see the "conn" statistic in mongostat rise until it hits this number per app server).
There is no need to set this higher than 100 in most cases but this setting is one of those "test it and see" things. Do note that you will have to make sure you set this low enough so that the total amount of connections to your server do not exceed
Found here How to configure MongoDB Java driver MongoOptions for production use?
I would like to know if the parameter maximumPoolSize on service WorkManagerThreadPool in jca-boss-beans.xml limit the maximum of threads we can instantiate ?
Thanks for your answers,
Nicolas Maujean
It's the maximum number of JCA connections that can be active at the same time (each connection is managed by a thread). So this property doesn't limit the overall number of threads JBoss is using, but just JCA connections. If you want more information about performance and pools you can check Best Practices for Performance Tunning.