How to clear connection pool from dotnet(C#) once the process/connection is closed ?
I am using sybase aseconnection , even after the connection is closed from ado.net , i could see some open connection in pool. Is there any way to clear those from dotnet code..Can anyone help me on this.
I am using sybase version ("Adaptive Server Enterprise/15.5/EBF 19397 SMP ESD#5/P/ia64/HP-UX B.11.23/asear155/2568/64-bit/FBO/Fri").
According to the API documentation on the Sybase page, you can control the lifetime of a connection in the pool by setting the Connection Lifetime property in the connection string. It defaults to 0, which means indefinately.
From the AseConnection docs:
Connection Lifetime
The time, in seconds, that connections can stay open. When a client
closes a connection that has reached or exceeded the defined
Connection Lifetime, before the driver closes the connection instead
of returning it to the connection pool. An idle connection is closed
and removed from the connection pool once it reaches the defined
Connection Lifetime.
The default value of Connection Lifetime is 0, which indicates that
the connection can remain open for an indefinite amount of time.
A second interesting property is:
Connection Idle Timeout
The time, in seconds, that a connection can stay idle in the connection pool before the driver closes the connection. A value of 0 allows connections to stay idle for an indefinite amount of time.
You should be able to set them using this connection string (1 minute lifetime and idle timeout):
"Server=srv;Port=5000;uid=u;pwd=p;Connection Lifetime=60;Connection Idle Timeout=60;"
Related
I know the general idea that a connection pool is a pool of reusable connections that speeds up traffic to the database because it can reuse connections instead of constantly creating new ones.
But this is a very high level explanation. It doesn't explain what is meant by a connection and why the connection pool works, since even with a connection pool such as for example client -> PgBouncer -> PostgreSQL, while the client does not have to create a connection to the databasee, it still has to connect to create a connection to the proxy.
So what is the connection created from (e.g.) client -> PgBouncer and why is creating this connection faster than creating the connection PgBouncer -> PostgreSQL?
There are two uses of a connection pool:
it prevents opening and closing database connections all the time
There is certainly a certain overhead with establishing a TCP connection to pgBouncer, but that is way cheaper than establishing a database connection. When you start a database connection, additional work is done:
a server process is started, which is way more expensive than a TCP connection
PostgreSQL loads cached metadata tables
it puts a limit on the number of client connections, thereby preventing database overload
The advantage over limiting max_connections is that connections in excess of the limit won't receive an error, but will be queued waiting for a connection to become free.
My Springboot application developed with version 2.1.5, when running it, its leaving the db connections in DB pool but after hikari max-lifetime all the connections that were created are elapsing. Any further REST request is not creating new connection and ending up with the error "Unable to connect JDBC"
What could be the problem and how can i overcome it?
If I understand you correctly, you may suffer from a DB connection leak. HikariCP won't close connection in use despite the maxLifetime property:
An in-use connection will never be retired, only when it is closed will it then be removed.
You need to find out if your connections are leaking. For starters you could use another HikariCP property: leakDetectionThreshold:
This property controls the amount of time that a connection can be out of the pool before a message is logged indicating a possible connection leak. A value of 0 means leak detection is disabled. Lowest acceptable value for enabling leak detection is 2000 (2 seconds). Default: 0
In pgjdbc we have:
loginTimeout
connectTimeout
socketTimeout
cancelSignalTimeout
But it isn't clear to me what's the difference (when are they applied) between loginTimeout, connectTimeout and socketTimeout.
As documented in the PostgreSQL JDBC documentation:
loginTimeout = int
Specify how long to wait for establishment of a database connection.
The timeout is specified in seconds.
connectTimeout = int
The timeout value used for socket connect operations. If connecting
to the server takes longer than this value, the connection is broken.
The timeout is specified in seconds and a value of zero means that it
is disabled.
socketTimeout = int
The timeout value used for socket read operations. If reading from
the server takes longer than this value, the connection is closed.
This can be used as both a brute force global query timeout and a
method of detecting network problems. The timeout is specified in
seconds and a value of zero means that it is disabled.
cancelSignalTimeout = int
Cancel command is sent out of band over its own connection, so
cancel message can itself get stuck. This property controls "connect
timeout" and "socket timeout" used for cancel commands. The timeout is
specified in seconds. Default value is 10 seconds.
The connectTimeout and socketTimeout are timeouts on low-level socket operations. The connectTimeout governs the time needed to establish a TCP socket connection. Establishing a TCP connection doesn't guarantee a login (it doesn't even guarantee that you're connecting to a PostgreSQL server, just that you connected to something that accepted your TCP connection). A socketTimeout governs the time a socket can be blocked waiting to read from a socket. This involves all reads from the server, not just during connect, but also during subsequent interaction with the server (eg executing queries).
On the other hand loginTimeout governs the PostgreSQL protocol operation of connecting and authenticating to the PostgreSQL server. This involves establishing a TCP connection followed by one or more exchanges of packets for the handshake and authentication to the PostgreSQL server (I'm not familiar with the details of the PostgreSQL protocol, so I can't be very specific).
Exchanging these packets can take additional time, or if you connected to something that isn't a PostgreSQL server the packet exchange may stall. It might be possible to solve this with careful control of both connectTimeout and socketTimeout, but there are no guarantees (eg data is being exchanged, but the login never completes). In addition, as the socketTimeout also governs all other operations on the connection, you may want to set it higher (eg for other operations that take a long time to get a response back) than you are willing to wait for the login to complete.
The cancelSignalTimeout is used as the connect and socket timeout of the separate TCP connection used for cancel commands.
After reading the source, I'd say it is like this:
connectTimeout specifies how long to wait for a TCP network connection to get established
loginTimeout specifies how long the whole process of logging into the database is allowed to take
socketTimeout specifies how long the client will wait for a response to a command from the server before throwing an error
The first two are related to establishing a connection, the third is relevant for the whole database session.
Establishing a TCP connection is part of establishing a database connection.
This question already has answers here:
Is there a timeout for idle PostgreSQL connections?
(7 answers)
Closed 8 years ago.
I had a user who was having a crashing issue with our app and was not closing his connection to the database properly - eventually reaching his connection limit and was unable to log back in.
That's ok. The problem was eventually fixed and the app is closing the connection properly now.
I was wondering, if a user reaches his connection limit, and without me doing anything, what is the default setting for Postgres 9.1 to drop the connection on its own?
Well, for your actual question:
if a user reaches his connection limit, and without me doing anything, what is the default setting for Postgres 9.1 to drop the connection on its own?
If the connections you are talking about are active connections to real clients (i.e. the client is still around, not crashed and exited, and can read and write to its connection socket), Postgres will not "drop the connection on its own". See this question if you're interested in ways to prune such idle clients.
On the other hand, it is possible for a Postgres backend to be nominally connected to a client which has really crashed or otherwise exited, as you alluded to earlier. In that case, Postgres may not notice the client is gone and not just in an idle state for some time. I believe the exact time would be controlled by a "TCP keepalive time" OS setting, or you could even fiddle with this setting in your client via connection parameters: see the various keepalives-related settings.
I've set up PGBouncer and configured it to connect to my postgres DB and it all connects just fine, however I'm not sure if its actually working.
I have a php script that runs as a daemon and picks up beanstalk jobs. The problem is for every distinct user/action on the system it opens a new connection to postgres and then leaves that connection idling because the daemon doesn't actually stop running so the connection is never terminated (a quick fix for this was to reset the connection at the end of the script loop, but that is going to be inefficient with many connects).
Anyway, this caused postgres to eventually run out of connections and lock up...
So PGBouncer seems like the answer.
But now when I run it, I see the same db connection multiple times when i do ps ax | grep postgres.
Isn't PGBouncer supposed to only ever have 1 connection open to the DB and route all traffic through that connection? Then open a new connection if that one is full?
At present I have 3 for one db connection (my access control system) and 2 for the other database (my client specific data).
To me, it feels like if I roll out these changes then I will just be faced with the same problem that the connections will just get eaten up again because they're not being released.
I hope that explains enough for someone to offer any advice.
It sounds like an important step here is to fix the scripts so they release the connections when they're done. Once you do that, PgBouncer will help reduce the connection setup/teardown overhead, but in connection pooling mode it won't give you the ability to maintain more connections to Pg than you otherwise could.
However, you can also use PgBouncer in transaction-pooling mode. When used for transaction pooling, PgBouncer keeps a pool of idle backend transactions and only assigns them when a client does a BEGIN. Connections are returned to the pool after the client does a COMMIT or ROLLBACK. That means that in transaction pooling mode you can have large numbers of open connections to PgBouncer; they don't each need a corresponding connection to a PostgreSQL backend, so long as most of them are idle at any point in time and don't have any transaction open.
The only real downside of transaction pooling mode is that it breaks applications that expect to SET session-level variables, keep prepared statements across transactions, etc. Applications may need modification, e.g. to use SET LOCAL.