I'm trying to figure out how to intercept the actions of a JDBC connection being taken out from the JDBC connection pool, and just before it's being returned.
Related
I know the general idea that a connection pool is a pool of reusable connections that speeds up traffic to the database because it can reuse connections instead of constantly creating new ones.
But this is a very high level explanation. It doesn't explain what is meant by a connection and why the connection pool works, since even with a connection pool such as for example client -> PgBouncer -> PostgreSQL, while the client does not have to create a connection to the databasee, it still has to connect to create a connection to the proxy.
So what is the connection created from (e.g.) client -> PgBouncer and why is creating this connection faster than creating the connection PgBouncer -> PostgreSQL?
There are two uses of a connection pool:
it prevents opening and closing database connections all the time
There is certainly a certain overhead with establishing a TCP connection to pgBouncer, but that is way cheaper than establishing a database connection. When you start a database connection, additional work is done:
a server process is started, which is way more expensive than a TCP connection
PostgreSQL loads cached metadata tables
it puts a limit on the number of client connections, thereby preventing database overload
The advantage over limiting max_connections is that connections in excess of the limit won't receive an error, but will be queued waiting for a connection to become free.
Can I retrieve a native connection from a hikari-cp connection pool or can I cast the proxy connection that it lends to a postgres connection?
I need to get a database connection from there and pass it to CopyManager. If yes, how?
Use the unwrap() API.
PgConnection pc = conn.unwrap(PgConnection.class);
EDIT: Be careful, do not call close() on the unwrapped Connection instance, because HikariCP will not be able to track it properly.
As far as I know when one establishes multiple Connection objects via JDBC to one database then each connection occupies a separate port on the machine where the Connection is established (they all connect to one port on the server where the DBMS is running).
I tried to extract the port that corresponds to the Connection objects. Unfortunately I did not find any way to do so.
Background: I'm doing performance analysis where I setup multiple clients which issue queries on the db. I'm logging the execution time of queries on the database server. In the resulting log I have - among others - information about the connection who initiated the query e.g. localhost.localdomain:44760 I hope it is possible to use this information to map each query to the client or more precisely the Connection object who initiated the query (which is my ultimate goal and serves analysis purposes).
Just run this select through the JDBC connection:
select inet_client_port()
More functions like that are in the manual:
http://www.postgresql.org/docs/current/static/functions-info.html
I am trying to use Queue-s, so I need to set a Connection Factory, and Queue.
But at the beggining, I can't go forward. I can't set the Connection Factory.
I am using this link to set up a connection factory:
link
But at the 9th step I don't know what to set, this is the interface to set:
If I set "localhost" to Hostname, then click NEXT and trying to test it, I get this message:
A connection could not be made to WebSphere MQ for the following
reason: JMSCMQ0001: WebSphere MQ call failed with compcode '2'
('MQCC_FAILED') reason '2059' ('MQRC_Q_MGR_NOT_AVAILABLE').
So.. How could create a connection factory?
The transport should not be client if you are trying to connect to a local qmgr in bindings mode. The application server will access the qmgr using IPC.
If you are trying to connect as a client then TCP will be used in that case you need to specify the port where the qmgr listener is listening and the SVRCONN (server connection) channel to use.
I get the following exception in my application after leaving a database connection idle for some amount of time:
... An I/O error occured while sending to the backend.; nested exception is org.postgresql.util.PSQLException: An I/O error occured while sending to the backend.] with root cause
java.net.SocketException: Operation timed out
at java.net.SocketInputStream.socketRead0(Native Method)
The same issue happens in psql AND I don't have issues connecting to a local database, so I'm pretty sure the problem is on RDS.
psql=> select 'ok';
SSL SYSCALL error: Operation timed out
psql=> select 'ok';
SSL SYSCALL error: EOF detected
The connection to the server was lost. Attempting reset: Succeeded.
I found this other question which suggests a work around that improved the situation (timeouts now take a lot longer) but didn't fix it.
I'm using Spring Boot with JDBC (tomcat connection pooling) and JDBCTemplate.
Is there a work around or a fix?
Perhaps forcing the connection pool to test and reconnect?
How do I do that in this environment?
EDIT:
This is my connection string
jdbc:postgresql://myhost.c2estvxozjm3.eu-west-1.rds.amazonaws.com/dashboard?tcpKeepAlive=true
SOLUTION:
Edited the RDS server side TCP_KeepAlive parameters as suggested in the selected answer. The parameters I'm using are:
tcp_keepalives_count 5
tcp_keepalives_idle 200
tcp_keepalives_interval 200
It looks like something - maybe a NAT router on your end, maybe something on AWS's end - is connection tracking, and is forgetting about connections after a while.
I suggest enabling TCP keepalives. You might be able to enable them server side in the AWS RDS configuration; if not, you can request them client-side in the JDBC driver.
TCP keepalives are a lot better than a validation/test query, because they're much lower overhead, and they don't result in unnecessary log spam in the server query logs.
Maybe try
spring.datasource.validation-query=SELECT 1
spring.datasource.test-on-borrow=true
(See AbstractDataSourceConfiguration for other options.)
In your connection string are you also including the port or just the endpoint? Try using the entire endpoint in your connection string. Also check to make sure the security group assigned to the RDS instance has the proper ports and Inbound CIDR defined.