I am using a Hikari data source in my application and database kills all the connections which are idle for more than 15 mins. So i wanted to set a connection test query. How do I know when this query is fired and how can I log when this query is fired?
Related
I've got Postgres 13 version and PG-bouncer.
How can I using connection and query to database test usefulness
of pgbouncer?
I mean start query which generate connections or other and
at the end of test we see that time of query is lower, or
workload on database without bouncer is higher?
I'm using SQLAlchemy inside a Lambda with RDS Proxy (PostgreSQL).
This configuration works, but when I'm invoking about 100 lambdas at the same time some lambdas receive a timeout (3 seconds) even when I'm increasing the timeout for 30 seconds I still get a timeout.
After investigating where this timeout comes from, I concluded that the SQLAlchemy hangs when it tries to get a connection to the database.
I thought the RDS proxy could easily handle 100+ concurrent connections.
Here is my engine configuration:
db_url = f"postgresql://{db_username}:{secret}#{db_hostname}:{db_port}/{db_name}"
engine = create_engine(
db_url,
echo=True,
echo_pool="debug",
poolclass=NullPool)
Usually, when SQLAlchemy creates a connection it logs the following:
DEBUG sqlalchemy.pool.impl.NullPool Created new connection <connection object at 0x7f074ebe5580; dsn: 'user=user password=xxx dbname=dbname host=rds-default.{rds-proxy-id}.{region}.rds.amazonaws.com port=port', closed: 0>
But sometimes it will just hang and won't log anything and eventually timeout.
Any help would be appreciated
I use mongodb client to connect Database like here. Notice, i only use one connect to DB, but when i start my application and check Mongodb connection by command.
db.serverStatus().connections
Before start app at 8 connection, after it increase more 20 connection. I miss something, any one help me to explain ?
After a period of inactivity, my go web service is getting a net.OpError with message read tcp x.x.x.x:52086->x.x.x.x:24414: read: connection reset by peer when executing the first postgres sql query. After the error, the subsequent requests will work fine.
The postgres database is hosted with compose.com which has haproxy in front of the postgres db. My go web app is using standard sql and sqlx.
I've tried running a ticker invoking db.Ping() every 15 minutes, but this hasn't fixed the issue.
Why is the go standard sql lib not handling these connection drops?
Since no one wrote that explicity. The solution to this problem is setting db.SetConnMaxLifetime(time.Minute). I tried it and it works. Connection reset occurs often on AWS where there is inactivity limit set to 350 seconds, after that TCP RST is returned.
I'm using JDBC to run queries against my PostgreSQL DB. When the queries take too long I want them to die. I set socketTimeout (as shown below), which does successfully kill the connection, but if I look at pg_stat_activity on the PostgreSQL server, the query often continues to run (for up to a few minutes).
My code looks a little like this:
Properties props = new Properties();
props.setProperty("user", datasource.getUsername());
props.setProperty("password", datasource.getPassword());
props.setProperty("socketTimeout", ""+(timeoutSeconds));
props.setProperty("loginTimeout", "10");
conn = datasource.getDbType().getDriver().connect(url, props);
conn.setReadOnly(true);
conn.setAutoCommit(false);
PreparedStatement statement = conn.prepareStatement(sql);
return statement.executeQuery();
My assumption is that socketTimeout literally just kills the connection, but not the running process on the postgres server. I.e., this timeout is driver-side not PostgreSQL-side.
Is there a way to set a timeout (on a per-statement level) that will actually make PostgreSQL kill the query itself after a given timeout?
Note: I have found one solution so far, which is to explictly cancel the query after a given timeout (i.e. using statement.cancel()). This works, but isn't ideal, since if my server dies, then nobody will kill the running queries.
You should set statement_timeout.