Currently have a project that is running Play + Play-Slick and we have issues in regards to running tests in parallel. The problem is that for every single Specification, it creates its own instance of a HikariCP connection pool which means that the tests end up failing with
org.postgresql.util.PSQLException: FATAL: sorry, too many clients already
You can tell this is happening because you get a lot of log messages that are saying
[info] - com.zaxxer.hikari.HikariDataSource - HikariCP pool db is starting.
Since its quite a large test suite (with many specifications), using the
override def is = args(sequential = true) ^ super.is
Isn't ideal, because that will only work for the specifications that are a subclass of a that specification (however we have many specifications that are in seperate packages that have no common super type).
Is there some way to configure specs2 to use a global connection pool, so that only a single HikariCP connection pool is used for the entire testing phase
Related
I have a Spring Boot application (v2.1.5) that uses JPA (Hibernate) to connect to a Postgres DB. Spring Boot uses HikariCP for connection pooling.
In my production environment, I see the following query executed every few seconds regardless of DB activity (almost as if they are some kind of health check?):
SET application_name = 'PostgreSQL JDBC Driver'
I am struggling to get to the bottom of why these queries are performed so often and if they can be avoided because in my local environment the above statement is only executed when a query to the DB is performed. I still do not understand why, but it is less frequent and different behaviour compared to production.
Are these queries necessary? Can they be avoided?
Thanks.
UPDATE:
Here is a screenshot of the queries received by the DB to which the Spring boot app is connecting using HikariCP. The time is shown as "Just now" because all of the query shown are only ~0.5 seconds apart and all within the "current minute".
This seems to be performed by the Hikari Connection Pool. See Default HikariCP connection pool starting Spring Boot application and it's answers.
I wouldn't bother about it since, it is not performing a "extremely high number" of these operations, but only every few seconds, possibly whenever a connection is handed out by or returned to the pool.
If it really bothers you, you could look into various places for disabling it.
The last comment here suggests that setting the connection property assumeMinServerVersion to 9.0 or higher might help.
Since this is probably triggered by the HikariConnectionPool it might be configurable there, by configuring the behaviour when starting, lending and returning a connection.
I run 3 processes at the same time , all of them are using the same DB (RDS postgres)
all of them are java application that uses JDBC for connecting to the DB
I am using PGPoolingDataSource in every process as a connection pool for the DB
every request is handled by the book - ended with
finally{
connection.close();
}
main problems:
1.
I ran out of connections really fast because I do a massive work
with the DB at the beginning (which is ok) but the pool never
shrinks.
I get some exceptions in the code because there are not enough connections and I wish I could expand the timeout when a requesting
a connection.
My insights:
the PGPoolinDataSource never shrinks by definition! I couldn't find any documentation about it, but I assume this is the case. So I tried the apache DBCP pool and again I am having the same problem .
I think there must be timeout when waiting for a connection - I would guess that this timeout can be configured, but I couldn't find this configuration on both pools .
My Questions:
why does the pool never shrinks?!
how to determine how many connections to allocate for each pool\process (here every process has one pool)
what happens if I don't close the pool (not the connections) and the app is dead does the connections on the pool are still alive? this happens a lot when I update the application I stop and start it so I never close the pool.
what would be a good JDBC connection pool that works best with postgres and that has an option to set the timeout for the getConnection ?
Thanks
I am using akka-persistence-jdbc plugin for Akka Persistence in some parts of application and slick directly in another parts.
After migrating the the hottest parts from direct slick usage to akka-persistence HikariCP started to throw exceptions:
WARN com.zaxxer.hikari.pool.LeakTask Connection leak detection triggered for connection org.postgresql.jdbc.PgConnection#3a3c8d5d, stack trace follows
java.lang.Exception: Apparent connection leak detected
That's not a problem of slow SQL queries. Explain analyze shows that PostgreSQL executes them in ~1 millisecond. But connection waits for something and not being released for minutes. Some more details on that are here.
akka-persistence-jdbc uses streaming for writes and reads, could it be a slick bug or something is wrong in the way akka-persistence-jdbc does such operations?
I am using slick 3.1.1, HikariCP 2.3.7, PostgreSQL 9.4 with max_connections: 120.
I'm new to Cassandra and JBoss, and am trying to create a connection pool. I've searched everywhere and found bits and pieces of information, but I'm still missing something.
I'm not clear on what I need in my standalone file, within the driver element. What should I specify for driver-class and xa-datasource-class?
And, in module..xml, what path should I be using in the resource-root element?
I have these 2 jar files - are they correct?
cassandra-driver-core-2.0.2.jar
cassandra-driver-dse-2.0.2.jar
I'm able to open a connection and execute cql queries from a standalone Java class, but now I need to create a connection pool in JBoss. Any help would be appreciated. Thanks.
Cassandra driver, itself maintains connection pool (atleast with datastax jars), which is configurable, in the run-time,
and also can be configured while making the session.
On top of that cassandra driver even lets you read connection pool status, if you have chosen to do that. So you can create your own monitoring service for connection pool status.
So, not sure, what you are trying to achieve here, pool on top of another pool?
Is there a way to configure the connection pool properties of Morphia? I see the connection count increase appropriately in the console as I run multiple, concurrent tests against my application. However, I have been unable to locate any documentation that explains how to configure the initial number of connections, timeouts, size of the pool, etc.
Any resources you can point me to that would explain how to do this?
You would tune that through the Mongo (or MongoClient) you use to connect. Morphia itself doesn't do any pooling. More documentation on the java driver can be found here: http://docs.mongodb.org/ecosystem/drivers/java/