Play tests with database: "Too many connections" - scala

Question

To have a database available in scalatest with evolutions I use this extension of the default PlaySpec inspired by this SO question:
trait ResetDbSpec extends PlaySpec with BeforeAndAfterAll {
lazy val appBuilder = new GuiceApplicationBuilder()
lazy val injector = appBuilder.injector()
lazy val databaseApi = injector.instanceOf[DBApi]
override def beforeAll() = {
Evolutions.applyEvolutions(databaseApi.database("default"))
}
override def afterAll() = {
Evolutions.cleanupEvolutions(databaseApi.database("default"))
databaseApi.database("default").shutdown()
}
}
It applies database evolutions when the suite starts, and reverts them when the suite ends. A test then looks like
class ProjectsSpec extends ResetDbSpec with OneAppPerSuite { ...
After adding more tests like this, I hit a point where some tests that succeed when I run them alone, fail with this error:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Data source rejected establishment of connection, message from server: "Too many connections"
As can be see in the code above, I tried to add the line
databaseApi.database("default").shutdown()
in afterAll() to mitigate that, but it had no effect. I tried to not run tests in parallel, but no effect either. Where is it that I open db connections without closing them, and where should I call shutdown()?
N.B. I use Play 2.5.10 and Slick 3.1.

Answer 1

I have a lot of tests (about 500) and I don't get this error, the only difference I have with your code is that I add
databaseApi.database("default").getConnection().close()
and
Play.stop(fakeApplication)
for the integration tests.
Let me know if it changes anything.

Answer 2

Although it does not answer to what is happening with the connections leakage, I finally managed to hack around this:
Add jdbc to you libraryDependencies, even if the Play-Slick FAQ tells you not to do it:
# build.sbt
libraryDependencies += jdbc
Restart sbt to take changes into account. In IntelliJ, you will want to refresh the project, too.
Disable the jdbc module that is conflicting with play-slick (credits: this SO answer):
# application.conf
play.modules.disabled += "play.api.db.DBModule"
At the same place you should have already configured something like
slick {
dbs {
default {
driver = "slick.driver.MySQLDriver$"
db.driver = "com.mysql.jdbc.Driver"
db.url = "jdbc:mysql://localhost/test"
db.user = "sa"
db.password = ""
}
}
}
Now you can use play.api.db.Databases from jdbc and its method withDatabase to run the evolutions.
import org.scalatest.BeforeAndAfterAll
import org.scalatestplus.play.PlaySpec
import play.api.db.{Database, Databases}
import play.api.db.evolutions.Evolutions
trait ResetDbSpec extends PlaySpec with BeforeAndAfterAll {
/**
* Here we use Databases.withDatabase to run evolutions without leaking connections.
* It slows down the tests considerably, though.
*/
private def withTestDatabase[T](block: Database => T) = {
Databases.withDatabase(
driver = "com.mysql.jdbc.Driver",
url = "jdbc:mysql://localhost/test",
name = "default",
config = Map(
"username" -> "sa",
"password" -> ""
)
)(block)
}
override def beforeAll() = {
withTestDatabase { database =>
Evolutions.applyEvolutions(database)
}
}
override def afterAll() = {
withTestDatabase { database =>
Evolutions.cleanupEvolutions(database)
}
}
}
Finally, call tests requiring a db reset like this:
class MySpec extends ResetDbSpec {...}
Of course it sucks repeating this config both in "application.test.conf" and in withDatabase(), plus it mixes two different APIs, not talking about performance. Also it adds this before and after each suite, which is annoying:
[info] application - Creating Pool for datasource 'default'
[info] application - Shutting down connection pool.
If somebody has a better suggestion, please improve on this answer! I have been struggling for months.