I have the following configuration in application.conf for an in-memory database HSQLDB:
db {
inmemory {
jndiName = jndiInMemory
driver = org.hsqldb.jdbc.JDBCDriver
url = "jdbc:hsqldb:mem:inmemory"
}
}
And connect in a controller with the following statements
val database = Database.forName("jndiInMemory")
val session = database.createSession
val conn = session.conn
// JDBC statements
Problem is that when the code runs several times, I get an exception in session.conn:
HikariPool-34 - Connection is not available, request timed out after
30000ms.
Since I'm using JNDI, I figured that the connections are reused. Do I have to drop the session after I finish using it? How to fix this code?
Hard to tell without looking at the actual code but in general: when you create a database connection at the start of the application, you usually reuse it until application ends - then you should close connection.
If you spawn a new connection every time you do query, without ending previous ones you will run at the connection limit pretty fast.
Easy to use pattern is: create a session in the beginning and then use dependency injection to pass it to wherever you need to run it.
BTW, I noticed that for some configurations e.g. Slick create connection statically (as in: stores them as static class properties). So, you need to create a handler, that closes session when application exits. It run ok... until you start it several times over in SBT, which by default uses the same JVM to run itself and spawned application. In such cases it is better to run things as fork. For tests I use Test / fork := true, for run I use sbt-revolver, though I am not sure how that would play out with Play.
Related
When my connection is open, the application won't exit, this causes some nasty problems for me (highly concurrent and nested using a shared sesssion, don't know when each part is finished) - is there a way to make sure that the cluster doesn't "hang" the application?
For example here:
object ZombieTest extends App {
val session= Cluster.builder().addContactPoint("localhost").build().connect()
// app doesn't exit unless doing:
session.getCluster.close() // won't exit unless this is called
}
In a slightly biased answer, you could look at https://github.com/outworkers/phantom instead of using the standard java driver.
You get scala.concurrent.Future, monix.eval.Task or even com.twitter.util.Future from a query automatically. You can choose between all three.
DB connection pools are better isolated inside ContactPoint and Database abstraction layers, which have shutdown methods you can easily wire in to your app lifecycle.
It's far faster than the Java driver, as the serialization an de-serialisation of types is wired in compile time via more advanced macro mechanisms.
The short answer is that you need to have a lifecycle way of calling session.close or session.closeAsync when you shut down everything else, it's how it's designed to work.
I have some automated tests that I run in order to test a MongoDB-related library. In order to do that, I start a Mongo server with a temporary data directory and on an ephemeral port, connect to it, and run some tests.
This leads to a race condition, obviously. So in my first version of these tests, I paused for a fixed amount of time and waited to make sure mongod had time to start before the tests began.
This was frustrating (and inefficient), so I decided to monitor the standard output and wait for a line on mongod's standard output stream matching the regular expression:
/\[initandlisten\] waiting for connections/
This got it working. So good, then I prepared to circle back and try to find a more robust way to do it. I recalled that a Java library called "embedmongo" ran MongoDB-based tests, and figured it must solve the problem. And it does this (GitHub):
protected String successMessage() {
return "waiting for connections on port";
}
... and uses that to figure out whether the process has started correctly.
So, are we right? Is examining the mongod process output log (is it ever internationalized? could the wording of the message ever change?) the very best way to do this? Or is there something more robust that we're both missing?
What we do in a similar scenario is:
Try to connect to the configured port (simply new Socket(host, port)) in a loop until it works (10 ms delay) - this ensures, that the mongo client, which starts an internal monitoring thread, does not throw exceptions due to "connection refused"
Connect to the mongodb and query something. This is important, as all mongo client objects are lazy init. (Simple listDatabaseNames() on the client is enough, but make sure to actually read the result.)
All the time, check the process for not being terminated.
I just wrote a small untilMongod command which does just that, which you can use in bash scripting: https://github.com/FGM/untilMongod
Includes a bash + Node.JS example use case.
I'm using NettyServerComponents to embed Play in my application server and I'm having problems with setting the required "application.secret" programatically.
The call I make is:
System.setProperty("application.secret", secret)
and I can verify that it is set via System.getProperty("application.secret"). However initialization will fail if I put the call inside the class that encapsulates and starts the server with this:
Exception in thread "main" #6m0lkl2h5: Configuration error
at play.api.libs.CryptoConfigParser.get$lzycompute(Crypto.scala:235)
at play.api.libs.CryptoConfigParser.get(Crypto.scala:204)
at play.api.BuiltInComponents$class.cryptoConfig(Application.scala:275)
...
If I move the same setProperty call earlier in the code, it works just fine.
Is there some import for play that causes system properties to be read and cached? Or some other reason why it's not seeing the value i can see via getProperty?
I've solved the initialization order problem and while it is rather specific to my setup, I wanted to post my answer in case someone ends up in a similar situation.
Configuration of play when using NettyServerComponents happens via this line:
lazy val configuration: Configuration = Configuration(ConfigFactory.load())
ConfigFactory.load() is part of com.typesafe.config and statically initializes configuration on first access. The daemon that I'm embedding Play in also uses this configuration via net.ceedubs.ficus.FicusConfig which means that even though the above line is a lazy initialization, my code had previously called ConfigFactory.load() for its own configuration, meaning that setting the application.secret via setProperty had no effect.
For bonus fun, since both sets of code use ConfigFactory.load(), i just had to put
application {
secret = "foo"
}
In my existing config file and Play picked it up from there.
I have a class inheriting from DbContext implementing a code-first Entity Framework object.
I would like to have a unit test that exercises the model builder in this dbcontext -- this would be useful for detecting things like 'key not defined on entity' errors.
But I may not have an actual database available during unit testing time. Is there a way to exercise this code without actually trying to make a database connection with the context. I tried something innocuous like:
var ctx = new MyDbContext("Data Source=(local);Initial Catalog=Dummy;<.......>");
var foo = ctx.GetValidationErrors(); //triggers DBModelBuilder, should throw if my DBModel is goofed up
This does technically work. However this takes a very long time to run -- if I pause and inspect the call stack it is triggering a call to System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin
Eventually it times out, swallows the connection error and finishes my test.
Is there any way to do this without the connection attempt?
The solution here is to just kind of go with the problem and use the lowest possible connection timeout. I'm using this connection string:
Server=localhost; Database=tempdb; Integrated Security=true;Connection Timeout=1;ConnectRetryCount=0
It still triggers the problem, but with a one second timeout and no automatic retries (for only this one test in the system) it's totally acceptable.
And if the developer has a local db installed, it will accidentally work even faster.
I am using Neo4j embedded in my Scala project. I have been including
ShutdownHookThread {
shutdown(ds)
}
the above piece of code in each and every function before the beginning of transaction. Do I need to include it in every function. What happens if I don't include it?
ShutdownHookThread registers a piece of code to be executed when your application is about to exit. You need to use it only once - somewhere in your app bootstrap code, cause there is no sense to shutdown the database more than one time.