I have a project use both slick and anorm.
I define a method for slick
object DBCache {
def apply(app: play.api.Application) = Cache.getOrElse[Database](app.hashCode.toString){
Database.forDataSource(PlayDB.getDataSource("default")(app))
}
}
private[persist] def inSession[T](block: Session => T) = DBCache(current).withSession(block(_))
And when I can a batch insert method use anorm
def batchInsert(customerAccounts: Seq[Customer]) = DB.withConnection { implicit conn =>
val sql = SQL(insertSql)
val batch = customerAccounts.foldLeft(sql.asBatch) {
(sql, c) => sql.addbatch(xxx)
}
}
It reports
play.api.Application$$anon$1: Execution exception[[MySQLNonTransientConnectionException: No operations allowed after connection closed.]
How to avoid this error
I found the master branch of playframework disable the tracking ,
datasource.setDisableConnectionTracking(
conf.getBoolean("disableConnectionTracking").getOrElse(true))
So I disable it in Global.scala.
There'are still some problem. When sql error happends, connection may be closed. Then I turned to use alibaba's data source https://github.com/alibaba/druid. It works fine!
Related
I've been trying to understand implicits for Scala and trying to use them at work - one particular place im stuck at is trying to pass implicits in the following manner
object DBUtils {
case class DB(val jdbcConnection: Connection) {
def execute[A](op: =>Unit): Any = {
implicit val con = jdbcConnection
op
}
}
object DB {
def SQL(query: String)(implicit jdbcConnection: Connection): PreparedStatement = {
jdbcConnection.prepareStatement(query)
}
}
val someDB1 = DB(jdbcConnection)
val someDB2 = DB(jdbcConnection2)
val someSQL = SQL("SOME SQL HERE")
someDB1.execute{someSQL}
someDB2.execute{someSQL}
Currently i get an execption saying that the SQL() function cannot find the implicit jdbcConnection.What gives and what do i do to make it work in the format i need?
Ps-:Im on a slightly older version of Scala(2.10.4) and cannot upgrade
Edit: Changed the problem statement to be more clear - I cannot use a single implicit connection in scope since i can have multiple DBs with different Connections
At the point where SQL is invoked there is no implicit value of type Connection in scope.
In your code snippet the declaration of jdbcConnection is missing, but if you change it from
val jdbcConnection = //...
to
implicit val jdbcConnection = // ...
then you will have an implicit instance of Connection in scope and the compiler should be happy.
Try this:
implicit val con = jdbcConnection // define implicit here
val someDB = DB(jdbcConnection)
val someSQL = SQL("SOME SQL HERE") // need implicit here
someDB.execute{someSQL}
The implicit must be defined in the scope where you need it. (In reality, it's more complicated, because there are rules for looking elsewhere, as you can find in the documentation. But the simplest thing is to make sure the implicit is available in the scope where you need it.)
Make the following changes
1) execute method take a function from Connection to Unit
2) Instead of this val someDB1 = DB(jdbcConnection) use this someDB1.execute{implicit con => someSQL}
object DBUtils {
case class DB(val jdbcConnection: Connection) {
def execute[A](op: Connection =>Unit): Any = {
val con = jdbcConnection
op(con)
}
}
Here is the complete code.
object DB {
def SQL(query: String)(implicit jdbcConnection: Connection): PreparedStatement = {
jdbcConnection.prepareStatement(query)
}
}
val someDB1 = DB(jdbcConnection)
val someDB2 = DB(jdbcConnection2)
val someSQL = SQL("SOME SQL HERE")
someDB1.execute{implicit con => someSQL}
someDB2.execute{implicit con => someSQL}
I have started to learn scala recently and trying to create simple api using akka HTTP and reactivemongo.
Have problems with simple operations. Spend a lot of time digging docks, official tutorials, stackoverflow etc. Probably I am missing something very simple.
My code:
object MongoDB {
val config = ConfigFactory.load()
val database = config.getString("mongodb.database")
val servers = config.getStringList("mongodb.servers").asScala
val credentials = Lis(Authenticate(database,config.getString("mongodb.userName"), config.getString("mongodb.password")))
val driver = new MongoDriver
val connection = driver.connection(servers, authentications = credentials)
//val db = connection.database(database)
}
Now I would like to make basic CRUD operations. I am trying different code snippets but can't get it working.
Here are some examples:
object TweetManager {
import MongoDB._
//taken from docs
val collection = connection.database("test").
map(_.collection("tweets"))
val document1 = BSONDocument(
"author" -> "Tester",
"body" -> "test"
)
//taken from reactivemongo tutorial, it had extra parameter as BSONCollection, but can't get find the way of getting it
def insertDoc1(doc: BSONDocument): Future[Unit] = {
//another try of getting the collection
//def collection = for ( db1 <- db) yield db1.collection[BSONCollection]("tweets")
val writeRes: Future[WriteResult] = collection.insert(doc)
writeRes.onComplete { // Dummy callbacks
case Failure(e) => e.printStackTrace()
case Success(writeResult) =>
println(s"successfully inserted document with result: $writeResult")
}
writeRes.map(_ => {}) // in this example, do nothing with the success
}
}
insertDoc1(document1)
I can't do any operation on the collection. IDE gives me: "cannot resolve symbol". Compiler gives error:
value insert is not a member of scala.concurrent.Future[reactivemongo.api.collections.bson.BSONCollection]
What is the correct way of doing it?
You are trying to call the insert operation on a Future[Collection], rather than on the underlying collection (calling operation on Future[T] rather than on T is not specific to ReactiveMongo).
It's recommanded to have a look at the documentation.
I'm connecting to MongoDb using following code :
def insert() = {
val mc = new com.mongodb.MongoClient("localhost", 27017);
val db = mc.getDatabase("MyDb");
//My insert code
mc.close();
} //> insert: ()Unit
I have various methods which open and close the connection.
Can the lines :
val mc = new com.mongodb.MongoClient("localhost", 27017);
val db = mc.getDatabase("MyDb");
mc.close();
be extracted so that they are implicitly called at beginning and end of method.
Does Scala implicits cater for this scenario or is reflection required ?
A common pattern would be to use a call-by-name method where you can pass a function that accepts a DB and does something with it. The call-by-name method can facilitate the creation of the client, etc, and execute the code within.
def withDB[A](block: DB => A): A = {
val mc = new com.mongodb.MongoClient("localhost", 27017);
val db = mc.getDatabase("MyDb");
try block(db) finally mc.close()
}
And use it:
def insert() = withDB { db =>
// do something with `db`
}
However, a look at the documentation says:
A MongoDB client with internal connection pooling. For most applications, you should have one MongoClient instance for the entire JVM.
Which makes the above approach look like a bad idea, assuming that is the version you are using. I can definitely see some concurrency issues trying to do this and having too many connections open.
But, you can follow the same pattern, stuffing the connection creating into a singleton object. You would need to manage the closing of the client when your application is shutdown, however.
object Mongo {
lazy val mc = new com.mongodb.MongoClient("localhost", 27017);
lazy val db = mc.getDatabase("MyDb");
def withDB[A](block: DB => A): A = block(db)
def close(): Unit = mc.close()
}
You can define a method that executes some 'work' function e.g.
def withMongoDb[T](work: DB => T): T = {
val mc = new com.mongodb.MongoClient("localhost", 27017)
// I don't actually know what type `db` is so I'm calling it `DB`
val db: DB = mc.getDatabase("MyDb")
try { work(db) }
finally { mc.close() }
}
Then you can use it like:
withMongoDb { db =>
db.insert(...)
db.query(...)
}
This is a similar approach to the one used in Slick, pre-3.0, i.e. withSession and withTransaction.
Now, if you implemented some convenience methods e.g.
def insertStuff(values: Seq[Int])(implicit db: DB) = {
db.insert(values)
}
Then you could call mark the db as implicit in your withMongoDb call, effectively making sure you never accidentally call insertStuff outside of that block.
withMongoDb { implicit db =>
insertStuff(Seq(1,2,3,4))
}
insertStuff(Seq(1,2,3,4)) // compile error
Instead of implicits you could do something like this:
def mongoConn(ip:String, port:Int, dbName:String):(Database => Unit) => Unit = {
f => {
val mc = new com.mongodb.MongoClient(ip, port)
val db = mc.getDatabase(dbName)
f(db)
mc.close()
}
}
val conn = mongoConn("localhost", 27017, "MyDb")
conn(db => {
//insert code
})
I have a problem with futures executing SQL queries.
While database connection is good futures asynchronously executing sql queries within sessions from connection pool. TypeSafe Slick helps me to get sessions from the pool.
When database connection broke down the new coming futures can't execute their queries and wait. I don't see any callbacks onComplete.
When the database connection is good again all previous futures are still waiting. Only new futures coming after reconnect can execute their work.
Please advise how to tell already called waiting futures about db reconnection and continue their work, or do callback onComplete Failure.
My configuration for c3p0 ComboPooledDataSource:
val ds = new ComboPooledDataSource
ds.setDriverClass("oracle.jdbc.OracleDriver")
ds.setJdbcUrl(jdbcUrl)
ds.setInitialPoolSize(20)
ds.setMinPoolSize(1)
ds.setMaxPoolSize(40)
ds.setAcquireIncrement(5)
ds.setMaxIdleTime(3600)
//Connection testing
ds.setTestConnectionOnCheckout(false)
ds.setTestConnectionOnCheckin(false)
ds.setIdleConnectionTestPeriod(10)
//Connection recovery
ds.setBreakAfterAcquireFailure(false)
ds.setAcquireRetryAttempts(30)
ds.setAcquireRetryDelay(10000)
val databasePool = Database.forDataSource(ds)
// Typesafe Slick session handling
def withClient[T](body: Session => T) = {
databasePool.withSession(body)
}
Futures creates here:
class RewardActivatorHelper {
private implicit val ec = new ExecutionContext {
val threadPool = Executors.newFixedThreadPool(1000)
def execute(runnable: Runnable) {threadPool.submit(runnable)}
def reportFailure(t: Throwable) {throw t}
}
case class FutureResult(spStart:Long, spFinish:Long)
def activateReward(msg:Msg, time:Long):Unit = {
msg.users.foreach {
user =>
val future:Future[FutureResult] = Future {
val (spStart, spFinish) = OracleClient.rewardActivate(user)
FutureResult(spStart, spFinish)
}
future.onComplete {
case Success(futureResult:FutureResult) =>
//do something
case Failure(e:Throwable) => log.error(e.getMessage())
}
}
}
Problem solved by setTestConnectionOnCheckout(true)
I am developing an application using Play framework and scala. I am using anorm for data-access layer. And I've got a problem I could not solve.
Brief: I want to be able to have methods in data-access objects (dao) to work inside transactions as well as being called alone.
Details:
I have data-access layer consist of class with methods that only executes particular SQL over database. Traditionally they looks like:
def list() = DB.withConnection { implicit cn =>
...
}
Now I want to have some methods to be executed in a transaction scope. Like traditional select-update service methods but still be able to run them alone. So, what I have in my mind is like this:
class Service {
def fooTransacted() = {
inTransaction {
val old = dao.select(id = 2)
val newObj = old.copy(isActive = true)
dao.update(newObj)
}
}
def fooSinle() = {
dao.select(id = 2)
}
}
I tried around several ways, but could not come up with any solution.
What about
class Dao {
def foo(id: Long)(implicit connection: Connection) = {
SQL("select * from foo where id={id}").on('id->id).as(...)
}
}
class Service{
def withConnection = {
DB.withConnection {implicit connection =>
Dao.foo(1)
Dao.foo(2)
}
}
def withTransaction = {
DB.withTransaction {implicit connection =>
Dao.foo(1)
Dao.foo(2)
}
}
The solution I've seen used elsewhere (principally in Squeryl), is roughly the following:
import java.sql.Connection
object Helper {
private val conn: ThreadLocal[Connection] = new ThreadLocal
def inTransaction[X](f: Connection => X) = {
conn.get() match {
case null =>
DB.withConnection { newConn =>
conn.set(newConn)
try f(newConn)
finally conn.set(null)
}
case c => f(c)
}
}
}
This way, the inTransaction method is re-entrant, so there's no harm in calling it redundantly inside dao.select.
If you prefer, you can expose conn via a public method, and change the signature of f to => X - you lose some compile-time safety, but the API is a little cleaner.
One pitfall with this approach is that connections are tied to threads, which may cause problems if you're using futures or actors, and a process can resume on a different thread (this is a tricky area anyway, but one you should be aware of).
You might want to look into Squeryl too - it may already do what you need.