Slick access to DataSource Programmatically - scala

I have a object like this :
object DatabaseFactory {
import slick.jdbc.PostgresProfile.api._
private val db = Database.forConfig("database.postgresql")
def getDatabase = db
}
and a configuration like this :
database {
postgresql {
connectionPool = "HikariCP"
dataSourceClass = "org.postgresql.ds.PGSimpleDataSource"
properties = {
serverName = "localhost"
portNumber = "5432"
databaseName = "myProject"
user = "user"
password = "userPass"
}
numThreads = 10
}
}
There is any way to get javax.sql.DataSource from slick ?
I need a instance of PGSimpleDataSource from slick .
I want to use that on Flyway configuration :
Flyway.configure()
.baselineOnMigrate(true)
.locations("filesystem:/etc/myProject/db-scripts")
.dataSource(??? Need DataSource ???)

I've just stumbled upon this and seen the comment by https://stackoverflow.com/users/337134/knows-not-much .
Basically, you'll need to implement your own instance of Datasource:
package slick.migration.api.flyway
import java.io.PrintWriter
import java.sql.{DriverManager, SQLException, SQLFeatureNotSupportedException}
import slick.jdbc.JdbcBackend
import javax.sql.DataSource
class DatabaseDatasource(database: JdbcBackend#Database) extends DataSource {
override def getConnection = database.createSession().conn
override def getConnection(username: String, password: String) = throw new SQLFeatureNotSupportedException()
override def unwrap[T](iface: Class[T]) =
if (iface.isInstance(this)) this.asInstanceOf[T]
else throw new SQLException(getClass.getName + " is not a wrapper for " + iface)
override def isWrapperFor(iface: Class[_]) = iface.isInstance(this)
override def getLogWriter = throw new SQLFeatureNotSupportedException()
override def setLogWriter(out: PrintWriter): Unit = throw new SQLFeatureNotSupportedException()
override def setLoginTimeout(seconds: Int): Unit = DriverManager.setLoginTimeout(seconds)
override def getLoginTimeout = DriverManager.getLoginTimeout
override def getParentLogger = throw new SQLFeatureNotSupportedException()
}

There is any way to get javax.sql.DataSource from slick ?
I don't think so but I was able to get an instance of slick.jdbc.JdbcDataSource which I then used the exact same way i always used javax.sql.DataSource. creating connection, then preparedstatement, then handling a resultset. all the same.
db.source
I know this is old but wanted to give an option to try that isn't as drastic as the other answer of creating your own instance of datasource.

Related

Determing if a MongoDB connection is unavailavble and creating a new connection if it is

I'm attempting to improve the below code that creates a MongoDB connection and inserts a document using the insertDocument method:
import com.typesafe.scalalogging.LazyLogging
import org.mongodb.scala.result.InsertOneResult
import org.mongodb.scala.{Document, MongoClient, MongoCollection, MongoDatabase, Observer, SingleObservable}
import play.api.libs.json.JsResult.Exception
object MongoFactory extends LazyLogging {
val uri: String = "mongodb+srv://*********"
val client: MongoClient = MongoClient(uri)
val db: MongoDatabase = client.getDatabase("db")
val collection: MongoCollection[Document] = db.getCollection("col")
def insertDocument(document: Document) = {
val singleObservable: SingleObservable[InsertOneResult] = collection.insertOne(document)
singleObservable.subscribe(new Observer[InsertOneResult] {
override def onNext(result: InsertOneResult): Unit = println(s"onNext: $result")
override def onError(e: Throwable): Unit = println(s"onError: $e")
override def onComplete(): Unit = println("onComplete")
})
}
}
The primary issue I see with the above code is that if the connection becomes stale due to MongoDB server going offline or some other condition then
the connection is not restarted.
An improvement to cater for this scenario is :
object MongoFactory extends LazyLogging {
val uri: String = "mongodb+srv://*********"
var client: MongoClient = MongoClient(uri)
var db: MongoDatabase = client.getDatabase("db")
var collection: MongoCollection[Document] = db.getCollection("col")
def isDbDown() : Boolean = {
try {
client.getDatabase("db")
false
}
catch {
case e: Exception =>
true
}
}
def insertDocument(document: Document) = {
if(isDbDown()) {
client = MongoClient(uri)
db = client.getDatabase("db")
collection = db.getCollection("col")
}
val singleObservable: SingleObservable[InsertOneResult] = collection.insertOne(document)
singleObservable.subscribe(new Observer[InsertOneResult] {
override def onNext(result: InsertOneResult): Unit = println(s"onNext: $result")
override def onError(e: Throwable): Unit = println(s"onError: $e")
override def onComplete(): Unit = println("onComplete")
})
}
}
I expect this to handle the scenario if the DB connection becomes unavailable but is there a more idiomatic Scala method of
determining
Your code does not create connections. It creates MongoClient instances.
As such you cannot "create a new connection". MongoDB drivers do not provide an API for applications to manage connections.
Connections are managed internally by the driver and are created and destroyed automatically as needed in response to application requests/commands. You can configure connection pool size and when stale connections are removed from the pool.
Furthermore, execution of a single application command may involve multiple connections (up to 3 easily, possibly over 5 if encryption is involved), and the connection(s) used depend on the command/query. Checking the health of any one connection, even if it was possible, wouldn't be very useful.

How to bind Slick dependency with Lagom?

So, I have this dependency which is used to create tables and interact with Postgres. Here is a Sample Class:
class ConfigTable {
this: DBFactory =>
import driver.api._
implicit val configKeyMapper = MappedColumnType.base[ConfigKey, String](e => e.toString, s => ConfigKey.withName(s))
val configs = TableQuery[ConfigMapping]
class ConfigMapping(tag: Tag) extends Table[Config](tag, "configs") {
def key = column[ConfigKey]("key")
def value = column[String]("value")
def * = (key, value) <> (Config.tupled, Config.unapply _)
}
/**
* add config
*
* #param config
* #return
*/
def add(config: Config): Try[Config] = try {
sync(db.run(configs += config)) match {
case 1 => Success(config)
case _ => Failure(new Exception("Unable to add config"))
}
} catch {
case ex: PSQLException =>
if (ex.getMessage.contains("duplicate key value")) Failure(new Exception("alt id already exists."))
else Failure(new Exception(ex.getMessage))
}
def get(key: ConfigKey): Option[Config] = sync(db.run(configs.filter(x => x.key === key).result)).headOption
def getAll(): Seq[Config] = sync(db.run(configs.result))
}
object ConfigTable extends ConfigTable with PSQLComponent
PSQLComponent is the Abstraction for Database meta configuration:
import slick.jdbc.PostgresProfile
trait PSQLComponent extends DBFactory {
val driver = PostgresProfile
import driver.api.Database
val db: Database = Database.forConfig("db.default")
}
DBFactory is again an abstraction:
import slick.jdbc.JdbcProfile
trait DBFactory {
val driver: JdbcProfile
import driver.api._
val db: Database
}
application.conf:
db.default {
driver = "org.postgresql.Driver"
url = "jdbc:postgresql://localhost:5432/db"
user = "user"
password = "pass"
hikaricp {
minimumIdle = ${db.default.async-executor.minConnections}
maximumPoolSize = ${db.default.async-executor.maxConnections}
}
}
jdbc-defaults.slick.profile = "slick.jdbc.PostgresProfile$"
lagom.persistence.jdbc.create-tables.auto=false
I compile and publish this dependency to nexus and trying to use this in my Lagom Microservice.
Here is the Loader Class:
class SlickExapleAppLoader extends LagomApplicationLoader {
override def load(context: LagomApplicationContext): LagomApplication = new SlickExampleApp(context) {
override def serviceLocator: ServiceLocator = NoServiceLocator
}
override def loadDevMode(context: LagomApplicationContext): LagomApplication = new SlickExampleApp(context) with LagomDevModeComponents {
}
override def describeService = Some(readDescriptor[SlickExampleLMSServiceImpl])
}
abstract class SlickExampleApp(context: LagomApplicationContext)
extends LagomApplication(context)
// No Idea which to use and how, nothing clear from doc too.
// with ReadSideJdbcPersistenceComponents
// with ReadSideSlickPersistenceComponents
// with SlickPersistenceComponents
with AhcWSComponents {
wire[SlickExampleScheduler]
}
I'm trying to implement it in this scheduler:
class SlickExampleScheduler #Inject()(lmsService: LMSService,
configuration: Configuration)(implicit ec: ExecutionContext) {
val brofile = `SomeDomainObject`
val gson = new Gson()
val concurrency = Runtime.getRuntime.availableProcessors() * 10
implicit val timeout: Timeout = 3.minute
implicit val system: ActorSystem = ActorSystem("LMSActorSystem")
implicit val materializer: ActorMaterializer = ActorMaterializer()
// Getting Exception Initializer here..... For ConfigTable ===> ExceptionLine
val schedulerImplDao = new SchedulerImplDao(ConfigTable)
def hitLMSAPI = {
println("=============>1")
schedulerImplDao.doSomething()
}
system.scheduler.schedule(2.seconds, 2.seconds) {
println("=============>")
hitLMSAPI
}
}
Not sure if it's the correct way, or if it's not what is the correct way of doing this. It is the project requirement to keep the Data Models separate from the service for the obvious reasons of re-usability.
Exception Stack:
17:50:38.666 [info] akka.cluster.Cluster(akka://lms-impl-application) [sourceThread=ForkJoinPool-1-worker-1, akkaTimestamp=12:20:38.665UTC, akkaSource=akka.cluster.Cluster(akka://lms-impl-application), sourceActorSystem=lms-impl-application] - Cluster Node [akka.tcp://lms-impl-application#127.0.0.1:45805] - Started up successfully
17:50:38.707 [info] akka.cluster.Cluster(akka://lms-impl-application) [sourceThread=lms-impl-application-akka.actor.default-dispatcher-6, akkaTimestamp=12:20:38.707UTC, akkaSource=akka.cluster.Cluster(akka://lms-impl-application), sourceActorSystem=lms-impl-application] - Cluster Node [akka.tcp://lms-impl-application#127.0.0.1:45805] - No seed-nodes configured, manual cluster join required
java.lang.ExceptionInInitializerError
at com.slick.init.impl.SlickExampleScheduler.<init>(SlickExampleScheduler.scala:29)
at com.slick.init.impl.SlickExampleApp.<init>(SlickExapleAppLoader.scala:42)
at com.slick.init.impl.SlickExapleAppLoader$$anon$2.<init>(SlickExapleAppLoader.scala:17)
at com.slick.init.impl.SlickExapleAppLoader.loadDevMode(SlickExapleAppLoader.scala:17)
at com.lightbend.lagom.scaladsl.server.LagomApplicationLoader.load(LagomApplicationLoader.scala:76)
at play.core.server.LagomReloadableDevServerStart$$anon$1.$anonfun$get$5(LagomReloadableDevServerStart.scala:176)
at play.utils.Threads$.withContextClassLoader(Threads.scala:21)
at play.core.server.LagomReloadableDevServerStart$$anon$1.$anonfun$get$3(LagomReloadableDevServerStart.scala:173)
at scala.Option.map(Option.scala:163)
at play.core.server.LagomReloadableDevServerStart$$anon$1.$anonfun$get$2(LagomReloadableDevServerStart.scala:149)
at scala.util.Success.flatMap(Try.scala:251)
at play.core.server.LagomReloadableDevServerStart$$anon$1.$anonfun$get$1(LagomReloadableDevServerStart.scala:147)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:658)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.NullPointerException
at com.example.db.models.LoginTable.<init>(LoginTable.scala:29)
at com.example.db.models.LoginTable$.<init>(LoginTable.scala:293)
at com.example.db.models.LoginTable$.<clinit>(LoginTable.scala)
... 24 more
This is how it is woking:
abstract class SlickExampleApp(context: LagomApplicationContext) extends LagomApplication(context)
with SlickPersistenceComponents with AhcWSComponents {
override implicit lazy val actorSystem: ActorSystem = ActorSystem("LMSActorSystem")
override lazy val materializer: ActorMaterializer = ActorMaterializer()
override lazy val lagomServer = serverFor[SlickExampleLMSService](wire[SlickExampleLMSServiceImpl])
lazy val externalService = serviceClient.implement[LMSService]
override def connectionPool: ConnectionPool = new HikariCPConnectionPool(environment)
override def jsonSerializerRegistry: JsonSerializerRegistry = new JsonSerializerRegistry {
override def serializers: immutable.Seq[JsonSerializer[_]] = Vector.empty
}
val loginTable = wire[LoginTable]
wire[SlickExampleScheduler]
}
> One thing I'd like to report is: Lagom docs about the application.conf configuration of slick is not correct, it misleaded me for two days, the I digged into the Liberary code and this is how it goes:
private val readSideConfig = system.settings.config.getConfig("lagom.persistence.read-side.jdbc")
private val jdbcConfig = system.settings.config.getConfig("lagom.persistence.jdbc")
private val createTables = jdbcConfig.getConfig("create-tables")
val autoCreateTables: Boolean = createTables.getBoolean("auto")
// users can disable the usage of jndiDbName for userland read-side operations by
// setting the jndiDbName to null. In which case we fallback to slick.db.
// slick.db must be defined otherwise the application will fail to start
val db = {
if (readSideConfig.hasPath("slick.jndiDbName")) {
new InitialContext()
.lookup(readSideConfig.getString("slick.jndiDbName"))
.asInstanceOf[Database]
} else if (readSideConfig.hasPath("slick.db")) {
Database.forConfig("slick.db", readSideConfig)
} else {
throw new RuntimeException("Cannot start because read-side database configuration is missing. " +
"You must define either 'lagom.persistence.read-side.jdbc.slick.jndiDbName' or 'lagom.persistence.read-side.jdbc.slick.db' in your application.conf.")
}
}
val profile = DatabaseConfig.forConfig[JdbcProfile]("slick", readSideConfig).profile
The configuration it requires is very much different than the suggested one on the Doc.

Close or shutdown of H2 database after tests is not working

I am facing a problem of database clean-up after each test when using scalatest with Slick.
Here is code of the test:
class H2DatabaseSpec extends WordSpec with Matchers with ScalaFutures with BeforeAndAfter {
implicit override val patienceConfig = PatienceConfig(timeout = Span(5, Seconds))
val h2DB: H2DatabaseService = new H2DatabaseService
var db: Database = _
before {
db = Database.forConfig("h2mem1")
h2DB.createSchema.futureValue
}
after {
db.shutdown.futureValue
}
"H2 database" should {
"query a question" in {
val newQuestion: QuestionEntity = QuestionEntity(Some(1L), "First question")
h2DB.insertQuestion(newQuestion).futureValue
val question = h2DB.getQuestionById(1L)
question.futureValue.get shouldBe newQuestion
}
}
it should {
"query all questions" in {
val newQuestion: QuestionEntity = QuestionEntity(Some(2L), "Second question")
h2DB.insertQuestion(newQuestion).futureValue
val questions = h2DB.getQuestions
questions.futureValue.size shouldBe 1
}
}
}
Database service is just invoking run method on defined database:
class H2DatabaseService {
val db = Database.forConfig("h2mem1")
val questions = TableQuery[Question]
def createSchema =
db.run(questions.schema.create)
def getQuestionById(id: Long): Future[Option[QuestionEntity]] =
db.run(questions.filter(_.id === id).result.headOption)
def getQuestions: Future[Seq[QuestionEntity]] =
db.run(questions.result)
def insertQuestion(question: QuestionEntity): Future[Int] =
db.run(questions += question)
}
class Question(tag: Tag) extends Table[QuestionEntity](tag, "QUESTION") {
def id = column[Option[Long]]("QUESTION_ID", O.PrimaryKey, O.AutoInc)
def title = column[String]("TITLE")
def * = (id, title) <> ((QuestionEntity.apply _).tupled, QuestionEntity.unapply)
}
case class QuestionEntity(id: Option[Long] = None, title: String) {
require(!title.isEmpty, "title.empty")
}
And the database is defined as follows:
h2mem1 = {
url = "jdbc:h2:mem:test1"
driver = org.h2.Driver
connectionPool = disabled
keepAliveConnection = true
}
I am using Scala 2.11.8, Slick 3.1.1, H2 database 1.4.192 and scalatest 2.2.6.
Error that appears when tests are executed is Table "QUESTION" already exists. So it looks like shutdown() method has no effect at all (but it is invoked - checked in debugger).
Anybody knows how to handle such scenario? How to clean-up database properly after each test?
Database has not been correctly cleaned-up because of invoking the method on different object.
H2DatabaseService has it's own db object and the test it's own. Issue was fixed after refactoring H2 database service and invoking:
after {
h2DB.db.shutdown.futureValue
}

Best Practise of Using Connection Pool in Slick 3.0.0 Together with Play Framework

I followed the documentation of Slick 3.0.0-RC1, using Typesafe Config as database connection configuration. Here is my conf:
database = {
driver = "org.postgresql.Driver"
url = "jdbc:postgresql://localhost:5432/postgre"
user = "postgre"
}
I established a file Locale.scala as:
package models
import slick.driver.PostgresDriver.api._
import scala.concurrent.Future
case class Locale(id: String, name: String)
class Locales(tag: Tag) extends Table[Locale](tag, "LOCALES") {
def id = column[String]("ID", O.PrimaryKey)
def name = column[String]("NAME")
def * = (id, name) <> (Locale.tupled, Locale.unapply)
}
object Locales {
private val locales = TableQuery[Locales]
val db = Database.forConfig("database")
def count: Future[Int] =
try db.run(locales.length.result)
finally db.close
}
Then I got confused that when and where the proper time is to create Database object using
val db = Database.forConfig("database")
If I create db like this, there will be as many Database objects as my models. So what is the best practice to get this work?
You can create an Object DBLocator and load it using lazy operator so that its loaded only on demand.
You can always invoke the method defined in DBLocator class to get an instance of Session.

Slick: create database

Is there a way to get slick to create the database if it doesn't already exist?
Database.forURL("jdbc:mysql://127.0.0.1/database", driver = "com.mysql.jdbc.Driver", user = "root") withSession {
// create tables, insert data
}
"database" doesn't exist, so I want slick to create it for me. Any ideas? Thanks.
The answer above is relevant to Slick 2.x where withSession is deprecated,
so this is how it is done with Slick 3.0.0 API :
import scala.concurrent.Await
import scala.concurrent.duration._
import org.postgresql.util.PSQLException
import slick.driver.PostgresDriver
import slick.driver.PostgresDriver.api._
object SlickPGUtils {
private val actionTimeout = 10 second
private val driver = "org.postgresql.Driver"
def createDb(host: String, port: Int, dbName: String, user: String, pwd: String) = {
val onlyHostNoDbUrl = s"jdbc:postgresql://$host:$port/"
using(Database.forURL(onlyHostNoDbUrl, user = user, password = pwd, driver = driver)) { conn =>
Await.result(conn.run(sqlu"CREATE DATABASE #$dbName"), actionTimeout)
}
}
def dropDb(host: String, port: Int, dbName: String, user: String, pwd: String) = {
val onlyHostNoDbUrl = s"jdbc:postgresql://$host:$port/"
try {
using(Database.forURL(onlyHostNoDbUrl, user = user, password = pwd, driver = driver)) { conn =>
Await.result(conn.run(sqlu"DROP DATABASE #$dbName"), actionTimeout)
}
} catch {
// ignore failure due to db not exist
case e:PSQLException => if (e.getMessage.equals(s""""database "$dbName" does not exist""")) {/* do nothing */}
case e:Throwable => throw e // escalate other exceptions
}
}
private def using[A <: {def close() : Unit}, B](resource: A)(f: A => B): B =
try {
f(resource)
} finally {
Try {
resource.close()
}.failed.foreach(err => throw new Exception(s"failed to close $resource", err))
}
}
You can connect to the database engine using only "jdbc:mysql://localhost/" as JDBC URL and then issue a pure SQL create database query:
import scala.slick.driver.MySQLDriver.simple._
import scala.slick.jdbc.{StaticQuery => Q}
object Main extends App
{
Database.forURL("jdbc:mysql://localhost/", driver = "com.mysql.jdbc.Driver") withSession {
implicit session =>
Q.updateNA("CREATE DATABASE `dataBaseName`").execute
.
.
.
}
}