Slick: create database - scala

Is there a way to get slick to create the database if it doesn't already exist?
Database.forURL("jdbc:mysql://127.0.0.1/database", driver = "com.mysql.jdbc.Driver", user = "root") withSession {
// create tables, insert data
}
"database" doesn't exist, so I want slick to create it for me. Any ideas? Thanks.

The answer above is relevant to Slick 2.x where withSession is deprecated,
so this is how it is done with Slick 3.0.0 API :
import scala.concurrent.Await
import scala.concurrent.duration._
import org.postgresql.util.PSQLException
import slick.driver.PostgresDriver
import slick.driver.PostgresDriver.api._
object SlickPGUtils {
private val actionTimeout = 10 second
private val driver = "org.postgresql.Driver"
def createDb(host: String, port: Int, dbName: String, user: String, pwd: String) = {
val onlyHostNoDbUrl = s"jdbc:postgresql://$host:$port/"
using(Database.forURL(onlyHostNoDbUrl, user = user, password = pwd, driver = driver)) { conn =>
Await.result(conn.run(sqlu"CREATE DATABASE #$dbName"), actionTimeout)
}
}
def dropDb(host: String, port: Int, dbName: String, user: String, pwd: String) = {
val onlyHostNoDbUrl = s"jdbc:postgresql://$host:$port/"
try {
using(Database.forURL(onlyHostNoDbUrl, user = user, password = pwd, driver = driver)) { conn =>
Await.result(conn.run(sqlu"DROP DATABASE #$dbName"), actionTimeout)
}
} catch {
// ignore failure due to db not exist
case e:PSQLException => if (e.getMessage.equals(s""""database "$dbName" does not exist""")) {/* do nothing */}
case e:Throwable => throw e // escalate other exceptions
}
}
private def using[A <: {def close() : Unit}, B](resource: A)(f: A => B): B =
try {
f(resource)
} finally {
Try {
resource.close()
}.failed.foreach(err => throw new Exception(s"failed to close $resource", err))
}
}

You can connect to the database engine using only "jdbc:mysql://localhost/" as JDBC URL and then issue a pure SQL create database query:
import scala.slick.driver.MySQLDriver.simple._
import scala.slick.jdbc.{StaticQuery => Q}
object Main extends App
{
Database.forURL("jdbc:mysql://localhost/", driver = "com.mysql.jdbc.Driver") withSession {
implicit session =>
Q.updateNA("CREATE DATABASE `dataBaseName`").execute
.
.
.
}
}

Related

Slick access to DataSource Programmatically

I have a object like this :
object DatabaseFactory {
import slick.jdbc.PostgresProfile.api._
private val db = Database.forConfig("database.postgresql")
def getDatabase = db
}
and a configuration like this :
database {
postgresql {
connectionPool = "HikariCP"
dataSourceClass = "org.postgresql.ds.PGSimpleDataSource"
properties = {
serverName = "localhost"
portNumber = "5432"
databaseName = "myProject"
user = "user"
password = "userPass"
}
numThreads = 10
}
}
There is any way to get javax.sql.DataSource from slick ?
I need a instance of PGSimpleDataSource from slick .
I want to use that on Flyway configuration :
Flyway.configure()
.baselineOnMigrate(true)
.locations("filesystem:/etc/myProject/db-scripts")
.dataSource(??? Need DataSource ???)
I've just stumbled upon this and seen the comment by https://stackoverflow.com/users/337134/knows-not-much .
Basically, you'll need to implement your own instance of Datasource:
package slick.migration.api.flyway
import java.io.PrintWriter
import java.sql.{DriverManager, SQLException, SQLFeatureNotSupportedException}
import slick.jdbc.JdbcBackend
import javax.sql.DataSource
class DatabaseDatasource(database: JdbcBackend#Database) extends DataSource {
override def getConnection = database.createSession().conn
override def getConnection(username: String, password: String) = throw new SQLFeatureNotSupportedException()
override def unwrap[T](iface: Class[T]) =
if (iface.isInstance(this)) this.asInstanceOf[T]
else throw new SQLException(getClass.getName + " is not a wrapper for " + iface)
override def isWrapperFor(iface: Class[_]) = iface.isInstance(this)
override def getLogWriter = throw new SQLFeatureNotSupportedException()
override def setLogWriter(out: PrintWriter): Unit = throw new SQLFeatureNotSupportedException()
override def setLoginTimeout(seconds: Int): Unit = DriverManager.setLoginTimeout(seconds)
override def getLoginTimeout = DriverManager.getLoginTimeout
override def getParentLogger = throw new SQLFeatureNotSupportedException()
}
There is any way to get javax.sql.DataSource from slick ?
I don't think so but I was able to get an instance of slick.jdbc.JdbcDataSource which I then used the exact same way i always used javax.sql.DataSource. creating connection, then preparedstatement, then handling a resultset. all the same.
db.source
I know this is old but wanted to give an option to try that isn't as drastic as the other answer of creating your own instance of datasource.

Play can't find database

So, for 3 days now I have had various problems with Play, I am new to the framework, but I don't get what is happening. So, after I was unable to use Slick, due to some futures never returning a value, I decided to switch to Anorm. It worked, until I decided to add a second repository... after which, I am now unable to load my page because I keep getting this:
#769g71c3d - Internal server error, for (GET) [/] ->
play.api.UnexpectedException: Unexpected exception[ProvisionException: Unable to provision, see the following errors:
1) Error injecting constructor, java.lang.IllegalArgumentException: Could not find database for pmf
at dao.ActivityTypeRepository.<init>(ActivityTypeDAO.scala:13)
at dao.ActivityTypeRepository.class(ActivityTypeDAO.scala:13)
while locating dao.ActivityTypeRepository
for the 1st parameter of services.ActivityTypeServiceImpl.<init>(ActivityTypeService.scala:17)
while locating services.ActivityTypeServiceImpl
while locating services.ActivityTypeService
The database is input correctly, I can connect to it via terminal and via datagrip... Has anyone ever had a similar issue?
As requested, here is my configuration:
slick.dbs.default.profile = "slick.jdbc.PostgresProfile$"
slick.dbs.default.db.driver = "org.postgresql.Driver"
slick.dbs.default.db.url = "jdbc:postgresql://localhost:5432/pmf"
These are my classes:
#Singleton
class ActivityTypeRepository #Inject()(dbapi: DBApi)(implicit ec: ExecutionContext) {
private val db = dbapi.database(RepositorySettings.dbName)
private[dao] val activityTypeMapping = {
get[Int]("activityType.id") ~
get[String]("activityType.name") map {
case id ~ name => ActivityType(id, name)
}
}
def listAll: Future[Seq[ActivityType]] = Future {
db.withConnection { implicit conn =>
SQL("SELECT * FROM ActivityType").as(activityTypeMapping *)
}
}
}
#Singleton
class UserRepository #Inject()(dbApi: DBApi)(implicit ec: ExecutionContext){
private val db = dbApi.database(RepositorySettings.dbName)
private[dao] val userMapping = {
get[Option[Long]]("users.id") ~
get[String]("users.email") ~
get[Option[String]]("users.name") ~
get[Option[String]]("users.surname") map {
case id ~ email ~ name ~ surname => User(id, email, name, surname)
}
}
def add(user: User): Future[Option[Long]] = Future {
db.withConnection { implicit conn =>
SQL"INSERT INTO users(id, email, name, surname) VALUES (${user.id}, ${user.email}, ${user.name}, ${user.surname})".executeInsert()
}
}
def find(id: Long): Future[Option[User]] = Future {
db.withConnection { implicit conn =>
SQL"SELECT * FROM User WHERE id = $id".as(userMapping *).headOption
}
}
def findByEmail(email: String): Future[Option[User]] = Future {
db.withConnection { implicit conn =>
SQL"SELECT * FROM User WHERE email = $email".as(userMapping *).headOption
}
}
def listAll: Future[Seq[User]] = Future {
db.withConnection { implicit conn =>
SQL("SELECT * FROM User").as(userMapping *)
}
}
}
EDIT:
Added to application.conf
db {
default.driver = org.postgresql.Driver
default.url = "jdbc:postgresql://localhost:5432/pmf_visualizations"
}
but no change.

connecting slick 3.1.1 to the database

I have the following code and I'm trying to connect to the MySQL database without success.
cat Database.scala
package com.github.odnanref.EmailFilter
import slick.driver.MySQLDriver._
import slick.driver.MySQLDriver.backend.Database
/**
* Created by andref on 12/05/16.
*/
class Database {
val url = "jdbc:mysql://localhost/playdb"
val db = Database.forURL(url, driver = "com.mysql.jdbc.Driver")
override def finalize() {
db.close()
super.finalize()
}
}
cat EmailMessageTable.scala
package com.github.odnanref.EmailFilter
import java.sql.Timestamp
import slick.driver.JdbcProfile
import slick.driver.MySQLDriver.api._
import scala.concurrent.Future
class EmailMessageTable(tag: Tag) extends Table[EmailMessage](tag, "email_message") {
def id = column[Option[Long]]("id", O.AutoInc, O.PrimaryKey)
def email = column[String]("email")
def subject = column[String]("subject")
def body = column[String]("body")
def datain = column[Timestamp]("datain")
def email_id= column[Long]("email_id")
def * = (id, email, subject, body, datain, email_id) <> ((EmailMessage.apply _).tupled, EmailMessage.unapply)
def ? = (id.get.?, email.?, subject.?, body.?, datain.?).shaped.<>({ r =>; _1.map(_ =>
EmailMessage.tupled((_1, _2.get, _3.get, _4.get, _5.get))) }, (_: Any) =>
throw new Exception("Inserting into ? projection not supported."))
}
I can't initialize the database and execute search query's or insert statements based on this code I try to do
val db = new Database()
db.db.run(TableQuery[EmailMessageTable] += EmailMessage(...) )
And it says, it doesn't know the method +=
Also I get this error:
Database.scala:4: imported `Database' is permanently hidden by definition of class Database in package EmailFilter
[warn] import slick.driver.MySQLDriver.backend.Database
What am I doing wrong?
Post EDIT>
package com.github.odnanref.EmailFilter
import java.sql.Timestamp
case class EmailMessage(
id: Option[Long],
email: String,
subject:String,
body:String,
datain: Timestamp,
email_id: Long
)
You are importing a class named Database inside a file that defines another class with the same name. You can:
rename your Database class:
class MyDatabase {
val url = ...
val db = ...
...
}
rename imported class:
import slick.driver.MySQLDriver.backend.{Database => SlickDB}
...
val db = SlickDB.forURL(url, driver = "com.mysql.jdbc.Driver")
avoid importing Database explicitly:
import slick.driver.MySQLDriver.backend
...
val db = backend.Database.forURL(url, driver = "com.mysql.jdbc.Driver")

NullPointerException on executing concurrent queries using Slick

I am working on a Scala application with Postgres 9.3 and Slick 3.1.1. I am getting Null Pointer Exception on slick driver when multiple queries execute at the same time.
Here is my simplified code. I am creating multiple actors which will call the same method to query from the database.
package com.app.repo
import java.sql.Timestamp
import akka.actor.{Actor, ActorSystem, Props}
import slick.driver.PostgresDriver
import slick.driver.PostgresDriver.api._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.FiniteDuration
import scala.util.{Failure, Success}
case class SampleData(id: Long, name: String, createDate: java.sql.Timestamp)
object Tables extends {
val profile = PostgresDriver
} with Tables
trait Tables {
val profile: PostgresDriver
import profile.api._
class SampleDataTable(_tableTag: Tag) extends Table[SampleData](_tableTag, Some("processing"), "SampleData") {
def * = (id, name, createDate) <>(SampleData.tupled, SampleData.unapply)
def ? = (Rep.Some(id), Rep.Some(name), Rep.Some(createDate)).shaped.<>({ r => import r._; _1.map(_ => SampleData.tupled((_1.get, _2.get, _3.get))) }, (_: Any) => throw new Exception("Inserting into ? projection not supported."))
val id: Rep[Long] = column[Long]("SampleId", O.AutoInc, O.PrimaryKey)
val name: Rep[String] = column[String]("Name")
val createDate: Rep[java.sql.Timestamp] = column[java.sql.Timestamp]("CreateDate")
}
lazy val sampleDataTable = new TableQuery(tag => new SampleDataTable(tag))
}
class SampleQueryingActor(delay: FiniteDuration, duration: FiniteDuration) extends Actor {
import scala.concurrent.duration._
override def preStart() = {
context.system.scheduler.schedule(0.second, duration, self, "tick")
}
override def receive: Receive = {
case "tick" => {
println("tick received.. ")
//val range = 1 until 1000
RepositoryImpl.reader.onComplete({
case Success(r) => println(s"got sum as ${r.getOrElse(0)}")
case Failure(ex) => ex.printStackTrace()
})
}
}
}
object DriverHelper {
val user = "postgres"
val url = "jdbc:postgresql://192.168.1.50:5432/MyDatabase"
val password = "password"
val jdbcDriver = "org.postgresql.Driver"
val db: PostgresDriver.backend.DatabaseDef = Database.forURL(url, user = user, password = password, driver = jdbcDriver)
}
object RepositoryImpl {
val db: PostgresDriver.backend.DatabaseDef = DriverHelper.db
val now = new Timestamp(System.currentTimeMillis())
def reader = {
db.run(Tables.sampleDataTable.filter(_.createDate > now).map(_.id).sum.result)
}
def insertBatchRecords(list: List[SampleData]) = {
db.run(Tables.sampleDataTable ++= list)
}
}
object PGConnectionTester extends App {
import scala.concurrent.duration._
val sys = ActorSystem("sys")
sys.actorOf(Props(classOf[SampleQueryingActor], 1.seconds, 10.seconds))
sys.actorOf(Props(classOf[SampleQueryingActor], 1.seconds, 10.seconds))
sys.actorOf(Props(classOf[SampleQueryingActor], 1.seconds, 10.seconds))
}
When I execute the above code, I get the error as below:
java.lang.NullPointerException
at slick.jdbc.DriverDataSource.getConnection(DriverDataSource.scala:98)
at slick.jdbc.DataSourceJdbcDataSource.createConnection(JdbcDataSource.scala:64)
at slick.jdbc.JdbcBackend$BaseSession.conn$lzycompute(JdbcBackend.scala:415)
at slick.jdbc.JdbcBackend$BaseSession.conn(JdbcBackend.scala:414)
at slick.jdbc.JdbcBackend$SessionDef$class.prepareStatement(JdbcBackend.scala:297)
at slick.jdbc.JdbcBackend$BaseSession.prepareStatement(JdbcBackend.scala:407)
at slick.jdbc.StatementInvoker.results(StatementInvoker.scala:33)
at slick.jdbc.StatementInvoker.iteratorTo(StatementInvoker.scala:22)
at slick.jdbc.Invoker$class.first(Invoker.scala:31)
at slick.jdbc.StatementInvoker.first(StatementInvoker.scala:16)
at slick.driver.JdbcActionComponent$QueryActionExtensionMethodsImpl$$anon$3.run(JdbcActionComponent.scala:228)
at slick.driver.JdbcActionComponent$SimpleJdbcDriverAction.run(JdbcActionComponent.scala:32)
at slick.driver.JdbcActionComponent$SimpleJdbcDriverAction.run(JdbcActionComponent.scala:29)
at slick.backend.DatabaseComponent$DatabaseDef$$anon$2.liftedTree1$1(DatabaseComponent.scala:237)
at slick.backend.DatabaseComponent$DatabaseDef$$anon$2.run(DatabaseComponent.scala:237)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
The actor will invoke the same method every 10 second. However, I am getting this error only for the first time. After that the queries are executing correctly. I am not able to understand why this is happening. In this sample case, there are only some simple read operations. But in my actual case, since the query is failing, some of the data is getting lost without processing correctly.
Is this error is something to do with connection pooling?
I think you have found this issue. Try to use lazy val for db so it only initializes once:
object DriverHelper {
val user = "postgres"
val url = "jdbc:postgresql://192.168.1.50:5432/MyDatabase"
val password = "password"
val jdbcDriver = "org.postgresql.Driver"
lazy val db: PostgresDriver.backend.DatabaseDef = Database.forURL(url, user = user, password = password, driver = jdbcDriver)
}
Just sharing the information for anyone else facing this issue.
There was a bug with Slick itself. It was reported here. Git user, mustajavi fixed this and was merged to latest Slick branch. With the latest update of 3.1.1, the issue is resolved for me.
Related Links in GitHub:
https://github.com/slick/slick/pull/1401
https://github.com/slick/slick/pull/1445

Slick code generation for only a single schema

Is there a way to have Slick's code generation generate code for only a single schema? Say, public? I have extensions that create a whole ton of tables (eg postgis, pg_jobman) that make the code that slick generates gigantic.
Use this code with appropriate values and schema name,
object CodeGenerator {
def outputDir :String =""
def pkg:String =""
def schemaList:String = "schema1, schema2"
def url:String = "dburl"
def fileName:String =""
val user = "dbUsername"
val password = "dbPassword"
val slickDriver="scala.slick.driver.PostgresDriver"
val JdbcDriver = "org.postgresql.Driver"
val container = "Tables"
def generate() = {
val driver: JdbcProfile = buildJdbcProfile
val schemas = createSchemaList
var model = createModel(driver,schemas)
val codegen = new SourceCodeGenerator(model){
// customize Scala table name (table class, table values, ...)
override def tableName = dbTableName => dbTableName match {
case _ => dbTableName+"Table"
}
override def code = {
//imports is copied right out of
//scala.slick.model.codegen.AbstractSourceCodeGenerator
val imports = {
"import scala.slick.model.ForeignKeyAction\n" +
(if (tables.exists(_.hlistEnabled)) {
"import scala.slick.collection.heterogenous._\n" +
"import scala.slick.collection.heterogenous.syntax._\n"
} else ""
) +
(if (tables.exists(_.PlainSqlMapper.enabled)) {
"import scala.slick.jdbc.{GetResult => GR}\n" +
"// NOTE: GetResult mappers for plain SQL are only generated for tables where Slick knows how to map the types of all columns.\n"
} else ""
) + "\n\n" //+ tables.map(t => s"implicit val ${t.model.name.table}Format = Json.format[${t.model.name.table}]").mkString("\n")+"\n\n"
}
val bySchema = tables.groupBy(t => {
t.model.name.schema
})
val schemaFor = (schema: Option[String]) => {
bySchema(schema).sortBy(_.model.name.table).map(
_.code.mkString("\n")
).mkString("\n\n")
}
}
val joins = tables.flatMap( _.foreignKeys.map{ foreignKey =>
import foreignKey._
val fkt = referencingTable.TableClass.name
val pkt = referencedTable.TableClass.name
val columns = referencingColumns.map(_.name) zip
referencedColumns.map(_.name)
s"implicit def autojoin${fkt + name.toString} = (left:${fkt} ,right:${pkt}) => " +
columns.map{
case (lcol,rcol) =>
"left."+lcol + " === " + "right."+rcol
}.mkString(" && ")
})
override def entityName = dbTableName => dbTableName match {
case _ => dbTableName
}
override def Table = new Table(_) {
table =>
// customize table value (TableQuery) name (uses tableName as a basis)
override def TableValue = new TableValue {
override def rawName = super.rawName.uncapitalize
}
// override generator responsible for columns
override def Column = new Column(_){
// customize Scala column names
override def rawName = (table.model.name.table,this.model.name) match {
case _ => super.rawName
}
}
}
}
println(outputDir+"\\"+fileName)
(new File(outputDir)).mkdirs()
val fw = new FileWriter(outputDir+File.separator+fileName)
fw.write(codegen.packageCode(slickDriver, pkg, container))
fw.close()
}
def createModel(driver: JdbcProfile, schemas:Set[Option[String]]): Model = {
driver.simple.Database
.forURL(url, user = user, password = password, driver = JdbcDriver)
.withSession { implicit session =>
val filteredTables = driver.defaultTables.filter(
(t: MTable) => schemas.contains(t.name.schema)
)
PostgresDriver.createModel(Some(filteredTables))
}
}
def createSchemaList: Set[Option[String]] = {
schemaList.split(",").map({
case "" => None
case (name: String) => Some(name)
}).toSet
}
def buildJdbcProfile: JdbcProfile = {
val module = currentMirror.staticModule(slickDriver)
val reflectedModule = currentMirror.reflectModule(module)
val driver = reflectedModule.instance.asInstanceOf[JdbcProfile]
driver
}
}
I encountered the same problem and I found this question. The answer by S.Karthik sent me in the right direction. However, the code in the answer is slightly outdated. And I think a bit over-complicated. So I crafted my own solution:
import slick.codegen.SourceCodeGenerator
import slick.driver.JdbcProfile
import slick.model.Model
import scala.concurrent.duration.Duration
import scala.concurrent.{Await, ExecutionContext}
val slickDriver = "slick.driver.PostgresDriver"
val jdbcDriver = "org.postgresql.Driver"
val url = "jdbc:postgresql://localhost:5432/mydb"
val outputFolder = "/path/to/src/test/scala"
val pkg = "com.mycompany"
val user = "user"
val password = "password"
object MySourceCodeGenerator {
def run(slickDriver: String, jdbcDriver: String, url: String, outputDir: String,
pkg: String, user: Option[String], password: Option[String]): Unit = {
val driver: JdbcProfile =
Class.forName(slickDriver + "$").getField("MODULE$").get(null).asInstanceOf[JdbcProfile]
val dbFactory = driver.api.Database
val db = dbFactory.forURL(url, driver = jdbcDriver, user = user.orNull,
password = password.orNull, keepAliveConnection = true)
try {
// **1**
val allSchemas = Await.result(db.run(
driver.createModel(None, ignoreInvalidDefaults = false)(ExecutionContext.global).withPinnedSession), Duration.Inf)
// **2**
val publicSchema = new Model(allSchemas.tables.filter(_.name.schema.isEmpty), allSchemas.options)
// **3**
new SourceCodeGenerator(publicSchema).writeToFile(slickDriver, outputDir, pkg)
} finally db.close
}
}
MySourceCodeGenerator.run(slickDriver, jdbcDriver, url, outputFolder, pkg, Some(user), Some(password))
I'll explain what's going on here:
I copied the run function from the SourceCodeGenerator class that's in the slick-codegen library. (I used version slick-codegen_2.10-3.1.1.)
// **1**: In the origninal code, the generated Model was referenced in a val called m. I renamed that to allSchemas.
// **2**: I created a new Model (publicSchema), using the options from the original model, and using a filtered version of the tables set from the original model. It turns out tables from the public schema don't get a schema name in the model. Hence the isEmpty. Should you need tables from one or more other schemas, you can easily create a different filter expression.
// **3**: I create a SourceCodeGenerator with the created publicSchema model.
Of course, it would even be better if the Slick codegenerator could incorporate an option to select one or more schemas.