The simple version
What's the preferred way to import and use generated Slick tables?
The detailed version and what I've tried
I used Slick 3.1.1 codegen to generate a Tables.scala from a MySQL (MariaDB) schema.
Tables.scala begins with this:
// AUTO-GENERATED Slick data model
/** Stand-alone Slick data model for immediate use */
object Tables extends {
val profile = slick.driver.MySQLDriver
} with Tables
What's the best way to use these classes? As per the Slick documentation:
The file contains an object Tables from which the code can be imported for use right away. ... The file also contains a trait Tables which can be used in the cake pattern.
... I've tried variations on this example
import Tables._
import Tables.profile.api._
import slick.jdbc.JdbcBackend
class Test(s: String)(implicit db: Database) {
def exec[T](action: DBIO[T])(implicit db: Database): T =
Await.result(db run action)
def run: Unit = exec(((ATable filter (_.id)).result)
}
object Test {
implicit val db = Database.forURL(url, user, password)
new Test("")
}
I get a compile error wherever I reference the class ATable:
could not find implicit value for parameter tables: Tables
I don't even see tables in Tables.scala. How do I get everything I need in scope to use my generated Slick classes?
I got the example to work: Tables._ and Tables.profile.api._ just need to be imported inside the class with an implicit Database available.
import slick.jdbc.JdbcBackend
class Test(s: String)(implicit db: Database) {
import Tables._
import Tables.profile.api._
def exec[T](action: DBIO[T])(implicit db: Database): T =
Await.result(db run action)
def run: Unit = exec(((ATable filter (_.id)).result)
}
object Test {
implicit val db = Database.forURL(url, user, password)
new Test("")
}
Related
I'm using slick with Postgresql 9.6.1, Plya! 2.5 and play-slick 2.0.2.
(I also use slick-pg 0.14.3 but I don't think that it changes anything here.)
I'm using insertOrUpdate in a very straight forward way but I still get a unique exception.
I have a very simple test using insertOrUpdate:
if I run it several times I always get an sql exception:
ERROR: duplicate key value violates unique constraint "ga_client_id_pkey"
Detail: Key (client_id)=(1885746393.1464005051) already exists
However, my table is defined with client_id as primary key:
def clientId = column[String]("client_id", O.PrimaryKey)
and defined in sql as followed:
client_id TEXT NOT NULL UNIQUE PRIMARY KEY
The function tested simply does:
db.run(gaClientIds.insertOrUpdate(gaClientId))
and the controller simply calls this methods and does nothing else.
A strange thing is that launching the methods itself several times don't lead to the error but the controller does although it only calls the method.
Is insertOrUpdate slick function not sure yet or am I missing something?
insertOrUpdate is only supported in MySQL driver
http://slick.lightbend.com/doc/3.2.1/supported-databases.html
You can try this library which gives you the implementation of insertOrUpdate/upsert
https://github.com/tminglei/slick-pg
This is how we use it on current project.
import com.github.tminglei.slickpg._
import com.typesafe.config.ConfigFactory
import slick.basic.Capability
object DBComponent {
private val config = ConfigFactory.load()
val driver = config.getString("rdbms.properties.driver") match {
case "org.h2.Driver" => slick.jdbc.H2Profile
case _ => MyPostgresProfile
}
import driver.api._
val db: Database = Database.forConfig("rdbms.properties")
}
trait MyPostgresProfile extends ExPostgresProfile {
// Add back `capabilities.insertOrUpdate` to enable native `upsert` support; for postgres 9.5+
override protected def computeCapabilities: Set[Capability] =
super.computeCapabilities + slick.jdbc.JdbcCapabilities.insertOrUpdate
override val api: MyAPI.type = MyAPI
object MyAPI extends API
}
object MyPostgresProfile extends MyPostgresProfile
I'm continuing my exploration of the Play framework and its related components. I used the template for CRUD application with a connection to a PostgreSQL database to begin with. It splits the application in models, repositories, controllers and views. This works well.
Now, I'm trying to create some tests for this application with Specs2. More precisely, I'm trying to test the repository. It is defined as follow:
package dal
import javax.inject.{ Inject, Singleton }
import play.api.db.slick.DatabaseConfigProvider
import slick.driver.JdbcProfile
import models.Cat
import scala.concurrent.{ Future, ExecutionContext }
#Singleton
class CatRepository #Inject() (dbConfigProvider: DatabaseConfigProvider)(implicit ec: ExecutionContext) {
...
}
I would like to set an in memory database which would be created (schema, evolutions) before all tests, destroyed after all tests, populated (data, probably with direct SQL) and flushed around each test. I would like to pass it on to my repository instance which I would then use to perform my test. Like:
val repo = new CatRepository(inMem_DB)
So how do I go about:
1) creating this db and applying the evolutions?
maybe:
trait TestDB extends BeforeAfterAll {
var database: Option[Database] = None
def before = {
database = Some(Databases.inMemory(name = "test_db"))
database match {
case Some(con) => Evolutions.applyEvolutions(con)
case _ => None
}
println("DB READY")
}
def after = {
database match {
case Some(con) => con.shutdown()
case _ => None
}
}
}
Using a var and always making a "match/case" when I need to use the db isn't convenient. I guess there is a much better to do this...
2) Populate and flush around each test?
Shall I create a trait extending Around, just as with BeforeAfterAll?
3) Create on of these play.api.db.slick.DatabaseConfigProvider from the database?
Any link showing how to do this?
I found few examples which where covering this with running a FakeApplication, but I assume there is a way to pass somehow the db to such repository object outside of a running application..?
Thank you for helping.
Cheers!
You can use a lazy val and AfterAll for the database setup/teardown, then BeforeAfterEach for each example:
trait TestDB extends AfterAll with BeforeAfterEach {
lazy val database: Database =
Databases.inMemory(name = "test_db")
def afterAll =
database.shutdown
def before =
database.populate
def after =
datbase.clean
}
I'm writing some specc2 integration tests for my spray.io project that uses dynamodb. I'm using sbt-dynamodb to load a local dynamodb into the environment. I use the following pattern to load my tables before the tests are run.
trait DynamoDBSpec extends SpecificationLike {
val config = ConfigFactory.load()
val client = new AmazonDynamoDBClient(new DefaultAWSCredentialsProviderChain())
lazy val db = {
client.setEndpoint(config.getString("campaigns.db.endpoint"))
new DynamoDB(client)
}
override def map(fs: =>Fragments): Fragments =
Step(beforeAll) ^ fs ^ Step(afterAll)
protected def beforeAll() = {
//load my tables
}
protected def afterAll() = {
//delete my tables
}
}
Then any test class can just be extended with DynamoDBSpec and the tables will be created. It all works fine, until extend DynamoDBSpec from more than one test class, during which it throws an ResourceInUseException: 'Cannot create preexisting table'. The reason is that they execute in parallel, thus it wants to execute table creation at the same time.
I tried to overcome it by running the tests in sequential mode, but beforeall and afterall are still executed in parallel.
Ideally I think it would be good to create the tables before the entire suite runs instead of each Spec class invocation, and then tear them down after the entire suite completes. Does anyone know how to accomplish that?
There are 2 ways to achieve this.
With an object
You can use an object to synchronize the creation of your database
object Database {
lazy val config = ConfigFactory.load()
lazy val client =
new AmazonDynamoDBClient(new DefaultAWSCredentialsProviderChain())
// this will only be done once in
// the same jvm
lazy val db = {
client.setEndpoint(config.getString("campaigns.db.endpoint"))
val database = new DynamoDB(client)
// drop previous tables if any
// and create new tables
database.create...
database
}
}
// BeforeAll is a new trait in specs2 3.x
trait DynamoDBSpec extends SpecificationLike with BeforeAll {
//load my tables
def beforeAll = Database.db
}
As you can see, in this model we don't remove tables when the specification is finished (because we don't know if all other specifications have been executed), we just remove then when we re-run the specifications. This can actually be a good thing because this will help you investigate failures if any.
The other way to synchronize specifications at a global level, and to properly clean-up at the end, is to use specification links.
With links
With specs2 3.3 you can create dependencies between specification with links. This means that you can define a "Suite" specification which is going to:
start the database
collect all the relevant specifications
delete the database
For example
import org.specs2._
import specification._
import core.Fragments
import runner.SpecificationsFinder
// run this specification with `all` to execute
// all linked specifications
class Database extends Specification { def is =
"All database specifications".title ^ br ^
link(new Create).hide ^
Fragments.foreach(specs)(s => link(s) ^ br) ^
link(new Delete).hide
def specs = specifications(pattern = ".*Db.*")
}
// start the database with this specification
class Create extends Specification { def is = xonly ^
step("create database".pp)
}
// stop the database with this specification
class Delete extends Specification { def is = xonly ^
step("delete database".pp)
}
// an example of a specification using the database
// it will be invoked by the "Database" spec because
// its name matches ".*Db.*"
class Db1Spec extends Specification { def is = s2"""
test $db
"""
def db = { println("use the database - 1"); ok }
}
class Db2Spec extends Specification { def is = s2"""
test $db
"""
def db = { println("use the database - 2"); ok }
}
When you run:
sbt> test-only *Database* -- all
You should see a trace like
create database
use the database - 1
use the database - 2
delete database
I'm running into problems when trying to populate a DB with tables (using Slick 2.0.0):
import scala.slick.driver.JdbcDriver
import scala.slick.lifted.TableQuery
case class DemoInit(slickDriver:JdbcDriver, tables:Seq[TableQuery[_]]) {
lazy val db = slickDriver.simple.Database.forURL("jdbc_url", "user", "pass", driver = "com.example.Driver")
import slickDriver.simple.{tableQueryToTableQueryExtensionMethods, ddlToDDLInvoker}
def init() {
db withSession { implicit session =>
tables.map(_.ddl).reduce(_ ++ _).create
}
}
}
Attempting to compile the code above leads to the following two errors:
value ddl is not a member of scala.slick.lifted.TableQuery[_$1]
value create is not a member of scala.slick.lifted.TableQuery[_$1]
I'm guessing that the type parameters aren't being inferred correctly. What am I doing wrong?
You need to add the right constraint for the table type:
tables: Seq[TableQuery[_ <: Table[_]]]
In such cases where an implicit conversion that you expect is not taken, it helps to add an explicit call to this conversion. This will give you a better error message that explains why the conversion is not applicable.
Then you'll run into the next issue that you're using reduce wrong. You'll want to use something like tables.map(_.ddl).reduce(_ ++ _) instead.
Since Table is path-dependent you cannot use it with your setup that tries to pass everything to a class as parameters. While you are allowed to refer to path-dependent types from a previous argument list in a later argument list of a def, this is not possible in a class. You'll have to structure your code differently to get the path-dependent types right, e.g.:
import scala.slick.driver.JdbcProfile
class DemoDAO(val slickDriver: JdbcProfile) {
import slickDriver.simple._
lazy val db = slickDriver.simple.Database.forURL("jdbc_url", "user", "pass", driver = "com.example.Driver")
def init(tables: Seq[TableQuery[_ <: Table[_]]]) {
db withSession { implicit session =>
tables.map(_.ddl).reduce(_ ++ _).create
}
}
}
}
What I ended up doing was creating a new trait:
import scala.slick.driver.JdbcProfile
trait TablesSupplier {
def tables(profile:JdbcProfile):Seq[JdbcProfile#SimpleQL#TableQuery[_ <: JdbcProfile#SimpleQL#Table[_]]]
}
I'm using Slick to generate the source code for table objects, so I have a Tables trait, which I use in the implementation of TablesSupplier:
import scala.slick.driver.JdbcProfile
object DemoTablesSupplier extends TablesSupplier {
def tables(profile:JdbcProfile) = {
val _profile = profile
val demoTables = new Tables { val profile = _profile }
import demoTables._
Seq(Table1, Table2, Table3)
}
}
So, my DemoInit now looks like this:
import scala.slick.driver.JdbcDriver
case class DemoInit(slickDriver:JdbcDriver, tables:Seq[TableQuery[_]]) {
lazy val db = slickDriver.simple.Database.forURL("jdbc_url", "user", "pass", driver = "com.example.Driver")
import slickDriver.simple.{tableQueryToTableQueryExtensionMethods, ddlToDDLInvoker}
def init() {
db withSession { implicit session =>
val profile = slickDriver.profile
//import the type of Table and TableQuery that are expected as well as two implicit methods that are necessary in order to use 'ddl' and 'create' methods
import profile.simple.{Table, TableQuery, tableQueryToTableQueryExtensionMethods, ddlToDDLInvoker}
//using asInstanceOf is very ugly but I couldn't figure out a better way
val tables = tablesCreator.tables(profile).asInstanceOf[Seq[TableQuery[_ <: Table[_]]]]
tables.map(_.ddl).reduce(_ ++ _).create
}
}
}
Emm...I'm trying out Slick with Play 2. The table creation process has become very frustrating because unlike other ORM (like ebean), Slick doesn't detect if the database is created, if there is already a table existing, it will report an exception. I simply just don't want to drop and create every time I restart the server, so I decide to write a small function to help me:
def databaseCreate(tables: Table*) = {
for (table <- tables) {
if (MTable.getTables(table.getClass.getName).list.isEmpty) table.ddl.create
}
}
What this does is to take in some objects like this one:
object Tag extends Table [(Option[Int], String)]("Tags") {
def id = column[Int]("TAG_ID", O.PrimaryKey, O.AutoInc)
def tag_name = column[String]("TAG_NAME")
def * = id.? ~ tag_name
}
And use MTable method from scala.slick.jdbc.meta.MTable to know if the table exists or not. Then I kinda run into a simple Java reflection problem. If databaseCreate method takes Strings, then I can invoke .ddl.create. So I decide to pass in objects and use relfection: table.getClass.getName. The only problem is there is a type mismatch: (from my IDE)
Expected: MySQLDriver.simple.type#Table, actual: BlogData.Tag.type
BlogData is the big object I used to store all smaller table objects. How do I solve this mismatch problem?? Use a whole bunch of asInstanceOf? It would make the command unbearably long and ugly...
Corrected:
The type mismatch is a false alarm, which comes from the IDE, not from the typesafe console complier. The real problem is:
type Table takes type parameters
def databaseCreate(tables: Table*) = {
^
one error found
Then I followed the advice and changed the code:
def databaseCreate(tables: Table[_]*)(implicit session: Session) = {
for (table <- tables) {
if (MTable.getTables(table.tableName).list.isEmpty) table.ddl.create
}
}
Then I got this error:
ambiguous implicit values: both value session of type slick.driver.MySQLDriver.simple.Session and method threadLocalSession in object Database of type => scala.slick.session.Session match expected type scala.slick.session.Session
if (MTable.getTables(table.tableName).list.isEmpty) table.ddl.create
^
one error found
My imports are here:
import play.api.GlobalSettings
import play.api.Application
import models.BlogData._
import scala.slick.driver.MySQLDriver.simple._
import Database.threadLocalSession
import play.api.Play.current
import scala.slick.jdbc.meta.MTable
In Slick 2: For the sake of completeness, see this and that related thread.
I use the following method:
def createIfNotExists(tables: TableQuery[_ <: Table[_]]*)(implicit session: Session) {
tables foreach {table => if(MTable.getTables(table.baseTableRow.tableName).list.isEmpty) table.ddl.create}
}
Then you can just create your tables with the implicit session:
db withSession {
implicit session =>
createIfNotExists(table1, table2, ..., tablen)
}
You are using SLICK v1 I assume, as you have Table objects. I have working code for v1. This version will drop any tables that are present before recreating them (but wont drop any tables you don't wish to recreate). It will also merge the DDL into one before creation in order to get the creation sequence correct:
def create() = database withSession {
import scala.slick.jdbc.{StaticQuery => Q}
val tables = Seq(TableA, TableB, TableC)
def existingTables = MTable.getTables.list().map(_.name.name)
tables.filter(t => existingTables.contains(t.tableName)).foreach{t =>
Q.updateNA(s"drop table ${t.tableName} cascade").execute
}
val tablesToCreate = tables.filterNot(t => existingTables.contains(t.tableName))
val ddl: Option[DDL] = tablesToCreate.foldLeft(None: Option[DDL]){(ddl, table) =>
ddl match {
case Some(d) => Some(d ++ table.ddl)
case _ => Some(table.ddl)
}
}
ddl.foreach{_.create}
}
Not sure why you get this error message. I would need to see your imports and the place where you call databaseCreate, but what is wrong is def databaseCreate(tables: Table*) should be def databaseCreate(tables: Table[_]*) and probably take a second argument list as well def databaseCreate(tables: Table[_]*)(implicit session: Session) = ....
Also, instead of table.getClass.getName, you can use table.tableName.
You should use asTry with the table create DBIOAction . It will check whether there is table existing... if not exist the DBIOAction will work for creating a table.
in HomePageDAO
val createHomePageTableAction: DBIOAction[Int, NoStream, Effect.Schema with Effect.Write] = homePageTable.schema.create >>(homePageTable+=Homepage("No Data","No Data",0l))
in ExplorePageDAO
val SoftwareTableCreateAction: FixedSqlAction[Unit, NoStream, Effect.Schema] = softwareTable.schema.create
And in createTablesController
package controllers
import javax.inject.{Inject, Singleton}
import play.api.mvc.{AbstractController, Action, AnyContent, ControllerComponents}
import services.dbServices.{ExplorePageDAO, HomePageDAO}
import scala.concurrent.ExecutionContext
#Singleton
class CreateTablesController #Inject()(cc: ControllerComponents)(implicit assetsFinder: AssetsFinder, executionContext: ExecutionContext)
extends AbstractController(cc) {
import services.dbServices.dbSandBox._
def createTable: Action[AnyContent] = Action.async {
dbAccess.run(HomePageDAO.createHomePageTableAction .asTry >> ExplorePageDAO.SoftwareTableCreateAction.asTry).map(_=> Ok("Table Created"))
}
}