Scala Slick, how to create Schema ONLY if it does not exist - scala

In Scala Slick, a database schema can be created with the following:
val schema = coffees.schema ++ suppliers.schema
db.run(DBIO.seq(
schema.create
))
From the bottom of this documentation page http://slick.typesafe.com/doc/3.0.0/schemas.html
However, if the database schema already exists then this throws an exception.
Is there a normal way or right way to create the schema IF AND ONLY IF it does not already exist?

In Slick 3.3.0 createIfNotExists and dropIfExists schema methods were added. So:
db.run(coffees.schema.createIfNotExists)
Googled this question and tried several solutions from answers until figured it out.

This is what I do for multiple tables, with slick 3.1.1 and Postgres
import slick.driver.PostgresDriver.api._
import slick.jdbc.meta.MTable
import scala.concurrent.Await
import scala.concurrent.duration.Duration
import scala.concurrent.ExecutionContext.Implicits.global
val t1 = TableQuery[Table1]
val t2 = TableQuery[Table2]
val t3 = TableQuery[Table3]
val tables = List(t1, t2, t3)
val existing = db.run(MTable.getTables)
val f = existing.flatMap( v => {
val names = v.map(mt => mt.name.name)
val createIfNotExist = tables.filter( table =>
(!names.contains(table.baseTableRow.tableName))).map(_.schema.create)
db.run(DBIO.sequence(createIfNotExist))
})
Await.result(f, Duration.Inf)

With Slick 3.0, Mtable.getTables is a DBAction so something like this would work:
val coffees = TableQuery[Coffees]
try {
Await.result(db.run(DBIO.seq(
MTable.getTables map (tables => {
if (!tables.exists(_.name.name == coffees.baseTableRow.tableName))
coffees.schema.create
})
)), Duration.Inf)
} finally db.close

As JoshSGoman comment points out about the answer of Mike-s, the table is not created. I managed to make it work by slightly modifying the first answer's code :
val coffees = TableQuery[Coffees]
try {
def createTableIfNotInTables(tables: Vector[MTable]): Future[Unit] = {
if (!tables.exists(_.name.name == events.baseTableRow.tableName)) {
db.run(coffees.schema.create)
} else {
Future()
}
}
val createTableIfNotExist: Future[Unit] = db.run(MTable.getTables).flatMap(createTableIfNotInTables)
Await.result(createTableIfNotExist, Duration.Inf)
} finally db.close
With the following imports :
import slick.jdbc.meta.MTable
import slick.driver.SQLiteDriver.api._
import scala.concurrent.{Await, Future}
import scala.concurrent.duration.Duration
import scala.concurrent.ExecutionContext.Implicits.global

why don't you simply check the existence before create?
val schema = coffees.schema ++ suppliers.schema
db.run(DBIO.seq(
if (!MTable.getTables.list.exists(_.name.name == MyTable.tableName)){
schema.create
}
))

cannot use createIfNotExists on schema composed of 3 tables with composite primary key on one of the tables. Here, the 3rd table has a primary key composed from the the primary key of each of the 1st and 2nd table. I get an error on this schema when .createIfNotExists is encountered a 2nd time. I am using slick 3.3.1 on scala 2.12.8.
class UserTable(tag: Tag) extends Table[User](tag, "user") {
def id = column[Long]("id", O.AutoInc, O.PrimaryKey)
def name = column[String]("name")
def email = column[Option[String]]("email")
def * = (id.?, name, email).mapTo[User]
}
val users = TableQuery[UserTable]
lazy val insertUser = users returning users.map(_.id)
case class Room(title: String, id: Long = 0L)
class RoomTable(tag: Tag) extends Table[Room](tag, "room") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def title = column[String]("title")
def * = (title, id).mapTo[Room]
}
val rooms = TableQuery[RoomTable]
lazy val insertRoom = rooms returning rooms.map(_.id)
case class Occupant(roomId: Long, userId: Long)
class OccupantTable(tag: Tag) extends Table[Occupant](tag, "occupant") {
def roomId = column[Long]("room")
def userId = column[Long]("user")
def pk = primaryKey("room_user_pk", (roomId, userId) )
def * = (roomId, userId).mapTo[Occupant]
}
val occupants = TableQuery[OccupantTable]
I can successfully create schema and add user, room and occupant at first. On the second usage of .createIfNotExists as follows below, I get an error on duplicate primary key:
println("\n2nd run on .createIfNotExists using different values for users, rooms and occupants")
val initdup = for {
_ <- users.schema.createIfNotExists
_ <- rooms.schema.createIfNotExists
_ <- occupants.schema.createIfNotExists
curlyId <- insertUser += User(None, "Curly", Some("curly#example.org"))
larryId <- insertUser += User(None, "Larry")
moeId <- insertUser += User(None, "Moe", Some("moe#example.org"))
shedId <- insertRoom += Room("Shed")
_ <- occupants += Occupant(shedId, curlyId)
_ <- occupants += Occupant(shedId, moeId)
} yield ()
The exception is as below:
2nd run on .createIfNotExists using different values for users, rooms and occupants
[error] (run-main-2) org.h2.jdbc.JdbcSQLException: Constraint "room_user_pk" already exists; SQL statement:
[error] alter table "occupant" add constraint "room_user_pk" primary key("room","user") [90045-197]
[error] org.h2.jdbc.JdbcSQLException: Constraint "room_user_pk" already exists; SQL statement:
[error] alter table "occupant" add constraint "room_user_pk" primary key("room","user") [90045-197]
[error] at org.h2.message.DbException.getJdbcSQLException(DbException.java:357)
[error] at org.h2.message.DbException.get(DbException.java:179)
[error] at org.h2.message.DbException.get(DbException.java:155)
[error] at org.h2.command.ddl.AlterTableAddConstraint.tryUpdate(AlterTableAddConstraint.java:110)
[error] at org.h2.command.ddl.AlterTableAddConstraint.update(AlterTableAddConstraint.java:78)
[error] at org.h2.command.CommandContainer.update(CommandContainer.java:102)
[error] at org.h2.command.Command.executeUpdate(Command.java:261)
[error] at org.h2.jdbc.JdbcPreparedStatement.execute(JdbcPreparedStatement.java:249)
[error] at slick.jdbc.JdbcActionComponent$SchemaActionExtensionMethodsImpl$$anon$6.$anonfun$run$7(JdbcActionComponent.scala:292)
[error] at slick.jdbc.JdbcActionComponent$SchemaActionExtensionMethodsImpl$$anon$6.$anonfun$run$7$adapted(JdbcActionComponent.scala:292)
[error] at slick.jdbc.JdbcBackend$SessionDef.withPreparedStatement(JdbcBackend.scala:425)
[error] at slick.jdbc.JdbcBackend$SessionDef.withPreparedStatement$(JdbcBackend.scala:420)
[error] at slick.jdbc.JdbcBackend$BaseSession.withPreparedStatement(JdbcBackend.scala:489)
[error] at slick.jdbc.JdbcActionComponent$SchemaActionExtensionMethodsImpl$$anon$6.$anonfun$run$6(JdbcActionComponent.scala:292)
[error] at slick.jdbc.JdbcActionComponent$SchemaActionExtensionMethodsImpl$$anon$6.$anonfun$run$6$adapted(JdbcActionComponent.scala:292)
[error] at scala.collection.Iterator.foreach(Iterator.scala:941)
[error] at scala.collection.Iterator.foreach$(Iterator.scala:941)
[error] at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
[error] at scala.collection.IterableLike.foreach(IterableLike.scala:74)
[error] at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
[error] at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
[error] at slick.jdbc.JdbcActionComponent$SchemaActionExtensionMethodsImpl$$anon$6.run(JdbcActionComponent.scala:292)
[error] at slick.jdbc.JdbcActionComponent$SchemaActionExtensionMethodsImpl$$anon$6.run(JdbcActionComponent.scala:290)
[error] at slick.jdbc.JdbcActionComponent$SimpleJdbcProfileAction.run(JdbcActionComponent.scala:28)
[error] at slick.jdbc.JdbcActionComponent$SimpleJdbcProfileAction.run(JdbcActionComponent.scala:25)
[error] at slick.basic.BasicBackend$DatabaseDef$$anon$3.liftedTree1$1(BasicBackend.scala:276)
[error] at slick.basic.BasicBackend$DatabaseDef$$anon$3.run(BasicBackend.scala:276)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] Nonzero exit code: 1
[error] (Compile / run) Nonzero exit code: 1
Additionally, I can use .createIfNotExists more than once on schema where all tables are created with O.PrimaryKey convention.
Am I able to do something to massage code? Is there a workaround so that .createIfNotExists is still usable on composite primary key case?

Related

Slick3.2 Error: No matching Shape found

I'm not sure what is wrong here.
The following code block is throwing error:
(for {
(e,r) <- tblDetail.joinLeft(tblMaster).on((e,r) => r.col1 === e.col3)
} yield (e.id)
Error
No matching Shape found.
[error] Slick does not know how to map the given types.
[error] Possible causes: T in Table[T] does not match your * projection,
[error] you use an unsupported type in a Query (e.g. scala List),
[error] or you forgot to import a driver api into scope.
[error] Required level: slick.lifted.FlatShapeLevel
[error] Source type: (slick.lifted.Rep[Int], slick.lifted.Rep[String],...)
[error] Unpacked type: T
[error] Packed type: G
[error] (e,r) <- tblDetail.joinLeft(tblMaster).on((e,r) => r.col1 === e.col3)
I checked the Slick Tables for tblDetail and tblMaster they seemed to be fine.
tblMaster
class TblMaster(tag:Tag)
extends Table[(Int,String,...)](tag, "tbl_master") {
def id = column[Int]("id")
def col3 = column[String]("col3")
def * = (id,col3)
}
tblDetail
class TblDetail(tag:Tag)
extends Table[Entity](tag, "tbl_detail") {
def id = column[Int]("id")
def col1 = column[String]("col1")
def * : ProvenShape[Entity] = (id,col1) <>
((Entity.apply _).tupled, Entity.unapply)
}
Any help would be appreciable.

Slick query by mapped colum

When you try to query by MappedTo[Long]:
for {
game <- GamblrGame.table
bet <- GamblrBet.table if game.id === bet.game
} yield (game, bet)
you get:
[error] /Volumes/Home/dev/gamblr/test/BotTest.scala:27: Cannot perform option-mapped operation
[error] with type: (slicky.Slicky.ID, slicky.fields.FK[models.GamblrGame]) => R
[error] for base type: (slicky.Slicky.ID, slicky.Slicky.ID) => Boolean
[error] bet <- GamblrBet.table if game.id === bet.game
How should I use mapped columns in queries?
The FK:
case class FK[E <: IdEntity[E]](id: ID)(implicit tag: TypeTag[E])
extends MappedTo[Long]
The game column in GamblrBet.table:
def game = column[FK[GamblrGame]]("GAME")

Race condition in Slick Code

I have written this slick DAO and its unit test in specs2.
My code has race conditions. When I run the same tests, I get different outputs.
The race conditions exist even though in both the functions I do Await.result(future, Duration.Inf)
DAO
package com.example
import slick.backend.DatabasePublisher
import slick.driver.H2Driver.api._
import scala.concurrent.ExecutionContext.Implicits.global
import slick.jdbc.meta._
import scala.concurrent._
import ExecutionContext.Implicits.global
import scala.concurrent.duration._
case class Person(id: Int, firstname: String, lastname: String)
class People(tag: Tag) extends Table[Person](tag, "PEOPLE") {
def id = column[Int]("PERSON_ID", O.PrimaryKey)
def firstname = column[String]("PERSON_FIRST_NAME")
def lastname = column[String]("PERSON_LAST_NAME")
def * = (id, firstname, lastname) <> (Person.tupled, Person.unapply _)
}
object PersonDAO {
private def createList(numRows: Int) : List[Person] = {
def recFunc(counter: Int, result: List[Person]) : List[Person] = {
counter match {
case x if x <= numRows => recFunc(counter + 1, Person(counter, "test" + counter, "user" + counter) :: result)
case _ => result
}
}
recFunc(1, List[Person]())
}
val db = Database.forConfig("test1")
val people = TableQuery[People]
def createAndPopulate(numRows: Int) = {
val action1 = people.schema.create
val action2 = people ++= Seq(createList(numRows) : _* )
val combined = db.run(action1 andThen action2)
val future1 = combined.map { result =>
result map {x =>
println(s"number of rows inserted $x")
x
}
}
Await.result(future1, Duration.Inf).getOrElse(0)
}
def printAll() = {
val a = people.result
val b = db.run(a)
val y = b map { result =>
result map {x => x}
}
val z = Await.result(y, Duration.Inf)
println(z)
println(z.length)
z
}
}
Unit Test
import org.specs2.mutable._
import com.example._
class HelloSpec extends Specification {
"This usecase " should {
"should insert rows " in {
val x = PersonDAO.createAndPopulate(100)
x === 100
}
}
"This usecase " should {
"return 100 rows" in {
val x = PersonDAO.printAll()
val y = PersonDAO.printAll()
y.length === 100
}
}
}
When I run this same code using activator test I see 2 different types of outputs on different runs
sometimes the code gets exception
number of rows inserted 100
[info] HelloSpec
[info]
[info] This usecase should
[info] + should insert rows
[info]
[info] This usecase should
[info] ! return 100 rows
[error] JdbcSQLException: : Table PEOPLE not found; SQL statement:
[error] select x2."PERSON_ID", x2."PERSON_FIRST_NAME", x2."PERSON_LAST_NAME" from "PEOPLE" x2 [42S02-60] (Message.java:84)
[error] org.h2.message.Message.getSQLException(Message.java:84)
[error] org.h2.message.Message.getSQLException(Message.java:88)
[error] org.h2.message.Message.getSQLException(Message.java:66)
Sometimes the 1st function call returns 0 rows and the 2nd function call returns 100 values
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
number of rows inserted 100
Vector()
0
Vector(Person(100,test100,user100), Person(99,test99,user99), Person(98,test98,user98), Person(97,test97,user97), Person(96,test96,user96), Person(95,test95,user95), Person(94,test94,user94), Person(93,test93,user93), Person(92,test92,user92), Person(91,test91,user91), Person(90,test90,user90), Person(89,test89,user89), Person(88,test88,user88), Person(87,test87,user87), Person
I don't understand why does my code have these race conditions because I block on future in each method.
Your assumption that two test cases should run serial, one after the other is not right. The test cases are running parallel. Just use sequential to verify that thats the case.

Combine Two Slick Futures and then execute them together

I have written this code and I am trying to combine two futures obtained from separate SQL operations.
package com.example
import tables._
import scala.concurrent.{Future, Await}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.Duration
import slick.backend.DatabasePublisher
import slick.driver.H2Driver.api._
object Hello {
def main(args: Array[String]): Unit = {
val db = Database.forConfig("h2mem1")
try {
val people = TableQuery[Persons]
val setupAction : DBIO[Unit] = DBIO.seq(
people.schema.create
)
val setupFuture : Future[Unit] = db.run(setupAction)
val populateAction: DBIO[Option[Int]] = people ++= Seq(
(1, "test1", "user1"),
(2, "test2", "user2"),
(3, "test3", "user3"),
(4, "test4", "user4")
)
val populateFuture : Future[Option[Int]] = db.run(populateAction)
val combinedFuture : Future[Option[Int]] = setupFuture >> populateFuture
val r = combinedFuture.flatMap { results =>
results.foreach(x => println(s"Number of rows inserted $x"))
}
Await.result(r, Duration.Inf)
}
finally db.close
}
}
But I get an error when I try to compile this code
[error] /Users/abhi/ScalaProjects/SlickTest2/src/main/scala/Hello.scala:29:
value >> is not a member of scala.concurrent.Future[Unit]
[error] val combinedFuture : Future[Option[Int]] = setupFuture >>
populateFuture
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
The same code works, If I nest the populateFuture inside the map function of the setupFuture. But I don't want to write nested code because it will become very messy once there are more steps to do.
So I need a way to combine all futures into a single future and then execute it.
Edit:: I also tried combining the two actions
val combinedAction = setupAction.andThen(populateAction)
val fut1 = combinedAction.map{result =>
result.foreach{x =>println(s"number or rows inserted $x")}
}
Await.result(fut1, Duration.Inf)
but got error
/Users/abhi/ScalaProjects/SlickTest/src/main/scala/com/example/Hello.scala:31: type mismatch;
[error] found : scala.concurrent.Future[Option[Int]]
[error] required: PartialFunction[scala.util.Try[Unit],?]
[error] val combinedAction = setupAction.andThen(populateAction)
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 3 s, completed Jun 26, 2015 3:50:51 PM
Mohitas-MBP:SlickTest abhi$
According to http://slick.typesafe.com/doc/3.0.0/api/index.html#slick.dbio.DBIOAction, andThen() is what you are looking for:
val combinedAction = setupAction.andThen(populateAction)
val results = db.run(combinedAction)
populateAction will only run after setupAction completed successfully. This is crucial in your case since slick is fully non-blocking. The code you have now will cause problems at runtime. Both actions in your code will run asynchronously at the same time. There is no way to determine which action is executed first. But because populateAction depends on setupAction, you must ensure setupAction is executed first. Therefore use andThen.

Scala specs2 mocking a trait method returns always Nullpointer exception

I have a trait that I want to mock and use that mocked Trait in another Service during testing. The problem is, that I receive a Nullpointerexception when I try to mock the return value of the indexDocuments function.
Testmethod:
"createDemand must return None if writing to es fails" in new WithApplication {
val demandDraft = DemandDraft(UserId("1"), "socken bekleidung wolle", Location(Longitude(52.468562), Latitude(13.534212)), Distance(30), Price(25.0), Price(77.0))
val es = mock[ElasticsearchClient]
val sphere = mock[SphereClient]
val productTypes = mock[ProductTypes]
sphere.execute(any[ProductCreateCommand]) returns Future.successful(product)
productTypes.demand returns ProductTypeBuilder.of("demand", ProductTypeDrafts.demand).build()
// this line throws the nullpointer exception
es.indexDocument(any[IndexName], any[TypeName], any[JsValue]) returns Future.failed(new RuntimeException("test exception"))
val demandService = new DemandService(es, sphere, productTypes)
demandService.createDemand(demandDraft) must be (Option.empty[Demand]).await
}
Trait:
sealed trait ElasticsearchClient {
implicit def convertListenableActionFutureToScalaFuture[T](x: ListenableActionFuture[T]): Future[T] = {
val p = Promise[T]()
x.addListener(new ActionListener[T] {
def onFailure(e: Throwable) = p.failure(e)
def onResponse(response: T) = p.success(response)
})
p.future
}
lazy val client = createElasticsearchClient()
def close(): Unit
def createElasticsearchClient(): Client
def indexDocument(esIndex: IndexName, esType: TypeName, doc: JsValue): Future[IndexResponse] =
client.prepareIndex(esIndex.value, esType.value).setSource(doc.toString()).execute()
def search(esIndex: IndexName, esType: TypeName, query: QueryBuilder): Future[SearchResponse] =
client.prepareSearch(esIndex.value).setTypes(esType.value).setQuery(query).execute()
}
Exception
[error] NullPointerException: (DemandServiceSpec.scala:89)
[error] services.DemandServiceSpec$$anonfun$1$$anonfun$apply$8$$anon$2$$anonfun$8.apply(DemandServiceSpec.scala:89)
[error] services.DemandServiceSpec$$anonfun$1$$anonfun$apply$8$$anon$2$$anonfun$8.apply(DemandServiceSpec.scala:89)
[error] services.DemandServiceSpec$$anonfun$1$$anonfun$apply$8$$anon$2.delayedEndpoint$services$DemandServiceSpec$$anonfun$1$$anonfun$apply$8$$anon$2$1(DemandServiceSpec.scala:89)
[error] services.DemandServiceSpec$$anonfun$1$$anonfun$apply$8$$anon$2$delayedInit$body.apply(DemandServiceSpec.scala:81)
[error] play.api.test.WithApplication$$anonfun$around$1.apply(Specs.scala:23)
[error] play.api.test.WithApplication$$anonfun$around$1.apply(Specs.scala:23)
[error] play.api.test.PlayRunners$class.running(Helpers.scala:49)
[error] play.api.test.Helpers$.running(Helpers.scala:403)
[error] play.api.test.WithApplication.around(Specs.scala:23)
[error] play.api.test.WithApplication.delayedInit(Specs.scala:20)
[error] services.DemandServiceSpec$$anonfun$1$$anonfun$apply$8$$anon$2.<init>(DemandServiceSpec.scala:81)
[error] services.DemandServiceSpec$$anonfun$1$$anonfun$apply$8.apply(DemandServiceSpec.scala:81)
[error] services.DemandServiceSpec$$anonfun$1$$anonfun$apply$8.apply(DemandServiceSpec.scala:81)
Please let me know if you need additional information.
I found out that the any[] Matchers in the indexDocuments call are the problem. When I replace them with the actual values it works:
"createDemand must return None if writing to es fails and deleteDemand should be called once with correct parameters" in new WithApplication {
val demandDraft = DemandDraft(UserId("1"), "socken bekleidung wolle", Location(Longitude(52.468562), Latitude(13.534212)), Distance(30), Price(25.0), Price(77.0))
val es = mock[ElasticsearchClient]
val sphere = mock[SphereClient]
val productTypes = mock[ProductTypes]
sphere.execute(any[ProductCreateCommand]) returns Future.successful(product)
sphere.execute(any[ProductDeleteByIdCommand]) returns Future.successful(product)
productTypes.demand returns ProductTypeBuilder.of("demand", ProductTypeDrafts.demand).build()
es.indexDocument(IndexName("demands"), TypeName("demands"), Json.toJson(demand)) returns Future.failed(new RuntimeException("test exception"))
val demandService = new DemandService(es, sphere, productTypes)
demandService.createDemand(demandDraft) must be (Option.empty[Demand]).await
}
I've had this happen a whole bunch and work around it by creating a class (rather than a trait) to feed to mock:
trait SomeTraitYouWantToMock {
…
}
class MockableSomeTraitYouWantToMock extends SomeTraitYouWantToMock
val whatever = mock[MockableSomeTraitYouWantToMock]