Slick 2.0 Generic CRUD operations - postgresql

I've been looking around on how to implement a generic trait for commons CRUD and other kinds of operations, I looked at this and this and the method specified are working well.
What I would like to have is a generic method for insertion, my class looks like this at the moment (non generic implementation):
object CampaignModel {
val campaigns = TableQuery[Campaign]
def insert(campaign: CampaignRow)(implicit s: Session) = {
campaigns.insert(campaign)
}
}
What I tried so far, following the first link, was this (generic implementation):
trait PostgresGeneric[T <: Table[A], A] {
val tableReference = TableQuery[T]
def insertGeneric(row: ? What type goes here ?)(implicit s: Session) = tableReference.insert(row)
}
When I inspect the insert method it looks like the right type should be T#TableElementType but my knowledge is pretty basic and I can't wrap my head around types, I tried T and A and the compiler says that the classtype does not conform to the trait one's.
Other infos, the tables are generated with the slick table generation tools
case class CampaignRow(id: Long, name: Option[String])
/** Table description of table campaign. Objects of this class serve as prototypes for rows in queries. */
class Campaign(tag: Tag) extends Table[CampaignRow](tag, "campaign") {
def * = (id, name) <>(CampaignRow.tupled, CampaignRow.unapply)
/** Maps whole row to an option. Useful for outer joins. */
def ? = (id.?, name).shaped.<>({
r => import r._; _1.map(_ => CampaignRow.tupled((_1.get, _2)))
}, (_: Any) => throw new Exception("Inserting into ? projection not supported."))
/** Database column id AutoInc, PrimaryKey */
val id: Column[Long] = column[Long]("id", O.AutoInc, O.PrimaryKey)
/** Database column name */
val name: Column[Option[String]] = column[Option[String]]("name")
}

I managed to make it work, this is my generic trait:
import scala.slick.driver.PostgresDriver
import scala.slick.driver.PostgresDriver.simple._
import path.to.RichTable
trait PostgresGeneric[T <: RichTable[A], A] {
val tableReference: TableQuery[T]
def insert(row: T#TableElementType)(implicit s: Session) =
tableReference.insert(row)
def insertAndGetId(row: T#TableElementType)(implicit s: Session) =
(tableReference returning tableReference.map(_.id)) += row
def deleteById(id: Long)(implicit s: Session): Boolean =
tableReference.filter(_.id === id).delete == 1
def updateById(id: Long, row: T#TableElementType)(implicit s: Session): Boolean =
tableReference.filter(_.id === id).update(row) == 1
def selectById(id: Long)(implicit s: Session): Option[T#TableElementType] =
tableReference.filter(_.id === id).firstOption
def existsById(id: Long)(implicit s: Session): Boolean = {
(for {
row <- tableReference
if row.id === id
} yield row).firstOption.isDefined
}
}
Where RichTable is an abstract class with an id field, this, with the upper bound constraint is useful to get the id field of T#TableElementType (see this for more info):
import scala.slick.driver.PostgresDriver.simple._
import scala.slick.jdbc.{GetResult => GR}
abstract class RichTable[T](tag: Tag, name: String) extends Table[T](tag, name) {
val id: Column[Long] = column[Long]("id", O.PrimaryKey, O.AutoInc)
}
And my campaign table now looks like this:
import scala.slick.driver.PostgresDriver.simple._
import scala.slick.jdbc.{GetResult => GR}
import scala.slick.lifted.TableQuery
case class CampaignRow(id: Long, name: Option[String])
class Campaign(tag: Tag) extends RichTable[CampaignRow](tag, "campaign") {
def * = (id, name) <>(CampaignRow.tupled, CampaignRow.unapply)
def ? = (id.?, name).shaped.<>({
r => import r._; _1.map(_ => CampaignRow.tupled((_1.get, _2)))
}, (_: Any) => throw new Exception("Inserting into ? projection not supported."))
override val id: Column[Long] = column[Long]("id", O.AutoInc, O.PrimaryKey)
val name: Column[Option[String]] = column[Option[String]]("name")
}
The model implementing the generic trait looks like this:
object CampaignModel extends PostgresGeneric[Campaign, CampaignRow] {
override val tableReference: PostgresDriver.simple.TableQuery[Tables.Campaign] =
TableQuery[Campaign]
def insertCampaign(row: CampaignRow) = {
insert(CampaignRow(0, "test"))
}
}

Related

Slick using mapped column type in update statement

I have a trouble in using slick MappedColumnType, the code snippet is as following:
private trait UserTable {
self: HasDatabaseConfigProvider[JdbcProfile] =>
import driver.api._
lazy val userTable = TableQuery[User]
class UserTable(tag: Tag)
extends Table[User](tag, "user") {
implicit def mapper = MappedColumnType.base[JsObject, String](
{ scope: JsObject => scope.toString }, { s: String => Json.parse(s).as[JsObject] }
)
val id = column[Long]("id", O.PrimaryKey, O.AutoInc)
val name = column[String]("name")
val hobby = column[JsObject]("hobby")
def * = (id, name, hobby) <> (User.tupled, User.unapply)
}
}
My User case class is defined as follow:
case class User(id: Long, name: String: hobby: JsObject)
I have corresponding insert and update statement for my database. However the following update statement is not working for me.
def updateQuery(id: Long, newUser: User) = {
userTable.filter(x => x.id === id)
.map(x => (x.hobby))
.update(newUser.hobby)
It will throw a compile error:
No matching Shape found.
Slick does not know how to map the given types.
Possible causes: T in Table[T] does not match your * projection. Or you use an unsupported type in a Query (e.g. scala List).
I think it's pretty straight-forward. Is there something I did wrong?

slick 3.2 - how to define an "abstract" trait with tables and queries definitions but no concrete profile?

I want to define the queries and tables without the direct import of the final database profile (H2, MySQL etc) - so in unit tests I would use H2, and for the staging/production I would use MySQL. So far I couldn't find a way to import all necessary abstract components to get this working:
import slick.jdbc.H2Profile.api._
class OAuthCredentialsTable(tag: Tag) extends Table[OAuth](tag, "credentials_oauth") {
def username: Rep[String] = column[String]("username", O.SqlType("VARCHAR"))
def service: Rep[String] = column[String]("service", O.SqlType("VARCHAR"))
def serviceId: Rep[String] = column[String]("service_id", O.SqlType("VARCHAR"))
def userRef: ForeignKeyQuery[UserTable, User] = foreignKey("oauth_user_fk", username, userTable)(_.username, onDelete = ForeignKeyAction.Cascade)
override def * = (username, service, serviceId) <> (OAuth.tupled, OAuth.unapply)
}
val oauthTable: TableQuery[OAuthCredentialsTable] = TableQuery[OAuthCredentialsTable]
Eventually I discovered that to accomplish the driver-agnostic setup, it could be done as simple as that:
object UserPersistence extends JdbcProfile {
import api._
class UserTable(tag: Tag) extends Table[User](tag, "users") {
def username: Rep[String] = column[String]("username", O.PrimaryKey, O.SqlType("VARCHAR"))
def password: Rep[String] = column[String]("password", O.SqlType("VARCHAR"))
def serverkey: Rep[String] = column[String]("serverkey", O.SqlType("VARCHAR"), O.Length(64))
def salt: Rep[String] = column[String]("salt", O.SqlType("VARCHAR"), O.Length(64))
def iterations: Rep[Int] = column[Int]("iterationcount", O.SqlType("INT"))
def created: Rep[Timestamp] = column[Timestamp]("created_at", O.SqlType("TIMESTAMP"))
val mkUser: ((String, String, String, String, Int, Timestamp)) ⇒ User = {
case ((name, pwd, _, _, _, created)) ⇒ User(name, pwd, created.toInstant)
}
def unMkUser(u: User) = Some(u.username, u.password, "", "", 0, new Timestamp(u.createdAt.toEpochMilli))
override def * = (username, password, serverkey, salt, iterations, created) <> (mkUser, unMkUser)
}
val userTable: TableQuery[UserTable] = TableQuery[UserTable]
}
and then in order to use different profiles - you need to supply different database implementation when you run something, e.g
trait UserPersistence {
protected def db: Database
protected implicit val ec: ExecutionContext
override def findCredentialsOrRegister(oauthCredentials: OAuth): Future[User] = {
val userFound = (for (
creds ← oauthTable.filter(x ⇒ x.service === oauthCredentials.service && x.serviceId === oauthCredentials.serviceId);
user ← userTable.filter(x ⇒ x.username === creds.username)
) yield user).result
val authenticate = userFound.flatMap {
case Seq(user) ⇒
DBIO.from(Future.successful(user))
case Seq() ⇒
val newUUID = UUID.randomUUID
val user = User(newUUID.toString, UUID.randomUUID().toString, Instant.now())
DBIO.seq(
userTable += user,
oauthTable += oauthCredentials.copy(username = newUUID.toString)
) andThen DBIO.from(Future.successful(user))
}
db.run(authenticate.transactionally)
}
and then in test
val impl = new UserPersistence {
override def db: H2Profile.api.Database = // initialize and populate the database
override implicit val ec: ExecutionContext = scala.concurrent.ExecutionContext.global
}
For MySql just assign the MySQL profile to db property (and change type).
I hope I got you right, you can achieve it with injections and your configuration, separating test config from prod,
So I think you can do something like this-
Create binding for injection-
class DbModule extends Module {
bind[slick.driver.JdbcProfile#Backend#Database] toNonLazy DatabaseConfigProvider.get[JdbcProfile](inject[Application]).db
}
Then (for example)-
abstract class BaseDaoImpl[T <: IdentifiableTable[S] : ClassTag, S <: RichEntity[S]](implicit injector: Injector)
extends BaseDao[S] with Injectable with Logger{
protected val db: slick.driver.JdbcProfile#Backend#Database = inject[slick.driver.JdbcProfile#Backend#Database]
protected val table: TableQuery[T]
def insert(obj: S): Future[S] = {
val insertQuery = table.returning(table.map(_.id)) into ((item, id) => item.withDifferentId(Some(id)))
val res = db.run(insertQuery += obj)
logger.debug(s"Inserting ${this.getClass} - ${res.toString}")
res
}
def all(): Future[Seq[S]] = {
db.run(table.result)
}
}
I used this configuration in order to change the DB profile in a application.conf file.
import slick.jdbc.JdbcProfile
import slick.basic._
trait DBComponent {
// use the application.conf to change the profile
val database = DatabaseConfig.forConfig[JdbcProfile]("h2")
val driver = database.profile
}
trait BankTable extends DBComponent {
import driver.api._
class BankTable(tag: Tag) extends Table[Bank](tag, "bank") {
val id = column[Int]("id", O.PrimaryKey, O.AutoInc)
val name = column[String]("name")
def * = (name, id.?) <> (Bank.tupled, Bank.unapply)
}
protected val bankTableQuery = TableQuery[BankTable]
protected def bankTableAutoInc = bankTableQuery returning
bankTableQuery.map(_.id)
}
class BankRepositoryBDImpl extends BankTable with BankRepository {
import driver.api._
val db = database.db
def createBank(bank: Bank): Future[Int] = db.run { bankTableAutoInc += bank }
}
and use a application.conf file
h2 = {
url = "jdbc:h2:mem:test1"
driver = org.h2.Driver
connectionPool = disabled
keepAliveConnection = true
}
sqlite = {
url = "jdbc:sqlite::memory:"
driver = org.sqlite.JDBC
connectionPool = disabled
keepAliveConnection = true
}
To accomplish this in my project I created an object out of the JdbcProfile trait and imported that everywhere:
JdbcProfile.scala:
package my.application.data.support
import slick.jdbc.JdbcProfile
object JdbcProfile extends JdbcProfile
Wherever I define tables I import it as follows:
import my.application.data.support.JdbcProfile.api._
...
In order to support extensions to slick, I created an abstraction called DatabaseSupport where each database type would have their own concrete implementation which is injected at startup-time depending on configuration.
DatabaseSupport.scala:
package my.application.data.support
/**
* Database support implicits and methods.
*/
trait DatabaseSupport {
/**
* Implicit database array type.
*/
implicit val seqStringType: JdbcProfile.DriverJdbcType[Seq[String]]
}
This way I can have seperate H2DatabaseSupport.scala and PostgresDatabaseSupport.scala implementations that specifies seqStringType in database-specific ways.

Scala type inference working with Slick Table

Have such models (simplified):
case class User(id:Int,name:String)
case class Address(id:Int,name:String)
...
Slick (2.1.0 version) table mapping:
class Users(_tableTag: Tag) extends Table[User](_tableTag, "users") with WithId[Users, User] {`
val id: Column[Int] = column[Int]("id", O.AutoInc, O.PrimaryKey)
...
}
trait WithId[T, R] {
this: Table[R] =>
def id: Column[Int]
}
Mixing trait WithId I want to implement generic DAO methods for different tables with column id: Column[Int] (I want method findById to work with both User and Address table mappings)
trait GenericSlickDAO[T <: WithId[T, R], R] {
def db: Database
def findById(id: Int)(implicit stk: SlickTableQuery[T]): Option[R] = db.withSession { implicit session =>
stk.tableQuery.filter(_.id === id).list.headOption
}
trait SlickTableQuery[T] {
def tableQuery: TableQuery[T]
}
object SlickTableQuery {
implicit val usersQ = new SlickTableQuery[Users] {
val tableQuery: Table Query[Users] = Users
}
}
The problem is that findById doesn't compile:
Error:(13, 45) type mismatch;
found : Option[T#TableElementType] required: Option[R]
stk.tableQuery.filter(_.id === id).list.headOption
As I see it T is of type WithId[T, R] and at the same time is of type Table[R]. Slick implements the Table type such that if X=Table[Y] then X#TableElementType=Y.
So in my case T#TableElementType=R and Option[T#TableElementType] should be inferred as Option[R] but it isn't. Where am I wrong?
Your assumption about WithId[T, R] being of type Table[R] is wrong. The self-type annotation in WithId[T, R] just requires a Table[R] to be mixed in, but that doesn't mean that WithId[T, R] is a Table[R].
I think you confuse the declaration of WithId with instances of WithId which eventually need to be an instance of a Table.
Your upper type bound constraint in the GenericSlickDAO trait also doesn't guarantee you the property of WithId to be an instance of Table, since any type is a subtype of itself.
See this question for a more elaborate explanation about the differences between self-types and subtypes.
I'm using play-slick and I tried to do exactly like you, with a trait and using self-type without success.
But I succeeded with the following:
import modelsunscanned.TableWithId
import scala.slick.jdbc.JdbcBackend
import scala.slick.lifted.TableQuery
import play.api.db.slick.Config.driver.simple._
/**
* #author Sebastien Lorber (lorber.sebastien#gmail.com)
*/
package object models {
private[models] val Users = TableQuery(new UserTable(_))
private[models] val Profiles = TableQuery(new ProfileTable(_))
private[models] val Companies = TableQuery(new CompanyTable(_))
private[models] val Contacts = TableQuery(new ContactTable(_))
trait ModelWithId {
val id: String
}
trait BaseRepository[T <: ModelWithId] {
def tableQuery: TableQuery[TableWithId[T]]
private val FindByIdQuery = Compiled { id: Column[String] =>
tableQuery.filter(_.id === id)
}
def insert(t: T)(implicit session: JdbcBackend#Session) = {
tableQuery.insert(t)
}
def getById(id: String)(implicit session: JdbcBackend#Session): T = FindByIdQuery(id).run.headOption
.getOrElse(throw new RuntimeException(s"Could not find entity with id=$id"))
def findById(id: String)(implicit session: JdbcBackend#Session): Option[T] = FindByIdQuery(id).run.headOption
def update(t: T)(implicit session: JdbcBackend#Session): Unit = {
val nbUpdated = tableQuery.filter(_.id === t.id).update(t)
require(nbUpdated == 1,s"Exactly one should have been updated, not $nbUpdated")
}
def delete(t: T)(implicit session: JdbcBackend#Session) = {
val nbDeleted = tableQuery.filter(_.id === t.id).delete
require(nbDeleted == 1,s"Exactly one should have been deleted, not $nbDeleted")
}
def getAll(implicit session: JdbcBackend#Session): List[T] = tableQuery.list
}
}
// play-slick bug, see https://github.com/playframework/play-slick/issues/227
package modelsunscanned {
abstract class TableWithId[T](tableTag: Tag,tableName: String) extends Table[T](tableTag,tableName) {
def id: Column[String]
}
}
I give you an exemple usage:
object CompanyRepository extends BaseRepository[Company] {
// Don't know yet how to avoid that cast :(
def tableQuery = Companies.asInstanceOf[TableQuery[TableWithId[Company]]]
// Other methods here
...
}
case class Company(
id: String = java.util.UUID.randomUUID().toString,
name: String,
mainContactId: String,
logoUrl: Option[String],
activityDescription: Option[String],
context: Option[String],
employeesCount: Option[Int]
) extends ModelWithId
class CompanyTable(tag: Tag) extends TableWithId[Company](tag,"COMPANY") {
override def id = column[String]("id", O.PrimaryKey)
def name = column[String]("name", O.NotNull)
def mainContactId = column[String]("main_contact_id", O.NotNull)
def logoUrl = column[Option[String]]("logo_url", O.Nullable)
def activityDescription = column[Option[String]]("description", O.Nullable)
def context = column[Option[String]]("context", O.Nullable)
def employeesCount = column[Option[Int]]("employees_count", O.Nullable)
//
def * = (id, name, mainContactId,logoUrl, activityDescription, context, employeesCount) <> (Company.tupled,Company.unapply)
//
def name_index = index("idx_name", name, unique = true)
}
Note that active-slick is also using something similar
This helped me out a lot. It's a pretty simple example of a genericdao https://gist.github.com/lshoo/9785645
package slicks.docs.dao
import scala.slick.driver.PostgresDriver.simple._
import scala.slick.driver._
trait Profile {
val profile: JdbcProfile
}
trait CrudComponent {
this: Profile =>
abstract class Crud[T <: Table[E] with IdentifiableTable[PK], E <: Entity[PK], PK: BaseColumnType](implicit session: Session) {
val query: TableQuery[T]
def count: Int = {
query.length.run
}
def findAll: List[E] = {
query.list()
}
def queryById(id: PK) = query.filter(_.id === id)
def findOne(id: PK): Option[E] = queryById(id).firstOption
def add(m: E): PK = (query returning query.map(_.id)) += m
def withId(model: E, id: PK): E
def extractId(m: E): Option[PK] = m.id
def save(m: E): E = extractId(m) match {
case Some(id) => {
queryById(id).update(m)
m
}
case None => withId(m, add(m))
}
def saveAll(ms: E*): Option[Int] = query ++= ms
def deleteById(id: PK): Int = queryById(id).delete
def delete(m: E): Int = extractId(m) match {
case Some(id) => deleteById(id)
case None => 0
}
}
}
trait Entity[PK] {
def id: Option[PK]
}
trait IdentifiableTable[I] {
def id: Column[I]
}
package slicks.docs
import slicks.docs.dao.{Entity, IdentifiableTable, CrudComponent, Profile}
case class User(id: Option[Long], first: String, last: String) extends Entity[Long]
trait UserComponent extends CrudComponent {
this: Profile =>
import profile.simple._
class UsersTable(tag: Tag) extends Table[User](tag, "users") with IdentifiableTable[Long] {
override def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def first = column[String]("first")
def last = column[String]("last")
def * = (id.?, first, last) <> (User.tupled, User.unapply)
}
class UserRepository(implicit session: Session) extends Crud[UsersTable, User, Long] {
override def query = TableQuery[UsersTable]
override def withId(user: User, id: Long): User = user.copy(id = Option(id))
}
}

Scala compiler infers Nothing for generic arguments

I'm trying something that I've seen in different shapes in different contexts before:
extending scala's query extensions with filterById(id: Id)
This is what I've tried:
trait TableWithId { self: Profile =>
import profile.simple._
trait HasId[Id] { self: Table[_] =>
def id: Column[Id]
}
implicit class HasIdQueryExt[Id: BaseColumnType, U]
(query: Query[Table[U] with HasId[Id], U]) {
def filterById(id: Id)(implicit s: Session) = query.filter(_.id === id)
def insertReturnId(m: U)(implicit s: Session): Id = query.returning(query.map(_.id)) += m
}
}
This works fine, no real magic there. But because there is no type constraint on the Table type, any query to which I apply filterById, looses it's specificness (is is now a generic Table with HasId[Id]), and I can no longer reach it's columns (except for _.id ofcourse).
I don't know how to type this implicit conversion, such that this is prevented. Is it possible? The following "naieve" solution does not work, because Scala infers Nothing for the Id type now:
implicit class HasIdQueryExt[Id: BaseColumnType, U, T <: Table[U] with HasId[Id]]
(query: Query[T, U]) { ... }
I find it kind of strange that suddenly the Id type is inferred as Nothing. How do I hint the compiler where to look for that Id type?
This is my solution for a similar problem. I did use specific type for id though.:
trait GenericComponent { this: Profile =>
import profile.simple._
abstract class TableWithId[A](tag:Tag, name:String) extends Table[A](tag:Tag, name) {
def id = column[Option[UUID]]("id", O.PrimaryKey)
}
abstract class genericTable[T <: Table[A] , A] {
val table: TableQuery[T]
/**
* generic methods
*/
def insert(entry: A)(implicit session:Session): A = session.withTransaction {
table += entry
entry
}
def insertAll(entries: List[A])(implicit session:Session) = session.withTransaction { table.insertAll(entries:_*) }
def all: List[A] = database.withSession { implicit session =>
table.list.map(_.asInstanceOf[A])
}
}
/**
* generic queries for any table which has id:Option[UUID]
*/
abstract class genericTableWithId[T <: TableWithId[A], A <:ObjectWithId ] extends genericTable[T, A] {
def forIds(ids:List[UUID]): List[A] = database.withSession { implicit session =>
ids match {
case Nil => Nil
case x::xs =>table.filter(_.id inSet(ids)).list.map(_.asInstanceOf[A])
}
}
def forId(id:UUID):Option[A] = database.withSession { implicit session =>table.filter(_.id === id).firstOption }
}
}
and then for your concrete component:
case class SomeObjectRecord(
override val id:Option[UUID] = None,
name:String) extends ObjectWithId(id){
// your function definitions there
}
trait SomeObjectComponent extends GenericComponent { this: Profile =>
import profile.simple._
class SomeObjects(tag: Tag) extends TableWithId[SomeObjectRecord](tag, "some_objects") {
def name = column[String]("name", O.NotNull)
def * = (id, name) <> (SomeObjectRecord.tupled, SomeObjectRecord.unapply)
def nameIdx = index("u_name", (name), unique = true)
}
object someobjects extends genericTableWithId[SomeObjects, SomeObjectRecord] {
val table = TableQuery[Units]
// your additional methods there; you get insert and insertAll from the parent
}
}

Return a mapped object using Slick

How can I return a mapped object using Slick? Using the following code my query returns List[(Int, String)] and not a List[Task] like I want it to. Is this not possible using Slick or am I thinking about Slick the wrong way is it not an ORM? I'm trying to return a query and use it in a view template using the Play2 framework. I'd like to end up accessing the objects like task.id task.label etc... Thanks.
import play.api.Play.current
import play.api.db._
import scala.slick.driver.H2Driver.simple._
case class Task(id: Int, label: String)
object Task extends Table[(Int, String)]("TASKS") {
lazy val database = Database.forDataSource(DB.getDataSource())
def id = column[Int]("ID", O.PrimaryKey, O.AutoInc)
def label = column[String]("LABEL")
def * = id ~ label
def all() : List[Task] = database.withSession { implicit db: Session =>
Query(Task).list
}
}
The issue is with how you defined your table. Try changing your table definition to:
case class Task(id: Int, label: String)
object Task extends Table[Task]("TASKS") {
lazy val database = Database.forDataSource(DB.getDataSource())
def id = column[Int]("ID", O.PrimaryKey, O.AutoInc)
def label = column[String]("LABEL")
def * = id ~ label <> (Task.apply _, Task.unapply _)
def all() : List[Task] = database.withSession { implicit db: Session =>
Query(Task).list
}
}
The difference is that the type param I am passing to the Table is Task instead of (Int, String). This should fix your issue.