Extending SLICK Tables in a DRY manner - scala

I have an interesting question around Slick/Scala that I am hoping that one of you nice chaps might be able to assist me with.
I have several tables and by extension in SLICK case classes
case class A(...)
case class B(...)
case class C(...)
that share these common fields
(id: String, livemode: Boolean, created: DateTime, createdBy : Option[Account]) .
Because these fields are repeated in every case class, I'd like to explore the possibility of extracting them into a single object or type.
However, when creating the SLICK table objects I would like things to end up where these common fields are included too so I can persist their individual values in each table.
object AsTable extends Table[A]("a_table") {
...
def id = column[String]("id", O.PrimaryKey)
def livemode = column[Boolean]("livemode", O.NotNull)
def created = column[DateTime]("created", O.NotNull)
def createdBy = column[Account]("created_by", O.NotNull)
...
}
Effectively, the end result I'm looking for is to allow me make changes to the common fields without having to update each table.
Is there a way to do this?
Thanks in advance

I have not tried this, but how about a trait you mix in:
trait CommonFields { this: Table[_] =>
def id = column[String]("id", O.PrimaryKey)
def livemode = column[Boolean]("livemode", O.NotNull)
def created = column[DateTime]("created", O.NotNull)
def createdBy = column[Account]("created_by", O.NotNull)
protected common_* = id ~ livemode ~ created ~ createdBy
}
Then you can do:
object AsTable extends Table[(String,Boolean,DateTime,Account,String)]("a_table")
with CommonFields {
def foo = column[String]("foo", O.NotNull)
def * = common_* ~ foo
}
The only thing you'll have to repeat now is the type of the elements.
UPDATE
If you want to do object-mapping and:
You map to case-classes
The fields in your case classes are in the same order
Just do:
case class A(
id: String,
livemode: Boolean,
created: DateTime,
createdBy: Account,
foo: String)
object AsTable extends Table[A]("a_table") with CommonFields {
def foo = column[String]("foo", O.NotNull)
def * = common_* ~ foo <> (A.apply _, A.unapply _)
}
This seems to be the most economical solution (rather then trying to define * in CommonFields and adding a type parameter). However, it requires you to change all case classes if your fields change.
We could try to mitigate this by using composition on the case classes:
case class Common(
id: String,
livemode: Boolean,
created: DateTime,
createdBy: Account)
case class A(
common: Common,
foo: String)
However, when constructing the mapper function, we will (somewhere) end up having to convert tuples of the form:
(CT_1, CT_2, ... CT_N, ST_1, ST_2, ..., ST_M)
CT Common type (known in CommonFields)
ST Specific type (known in AsTable)
To:
(CT_1, CT_2, ... CT_N), (ST_1, ST_2, ..., ST_M)
In order to pass them to subroutines individually converting Common and A to and from their tuples.
We have to do this without knowing the number or the exact types of either CT (when implementing in AsTable) or ST (when implementing in CommonFields). The tuples in the Scala standard library are unable to do that. You would need to use HLists as for exampled offered by shapeless to do this.
It is questionable whether this is worth the effort.
The basic outline could look like this (without all the implicit mess which will be required). This code will not compile like this.
trait CommonFields { this: Table[_] =>
// like before
type ElList = String :: Boolean :: DateTime :: Account :: HNil
protected def toCommon(els: ElList) = Common.apply.tupled(els.tupled)
protected def fromCommon(c: Common) = HList(Common.unapply(c))
}
object AsTable extends Table[A] with CommonFields {
def foo = column[String]("foo", O.NotNull)
def * = common_* ~ foo <> (x => toA(HList(x)), x => fromA(x) tupled)
// convert HList to A
protected def toA[L <: HList](els: L) = {
// Values for Common
val c_els = els.take[Length[ElList]]
// Values for A
val a_els = toCommon(c_els) :: els.drop[Length[ElList]]
A.apply.tupled(a_els.tupled)
}
// convert A to HList
protected def fromA(a: A) =
fromCommon(a.common) :: HList(A.unapply(a)).drop[One]
}
Using some more type magic you can probably resolve the last two issues:
Put toA and fromA into the base trait (by using type parameters in the trait, or using abstract type members)
Avoid defining ElList explicitly by extracting it from Common.apply by using this technique

Related

Scala Slick single filter for multiple TableQuery instances

I'm using Scala Slick-3.1.0 lib.
How is it possible to make a generic Slick filter function that takes TableQuery instance as an input and makes same slick filter on it?
I have several case classes (two for example) representing data stored in DB.
Some fields are the same so classes could possibly extend common ancestor:
case class FirstData(symbol: String,
date: Timestamp,
firstField: Double)
case class SecondData(symbol: String,
date: Timestamp,
secondField: String)
Each of them has its own SQL Table in DB and is represented by separate Slick Table class. Also they have same primary keys:
class FirstDataTable(tag: Tag) extends Table[FirstData](tag, "firstData") {
def symbol = column[String]("symbol")
def date = column[Timestamp]("date")
def firstField= column[Double]("firstField")
def * = (symbol, date, firstField) <> ((FirstData.apply _).tupled, FirstData.unapply)
def pk = primaryKey("pk_firstData", (symbol, date))
}
class SecondDataTable(tag: Tag) extends Table[SecondData](tag, "secondData"){
def symbol = column[String]("symbol")
def date = column[Timestamp]("date")
def secondField= column[String]("secondField")
def * = (symbol, date, secondField) <> ((SecondData.apply _).tupled, SecondData.unapply)
def pk = primaryKey("pk_secondData", (symbol, date))
}
Finally TableQuery classes are:
val firstDataTableQuery = TableQuery[FirstDataTable]
val secondDataTableQuery = TableQuery[SecondDataTable]
etc ...
How is it possible to make a generic Slick filter query function that takes firstDataTableQuery or secondDataTableQuery as an argument and makes same slick query on input. Filtering only on their common fields or another way saying on their SQL Table representations common columns. For example like this:
def filter(genericTableQuery: TableQuery) = {
genericTableQuery.filter { data => dataFilterFunction(data.symbol)
}.filter(_.date >= someDate).sortBy(data => data.date.asc)
}
val firstTableResult = filter(firstDataTableQuery)
val seconfTableResult = filter(secondDataTableQuery)
etc ...
I looked on this topics, but still could not make up a solution:
Slick 3 reusable generic repository
Scala reflection to instantiate scala.slick.lifted.TableQuery
How about making the data table classes extend a common trait, which in turn extends a common trait for data classes, like for example:
trait Data {
def symbol: String
def date: Timestamp
}
// make FirstData and SecondData extend the above trait
trait GenericDataTable extends Data {
def symbol: Rep[String]
def date: Rep[Timestamp]
def pk: PrimaryKey
}
class FirstDataTable(tag: Tag) extends Table[FirstData](tag, "firstData") with GenericDataTable {
// ...
and then:
def filter[T <: GenericDataTable](genericTableQuery: TableQuery[T]) = // ...
invariant, thanks a lot. You pointed me to the right direction. I slightly modified your answer and everything works )
Traits:
trait Data {
def symbol: String
def date: Timestamp
}
trait GenericDataTable[T <: Data] extends Table[T] {
def symbol: Rep[String]
def date: Rep[Timestamp]
def pk: PrimaryKey
}
FirstData and FirstDataTable classes look like this:
case class FirstData(
symbol: String,
date: Timestamp,
firstField: Double) extends Data
class FirstDataTable(tag: Tag) extends Table[FirstData(tag,"firstData")
with GenericDataTable[FirstData] {
def symbol = column[String]("symbol")
def date = column[Timestamp]("date")
def firstField= column[Double]("firstField")
def * = (symbol, date, firstField) <> ((FirstData.apply _).tupled, FirstData.unapply)
def pk = primaryKey("pk_firstData", (symbol, date))
}
Finally function looks like this:
private def filter[M <: Data, T <: GenericDataTable[M]] (genericTableQuery: TableQuery[T]) = {
genericTableQuery.filter { data => dataFilterFunction(data.symbol)}.
filter(_.date >= someDate).sortBy(data => data.date.asc)
}

In Slick, how to provide default columns for Table

I have some columns all my tables share, so I'd like to provide the default columns for all the tables. Following is what I have tried so far. I am using Slick 3.0.
// created_at and updated_at are the columns every table share
abstract class BaseTable[T](tag: Tag, tableName: String)
extends Table[T](tag, tableName) {
def currentWhenInserting = new Timestamp((new Date).getTime)
def createdAt = column[Timestamp]("created_at", O.Default(currentWhenInserting))
def updatedAt = column[Timestamp]("updated_at", O.Default(currentWhenInserting))
}
And the simple way to implement this seems like the following.
case class Student(
name: String, age: Int,
created_at: Timestamp, updated_at: Timestamp
)
class Students(tag: Tag) extends BaseTable[Student](tag, "students"){
def name = column[String]("name")
def age = column[Int]("age")
override def * : ProvenShape[Student] =
(name, age, createdAt, updatedAt).shaped <>
(Student.tupled, Student.unapply _)
}
But it's not desirable.
First, force every table row case class to incoporate created_at and updated_at. If I have more fields that would be totally unacceptable from the API design angle.
Second, write the two createdAt, updatedAt in (name, age, createdAt, updatedAt) explicitly. This is not what I expect about Default row.
My ideal way of solving this would be like the following:
case class Student(name: String, age: Int)
class Students(tag: Tag) extends BaseTable[Student](tag, "students"){
def name = column[String]("name")
def age = column[Int]("age")
override def * : ProvenShape[Student] =
(name, age).shaped <>
(Student.tupled, Student.unapply _)
}
That is, write some method in BaseTable or Define BaseCaseClass to avoid explicitly writing the extra two fields in table definition like Students and row case class definition Student.
However, after a painful struggle, still can get it done.
Any help would be greatly appreciated.
I'm using the following pattern:
case class Common(arg0: String, arg1: String)
trait Commons { this: Table[_] =>
def arg0 = column[String]("ARG0", O.Length(123))
def arg1 = column[String]("ARG1", O.Length(123))
def common = (arg0, arg1).<> [Meta, (String, String)](
r => {
val (arg0, arg1) = r
Meta(arg0, arg1)
},
o => Some(o.arg0, o.arg1)
)
}
case class MyRecord(a: String, b: String, common: Common)
class MyRecords(tag: Tag) extends Table[MyRecord](tag, "MY_RECORDS") with Commons {
def a = column[String]("A", O.PrimaryKey, O.Length(123))
def b = column[String]("B", O.Length(123))
def * = (a, b, common) <> (MyRecord.tupled, MyRecord.unapply)
}
It's not perfect but it helps avoid duplication without it being to difficult to understand.

How can I define a class that is almost the same as List

In Scala, suppose I would like a class that is just like List[String] except for a few methods, for example toString.
What I have now is
case class FQN(val names : List[String]) extends LinearSeq[String] {
def this( params : String* ) { this( List(params : _* )) }
override def toString() : String = {
names.reduce((a,b) => b++"."++a)
}
override def apply( i : Int ) = names.apply( i )
override def length() = names.length
override def tail = new FQN( names.tail )
override def head = names.head
override def isEmpty = names.isEmpty
def ::( str : String) = new FQN( str :: names )
}
However, what I would like is for fqn0 ++ fqn1 to give me an FQN, just as list0 ++ list1 gives a List'; it currently gives aLinearSeq[String]`. So something is not as I would like.
Also I made FQN a case class in order to get equals and hashCode to be the way I want, but I'm not sure whether this is the best way to do it.
What is the best way to make a class that is just like List, but implements certain methods differently?
My guts tell me that you are probably better off not implementing a specialized FQN collection, but hide the collection as part of the internal implementation of FQN, effectivly removing the corresponding operations from the interface of FQN. What I mean is, you should ask yourself whether the world must really see FQN as a Seq and be able use all corresponding operations, or whether it is enough to maybe add solely the operation ++ with a dispatch to the underyling list's ++ operation to FQN's interface.
However, the Scala guys apparently did a great job in generalizing their collections and at second glance, implementing your own collection doesn't seem like a huge pain. The documentation I posted in a comment explains how to reuse code when implementing your own collection and I recommend you read it.
In short, you'd want to mix-in traits with a name of the form xxxLike; in your specific case LinearSeqLike[String, FQN] and override the method newBuilder: mutable.Builder[String, FQN] to make methods like drop return a FQN.
The builder alone however isn't powerful enough on its own: Methods such as map (and to my surprise ++) may map the elements contained in the collection to a different type that is not necessary supported by your collection. In your example, FQN("foo").map(_.length) is not a legal FQN, so the result cannot be a FQN, however FQN("Foo").map(_.toLowercase) is legal (at least regarding the types). This problem is solved by bringing an implicit CanBuildFrom value to the scope.
The final implementation could look like this:
final class FQN private (underlying: List[String]) extends LinearSeq[String] with LinearSeqLike[String, FQN] {
def apply(i: Int): String = underlying.apply(i)
def length = underlying.length
/** From the documentation of {LinearSeqLike}:
* Linear sequences are defined in terms of three abstract methods, which are assumed
* to have efficient implementations. These are:
* {{{
* def isEmpty: Boolean
* def head: A
* def tail: Repr
* }}}
*/
override def isEmpty: Boolean = underlying.isEmpty
override def head: String = underlying.head
override def tail: FQN = FQN.fromSeq(underlying.tail)
override def newBuilder: mutable.Builder[String, FQN] = FQN.newBuilder
def ::(str: String) = new FQN(str :: underlying)
override def toString(): String = {
underlying.mkString(".")
}
}
object FQN {
implicit def canBuildFrom: CanBuildFrom[FQN, String, FQN] =
new CanBuildFrom[FQN, String, FQN] {
def apply(): mutable.Builder[String, FQN] = newBuilder
def apply(from: FQN): mutable.Builder[String, FQN] = newBuilder
}
def newBuilder: mutable.Builder[String, FQN] =
new ArrayBuffer mapResult fromSeq
def fromSeq(s: Seq[String]): FQN = new FQN(s.toList)
def apply(params: String*): FQN = fromSeq(params)
}
Based on comments from #Ryan and #KuluLimpa (or at any rate my understanding of them), here is an approach
type FQN = List[String]
implicit class FQNWithFQNToString( names : List[String] ) {
def fqnToString() : String = {
names.reduce((a,b) => b++"."++a)
}
}
Now everything you can do with List[String], you can do with FQN. For example fqn1++fqn2 gives an FQN. In order to get the string I want, I can use fqn.fqnToString.
I find this answer less than satisfactory for two reasons
Weak type checking. There is no error if any old list of Strings is treated as an FQN. In particular I can't define a type QN (for qualified name) and have a compile time error if I use a QN where a FQN is needed.
There are now two ways to convert an FQN to a string. I have to be sure to use the right one in each case. If I create another name QN for List[String], I'll then have three ways to convert an object to a string. This is starting to feel very non-object-oriented.

How to define generic type in Scala?

In Slick 2, we can map tables like this:
case class Cooler(id: Option[Int], minTemp: Option[Double], maxTemp: Option[Double])
/**
* Define table "cooler".
*/
class Coolers(tag: Tag) extends Table[Cooler](tag, "cooler") {
def id = column[Int]("id", O.PrimaryKey, O.AutoInc)
def minTemp = column[Double]("min_temp", O.Nullable)
def maxTemp = column[Double]("max_temp", O.Nullable)
def * = (id.?, minTemp.?, maxTemp.?) <> (Cooler.tupled, Cooler.unapply _)
}
object Coolers {
val tableQuery = TableQuery[Coolers]
}
because I have a lot of tables, I want to define generic methods for them, like find, delete, update so I have to define these methods in a super class from where to extend my objects (object Coolers extends TableUtils[Coolers, Cooler]). In order to define those methods, I need tableQuery to move out of my object in this super class, so I tried it like:
abstract class TableUtils[T <: Table[A] , A] {
val tableQuery = TableQuery[T]
}
but I receive an error on tableQuery definition:
class type required but T found
Does anybody know what I am doing wrong?
When you do TableQuery[T] you are in fact calling TableQuery.apply, which is actually a macro.
The body of this macro tries to instantiate T, but in your case T has become an (unknown) type parameter that the compiler does not know how to instantiate. The problem is similar to trying to compile this:
def instantiate[T]: T = new T
// Does not compile ("class type required but T found")
The net effect is that TableQuery.apply can only be used on concrete types.
You could work around that using a type class to capture the call to TableQuery.apply (at the point where the concrete type is known) along with an implicit macro to provide an instance of this type class. Then you would have something like:
abstract class TableUtils[T <: Table[A] : TableQueryBuilder, A] {
val tableQuery = BuildTableQuery[T]
}
Where TableQueryBuilder is the type class and BuildTableQuery is an alternate version of TableQuery.apply that will forward to the TableQueryBuilder instance to perform the actual instantiation.
I've added an implementation as part of another answer here.
It will be much easier (if less convenient) to just declare tableQuery as an abstract value and define it in every concrete derived class of TableUtils:
abstract class TableUtils[T <: Table[A] , A] {
val tableQuery: TableQuery[T, T#TableElementType]
// define here your helper methods operating on `tableQuery`
}
object Coolers extends TableUtils[Coolers, Cooler] {
val tableQuery = TableQuery[Coolers]
}
Here is one solution:
At first, define this to avoid class type issue..
class Service[T <: Table[_]](path: String, cons: Tag => T){
lazy val db = Database.forConfig(path)
def query = TableQuery[T](cons)
}
Then use it this way, Post is sub class of Table:
object Abcd {
object Def extends Service[Post]("mydb", abc) {
def test = {
//db
val q = query.drop(1).take(20)
val r = db.run(q.result)
println(q.result.statements.head)
println(r)
r
}
}
private def abc(tag: Tag) = new Post(tag)
}
This solution tested ok in slick 3.x, and Play slick 1.x, since the slick 2.0 Query.scala comply to slick 3.0 Query.scala, this might work at 2 too.

How can I compose queries in ScalaQuery in order to create reusable traits?

I'm having some trouble composing different query components into a single Query. My goal is to create a set of traits (e.g. SoftDeletable, HasName, SortedByName, WithTimestamps) that I can simply mix-in to Table objects to add that behavior.
The ideal would look like:
abstract class BaseModel[Tuple <: Product,CaseClass](tableName: String)
extends Table[Tuple](tableName) {
def id = column[Int]("id", O.AutoInc, O.PrimaryKey)
def mapped: MappedProjection[CaseClass, TupleClass]
def allQuery = this.map(_.mapped)
final def all = database.withSession { implicit session: Session =>
allQuery.list()
}
...
}
trait SoftDeletable[Tuple <: Product, CaseClass]
extends BaseModel[Tuple,CaseClass] {
def isActive = column[String]("is_active")
def * = super.* ~ isActive
def allQuery = /* here, I'd like to compose super.allQuery
with a filter that returns rows where isActive is true */
}
trait HasName[Tuple <: Product] extends Table[Tuple] {
def name = column[String]("name")
def * = super.* ~ name
}
trait SortedByName[Tuple <: Product] extends HasName[Tuple {
override def allQuery = super.allQuery /* compose somehow
with (_ <- Query orderBy name */
}
Can I do these kinds of things with ScalaQuery? The main sticking points are:
How do I cleanly compose the filters in SoftDeletable.allQuery and the sort in SortedByName.allQuery with BaseModel.allQuery?
By adding columns in subclass implementations of the * method, the tuple type parameter to Table no latter matches - is there a way for these traits to incrementally add new types to the columns tuple in the ultimate concrete class? (I don't expect there to be, but it would be nice if there was something I'm missing).
I need to repeat the long tuple declaration in every trait, which becomes very unwieldy if a table has five or six columns. Is there something I can do with type members to avoid having to do things like:
case class Foo
class Foos[(Int,Int,Boolean,String), Foo] extends
Table[(Int,Int,Boolean,String)] with
SoftDeletable[(Int,Int,Boolean,String), Foo] with
SortedByName[(Int,Int,Boolean,String), Foo] with
HasName[(Int,Int,Boolean,String)] {
}
Can I avoid all this repetition? Based on a suggestion from jesnor on IRC, I was able to avoid some of this like so:
abstract class SoftDeletableBaseModel[TupleClass <: Product, CaseClass](tableName: String)
extends BaseModel[TupleClass, CaseClass](tableName)
with SoftDeletable[TupleClass,CaseClass]
In other words, by combining specific traits together, I don't need to repeat the entire tuple declaration; of course, the disadvantage is that easy mixing-in of various traits is no longer possible - I need to create lots of specific subclasses to avoid this repetition. Is there another way?
Update: So I realized that I don't need to use separate CaseClass and TupleClass type parameters. Since case classes implement Product*, you can just pass the case class name into Table, which solves the problem in 3:
trait SoftDeletable[CaseClass] extends BaseModel[CaseClass] { ... }
class Models extends BaseModel[Model]("models") with SoftDeletable[Model] { ... }
If your issue is only adding the sort, isn't it just a matter of flatMap?
def sortBy[T,U,C](q: Query[T,U], col: NamedColumn[C]) = q.flatMap(_ => Query orderBy col)