Silently dropping a nullable database column with slick - scala

I have a database for objects called Campaigns containing three fields :
Id (int, not nullable)
Version (int, not nullable)
Stuff (Text, nullable)
Let's call CampaignsRow the corresponding slick entity class
When I select line from Campaigns, I don't always need to read stuff, which contains big chunks of text.
However, I'd very much like to work in the codebase with the class CampaignsRow instead of a tuple, and so to be able to sometimes just drop the Stuff column, while retaining the original type
Basically, I'm trying to write the following function :
//Force dropping the Stuff column from the current Query
def smallCampaign(campaigns: Query[Campaigns, CampaignsRow, Seq]): Query[Campaigns, CampaignsRow, Seq] = {
val smallCampaignQuery = campaigns.map {
row => CampaignsRow(row.id, row.version , None : Option[String])
}
smallCampaignQuery /* Fails because the type is now wrong, I have a Query[(Rep[Int], Rep[Int], Rep[Option[String]), (Int, Int, Option[String], Seq] */
}
Any idea how to do this ? I suspect this has to do with Shape in slick, but I can't find a resource to start understanding this class, and the slick source code is proving too complex for me to follow.

You're actually already doing almost what you want in def *, the default mapping. You can use the same tools in the map method. Your two tools are mapTo and <>.
As you've found, there is the mapTo method which you can only use if your case class exactly matches the shape of the tuple, so if you wanted a special case class just for this purpose:
case class CampaignLite(id: Int, version: Int)
val smallCampaignQuery = campaigns.map {
row => (row.id, row.version).mapTo[CampaignLite]
}
As you want to reuse your existing class, you can write your own convert functions instead of using the standard tupled and unapply and pass those to <>:
object CampaignRow {
def tupleLite(t: (Int, Int)) = CampaignRow(t._1, t._2, None)
def unapplyLite(c: CampaignRow) = Some((c.id, c.version))
}
val smallCampaignQuery = campaigns.map {
row => (row.id, row.version) <> (CampaignRow.tupleLite, CampaignRow.unapplyLite)
}
This gives you the most flexibility, as you can do whatever you like in your convert functions, but it's a bit more wordy.
As row is an instance of the Campaigns table you could always define it there alongside *, if you need to use it regularly.
class Campaigns ... {
...
def * = (id, version, stuff).mapTo[CampaignRow]
def liteMapping = (id, version) <> (CampaignRow.tupleLite, CampaignRow.unapplyLite)
}
val liteCampaigns = campaigns.map(_.liteMapping)
Reference: Essential Slick 3, section 5.2.1

If I understand your requirement correctly, you could consider making CampaignRow a case class that models your Campaigns table class by having Campaigns extend Table[CampaignRow] and providing the bidirectional mapping for the * projection:
case class CampaignRow(id: Int, version: Int, stuff: Option[String])
class Campaigns(tag: Tag) extends Table[CampaignRow](tag, "CAMPAIGNS") {
// ...
def * = (id, version, stuff) <> (CampaignRow.tupled, CampaignRow.unapply)
}
You should then be able to do something like below:
val campaigns = TableQuery[CampaignRow]
val smallCampaignQuery = campaigns.map( _.copy(stuff = None) )
For a relevant example, here's a Slick doc.

Related

Extracting DDL from case classes

I'm experimenting with scalikejdbc (trying to move from Slick), and I'm stuck on creating my schema from the entities (read: case classes).
// example Slick equivalent
case class X(id: Int, ...)
class XTable(tag: Tag) extends Table[X] (tag, "x") {
def id = column[Int]("id")
... //more columns
def * = (id, ...) <> (X.tupled, X.unapply)
}
val xTable = TableQuery[XTable]
db.run(xtable.schema.create) //creates in the DB a table named "x", with "id" and all other columns
It seemed like using SQLSyntaxSupport could be a step in the right direction, with something like
// scalikejdbc
case class X (id: Int, ...)
object X extends SQLSyntaxSupport[X] {
def apply (x: ResultName[X])(rs: WrappedResultSet): X = new X(id = rs.get(x.id, ...))
}
X.table.??? // what to do now?
but could not figure out the next step.
What I'm looking for is the opposite of the tool described under [Reverse-engineering]: http://scalikejdbc.org/documentation/reverse-engineering.html
Any help/ideas, in particular directions to a relevant part of the documentation, will be appreciated
You can use the the statements method to get the SQL code, like for
most other SQL-based Actions. Schema Actions are currently the only
Actions that can produce more than one statement.
schema.create.statements.foreach(println)
schema.drop.statements.foreach(println)
http://slick.typesafe.com/doc/3.0.0/schemas.html

Slick - create columns NOT present in the projection

When using tableQuery.ddl.create, it creates only the columns in the projection. However in our use case there are columns which are used ONLY for filtering and/or ordering, so they are not part of the projection, but they need to be created:
case class CacheEntry(source: String, destination: String, hops: Int)
class CacheTable(tag: Tag) extends Table[CacheEntry](tag, "cache") {
def source = column[String]("source")
def destination = column[String]("dest")
def hops = column[Int]("hops")
def timestamp = column[LocalDateTime]("ts", O.DBType("timestamp default now"))
def * = (source, destination, hops) <> ((CacheEntry.apply _).tupled, CacheEntry.unapply)
}
How can we convince Slick to create the timstamp column with TableQuery[CacheTable].ddl.create?
Are we approaching this in the wrong way? We definately do NOT want the ts to show up in the CacheEntry (we could live with it in this case, but we have more complicated cases where this is not desirable)
You could define something like a case class TimestampedEE](entry: E, timestamp: LocalDateTime), and change your CacheTable to a Table[Timestamped[CacheEntry]]. The def * projection is probably going to look ugly (if don't rely on some shapeless magic), but that's one way to do it.

Listing columns on a Slick table

I have a Slick 3.0 table definition similar to the following:
case class Simple(a: String, b: Int, c: Option[String])
trait Tables { this: JdbcDriver =>
import api._
class Simples(tag: Tag) extends Table[Simple](tag, "simples") {
def a = column[String]("a")
def b = column[Int]("b")
def c = column[Option[String]]("c")
def * = (a, b, c) <> (Simple.tupled, Simple.unapply)
}
lazy val simples = TableQuery[Simples]
}
object DB extends Tables with MyJdbcDriver
I would like to be able to do 2 things:
Get a list of the column names as Seq[String]
For an instance of Simple, generate a Seq[String] that would correspond to how the data would be inserted into the database using a raw query (e.g. Simple("hello", 1, None) becomes Seq("'hello'", "1", "NULL"))
What would be the best way to do this using the Slick table definition?
First of all it is not possible to trick Slick and change the order on the left side of the <> operator in the * method without changing the order of values in Simple, the row type of Simples, i.e. what Ben assumed is not possible. The ProvenShape return type of the * projection method ensures that there is a Shape available for translating between the Column-based type in * and the client-side type and if you write def * = (c, b, a) <> Simple.tupled, Simple.unapply) having Simple defined as case class Simple(a: String, b: Int, c: Option[String]), Slick will complain with an error "No matching Shape found. Slick does not know how to map the given types...". Ergo, you can iterate over all the elements of an instance of Simple with its productIterator.
Secondly, you already have the definition of the Simples table in your code and querying metatables to get the same information you already have is not sensible. You can get all you column names with a one-liner simples.baseTableRow.create_*.map(_.name). Note that the * projection of the table also defines the columns generated when you create the table schema. So the columns not mentioned in the projection are not created and the statement above is guaranteed to return exactly what you need and not to drop anything.
To recap briefly:
To get a list of the column names of the Simples table as Seq[String] use
simples.baseTableRow.create_*.map(_.name).toSeq
To generate a Seq[String] that would correspond to how the data
would be inserted into the database using a raw query for aSimple,
an instance of Simple use aSimple.productIterator.toSeq
To get the column names, try this:
db.run(for {
metaTables <- slick.jdbc.meta.MTable.getTables("simples")
columns <- metaTables.head.getColumns
} yield columns.map {_.name}) foreach println
This will print
Vector(a, b, c)
And for the case class values, you can use productIterator:
Simple("hello", 1, None).productIterator.toVector
is
Vector(hello, 1, None)
You still have to do the value mapping, and guarantee that the order of the columns in the table and the values in the case class are the same.

SLICK: A simple query using union and parameter lists

I'm new to SLICK (2.1) and am lost in creating my first query using union. Because the parameters are provided from external (via a web interface) eventually, I set them as optional. Please see the comment in the code below. How to create an appropriate query?
My actual class is more complex which I simplified for the sake of this question.
case class MyStuff(id: Int, value: Int, text: String)
class MyTable (tag: Tag) extends Table[MyStuff](tag, "MYSTUFF"){
def id = column[Int]("ID", O NotNull)
def value = column[Int]("VALUE", O NotNull)
def text = column[String]("TEXT", O NotNull)
def * =
(id,
value,
text).shaped <> ((MyStuff.apply _).tupled, MyStuff.unapply)
}
object myTable extends TableQuery(new MyTable(_)){
def getStuff(ids: Option[List[Int]], values: Option[List[Int]])(implicit session: Session): Option[List[MyStuff]] = {
/*
1) If 'ids' are given, retrieve all matching entries, if any.
2) If 'values' are given, retrieve all matching entries (if any), union with the results of the previous step, and remove duplicate entries.
4) If neither 'ids' nor 'values' are given, retrieve all entries.
*/
}
}
getStuff is called like this:
db: Database withSession { implicit session => val myStuff = myTable.getStuff(...) }
You can use inset if the values are Some, otherwise a literal false and only filter when something is not None.
if(ids.isDefined || values.isDefined)
myTable.filter(row =>
ids.map(row.id inSet _).getOrElse(slick.lifted.LiteralColumn(false))
) union myTable.filter(row =>
values.map(row.value inSet _).getOrElse(slick.lifted.LiteralColumn(false))
)
else myTable
If I understand you correctly you want to build a filter at runtime from the given input. You can look at the extended docs for 3.0 (http://slick.typesafe.com/doc/3.0.0-RC1/queries.html#sorting-and-filtering) at "building criteria using a "dynamic filter" e.g. from a webform". This part of the docs is also valid for version 2.1.

Easy idiomatic way to define Ordering for a simple case class

I have a list of simple scala case class instances and I want to print them in predictable, lexicographical order using list.sorted, but receive "No implicit Ordering defined for ...".
Is there exist an implicit that provides lexicographical ordering for case classes?
Is there simple idiomatic way to mix-in lexicographical ordering into case class?
scala> case class A(tag:String, load:Int)
scala> val l = List(A("words",50),A("article",2),A("lines",7))
scala> l.sorted.foreach(println)
<console>:11: error: No implicit Ordering defined for A.
l.sorted.foreach(println)
^
I am not happy with a 'hack':
scala> l.map(_.toString).sorted.foreach(println)
A(article,2)
A(lines,7)
A(words,50)
My personal favorite method is to make use of the provided implicit ordering for Tuples, as it is clear, concise, and correct:
case class A(tag: String, load: Int) extends Ordered[A] {
// Required as of Scala 2.11 for reasons unknown - the companion to Ordered
// should already be in implicit scope
import scala.math.Ordered.orderingToOrdered
def compare(that: A): Int = (this.tag, this.load) compare (that.tag, that.load)
}
This works because the companion to Ordered defines an implicit conversion from Ordering[T] to Ordered[T] which is in scope for any class implementing Ordered. The existence of implicit Orderings for Tuples enables a conversion from TupleN[...] to Ordered[TupleN[...]] provided an implicit Ordering[TN] exists for all elements T1, ..., TN of the tuple, which should always be the case because it makes no sense to sort on a data type with no Ordering.
The implicit ordering for Tuples is your go-to for any sorting scenario involving a composite sort key:
as.sortBy(a => (a.tag, a.load))
As this answer has proven popular I would like to expand on it, noting that a solution resembling the following could under some circumstances be considered enterprise-grade™:
case class Employee(id: Int, firstName: String, lastName: String)
object Employee {
// Note that because `Ordering[A]` is not contravariant, the declaration
// must be type-parametrized in the event that you want the implicit
// ordering to apply to subclasses of `Employee`.
implicit def orderingByName[A <: Employee]: Ordering[A] =
Ordering.by(e => (e.lastName, e.firstName))
val orderingById: Ordering[Employee] = Ordering.by(e => e.id)
}
Given es: SeqLike[Employee], es.sorted() will sort by name, and es.sorted(Employee.orderingById) will sort by id. This has a few benefits:
The sorts are defined in a single location as visible code artifacts. This is useful if you have complex sorts on many fields.
Most sorting functionality implemented in the scala library operates using instances of Ordering, so providing an ordering directly eliminates an implicit conversion in most cases.
object A {
implicit val ord = Ordering.by(unapply)
}
This has the benefit that it is updated automatically whenever A changes. But, A's fields need to be placed in the order by which the ordering will use them.
To summarize, there are three ways to do this:
For one-off sorting use .sortBy method, as #Shadowlands have showed
For reusing of sorting extend case class with Ordered trait, as #Keith said.
Define a custom ordering. The benefit of this solution is that you can reuse orderings and have multiple ways to sort instances of the same class:
case class A(tag:String, load:Int)
object A {
val lexicographicalOrdering = Ordering.by { foo: A =>
foo.tag
}
val loadOrdering = Ordering.by { foo: A =>
foo.load
}
}
implicit val ord = A.lexicographicalOrdering
val l = List(A("words",1), A("article",2), A("lines",3)).sorted
// List(A(article,2), A(lines,3), A(words,1))
// now in some other scope
implicit val ord = A.loadOrdering
val l = List(A("words",1), A("article",2), A("lines",3)).sorted
// List(A(words,1), A(article,2), A(lines,3))
Answering your question Is there any standard function included into the Scala that can do magic like List((2,1),(1,2)).sorted
There is a set of predefined orderings, e.g. for String, tuples up to 9 arity and so on.
No such thing exists for case classes, since it is not easy thing to roll off, given that field names are not known a-priori (at least without macros magic) and you can't access case class fields in a way other than by name/using product iterator.
The unapply method of the companion object provides a conversion from your case class to an Option[Tuple], where the Tuple is the tuple corresponding to the first argument list of the case class. In other words:
case class Person(name : String, age : Int, email : String)
def sortPeople(people : List[Person]) =
people.sortBy(Person.unapply)
The sortBy method would be one typical way of doing this, eg (sort on tag field):
scala> l.sortBy(_.tag)foreach(println)
A(article,2)
A(lines,7)
A(words,50)
Since you used a case class you could extend with Ordered like such:
case class A(tag:String, load:Int) extends Ordered[A] {
def compare( a:A ) = tag.compareTo(a.tag)
}
val ls = List( A("words",50), A("article",2), A("lines",7) )
ls.sorted
My personal favorite method is using the SAM(Single abstraction method) with 2.12 as mentioned over the below example:
case class Team(city:String, mascot:String)
//Create two choices to sort by, city and mascot
object MyPredef3 {
// Below used in 2.11
implicit val teamsSortedByCity: Ordering[Team] = new Ordering[Team] {
override def compare(x: Team, y: Team) = x.city compare y.city
}
implicit val teamsSortedByMascot: Ordering[Team] = new Ordering[Team] {
override def compare(x: Team, y: Team) = x.mascot compare y.mascot
}
/*
Below used in 2.12
implicit val teamsSortedByCity: Ordering[Team] =
(x: Team, y: Team) => x.city compare y.city
implicit val teamsSortedByMascot: Ordering[Team] =
(x: Team, y: Team) => x.mascot compare y.mascot
*/
}
object _6OrderingAList extends App {
//Create some sports teams
val teams = List(Team("Cincinnati", "Bengals"),
Team("Madrid", "Real Madrid"),
Team("Las Vegas", "Golden Knights"),
Team("Houston", "Astros"),
Team("Cleveland", "Cavaliers"),
Team("Arizona", "Diamondbacks"))
//import the implicit rule we want, in this case city
import MyPredef3.teamsSortedByCity
//min finds the minimum, since we are sorting
//by city, Arizona wins.
println(teams.min.city)
}