How to write nested queries in select clause - scala

I'm trying to produce this SQL with SLICK 1.0.0:
select
cat.categoryId,
cat.title,
(
select
count(product.productId)
from
products product
right join products_categories productCategory on productCategory.productId = product.productId
right join categories c on c.categoryId = productCategory.categoryId
where
c.leftValue >= cat.leftValue and
c.rightValue <= cat.rightValue
) as productCount
from
categories cat
where
cat.parentCategoryId = 2;
My most successful attempt is (I dropped the "joins" part, so it's more readable):
def subQuery(c: CategoriesTable.type) = (for {
p <- ProductsTable
} yield(p.id.count))
for {
c <- CategoriesTable
if (c.parentId === 2)
} yield(c.id, c.title, (subQuery(c).asColumn))
which produces the SQL lacking parenthesis in subquery:
select
x2.categoryId,
x2.title,
select count(x3.productId) from products x3
from
categories x2
where x2.parentCategoryId = 2
which is obviously invalid SQL
Any thoughts how to have SLICK put these parenthesis in the right place? Or maybe there is a different way to achieve this?

I never used Slick or ScalaQuery so it was quite an adventure to find out how to achieve this. Slick is very extensible, but the documentation on extending is a bit tricky. It might already exist, but this is what I came up with. If I have done something incorrect, please correct me.
First we need to create a custom driver. I extended the H2Driver to be able to test easily.
trait CustomDriver extends H2Driver {
// make sure we create our query builder
override def createQueryBuilder(input: QueryBuilderInput): QueryBuilder =
new QueryBuilder(input)
// extend the H2 query builder
class QueryBuilder(input: QueryBuilderInput) extends super.QueryBuilder(input) {
// we override the expr method in order to support the 'As' function
override def expr(n: Node, skipParens: Boolean = false) = n match {
// if we match our function we simply build the appropriate query
case CustomDriver.As(column, LiteralNode(name: String)) =>
b"("
super.expr(column, skipParens)
b") as ${name}"
// we don't know how to handle this, so let super hanle it
case _ => super.expr(n, skipParens)
}
}
}
object CustomDriver extends CustomDriver {
// simply define 'As' as a function symbol
val As = new FunctionSymbol("As")
// we override SimpleSql to add an extra implicit
trait SimpleQL extends super.SimpleQL {
// This is the part that makes it easy to use on queries. It's an enrichment class.
implicit class RichQuery[T: TypeMapper](q: Query[Column[T], T]) {
// here we redirect our as call to the As method we defined in our custom driver
def as(name: String) =
CustomDriver.As.column[T](Node(q.unpackable.value), name)
}
}
// we need to override simple to use our version
override val simple: SimpleQL = new SimpleQL {}
}
In order to use it we need to import specific things:
import CustomDriver.simple._
import Database.threadLocalSession
Then, to use it you can do the following (I used the tables from the official Slick documentation in my example).
// first create a function to create a count query
def countCoffees(supID: Column[Int]) =
for {
c <- Coffees
if (c.supID === supID)
} yield (c.length)
// create the query to combine name and count
val coffeesPerSupplier =
for {
s <- Suppliers
} yield (s.name, countCoffees(s.id) as "test")
// print out the name and count
coffeesPerSupplier foreach { case (name, count) =>
println(s"$name has $count type(s) of coffee")
}
The result is this:
Acme, Inc. has 2 type(s) of coffee
Superior Coffee has 2 type(s) of coffee
The High Ground has 1 type(s) of coffee

Related

Combine Scala Future[Seq[X]] with Seq[Future[Y]] to produce Future[(X,Seq[Y])]

I have below relationship in my entity classes,
customer -> * Invoice
now I have to implement a method which returns customers with their invoices
type CustomerWithInvoices = (Custimer,Seq[Invoice])
def findCustomerWitnInvoices:Future[Seq[CustomerWithInvoices]] = {
for{
customers <- findCustomers
eventualInvoices: Seq[Future[Seq[Invoice]]] = customers.map(customer => findInvoicesByCustomer(customer))
} yield ???
}
using existing repository methods as below
def findCustomers:Future[Seq[Customers]] = {...}
def findInvoicesByCustomer(customer:Customer):Future[Seq[Invoice]] = {...}
I try to use for expression as above but I can't figure the proper way to do it, as I'm fairly new to Scala, highly appreciate any help..
i would use Future.sequence, the simplified method signature is
sequence takes M[Future[A]] and returns Future[M[A]]
That is what we need to solve your problem, here's the code i would write:
val eventualCustomersWithInvoices: Future[Seq[(Customer, Seq[Invoice])]] = for {
customers <- findCustomers()
eventualInvoices <- Future.sequence(customers.map(customer => findInvoicesByCustomer(customer)))
} yield eventualInvoices
note that the type of eventualInvoices is Future[Seq[(Customer, Seq[Invoice])]] hence Future[Seq[CustomerWithInvoices]]

Slick: how to implement find by example i.e. findByExample generically?

I'm exploring the different possibilities on how to implement a generic DAO using the latest Slick 3.1.1 to boost productivity and yes there is need for it because basing the service layer of my Play Web application on TableQuery alone leads to a lot of boilerplate code. One of the methods I'd like to feature in my generic DAO implementation is the findByExample, possible in JPA with the help of the Criteria API. In my case, I'm using the Slick Code Generator to generate the model classes from a sql script.
I need the following to be able to dynamically access the attribute names, taken from Scala. Get field names list from case class:
import scala.reflect.runtime.universe._
def classAccessors[T: TypeTag]: List[MethodSymbol] = typeOf[T].members.collect {
case m: MethodSymbol if m.isCaseAccessor => m
}.toList
A draft implementation for findByExample would be:
def findByExample[T, R](example: R) : Future[Seq[R]] = {
var qt = TableQuery[T].result
val accessors = classAccessors[R]
(0 until example.productArity).map { i =>
example.productElement(i) match {
case None => // ignore
case 0 => // ignore
// ... some more default values => // ignore
// handle a populated case
case Some(x) => {
val columnName = accessors(i)
qt = qt.filter(_.columnByName(columnName) == x)
}
}
}
qt.result
}
But this doesn't work because I need better Scala Kungfu. T is the entity table type and R is the row type that is generated as a case class and therefore a valid Scala Product type.
The first problem in that code is that would be too inefficient because instead of doing e.g.
qt.filter(_.firstName === "Juan" && _.streetName === "Rosedale Ave." && _.streetNumber === 5)
is doing:
// find all
var qt = TableQuery[T].result
// then filter by each column at the time
qt = qt.filter(_.firstName === "Juan")
qt = qt.filter(_.streetName === "Rosedale Ave.")
qt = qt.filter(_.streetNumber === 5)
Second I can't see how to dynamically access the column name in the filter method i.e.
qt.filter(_.firstName == "Juan")
I need to instead have
qt.filter(_.columnByName("firstName") == "Juan")
but apparently there is no such possibility while using the filter function?
Probably the best ways to implement filters and sorting by dynamically provided column names would be either plain SQL or extending the code generator to generate extension methods, something like this:
implicit class DynamicPersonQueries[C[_]](q: Query[PersonTable, PersonRow, C]){
def dynamicFilter( column: String, value: String ) = column {
case "firstName" => q.filter(_.firstName === value)
case "streetNumber" => q.filter(_.streetNumber === value.toInt)
...
}
}
You might have to fiddle with the types a bit to get it to compile (and ideally update this post afterwards :)).
You can then filter by all the provided values like this:
val examples: Map[String, String] = ...
val t = TableQuery[PersonTable]
val query = examples.foldLeft(t){case (t,(column, value)) => t.dynamicFilter(column, value)
query.result
Extending the code generator is explained here: http://slick.lightbend.com/doc/3.1.1/code-generation.html#customization
After further researching found the following blog post Repository Pattern / Generic DAO Implementation.
There they declare and implement a generic filter method that works for any Model Entity type and therefore it is in my view a valid functional replacement to the more JPA findByExample.
i.e.
T <: Table[E] with IdentifyableTable[PK]
E <: Entity[PK]
PK: BaseColumnType
def filter[C <: Rep[_]](expr: T => C)(implicit wt: CanBeQueryCondition[C]) : Query[T, E, Seq] = tableQuery.filter(expr)

How can I write a query where my order by clause varies?

I have a query that looks like:
object CategorySort extends Enumeration {
type CategorySort = Value
val ID, Price, CostPrice = Value
}
object SortDirection extends Enumeration {
type SortDirection = Value
val Asc, Desc = Value
}
def getProducts(categoryId: Int, sort: CategorySort, sortDirection: SortDirection): List[Product] = {
val q = for {
p <- products if p.categoryid === categoryId
} yield p
p.sortBy(o => o.id.desc).list()
}
The above function doesn't currently use my sort and sortDirection parameters.
Is there a way for me to modify my function so it can handle all the sort cases in an elegant manner?
The only way I know how currently is to use a switch statement but it will get very complicated because of the sort direction.
I am sure there is a smarter way to do this.
That thing bugged me too. I ended up implementing this trait, which I implement in my table classes. In your sortBy you can then call o.colByName(...) and sort according to the direction.
trait Sortable {
protected val orderColMap: Map[String, Column[_]]
def colByName(colName: String) = orderColMap.get(colName)
}
still not very elegant, because it forces you to implement a map for each table, but at least saves you the switch/match.

Scala slick query where in list

I am attempting to learn to use Slick to query MySQL. I have the following type of query working to get a single Visit object:
Q.query[(Int,Int), Visit]("""
select * from visit where vistor = ? and location_code = ?
""").firstOption(visitorId,locationCode)
What I would like to know is how can I change the above to query to get a List[Visit] for a collection of Locations...something like this:
val locationCodes = List("loc1","loc2","loc3"...)
Q.query[(Int,Int,List[String]), Visit]("""
select * from visit where vistor = ? and location_code in (?,?,?...)
""").list(visitorId,locationCodes)
Is this possible with Slick?
As the other answer suggests, this is cumbersome to do with static queries. The static query interface requires you to describe the bind parameters as a Product. (Int, Int, String*)
is not valid scala, and using (Int,Int,List[String]) needs some kludges as well. Furthermore, having to ensure that locationCodes.size is always equal to the number of (?, ?...) you have in your query is brittle.
In practice, this is not too much of a problem because you want to be using the query monad instead, which is the type-safe and recommended way to use Slick.
val visitorId: Int = // whatever
val locationCodes = List("loc1","loc2","loc3"...)
// your query, with bind params.
val q = for {
v <- Visits
if v.visitor is visitorId.bind
if v.location_code inSetBind locationCodes
} yield v
// have a look at the generated query.
println(q.selectStatement)
// run the query
q.list
This is assuming you have your tables set up like this:
case class Visitor(visitor: Int, ... location_code: String)
object Visitors extends Table[Visitor]("visitor") {
def visitor = column[Int]("visitor")
def location_code = column[String]("location_code")
// .. etc
def * = visitor ~ .. ~ location_code <> (Visitor, Visitor.unapply _)
}
Note that you can always wrap your query in a method.
def byIdAndLocations(visitorId: Int, locationCodes: List[String]) =
for {
v <- Visits
if v.visitor is visitorId.bind
if v.location_code inSetBind locationCodes
} yield v
}
byIdAndLocations(visitorId, List("loc1", "loc2", ..)) list
It doesn't work because the StaticQuery object (Q) expects to implicitly set the parameters in the query string, using the type parameters of the query method to create a sort of setter object (of type scala.slick.jdbc.SetParameter[T]).
The role of SetParameter[T] is to set a query parameter to a value of type T, where the required types are taken from the query[...] type parameters.
From what I see there's no such object defined for T = List[A] for a generic A, and it seems a sensible choice, since you can't actually write a sql query with a dynamic list of parameters for the IN (?, ?, ?,...) clause
I did an experiment by providing such an implicit value through the following code
import scala.slick.jdbc.{SetParameter, StaticQuery => Q}
def seqParam[A](implicit pconv: SetParameter[A]): SetParameter[Seq[A]] = SetParameter {
case (seq, pp) =>
for (a <- seq) {
pconv.apply(a, pp)
}
}
implicit val listSP: SetParameter[List[String]] = seqParam[String]
with this in scope, you should be able to execute your code
val locationCodes = List("loc1","loc2","loc3"...)
Q.query[(Int,Int,List[String]), Visit]("""
select * from visit where vistor = ? and location_code in (?,?,?...)
""").list(visitorId,locationCodes)
But you must always manually guarantee that the locationCodes size is the same as the number of ? in your IN clause
In the end I believe that a cleaner workaround could be created using macros, to generalize on the sequence type. But I'm not sure it would be a wise choice for the framework, given the aforementioned issues with the dynamic nature of the sequence size.
You can generate in clause automaticly like this:
def find(id: List[Long])(implicit options: QueryOptions) = {
val in = ("?," * id.size).dropRight(1)
Q.query[List[Long], FullCard](s"""
select
o.id, o.name
from
organization o
where
o.id in ($in)
limit
?
offset
?
""").list(id ::: options.limits)
}
And use implicit SetParameter as pagoda_5b says
def seqParam[A](implicit pconv: SetParameter[A]): SetParameter[Seq[A]] = SetParameter {
case (seq, pp) =>
for (a <- seq) {
pconv.apply(a, pp)
}
}
implicit def setLongList = seqParam[Long]
If you have a complex query and the for comprehension mentioned above is not an option, you can do something like the following in Slick 3. But you need to make sure you validate the data in your list query parameter yourself to prevent SQL injection:
val locationCodes = "'" + List("loc1","loc2","loc3").mkString("','") + "'"
sql"""
select * from visit where visitor = $visitor
and location_code in (#$locationCodes)
"""
The # in front of the variable reference disables the type validation and allows you to solve this without supplying a function for the implicit conversion of the list query parameter.

Unpack a NamedColumn in a condition of a for-loop in ScalaQuery

I am quite new to Scala and ScalaQuery, using it a couple of weeks now. I am trying to figure out a condition in a query by calling a function, but I get a NamedColumn[T] instead of T, how to unpack it?
See 2nd link, line 20:
package with typemapper: https://gist.github.com/3469291
table object: https://gist.github.com/3469291
case class MyObject (
id: Long,
created: JodaTime
modified: JodaTime
special: JodaTime
)
object MyObjects extends Table[MyObject]("my_objects") {
lazy val database = Database.forDataSource(DB.getDataSource())
def id = column[Long]("id", O PrimaryKey, O AutoInc, O NotNull)
def created = column[JodaTime]("created", O NotNull)
def modified = column[JodaTime]("modified", O NotNull)
def special = column[JodaTime]("special", O NotNull)
def * = id ~ created <> (MyObject, MyObject.unapply _)
def getMarker(time: JodaTime) = database.withSession { implicit db:Session =>
(for {
e <- MyObjects if (new org.joda.time.Interval(e.created, e.modified).contains(e.special)
} yield (e.id, e.created)).firstOption
}
}
e.created / modified /special are NamedColumns, so the constructor and functioncall won't work. How do I make this work?
I did not test my object, I just grabbed a class and stripped and renamed things, but just to show what I have and want to do.
Thanks.
I think you could tricked by ScalaQueries nice syntax. You can't jut put arbitrary conditions in a for comprehension based on ScalaQuery tables.
The conditions don't get executed by the JVM, but get translated into SQL. This obviously doesn't work for arbitrary Scala code, but only special operations provided by ScalaQuery.
The following version should work:
for {
e <- MyObjects
if (e.created < e.special)
if (e.modified > e.special)
}
Note that I have no idea about the semantics of Interval.contains, so you might have to throw some >= or <= in there.