Group by in many-to-many join with Quill - postgresql

I am trying to achieve with Quill what the following PostgreSQL query does:
select books.*, array_agg(authors.name) from books
join authors_books on(books.id = authors_books.book_id)
join authors on(authors.id = authors_books.author_id)
group by books.id
For now I have this in my Quill version:
val books = quote(querySchema[Book]("books"))
val authorsBooks = quote(querySchema[AuthorBook]("authors_books"))
val authors = quote(querySchema[Author]("authors"))
val q: db.Quoted[db.Query[(db.Query[Book], Seq[String])]] = quote{
books
.join(authorsBooks).on(_.id == _.book_id)
.join(authors).on(_._2.author_id == _.id)
.groupBy(_._1._1.id)
.map {
case (bId, q) => {
(q.map(_._1._1), unquote(q.map(_._2.name).arrayAgg))
}
}
}
How can I get rid of the nested query in the result (db.Query[Book]) and get a Book instead?

I might be a little bit rusty with SQL but are you sure that your query is valid? Particularly I find suspicious that you do select books.* while group by books.id i.e. you directly return fields that you didn't group by. And attempt to translate that wrong query directly is what makes things go wrong
One way to fix it is to do group by by all fields. Assuming Book is declared as:
case class Book(id: Int, name: String)
you can do
val qq: db.Quoted[db.Query[((Index, String), Seq[String])]] = quote {
books
.join(authorsBooks).on(_.id == _.book_id)
.join(authors).on(_._2.author_id == _.id)
.groupBy(r => (r._1._1.id, r._1._1.name))
.map {
case (bId, q) => {
// (Book.tupled(bId), unquote(q.map(_._2.name).arrayAgg)) // doesn't work
(bId, unquote(q.map(_._2.name).arrayAgg))
}
}
}
val res = db.run(qq).map(r => (Book.tupled(r._1), r._2))
Unfortunately it seems that you can't apply Book.tupled inside quote because you get error
The monad composition can't be expressed using applicative joins
but you can easily do it after db.run call to get back your Book.
Another option is to do group by just Book.id and then join the Book table again to get all the fields back. This might be actually cleaner and faster if there are many fields inside Book

Related

How to group results with Slick

Let's say I have those two models:
case class Department(id: Long, title: String)
case class Program(id: Long, departmentId: Long, title: String)
And two TableQuery, departments and programs, based on Table mapped to those case classes respectively.
I would like to make a query returning a Seq[(Department, Seq[Program])], where I have a list of departments with their corresponding programs.
I started like this:
val query =
(departments join programs on ((d, p) => d.id === p.departmentId))
.groupBy {...}
But what ever I put in the group by clause just doesn't make sense.
please help.
#osehyum, thanks for your answer. Your query is returning this:
Map[Long, Seq[(Department, Program)]]
The Long here is for the Department Id.
I managed to turn it with this:
val query = departments.joinLeft(programs).on(_.id === _.departmentId).result
.map(_.groupBy(_._1).toSeq)
.map(items => items.map { case (dep, rows) =>
(dep, rows.map(_._2).filter(_.isDefined).map(_.get))
})
Note that I used joinLeft this time, so Program became Option[Program], so I had to filter and map. It is tested.
How about this?
val query = departments.join(programs).on(_.id === _.departmentId)
.result
.map(_.groupBy(_._1.id))
db.run(query)

How to filter on an optional table produced by a left join in slick

I need to apply a filter on an attribute of an optional table produced by a left join in scala slick. I could not find any documentation on this or any similar questions online.
Consider the following query:
val query = FirstTable joinLeft SecondTable on (_.foreignId === _.id)
I would like to filter is by an attribute of the SecondTable:
query.filter {
case (firstTable, secondTableOpt) => secondTableOpt.attribute === "value"
}
Obviously this does not compile since secondTableOpt is a Rep[Option[SecondTable]]. There does not seem to be a .get method on the Rep object.
There should be a way to write this in slick, does anyone know how to achieve this?
Thank you
As you need to filter the results in your SecondTable in the result, it is better if you do it before the left join. So the code will be something like this:
val filteredSecondTable = SecondTable.filter(_.attribute === "value")
val query = FirstTable joinLeft filteredSecondTable on (_.foreignId === _.id)

Slick 3.0.0: How to query one-to-many / many-to-many relations

Basically the same question has been asked about a year ago for slick 2.x (scala slick one-to-many collections). I'm wondering if there has any progression been made with the release of reactive slick.
Let's say for example we have three tables. library, book and library_to_book where a library has many books. What I want is a list of libraries with their books. In scala this would be something like Seq[(Library, Seq[Book])]. The query I have is as follows:
val q = (for {
l <- libraries
ltb <- libraryToBooks if l.id === ltb.libraryId
b <- books if ltb.bookId === b.id
} yield (l, b)
db.run(q.result).map( result => ??? )
results in this case is of type Seq[(Library, Book)]. How do I have to change my query to get a result of type Seq[(Library, Seq[Book])] instead? What is the "slick way" of writing such queries?
IMO your code looks fine. It really depends on what feels more readable to you. Alternatively, you can use join as well:
val findBooksQuery = libraries
.join(libraryToBooks).on(_.id === _.libraryId)
.join(books).on(_.id === _._2.bookId)
.result
val action = (for {
booksResult <- findBooksQuery
} yield {
booksResult.map { row =>
val (libraryTableRow, libraryToBooksTableRow) = row._1
val booksTableRow = row._2
// TODO: Access all data from the rows and construct desired DS
}
}
db.run(action)
You can then do a groupBy on a particular key to get the kind of data structure you are looking for. In this case, it would be more evolved as it is join across three tables. Example, add following to your query:
val findBooksQuery = libraries
.join(libraryToBooks).on(_.id === _.libraryId)
.join(books).on(_.id === _._2.bookId)
// To group by libraries.id
.groupBy(_._1.id)
.result
To what you want to map to, db.run returns a Future(of something), a Future[Seq[(Library, Seq[Book])]] in your case. When mapping over a future you have access to the Seq and you can transform it to something else to get a new Future.

Dynamically created slick queries

I have three tables: users, groups and users_groups. There is a many to many relation from groups and users because one user can belong in multiple groups and a group is constituted by multiple users.
I have a GET query like /group?name=X&user=Y
From that I'm searching from groups with name like X, but the tricky part is searching for the groups which the Y user doesnt belong.
def findUserGroups(id: Long) = {
users_groups.filter(ug => ug.userID === id)
}
From that I get all the groups which the user belongs, then I do this
var queries : List[Query[GroupsTable, GroupsTable#TableElementType, Seq]]= List[Query[GroupsTable, GroupsTable#TableElementType, Seq]]()
userGroups map { userGroup =>
val query : Query[GroupsTable, GroupsTable#TableElementType, Seq] = groups.filter(_.id =!= userGroup.group.get)
queries = query :: queries
}
If I println userGroup it gives me the correct groups.
Finally I've been trying a union
def findGroupByNameSynthFunction(name: String, queries: List[Query[GroupsTable, GroupsTable#TableElementType, Seq]]) = {
val query1 = groups.filter(g => g.name like ("%" + name + "%"))
val unionQuery: Query[GroupsTable, GroupsTable#TableElementType, Seq] = query1
queries map { query =>
unionQuery ++ query
}
unionQuery
}
I execute it
val found = GroupsTable.findGroupByNameSynthFunction(name, queries).run
But I get all groups anyway.
Can Someone explained me what I'm doing very wrong!? :)
I am not sure where exactly you had the wrong expectation, but it probably revolves around the expression
queries map { query =>
unionQuery ++ query
}
This doesn't have any side-effect. Unless you are doing something with the resulting value (and you are not) this doesn't do anything. You probably want something like
def findGroupByNameSynthFunction(
name: String,
queries: List[Query[GroupsTable, GroupsTable#TableElementType, Seq]]
) = queries.map(_.filter(_.name like ("%" + name + "%")))
.reduce(_ union _)

Slick/Scala - how do I access fields of the mapped projection/projected table part of a join in a where query

I have a number of basic queries define, and am using query composition to add stuff such as ordering, paging, where clauses and so on...
But I have a problem accessing the fields of the joined 2nd table in the where clause...
Here's my table queries and my table. All tables are mapped to case classes.
val basicCars = TableQuery[CarTable]
val basicCarValues = TableQuery[CarValueTable]
val carsWithValues = for {
(c, v) <- basicCars leftJoin basicCarValues on (_.id === _.carId)
} yield (c, v.?)
Now I reuse/compose queries by doing stuff such as
carsWithValues.where(_._1.id === someId)
which works perfectly...
But if I want to access any value of the 2nd table... and I try
carsWithValues.where(_._2.latestPrice === somePrice)
It tells me that somePrice is not a member of MappedProjection......
error: value somePrice is not a member of scala.slick.lifted.MappedProjection[Option[com......datastore.slick.generated.Tables.CarValue],(Option[Long], Option[Long], Option[String],.....
I understand that this kind of can't work, cause _._2 is a MappedProjection and not just a CarValue sitting in the tuple..
But I can't figure out how to use any field of the table that is in the MappedProjection in a where clause?
The .? from the Slick code generator is implemented using a MappedProjection, which doesn't have the members anymore. If you postpone the call to .? it works:
val carsWithValues = for {
(c, v) <- basicCars leftJoin basicCarValues on (_.id === _.carId)
} yield (c, v)
carsWithValues.where(_._2.latestPrice === somePrice).map{ case (c,v) => (c,v.?) }