Troublesome conversion of plain PostgreSQL query to Slick query - postgresql

I have a plain PostgreSQL query I'm having trouble translating into slick query. I go stuck in syntax soup when using groupBy clause.
SELECT u.id AS investor_id,
u.account_type,
i.first_name,
issuer_user.display_name AS issuer_name,
p.legal_name AS product_name,
v.investment_date,
iaa.as_of AS CCO_approval_date,
v.starting_investment_amount,
v.maturity_date,
v.product_interest_rate,
v.product_term_length,
i.user_information_id,
v.id AS investment_id
FROM investors u
JOIN
( SELECT ipi.investor_id,
ipi.first_name,
ipi.user_information_id
FROM investor_personal_information ipi
JOIN
( SELECT investor_id,
MAX(id) AS Max_Id
FROM investor_personal_information
GROUP BY investor_id ) M ON ipi.investor_id = m.investor_id
AND ipi.id = m.Max_Id ) i ON u.id = i.investor_id
JOIN investments v ON u.id = v.investor_id
JOIN sub_products AS sp ON v.sub_product_id = sp.id
JOIN products AS p ON p.id = sp.product_id
JOIN company AS c ON c.id = p.company_id
JOIN issuers AS issuer ON issuer.id = c.issuer_id
JOIN users AS issuer_user ON issuer.owner = issuer_user.id
JOIN investment_admin_approvals AS iaa ON iaa.investment_id = v.id
ORDER BY i.first_name DESC;
I've started writing it
val query = {
val investorInfoQuery = (for {
i <- InvestorPersonalInformation
} yield (i)).groupBy {
_.investorId
}.map {
case (id, rest) => {
id -> rest.map(_.id).max
}
}
}
I know I've to create base queries into one big query and apply joins on them separately. Can anybody help guiding me or providing me some examples? Slick is hard.

Looks pretty simple to write. I am not going to help you write the whole query, I will just give you an example which you can follow to wrtie your query.
Lets say you had following structure and respective table queries defined as employees, emplayeePackages and employeeSalaryCredits
case class Employee(id: String, name: String)
case class EmployeePackage(id: String, employeeId: String, baseSalary: Double)
case class EmployeeSalaryCredit(id: String, employeeId: String, salaryCredited: Double, date: ZonedDateTime)
Now lets say you want all salary credits for all employees with employee's id, employee's name, base salary, actual credited salary and date of salary credit then your query will look like
val queryExample = employees
.join(employeePackages)
.on({ case (e, ep ) => e.id === ep.employeeId })
.join(employeeSalaryCredits)
.on({ case ((e, ep), esc) => e.id === esc.employeeId })
.map({ case ((e, ep), esc) =>
(e.id, e.name, ep.baseSalary, esc.salaryCredited, esc.date)
})

Related

Slick group by count

Let's say I have organizations, each organization has different groups, and users subscribe to groups.
case class OrganizationEntity(id: Option[Int], name: String)
case class GroupEntity(id: Option[Int], organizationId: Int, name: String)
case class GroupUserEntity(groupId: Int, userId: Int)
I need to get all groups of an organization, with the organizationName, and the quantity of users subscribed to that group.
In SQL, this can be easily done with this query:
SELECT g.*, o.organizationname, COUNT(DISTINCT gu.userid) FROM `group` g
LEFT JOIN organization o ON g.orgid = o.organizationid
LEFT JOIN group_user gu ON g.groupid = gu.groupid
WHERE g.orgid = 1234
GROUP BY g.groupid;
But I am struggling to replicate that in slick,
I've started writing this, but I am stuck now:
def findByOrganizationId(organizationId: Int) = {
(for {
g <- groups if g.organizationId === organizationId
o <- organizations if o.id === organizationId
gu <- groupUsers if g.id === gu.groupid
} yield (g, o.name, gu)).groupBy(_._3.groupid).map { case (_, values) => (values.map { case (g, orgname, users) => (g, orgname, users.) } }.result
}
You can just add .length to do the count in your code.
I think should also work directly in the yield so you don't need the groupBy:
def findByOrganizationId(organizationId: Int) = {
(for {
g <- groups if g.organizationId === organizationId
o <- organizations if o.id === organizationId
gu <- groupUsers if g.id === gu.groupid
} yield (g, o.name, gu.length)).result
}

Quill way of doing INSERT INTO ... SELECT FROM

I'm trying to translate simple INSERT INTO...SELECT FROM query into a quote in Quill. First of all, I am failing to find a built-in way to do this, so ended up trying out to use infix query
val rawQuery = quote { (secondTableValues: List[Int]) => {
infix"""
INSERT INTO my_table (second_table_id)
VALUES (
${secondTableValues.map(stid => (SELECT id FROM second_table where id = $stid)).mkString(",")}}
)
""".as[Insert[Any]]
}}
databaseClient.run(rawQuery(List(1,2,3)))
This however does not compile as Quill can't build Ast for the query.
What I ended up doing is have a raw query and not using quotes and run it with executeAction.
So two questions
How would you do INSERT INTO...SELECT FROM in a built-in way?
What is wrong with the infix version above?
import io.getquill._
val ctx = new SqlMirrorContext(MirrorSqlDialect, Literal)
import ctx._
case class Table1(id: Int)
case class Table2(id: Int, name: String)
def contains(list: List[Int]) = {
//SELECT e.id,'hello' FROM Table1 e WHERE e.id IN (?)
val q = quote(
query[Table1].filter(e => liftQuery(list).contains(e.id)).map(e => Table2(e.id, "hello"))
)
//insert into table2 SELECT e.id, 'hello' FROM Table1 e WHERE e.id IN (?)
// `${..}` is expect ast , not string
quote(infix"insert into table2 ${q}".as[Insert[Any]])
}
// test
val list = List(1)
ctx.run(contains(list))

Group by in many-to-many join with Quill

I am trying to achieve with Quill what the following PostgreSQL query does:
select books.*, array_agg(authors.name) from books
join authors_books on(books.id = authors_books.book_id)
join authors on(authors.id = authors_books.author_id)
group by books.id
For now I have this in my Quill version:
val books = quote(querySchema[Book]("books"))
val authorsBooks = quote(querySchema[AuthorBook]("authors_books"))
val authors = quote(querySchema[Author]("authors"))
val q: db.Quoted[db.Query[(db.Query[Book], Seq[String])]] = quote{
books
.join(authorsBooks).on(_.id == _.book_id)
.join(authors).on(_._2.author_id == _.id)
.groupBy(_._1._1.id)
.map {
case (bId, q) => {
(q.map(_._1._1), unquote(q.map(_._2.name).arrayAgg))
}
}
}
How can I get rid of the nested query in the result (db.Query[Book]) and get a Book instead?
I might be a little bit rusty with SQL but are you sure that your query is valid? Particularly I find suspicious that you do select books.* while group by books.id i.e. you directly return fields that you didn't group by. And attempt to translate that wrong query directly is what makes things go wrong
One way to fix it is to do group by by all fields. Assuming Book is declared as:
case class Book(id: Int, name: String)
you can do
val qq: db.Quoted[db.Query[((Index, String), Seq[String])]] = quote {
books
.join(authorsBooks).on(_.id == _.book_id)
.join(authors).on(_._2.author_id == _.id)
.groupBy(r => (r._1._1.id, r._1._1.name))
.map {
case (bId, q) => {
// (Book.tupled(bId), unquote(q.map(_._2.name).arrayAgg)) // doesn't work
(bId, unquote(q.map(_._2.name).arrayAgg))
}
}
}
val res = db.run(qq).map(r => (Book.tupled(r._1), r._2))
Unfortunately it seems that you can't apply Book.tupled inside quote because you get error
The monad composition can't be expressed using applicative joins
but you can easily do it after db.run call to get back your Book.
Another option is to do group by just Book.id and then join the Book table again to get all the fields back. This might be actually cleaner and faster if there are many fields inside Book

EF code How to use lambda expresssion to implement left join?

Sql:
select a.id, b.name
from a
left join b on a.id = b.id
I want to use lambda in EF to get the same result in this sql
Here is what I've done:
var list = entities.a
.GroupJoin(
entities.b,
a => a.id,
b => b.id,
(a, b) => new { a, b })
.Select(o => o)
.ToList();
Here Select(o => o), I just don't know how to get the same result in sql
select a.id, b.name
I think this article from MSDN would be a lot helpful...
http://msdn.microsoft.com/en-us/library/bb397895.aspx
This is a quote from above article, that might help...
var query = from person in people
join pet in pets on person equals pet.Owner into gj
from subpet in gj.DefaultIfEmpty()
select new { person.FirstName, PetName = (subpet == null ? String.Empty : subpet.Name) };
UPDATE:
As per your request, the Lambda Expression would be something like this...
.SelectMany(#a => #a.#a.b.DefaultIfEmpty(), (#a, joineda) => new {#a, joineda})
Not sure, if it is correct, but it is a starting point, at least...

Recursive tree-like table query with Slick

My table data forms a tree structure where one row can reference a parent row in the same table.
What I am trying to achieve, using Slick, is to write a query that will return a row and all it's children. Also, I would like to do the same, but write a query that will return a child and all it's ancestors.
In other words:
findDown(1) should return
List(Group(1, 0, "1"), Group(3, 1, "3 (Child of 1)"))
findUp(5) should return
List(Group(5, 2, "5 (Child of 2)"), Group(2, 0, "2"))
Here is a fully functional worksheet (except for the missing solutions ;-).
package com.exp.worksheets
import scala.slick.driver.H2Driver.simple._
object ParentChildTreeLookup {
implicit val session = Database.forURL("jdbc:h2:mem:test1;", driver = "org.h2.Driver").createSession()
session.withTransaction {
Groups.ddl.create
}
Groups.insertAll(
Group(1, 0, "1"),
Group(2, 0, "2"),
Group(3, 1, "3 (Child of 1)"),
Group(4, 3, "4 (Child of 3)"),
Group(5, 2, "5 (Child of 2)"),
Group(6, 2, "6 (Child of 2)"))
case class Group(
id: Long = -1,
id_parent: Long = -1,
label: String = "")
object Groups extends Table[Group]("GROUPS") {
def id = column[Long]("ID", O.PrimaryKey, O.AutoInc)
def id_parent = column[Long]("ID_PARENT")
def label = column[String]("LABEL")
def * = id ~ id_parent ~ label <> (Group, Group.unapply _)
def autoInc = id_parent ~ label returning id into {
case ((_, _), id) => id
}
def findDown(groupId: Long)(implicit session: Session) = { ??? }
def findUp(groupId: Long)(implicit session: Session) = { ??? }
}
}
A really bad, and static attempt at findDown may be something like:
private def groupsById = for {
group_id <- Parameters[Long]
g <- Groups; if g.id === group_id
} yield g
private def childrenByParentId = for {
parent_id <- Parameters[Long]
g <- Groups; if g.id_parent === parent_id
} yield g
def findDown(groupId: Long)(implicit session: Session) = { groupsById(groupId).list union childrenByParentId(groupId).list }
But, I'm looking for a way for Slick to recursively search the same table using the id and id_parent link. Any other good ways to solve the problem is really welcome. Keep in mind though, that it would be best to minimise the number of database round-trips.
You could try calling SQL from slick. The SQL call to go up the hierarchy would look something like this (This is for SQL Server):
WITH org_name AS
(
SELECT DISTINCT
parent.id AS parent_id,
parentname.label as parent_label,
child.id AS child_id,
childname.ConceptName as child_label
FROM
Group parent RIGHT OUTER JOIN
Group child ON child.parent_id = parent.id
),
jn AS
(
SELECT
parent_id,
parent_label,
child_id,
child_label
FROM
org_name
WHERE
parent_id = 5
UNION ALL
SELECT
C.parent_id,
C.parent_label,
C.child_id,
C.child_label
FROM
jn AS p JOIN
org_name AS C ON C.child_id = p.parent_id
)
SELECT DISTINCT
jn.parent_id,
jn.parent_label,
jn.child_id,
jn.child_label
FROM
jn
ORDER BY
1;
If you want to go down the hierarchy change the line:
org_name AS C ON C.child_id = p.parent_id
to
org_name AS C ON C.parent_id = p.child_id
In plain SQL this would be tricky. You would have multiple options:
Use a stored procedure to collect the correct records (recursively). Then convert those records into a tree using code
Select all the records and convert those into a tree using code
Use more advanced technique as described here (from Optimized SQL for tree structures) and here. Then convert those records into a tree using code
Depending on the way you want to do it in SQL you need to build a Slick query. The concept of Leaky Abstractions is very evident here.
So getting the tree structure requires two steps:
Get the correct (or all records)
Build (using regular code) a tree from those records
Since you are using Slick I don't think it's an option, but another database type might be a better fit for your data model. Check out NoSQL for the differences between the different types.