Slick group by count - scala

Let's say I have organizations, each organization has different groups, and users subscribe to groups.
case class OrganizationEntity(id: Option[Int], name: String)
case class GroupEntity(id: Option[Int], organizationId: Int, name: String)
case class GroupUserEntity(groupId: Int, userId: Int)
I need to get all groups of an organization, with the organizationName, and the quantity of users subscribed to that group.
In SQL, this can be easily done with this query:
SELECT g.*, o.organizationname, COUNT(DISTINCT gu.userid) FROM `group` g
LEFT JOIN organization o ON g.orgid = o.organizationid
LEFT JOIN group_user gu ON g.groupid = gu.groupid
WHERE g.orgid = 1234
GROUP BY g.groupid;
But I am struggling to replicate that in slick,
I've started writing this, but I am stuck now:
def findByOrganizationId(organizationId: Int) = {
(for {
g <- groups if g.organizationId === organizationId
o <- organizations if o.id === organizationId
gu <- groupUsers if g.id === gu.groupid
} yield (g, o.name, gu)).groupBy(_._3.groupid).map { case (_, values) => (values.map { case (g, orgname, users) => (g, orgname, users.) } }.result
}

You can just add .length to do the count in your code.
I think should also work directly in the yield so you don't need the groupBy:
def findByOrganizationId(organizationId: Int) = {
(for {
g <- groups if g.organizationId === organizationId
o <- organizations if o.id === organizationId
gu <- groupUsers if g.id === gu.groupid
} yield (g, o.name, gu.length)).result
}

Related

Scala Slick joinLeft and combined conditions

I want to be able to create a query with Slick that let me filter left joins in a dynamic way
case class Player(
id: Long,
createdAt: DateTime,
lastModificationDate: DateTime,
name: String
)
class PlayerTable(tag: Tag) extends Table[Player](tag, "players") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def createdAt = column[DateTime]("createdAt")
def lastModificationDate = column[DateTime]("lastModificationDate")
def name = column[String]("name")
override def * : ProvenShape[Player] = (
id,
createdAt,
lastModificationDate,
updatedAt,
name
) <> (Player.tupled, Player.unapply)
}
case class PlayerGame(
id: Long,
createdAt: DateTime,
lastModificationDate: DateTime,
playerId: Long,
level: Int,
status: String
)
class PlayerGameTable(tag: Tag) extends Table[PlayerGame](tag, "player_games") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def createdAt = column[DateTime]("createdAt")
def lastModificationDate = column[DateTime]("lastModificationDate")
def playerId = column[Long]("playerId")
def level = column[Int]("level")
def status = column[String]("status")
override def * : ProvenShape[PlayerGame] = (
id,
createdAt,
lastModificationDate,
playerId,
level,
status
) <> (PlayerGame.tupled, PlayerGame.unapply)
}
I want to write a query like this with Slick, where the WHERE CLAUSE is dynamic. I wrote two examples
SELECT *
FROM players
LEFT JOIN player_games AS playerGamesOne ON players.id = playerGamesOne.playerId AND playerGamesOne.level = 1
LEFT JOIN player_games AS playerGamesTwo ON players.id = playerGamesTwo.playerId AND playerGamesTwo.level = 2
WHERE playerGamesOne.status LIKE 'gameOver'
OR playerGamesTWO.status LIKE 'gameOver'
SELECT *
FROM players
LEFT JOIN player_games AS playerGamesOne ON players.id = playerGamesOne.playerId AND playerGamesOne.level = 1
LEFT JOIN player_games AS playerGamesTwo ON players.id = playerGamesTwo.playerId AND playerGamesTwo.level = 2
WHERE playerGamesOne.status LIKE 'playing'
OR playerGamesTwo.status NOT LIKE 'gameOver'
I was trying something like this, but I get Rep[Option[PlayerGameTable]] as the parameter. Maybe there is a different way of doing something like this
val baseQuery = for {
((p, g1), g2) <- PlayerTable.playerQuery joinLeft
PlayerGameTable.playerGameQuery ON ((x, y) => x.id === y.playerId && y.level === 1) joinLeft
PlayerGameTable.playerGameQuery ON ((x, y) => x._1.id === y.playerId && y.level === 2)
} yield (p, g1, g2)
private def filterPlayerGames(gameStatus: String, playerGamesOneOpt: Option[PlayerGameTable], playerGamesTwoOpt: Option[PlayerGameTable]) = {
(gameStatus, playerGamesOneOpt, playerGamesOneOpt) match {
case (gameStatus: String, Some(playerGamesOne: PlayerGameTable), Some(playerGamesOne: PlayerGameTable)) if gameStatus == "gameOver" => playerGamesOne.status === "gameOver" || playerGamesTwo.status === "gameOver"
}
}
It is a complex question, if soemthing is not clear please let me know and I will try to clarify it
There are a couple of issues:
With multiple conditions, the underscore placeholder used within your ON clause would not work the way intended
_.level = something is an assignment, not a condition
Assuming PlayerTable.playerQuery is TableQuery[PlayerTable] and PlayerGameTable.playerGameQuery is TableQuery[PlayerGameTable], your baseQuery should look like this:
val baseQuery = for {
((p, g1), g2) <- PlayerTable.playerQuery joinLeft
PlayerGameTable.playerGameQuery on ((x, y) => x.id === y.playerId && y.level === 1) joinLeft
PlayerGameTable.playerGameQuery on ((x, y) => x._1.id === y.playerId && y.level === 2)
} yield (p, g1, g2)
It's not entirely clear to me how your filterPlayerGames method is going to handle dynamic conditions. Nor do I think any filtering wrapper method will be flexible enough to cover multiple conditions with arbitrary and/or/negation operators. I would suggest that you use the baseQuery for the necessary joins and build filtering queries on top of it, similar to something like below:
val query1 = baseQuery.filter{ case (_, g1, g2) =>
g1.filter(_.status === "gameOver").isDefined || g2.filter(_.status === "gameOver").isDefined
}
val query2 = baseQuery.filter{ case (_, g1, g2) =>
g1.filter(_.status === "playing").isDefined || g2.filter(_.status =!= "gameOver").isDefined
}
Note that with the left joins, g1 and g2 are of Option type, thus isDefined is applied for the or operation.
On a separate note, given that your filtering conditions are only on PlayerGameTable, it would probably be more efficient to perform filtering before the joins.

Slick one to many and grouping

I'm trying to model the following with Slick 3.1.0;
case class Review(txt: String, userId: Long, id: Long)
case class User(name: String, id: Long)
case class ReviewEvent(event: String, reviewId: Long)
I need to populate a class called a FullReview, which looks like;
case class FullReview(r: Review, user: User, evts: Seq[ReviewEvent])
Assuming I have the right tables for each of the models, I'm trying to fetch a FullReview using a combination of join and group by, like so:
val withUser = for {
(r, u) <- RTable join UTable on (_.userId === _.id)
}
val withUAndEvts = (for {
((r, user), evts) <- withUser joinLeft ETable on {
case ((r, _), ev) => r.id === ev.reviewId
}
} yield (r, user, events)).groupBy(_._1._id)
This seems to yield, when a nested Query type, from what I can see. What am I doing wrong here?
If I understand you correctly, you can use following example:
val users = TableQuery[Users]
val reviews = TableQuery[Reviews]
val events = TableQuery[ReviewEvents]
override def findAllReviews(): Future[Seq[FullReview]] = {
val query = reviews
.join(users).on(_.userId === _.id)
.joinLeft(events).on(_._1.id === _.reviewId)
db.run(query.result).map { a =>
a.groupBy(_._1._1.id).map { case (_, tuples) =>
val ((review, user), _) = tuples.head
val reviewEvents = tuples.flatMap(_._2)
FullReview(review, user, reviewEvents)
}.toSeq
}
}
If you want to add pagination to this request, I've already answered here and here is full example.
From some tinkering around, I figured it would just be better to do the aggregation on the client. What that would mean, indirectly, is that if 100 rows on the table ETable would match a single row on the RTable, you would get multiple rows on the client. The client then has to implement its own aggregation to group all the ReviewEvent by Review.
As far as pagination is concerned, you may do something like;
def withUser(page: Int, pageSize: Int) = for {
(r, u) <- RTable.drop(page * pageSize).take(pageSize) join UTable on (_.userId === _.id)
}
I guess this is elegant enough for now. If someone has a better answer, I'd be happy to hear it.

Troublesome conversion of plain PostgreSQL query to Slick query

I have a plain PostgreSQL query I'm having trouble translating into slick query. I go stuck in syntax soup when using groupBy clause.
SELECT u.id AS investor_id,
u.account_type,
i.first_name,
issuer_user.display_name AS issuer_name,
p.legal_name AS product_name,
v.investment_date,
iaa.as_of AS CCO_approval_date,
v.starting_investment_amount,
v.maturity_date,
v.product_interest_rate,
v.product_term_length,
i.user_information_id,
v.id AS investment_id
FROM investors u
JOIN
( SELECT ipi.investor_id,
ipi.first_name,
ipi.user_information_id
FROM investor_personal_information ipi
JOIN
( SELECT investor_id,
MAX(id) AS Max_Id
FROM investor_personal_information
GROUP BY investor_id ) M ON ipi.investor_id = m.investor_id
AND ipi.id = m.Max_Id ) i ON u.id = i.investor_id
JOIN investments v ON u.id = v.investor_id
JOIN sub_products AS sp ON v.sub_product_id = sp.id
JOIN products AS p ON p.id = sp.product_id
JOIN company AS c ON c.id = p.company_id
JOIN issuers AS issuer ON issuer.id = c.issuer_id
JOIN users AS issuer_user ON issuer.owner = issuer_user.id
JOIN investment_admin_approvals AS iaa ON iaa.investment_id = v.id
ORDER BY i.first_name DESC;
I've started writing it
val query = {
val investorInfoQuery = (for {
i <- InvestorPersonalInformation
} yield (i)).groupBy {
_.investorId
}.map {
case (id, rest) => {
id -> rest.map(_.id).max
}
}
}
I know I've to create base queries into one big query and apply joins on them separately. Can anybody help guiding me or providing me some examples? Slick is hard.
Looks pretty simple to write. I am not going to help you write the whole query, I will just give you an example which you can follow to wrtie your query.
Lets say you had following structure and respective table queries defined as employees, emplayeePackages and employeeSalaryCredits
case class Employee(id: String, name: String)
case class EmployeePackage(id: String, employeeId: String, baseSalary: Double)
case class EmployeeSalaryCredit(id: String, employeeId: String, salaryCredited: Double, date: ZonedDateTime)
Now lets say you want all salary credits for all employees with employee's id, employee's name, base salary, actual credited salary and date of salary credit then your query will look like
val queryExample = employees
.join(employeePackages)
.on({ case (e, ep ) => e.id === ep.employeeId })
.join(employeeSalaryCredits)
.on({ case ((e, ep), esc) => e.id === esc.employeeId })
.map({ case ((e, ep), esc) =>
(e.id, e.name, ep.baseSalary, esc.salaryCredited, esc.date)
})

Scala Slick 2 join on multiple fields?

how can do joins on multiple fields like in example beneath?
val ownerId = 1
val contactType = 1
...
val contact = for {
(t, c) <- ContactTypes leftJoin Contacts on (_.id === _.typeId && _.ownerId === ownerId)
if t.id === contactType
} yield (c.?, t)
How can I achieve this with Slick 2.0.1? Idelly I need slick to generate this kind of query
SELECT
x2."contact_id",
x2."type_id",
x2."owner_id",
x2."value",
x2."created_on",
x2."updated_on",
x3."id",
x3."type",
x3."model"
FROM
(
SELECT
x4."id" AS "id",
x4."type" AS "type",
x4."model" AS "model"
FROM
"contact_types" x4
)x3
LEFT OUTER JOIN(
SELECT
x5."created_on" AS "created_on",
x5."value" AS "value",
x5."contact_id" AS "contact_id",
x5."updated_on" AS "updated_on",
x5."type_id" AS "type_id",
x5."owner_id" AS "owner_id"
FROM
"contacts" x5
)x2 ON x3."id" = x2."type_id" AND x2.owner_id = 1
WHERE
(x3."id" = 3)
Please note ON x3."id" = x2."type_id" AND x2.owner_id = 16
Ok, so after digging through websites and source code I think I finally found the solution
leftJoin on() method accepts following parameter pred: (E1, E2) => T, so we simply can do like this
val contacts = for {
(t, c) <- ContactTypes leftJoin Contacts on ( (type, contact) => {
type.id === contact.typeId && contact.ownerId === ownerId
} )
} yield (c.?, t)
Which generated sql query as needed.

Scala Slick: Issues with groupBy and missing shapes

I'm trying to use Slick to query a many-to-many relationship, but I'm running into a variety of errors, the most prominent being "Don't know how to unpack (User, Skill) to T and pack to G".
The structure of the tables is similar to the following:
case class User(val name: String, val picture: Option[URL], val id: Option[UUID])
object Users extends Table[User]("users") {
def name = column[String]("name")
def picture = column[Option[URL]]("picture")
def id = column[UUID]("id")
def * = name ~ picture ~ id.? <> (User, User.unapply _)
}
case class Skill(val name: String, val id: Option[UUID])
object Skills extends Table[Skill]("skill") {
def name = column[String]("name")
def id = column[UUID]("id")
def * = name ~ id.? <> (Skill, Skill.unapply _)
}
case class UserSkill(val userId: UUID, val skillId: UUID, val id: Option[UUID])
object UserSkills extends Table[UserSkill]("user_skill") {
def userId = column[UUID]("userId")
def skillId = column[UUID]("skillId")
def id = column[UUID]("id")
def * = userId ~ skillId ~ id.? <> (UserSkill, UserSkill.unapply _)
def user = foreignKey("userFK", userId, Users)(_.id)
def skill = foreignKey("skillFK", skillId, Skills)(_.id)
}
Ultimately, what I want to achieve is something of the form
SELECT u.*, group_concat(s.name) FROM user_skill us, users u, skills s WHERE us.skillId = s.id && us.userId = u.id GROUP BY u.id
but before I spend the time trying to get group_concat to work as well, I have been trying to produce the simpler query (which I believe is still valid...)
SELECT u.* FROM user_skill us, users u, skills s WHERE us.skillId = s.id && us.userId = u.id GROUP BY u.id
I've tried a variety of scala code to produce this query, but an example of what causes the shape error above is
(for {
us <- UserSkills
user <- us.user
skill <- us.skill
} yield (user, skill)).groupBy(_._1.id).map { case(_, xs) => xs.first }
Similarly, the following produces a packing error regarding "User" instead of "(User, Skill)"
(for {
us <- UserSkills
user <- us.user
skill <- us.skill
} yield (user, skill)).groupBy(_._1.id).map { case(_, xs) => xs.map(_._1).first }
If anyone has any suggestions, I would be very grateful: I've spent most of today and yesterday scouring google/google groups as well as the slick source, but I haven't a solution yet.
(Also, I'm using postgre so group_concat would actually be string_agg)
EDIT
So it seems like when groupBy is used, the mapped projection gets applied because something like
(for {
us <- UserSkills
u <- us.user
s <- us.skill
} yield (u,s)).map(_._1)
works fine because _._1 gives the type Users, which has a Shape since Users is a table. However, when we call xs.first (as we do when we call groupBy), we actually get back a mapped projection type (User, Skill), or if we apply map(_._1) first, we get the type User, which is not Users! As far as I can tell, there is no shape with User as the mixed type because the only shapes defined are for Shape[Column[T], T, Column[T]] and for a table T <: TableNode, Shape[T, NothingContainer#TableNothing, T] as defined in slick.lifted.Shape. Furthermore, if I do something like
(for {
us <- UserSkills
u <- us.user
s <- us.skill
} yield (u,s))
.groupBy(_._1.id)
.map { case (_, xs) => xs.map(_._1.id).first }
I get a strange error of the form "NoSuchElementException: key not found: #1515100893", where the numeric key value changes each time. This is not the query I want, but it is a strange issue none the less.
I've run up against similar situations as well. While I love working with Scala and Slick, I do believe there are times when it is easier to denormalize an object in the database itself and link the Slick Table to a view.
For example, I have an application that has a Tree object that is normalized into several database tables. Since I'm comfortable with SQL, I think it is a cleaner solution than writing a plain Scala Slick query. The Scala code:
case class DbGFolder(id: String,
eTag: String,
url: String,
iconUrl: String,
title: String,
owner: String,
parents: Option[String],
children: Option[String],
scions: Option[String],
created: LocalDateTime,
modified: LocalDateTime)
object DbGFolders extends Table[DbGFolder]("gfolder_view") {
def id = column[String]("id")
def eTag = column[String]("e_tag")
def url = column[String]("url")
def iconUrl = column[String]("icon_url")
def title = column[String]("title")
def owner = column[String]("file_owner")
def parents = column[String]("parent_str")
def children = column[String]("child_str")
def scions = column[String]("scion_str")
def created = column[LocalDateTime]("created")
def modified = column[LocalDateTime]("modified")
def * = id ~ eTag ~ url ~ iconUrl ~ title ~ owner ~ parents.? ~
children.? ~ scions.? ~ created ~ modified <> (DbGFolder, DbGFolder.unapply _)
def findAll(implicit s: Session): List[GFolder] = {
Query(DbGFolders).list().map {v =>
GFolder(id = v.id,
eTag = v.eTag,
url = v.url,
iconUrl = v.iconUrl,
title = v.title,
owner = v.owner,
parents = v.parents.map { parentStr =>
parentStr.split(",").toSet }.getOrElse(Set()),
children = v.children.map{ childStr =>
childStr.split(",").toSet }.getOrElse(Set()),
scions = v.scions.map { scionStr =>
scionStr.split(",").toSet }.getOrElse(Set()),
created = v.created,
modified = v.modified)
}
}
}
And the underlying (postgres) view:
CREATE VIEW scion_view AS
WITH RECURSIVE scions(id, scion) AS (
SELECT c.id, c.child
FROM children AS c
UNION ALL
SELECT s.id, c.child
FROM children AS c, scions AS s
WHERE c.id = s.scion)
SELECT * FROM scions ORDER BY id, scion;
CREATE VIEW gfolder_view AS
SELECT
f.id, f.e_tag, f.url, f.icon_url, f.title, m.name, f.file_owner,
p.parent_str, c.child_str, s.scion_str, f.created, f.modified
FROM
gfiles AS f
JOIN mimes AS m ON (f.mime_type = m.name)
LEFT JOIN (SELECT DISTINCT id, string_agg(parent, ',' ORDER BY parent) AS parent_str
FROM parents GROUP BY id) AS p ON (f.id = p.id)
LEFT JOIN (SELECT DISTINCT id, string_agg(child, ',' ORDER BY child) AS child_str
FROM children GROUP BY id) AS c ON (f.id = c.id)
LEFT JOIN (SELECT DISTINCT id, string_agg(scion, ',' ORDER BY scion) AS scion_str
FROM scion_view GROUP BY id) AS s ON (f.id = s.id)
WHERE
m.category = 'folder';
Try this. Hope it may yield what you expected. Find the Slick Code below the case classes.
click here for the reference regarding lifted embedding .
case class User(val name: String, val picture: Option[URL], val id: Option[UUID])
class Users(_tableTag: Tag) extends Table[User](_tableTag,"users") {
def name = column[String]("name")
def picture = column[Option[URL]]("picture")
def id = column[UUID]("id")
def * = name ~ picture ~ id.? <> (User, User.unapply _)
}
lazy val userTable = new TableQuery(tag => new Users(tag))
case class Skill(val name: String, val id: Option[UUID])
class Skills(_tableTag: Tag) extends Table[Skill](_tableTag,"skill") {
def name = column[String]("name")
def id = column[UUID]("id")
def * = name ~ id.? <> (Skill, Skill.unapply _)
}
lazy val skillTable = new TableQuery(tag => new Skills(tag))
case class UserSkill(val userId: UUID, val skillId: UUID, val id: Option[UUID])
class UserSkills(_tableTag: Tag) extends Table[UserSkill](_tableTag,"user_skill") {
def userId = column[UUID]("userId")
def skillId = column[UUID]("skillId")
def id = column[UUID]("id")
def * = userId ~ skillId ~ id.? <> (UserSkill, UserSkill.unapply _)
def user = foreignKey("userFK", userId, Users)(_.id)
def skill = foreignKey("skillFK", skillId, Skills)(_.id)
}
lazy val userSkillTable = new TableQuery(tag => new UserSkills(tag))
(for {((userSkill, user), skill) <- userSkillTable join userTable.filter on
(_.userId === _.id) join skillTable.filter on (_._1.skillId === _.id)
} yield (userSkill, user, skill)).groupBy(_.2.id)