I have a database that contain activities with a one-to-many registrations relation.
The goal is to get all activities, with a list of their registrations.
By creating a cartesian product of activities with registrations, all necessary data to get that data is out is there.
But I can't seem to find a nice way to get it into a scala collection properly;
let's of type: Seq[(Activity, Seq[Registration])]
case class Registration(
id: Option[Int],
user: Int,
activity: Int
)
case class Activity(
id: Option[Int],
what: String,
when: DateTime,
where: String,
description: String,
price: Double
)
Assuming the appropriate slick tables and tablequeries exist, I would write:
val acts_regs = (for {
a <- Activities
r <- Registrations if r.activityId === a.id
} yield (a, r))
.groupBy(_._1.id)
.map { case (actid, acts) => ??? }
}
But I cannot seem to make the appropriate mapping. What is the idiomatic way of doing this? I hope it's better than working with a raw cartesian product...
In Scala
In scala code it's easy enough, and would look something like this:
val activities = db withSession { implicit sess =>
(for {
a <- Activities leftJoin Registrations on (_.id === _.activityId)
} yield a).list
}
activities
.groupBy(_._1.id)
.map { case (id, set) => (set(0)._1, set.map(_._2)) }
But this seems rather inefficient due to the unnecessary instantiations of Activity which the table mapper will create for you.
Neither does it look really elegant...
Getting a count of registrations
The in scala method is even worse when only interested in a count of registrations like so:
val result: Seq[Activity, Int] = ???
In Slick
My best attempt in slick would look like this:
val activities = db withSession { implicit sess =>
(for {
a <- Activities leftJoin Registrations on (_.id === _.activityId)
} yield a)
.groupBy(_._1.id)
.map { case (id, results) => (results.map(_._1), results.length) }
}
But this results in an error that slick cannot map the given types in the "map"-line.
I would suggest:
val activities = db withSession { implicit sess =>
(for {
a <- Activities leftJoin Registrations on (_.id === _.activityId)
} yield a)
.groupBy(_._1)
.map { case (activity, results) => (activity, results.length) }
}
The problem with
val activities = db withSession { implicit sess =>
(for {
a <- Activities leftJoin Registrations on (_.id === _.activityId)
} yield a)
.groupBy(_._1.id)
.map { case (id, results) => (results.map(_._1), results.length) }
}
is that you can't produce nested results in group by. results.map(_._1) is a collection of items. SQL does implicit conversions from collections to single rows in some cases, but Slick being type-safe doesn't. What you would like to do in Slick is something like results.map(_._1).head, but that is currently not supported. The closest you could get is something like (results.map(_.id).max, results.map(_.what).max, ...), which is pretty tedious. So grouping by the whole activities row is probably the most feasible workaround right now.
A solution for getting all registrations per activity:
// list of all activities
val activities = Activities
// map of registrations belonging to those activities
val registrations = Registrations
.filter(_.activityId in activities.map(_.id))
.list
.groupBy(_.activityId)
.map { case (aid, group) => (aid, group.map(_._2)) }
.toMap
// combine them
activities
.list
.map { a => (a, registrations.getOrElse(a.id.get, List()))
Which gets the job done in 2 queries. It should be doable to abstract this type of "grouping" function into a scala function.
Related
The below code works fine, but as you can see the 2nd clause in the for comprehension has a call that is unsafe.
case class ProductView(product: Product, stores: List[Store], warehosue: Option[Warehosue])
def loadView(...): ConnectionIO[Option[ProductView]] =
for {
product <- getProductById(id) // ConnectionIO[Option[Product]]
warehouse <- getWarehouse(product.get.warehouseId.get.id) // ConnectionIO[Option[Warehouse]]
stores <- loadStores(...) // ConnectionIO[List[Store]]
} yield product map { p =>
ProductView(p, stores, warehouse)
}
I tried to make that a safe call, but my types don't seem to line up.
warehouse <- getWarehouse(product.get.warehouseId.get.id)
How can I improve this, if any of the options is a None, I just want to return a None.
This is suppose to return a Option[Warehouse]
I tried this:
warehouse <- product.map(p => p.warehouseId.map(id => getWarehouse(id)))
Hoping someone can help with this part of my for comprehension.
The easiest way is using OptionT and flatTraverse
def loadView(id: Int):
ConnectionIO[Option[ProductView]] =
(for {
product <- OptionT(getProductById(id))
warehouse <- OptionT.liftF(product.warehouseId.flatTraverse(getWarehouse))
stores <- OptionT.liftF(loadStores(...))
} yield ProductView(product, stores, warehouse)).value
Also alternative variant without OptionT
def loadView(id: Int): ConnectionIO[Option[ProductView]] = {
getProductById(id).flatMap {
_.traverse { product =>
for {
warehouse <- product.warehouseId.flatTraverse(getWarehouse)
stores <- loadStores(...)
} yield ProductView(product, stores, warehouse)
}
}
}
I am having tuples of format as (DBIO[Seq[Person]], DBIO[Seq[Address]]) as one to one mapping. Person and Address is separate table in RDBMS. Profile definition is Profile(person: Person, address: Address). Now I want to convert the former into DBIO[Seq[Profile]]. Following is code snippet for how I have got (DBIO[Seq[Person]], DBIO[Seq[Address]])
for {
person <- personQuery if person.personId === personId
address <- addressQuery if address.addressId === profile.addressId
} yield (person.result, address.result)
Need help with this transformation to DBIO[Seq[Profile].
Assuming you can't use a join and you need to work with two actions (two DBIOs), what you can do is combine the two actions into a single one:
// Combine two actions into a single action
val pairs: DBIO[ ( Seq[Person], Seq[Address] ) ] =
(person.result).zip(address.result)
(zip is just one of many combinators you can use to manipulate DBIO).
From there you can use DBIO.map to convert the pair into the datastructure you want.
For example:
// Use Slick's DBIO.map to map the DBIO value into a sequence of profiles:
val profiles: DBIO[Seq[Profile]] = pairs.map { case (ppl, places) =>
// We now use a regular Scala `zip` on two sequences:
ppl.zip(places).map { case (person, place) => Profile(person, place) }
}
I am unfamiliar with whatever DBIO is. Assuming it is a case class of some type T :
val (DBIO(people), DBIO(addresses)) = for {
person <- personQuery if person.personId === personId
address <- addressQuery if address.addressId === profile.addressId
} yield (person.result, address.result)
val profiles = DBIO(people.zip(addresses).map{ case (person, address) => Profile(person, address)})
I'm trying to model the following with Slick 3.1.0;
case class Review(txt: String, userId: Long, id: Long)
case class User(name: String, id: Long)
case class ReviewEvent(event: String, reviewId: Long)
I need to populate a class called a FullReview, which looks like;
case class FullReview(r: Review, user: User, evts: Seq[ReviewEvent])
Assuming I have the right tables for each of the models, I'm trying to fetch a FullReview using a combination of join and group by, like so:
val withUser = for {
(r, u) <- RTable join UTable on (_.userId === _.id)
}
val withUAndEvts = (for {
((r, user), evts) <- withUser joinLeft ETable on {
case ((r, _), ev) => r.id === ev.reviewId
}
} yield (r, user, events)).groupBy(_._1._id)
This seems to yield, when a nested Query type, from what I can see. What am I doing wrong here?
If I understand you correctly, you can use following example:
val users = TableQuery[Users]
val reviews = TableQuery[Reviews]
val events = TableQuery[ReviewEvents]
override def findAllReviews(): Future[Seq[FullReview]] = {
val query = reviews
.join(users).on(_.userId === _.id)
.joinLeft(events).on(_._1.id === _.reviewId)
db.run(query.result).map { a =>
a.groupBy(_._1._1.id).map { case (_, tuples) =>
val ((review, user), _) = tuples.head
val reviewEvents = tuples.flatMap(_._2)
FullReview(review, user, reviewEvents)
}.toSeq
}
}
If you want to add pagination to this request, I've already answered here and here is full example.
From some tinkering around, I figured it would just be better to do the aggregation on the client. What that would mean, indirectly, is that if 100 rows on the table ETable would match a single row on the RTable, you would get multiple rows on the client. The client then has to implement its own aggregation to group all the ReviewEvent by Review.
As far as pagination is concerned, you may do something like;
def withUser(page: Int, pageSize: Int) = for {
(r, u) <- RTable.drop(page * pageSize).take(pageSize) join UTable on (_.userId === _.id)
}
I guess this is elegant enough for now. If someone has a better answer, I'd be happy to hear it.
I'm using Slick 3.0, and following is my codes:
def registerMember(newMember: TeamMember): Future[Long] = {
db.run(
teamProfileTable.filter(u => u.ID === newMember.ID).result.headOption
).flatMap {
case None => Future(-1)
case _ => db.run(
(teamProfileTable returning teamProfileTable.map(_.staffID)) += newMember.toTeamRecord
)
}
}
This may look ok. But when there are more layers of callback, the codes may become hard to read. I tried to simplify the codes using for-expression or andThen.. But due to the pattern matching part, I can only use flatMap to implement this..
Does anyone have ideas about how to refactor this?
I think a for comprehension should be okay here, you just need conditional handling of the Option in the result of the first Future. Something like this should work (note I did not compile check this):
def registerMember(newMember: TeamMember): Future[Long] = {
for{
r1Opt <- db.run(teamProfileTable.filter(u => u.ID === newMember.ID).result.headOption
r2 <- r1Opt.fold(Future.successful(-1L))(r1 => db.run((teamProfileTable returning teamProfileTable.map(_.staffID)) += newMember.toTeamRecord)
} yield r2
}
You can see on the right side of the fold that I have access to the result of the first Future if it was a Some (as r1).
I would even take this a step further and create separate methods for the steps of the for comprehension to clean things up, like so:
def registerMember(newMember: TeamMember): Future[Long] = {
def findMember =
db.run(teamProfileTable.filter(u => u.ID === newMember.ID).result.headOption
def addMember(r1Opt:Option[TeamMember]) = {
r1Opt.fold(Future.successful(-1L)){r1 =>
db.run((teamProfileTable returning teamProfileTable.map(_.staffID)) +=
newMember.toTeamRecord)
}
}
for{
r1Opt <- findMember
r2 <- addMember(r1Opt)
} yield r2
}
Another approach to simplify nested db.runs in Slick 3.0 when the query spans two tables could be to join the queries into a single query. Joining and Zipping. However, the OP seems to have the somewhat rarer case of nested queries on the same table so this approach may not be helpful in that particular case.
val query = slickLoginInfos join slickUserLoginInfos on
((l,ul) => l.id === ul.loginInfoId)
db.run((for { (l, ul) <- query } yield (ul)).result.headOption)
I have a number of basic queries define, and am using query composition to add stuff such as ordering, paging, where clauses and so on...
But I have a problem accessing the fields of the joined 2nd table in the where clause...
Here's my table queries and my table. All tables are mapped to case classes.
val basicCars = TableQuery[CarTable]
val basicCarValues = TableQuery[CarValueTable]
val carsWithValues = for {
(c, v) <- basicCars leftJoin basicCarValues on (_.id === _.carId)
} yield (c, v.?)
Now I reuse/compose queries by doing stuff such as
carsWithValues.where(_._1.id === someId)
which works perfectly...
But if I want to access any value of the 2nd table... and I try
carsWithValues.where(_._2.latestPrice === somePrice)
It tells me that somePrice is not a member of MappedProjection......
error: value somePrice is not a member of scala.slick.lifted.MappedProjection[Option[com......datastore.slick.generated.Tables.CarValue],(Option[Long], Option[Long], Option[String],.....
I understand that this kind of can't work, cause _._2 is a MappedProjection and not just a CarValue sitting in the tuple..
But I can't figure out how to use any field of the table that is in the MappedProjection in a where clause?
The .? from the Slick code generator is implemented using a MappedProjection, which doesn't have the members anymore. If you postpone the call to .? it works:
val carsWithValues = for {
(c, v) <- basicCars leftJoin basicCarValues on (_.id === _.carId)
} yield (c, v)
carsWithValues.where(_._2.latestPrice === somePrice).map{ case (c,v) => (c,v.?) }