How can I run inter-dependent queries alongside a non-DB operation in the same transaction using slick - postgresql

Given the data model (<- indicating a foreign key dependency)
TableA <- TableB <- TableC
^ v
-----------------
I need to execute an api DELETE operation that soft-deletes a row in TableC. This delete must also trigger a call to another service (requiring values from TableA and TableB) if there are no more undeleted TableC entries that reference that row's parent in TableB. If the external call fails, I want to rollback the soft-delete. I want to do all of this in an idiomatic fashion (I'm effectively brand new to scala/slick), and use transactions for the rollback
Based on what I've read, I need to be using for comprehension to assemble the queries, but I'm having issues getting the database operations to gel nicely with the external service call. My original plan was:
val select = for {
tableCRow <- tableBDao.TableQueryC.filter(_.id === idParam)
tableBRow <- tableBDao.TableQueryB if tableCRow.tableBForeignKey === tableBRow.id
tableARow <- TableADao.TableQueryA if tableCRow.tableAForeignKey === tableARow.id
count <- tableBDao.TableQueryC.filter(_.tableBForeignKey === tableBRow.id).map(_.id).countDefined
_ <- tableBDao.softDeleteRow(idParam)
_ <- if (count > 1) DBIO.successful(httpRequestService.deleteOtherResource(tableARow.someValue, tableBRow.someValue))
} yield ()
db.run(select.result)
But this had problems because I couldn't pass Slick's Rep[T] values to my httpRequestService method. I then tried to break it down into two portions - SELECT first, then DELETE, like so:
val select = for {
tableCRow <- tableBDao.TableQueryC.filter(_.id === idParam)
tableBRow <- tableBDao.TableQueryB if tableCRow.tableBForeignKey === tableBRow.id
tableARow <- TableADao.TableQueryA if tableCRow.tableAForeignKey === tableARow.id
count <- tableBDao.TableQueryC.filter(_.tableBForeignKey === tableBRow.id).map(_.id).countDefined
} yield (tableBRow.date.formatted("yyyy-MM-DD"), tableARow.externalServiceId, count)
val result: Future[Option[(String, Long, Integer)]] = db.run(select.result.headOption)
result.map {
case None => throw new IllegalArgumentException("exception message")
case Some(data) =>
val delete = for {
_ <- tableBDao.softDeleteRow(idParam)
_ <- if (data._3 > 1) DBIO.successful(httpRequestService.cancelSchedulerJob(data._2, data._1))
} yield numRows
db.run(delete.transactionally)
}
But, despite this actually passing IntelliJ IDEA checks, it won't compile as my count query returns a Rep[Int], which lacks a map function. Additionally, each of my table(A|B|C)Row maps raises an error because they're expecting slick.lifted.Query[Nothing,Nothing,Seq] and they're getting slick.lifted.Query[Nothing,T,Seq]. Finally, the db.run statement doesn't want to use headOption, and apparently returns Any which doesn't support map
halp

Solved this by finally understanding how slick puts things together in a for-comprehension. I had to pull the count out of the query and into a followup groupby->map function, which accumulated my list of things I wanted to count and THEN counted them as opposed to counting as part of the query. This fixed all the rest of the problems too, as the count query was throwing off the expected return types of everything else.
basically, the solution looked like (thing1 was for a join):
val select = (for {
thing1 <- query1
thing2 <- query2
thing3 <- query3
listToCount <- query4 selecting everything I wanted to count
} yield (thing2, thing3, listToCount))
.groupBy({
case (thing2, thing3, listToCount) =>
(thing2, thing3)
})
.map({
case ((thing2, thing3), list) =>
(thing2.deliveryDate, thing3.schedulerJobId, list.map(_._3).length)
})

Related

Doobie. Compose .update.withGeneratedKeys() and .update.run

Referencing to this question.
I want to insert some entity by some condition. It can either be inserted or not. If the condition is true the entity is inserted. I want to insert some other data in various tables. It looks like this:
val q = sql"insert into some_table (some_field) select 42 where ...(some condition)"
val inserts = List(
sql"insert ...",
sql"insert ...",
sql"insert ..."
)
for {
id <- q.update.withGeneratedKeys[Long]("id")
_ <- inserts.reduce(_ ++ _).update.run
} yield id
The problem is this does not compile because the first insert is a fs2.Stream but the second one is not.
I was trying to replace _ <- inserts.reduce... with _ = inserts.reduce. The app can compile but inserts in the second line does not occur.
UPD
My possible way to solve this problem:
...
for {
idOpt <- q.update.withGeneratedKeys[Long]("id").compile.last
_ <- idOpt.fold(0.pure[ConnectionIO])(_ => inserts.reduce(_ ++ _).update.run)
} yield idOpt
This works, but IMHO this is not pretty. Is there a better way to do it?
One way to perform your batch inserts - if you have similar data - is to use updateMany - see doc:
import doobie._
type PersonInfo = (String, Option[Short])
def insertMany(ps: List[PersonInfo]): ConnectionIO[Int] = {
val sql = "insert into person (name, age) values (?, ?)"
Update[PersonInfo](sql).updateMany(ps)
}
// Some rows to insert
val data = List[PersonInfo](
("Frank", Some(12)),
("Daddy", None))
Also, if you remove.compile.last, you can use the fact that if your resulting Stream q.update.withGeneratedKeys[Long]("id") is empty, you 'exit early' the for-comprehension.
So all in all, here is what you could do:
import fs2.Stream
val result =
// Now the for-comprehension operates on a Stream instead of an Option
for {
r <- q.update.withGeneratedKeys[Long]("id")
_ <- Stream.eval(insertMany(data)) // insertMany(data) is defined like in the snippet above
} yield r
result.compile.last

How to mix select and delete in a Slick transaction

Why does it not work to combine a SELECT and a DELETE statement in a Slick query? as in:
val query = (for {
item <- SomeTable
_ <- OtherTable.filter(_.id === item.id).delete
} yield ()).transactionally
"Cannot resolve symbol 'transactionally'"
(without .transactionally, it is a Query[Nothing, Nothing, Seq], if that helps)
while the two actions work separately:
val query = (for {
item <- SomeTable
} yield ()).transactionally
,
val query = (for {
_ <- OtherTable.filter(_.id === 2).delete
} yield ()).transactionally
OK so this is a classic example of mixing DBIO with Query.
In your first case:
val query = (for {
item <- SomeTable // this is `Query`
_ <- OtherTable.filter(_.id === item.id).delete // this is `DBIO`
} yield ()).transactionally
Obviously for DML you can use only actions (Query is for DQL - being simply SELECT).
So first thing is - change your code to use only DBIOs. Below example is incorrect.
val query = (for {
item <- SomeTable.result // this is `DBIO` now
_ <- OtherTable.filter(_.id === item.id).delete // but this won't work !!
} yield ()).transactionally
OK, we are nearly there - the problem is that it doesn't compile. What you need to do is to be aware that now this part:
item <- SomeTable.result
returns Seq of your SomeTable case class (which among other things contains your id).
So let's take into account:
val query = (for {
items <- SomeTable.result // I changed the name to `items` to reflect it's plural nature
_ <- OtherTable.filter(_.id.inset(items.map(_.id))).delete // I needed to change it to generate `IN` query
} yield ()).transactionally

In Slick 3.0, how to simplify nested `db.run`?

I'm using Slick 3.0, and following is my codes:
def registerMember(newMember: TeamMember): Future[Long] = {
db.run(
teamProfileTable.filter(u => u.ID === newMember.ID).result.headOption
).flatMap {
case None => Future(-1)
case _ => db.run(
(teamProfileTable returning teamProfileTable.map(_.staffID)) += newMember.toTeamRecord
)
}
}
This may look ok. But when there are more layers of callback, the codes may become hard to read. I tried to simplify the codes using for-expression or andThen.. But due to the pattern matching part, I can only use flatMap to implement this..
Does anyone have ideas about how to refactor this?
I think a for comprehension should be okay here, you just need conditional handling of the Option in the result of the first Future. Something like this should work (note I did not compile check this):
def registerMember(newMember: TeamMember): Future[Long] = {
for{
r1Opt <- db.run(teamProfileTable.filter(u => u.ID === newMember.ID).result.headOption
r2 <- r1Opt.fold(Future.successful(-1L))(r1 => db.run((teamProfileTable returning teamProfileTable.map(_.staffID)) += newMember.toTeamRecord)
} yield r2
}
You can see on the right side of the fold that I have access to the result of the first Future if it was a Some (as r1).
I would even take this a step further and create separate methods for the steps of the for comprehension to clean things up, like so:
def registerMember(newMember: TeamMember): Future[Long] = {
def findMember =
db.run(teamProfileTable.filter(u => u.ID === newMember.ID).result.headOption
def addMember(r1Opt:Option[TeamMember]) = {
r1Opt.fold(Future.successful(-1L)){r1 =>
db.run((teamProfileTable returning teamProfileTable.map(_.staffID)) +=
newMember.toTeamRecord)
}
}
for{
r1Opt <- findMember
r2 <- addMember(r1Opt)
} yield r2
}
Another approach to simplify nested db.runs in Slick 3.0 when the query spans two tables could be to join the queries into a single query. Joining and Zipping. However, the OP seems to have the somewhat rarer case of nested queries on the same table so this approach may not be helpful in that particular case.
val query = slickLoginInfos join slickUserLoginInfos on
((l,ul) => l.id === ul.loginInfoId)
db.run((for { (l, ul) <- query } yield (ul)).result.headOption)

Slick/Scala - how do I access fields of the mapped projection/projected table part of a join in a where query

I have a number of basic queries define, and am using query composition to add stuff such as ordering, paging, where clauses and so on...
But I have a problem accessing the fields of the joined 2nd table in the where clause...
Here's my table queries and my table. All tables are mapped to case classes.
val basicCars = TableQuery[CarTable]
val basicCarValues = TableQuery[CarValueTable]
val carsWithValues = for {
(c, v) <- basicCars leftJoin basicCarValues on (_.id === _.carId)
} yield (c, v.?)
Now I reuse/compose queries by doing stuff such as
carsWithValues.where(_._1.id === someId)
which works perfectly...
But if I want to access any value of the 2nd table... and I try
carsWithValues.where(_._2.latestPrice === somePrice)
It tells me that somePrice is not a member of MappedProjection......
error: value somePrice is not a member of scala.slick.lifted.MappedProjection[Option[com......datastore.slick.generated.Tables.CarValue],(Option[Long], Option[Long], Option[String],.....
I understand that this kind of can't work, cause _._2 is a MappedProjection and not just a CarValue sitting in the tuple..
But I can't figure out how to use any field of the table that is in the MappedProjection in a where clause?
The .? from the Slick code generator is implemented using a MappedProjection, which doesn't have the members anymore. If you postpone the call to .? it works:
val carsWithValues = for {
(c, v) <- basicCars leftJoin basicCarValues on (_.id === _.carId)
} yield (c, v)
carsWithValues.where(_._2.latestPrice === somePrice).map{ case (c,v) => (c,v.?) }

How do you change lifted types back to Scala types when using Slick lifted embedding?

How do you 'un-lift' a value inside a query in Slick when using lifted embedding? I was hoping a 'get', 'toLong' or something like that may do the trick, but no such luck.
The following code does not compile:
val userById = for {
uid <- Parameters[Long]
u <- Users if u.id === uid
} yield u
val userFirstNameById = for {
uid <- Parameters[Long]
u <- userById(uid)
---------------^
// type mismatch; found : scala.slick.lifted.Column[Long] required: Long
} yield u.name
You can't, for 2 reasons:
1) with val this is happening at compile time, there is no Long
value uid. userById(uid) binds a Long uid to the compile time
generated prepared statement, and then .list, .first, etc. invoke
the query.
2) the other issue is as soon as you Parameterize a query,
composition is no longer possible -- it's a limitation dating back to
ScalaQuery.
Your best bet is to delay Parameterization until the final composed query:
val forFooBars = for{
f <- Foos
b <- Bars if f.id is b.fooID
} yield(f,b)
val allByStatus = for{ id ~ active <- Parameters[(Long,Boolean)]
(f,b) <- forFooBars if (f.id is id) && (b.active is active)
} yield(f,b)
def findAllByActive(id: Long, isActive: Boolean) = allByStatus(id, isActive).list
At any rate, in your example you could just as well do:
val byID = Users.createFinderBy(_.id)
The only way that I know to get this kind of thing to work is wrap the query val in a def and pass in a runtime variable, which means Slick has to re-generate the sql on every request, and no prepared statement is sent to underlying DBMS. In some cases you have to do this, like passing in a List(1,2,3) for inList.
def whenNothingElseWorks(id: Long) = {
val userFirstNameById = for {u <- userById(id.bind)} yield u.name
}