How to mix select and delete in a Slick transaction - scala

Why does it not work to combine a SELECT and a DELETE statement in a Slick query? as in:
val query = (for {
item <- SomeTable
_ <- OtherTable.filter(_.id === item.id).delete
} yield ()).transactionally
"Cannot resolve symbol 'transactionally'"
(without .transactionally, it is a Query[Nothing, Nothing, Seq], if that helps)
while the two actions work separately:
val query = (for {
item <- SomeTable
} yield ()).transactionally
,
val query = (for {
_ <- OtherTable.filter(_.id === 2).delete
} yield ()).transactionally

OK so this is a classic example of mixing DBIO with Query.
In your first case:
val query = (for {
item <- SomeTable // this is `Query`
_ <- OtherTable.filter(_.id === item.id).delete // this is `DBIO`
} yield ()).transactionally
Obviously for DML you can use only actions (Query is for DQL - being simply SELECT).
So first thing is - change your code to use only DBIOs. Below example is incorrect.
val query = (for {
item <- SomeTable.result // this is `DBIO` now
_ <- OtherTable.filter(_.id === item.id).delete // but this won't work !!
} yield ()).transactionally
OK, we are nearly there - the problem is that it doesn't compile. What you need to do is to be aware that now this part:
item <- SomeTable.result
returns Seq of your SomeTable case class (which among other things contains your id).
So let's take into account:
val query = (for {
items <- SomeTable.result // I changed the name to `items` to reflect it's plural nature
_ <- OtherTable.filter(_.id.inset(items.map(_.id))).delete // I needed to change it to generate `IN` query
} yield ()).transactionally

Related

Doobie. Compose .update.withGeneratedKeys() and .update.run

Referencing to this question.
I want to insert some entity by some condition. It can either be inserted or not. If the condition is true the entity is inserted. I want to insert some other data in various tables. It looks like this:
val q = sql"insert into some_table (some_field) select 42 where ...(some condition)"
val inserts = List(
sql"insert ...",
sql"insert ...",
sql"insert ..."
)
for {
id <- q.update.withGeneratedKeys[Long]("id")
_ <- inserts.reduce(_ ++ _).update.run
} yield id
The problem is this does not compile because the first insert is a fs2.Stream but the second one is not.
I was trying to replace _ <- inserts.reduce... with _ = inserts.reduce. The app can compile but inserts in the second line does not occur.
UPD
My possible way to solve this problem:
...
for {
idOpt <- q.update.withGeneratedKeys[Long]("id").compile.last
_ <- idOpt.fold(0.pure[ConnectionIO])(_ => inserts.reduce(_ ++ _).update.run)
} yield idOpt
This works, but IMHO this is not pretty. Is there a better way to do it?
One way to perform your batch inserts - if you have similar data - is to use updateMany - see doc:
import doobie._
type PersonInfo = (String, Option[Short])
def insertMany(ps: List[PersonInfo]): ConnectionIO[Int] = {
val sql = "insert into person (name, age) values (?, ?)"
Update[PersonInfo](sql).updateMany(ps)
}
// Some rows to insert
val data = List[PersonInfo](
("Frank", Some(12)),
("Daddy", None))
Also, if you remove.compile.last, you can use the fact that if your resulting Stream q.update.withGeneratedKeys[Long]("id") is empty, you 'exit early' the for-comprehension.
So all in all, here is what you could do:
import fs2.Stream
val result =
// Now the for-comprehension operates on a Stream instead of an Option
for {
r <- q.update.withGeneratedKeys[Long]("id")
_ <- Stream.eval(insertMany(data)) // insertMany(data) is defined like in the snippet above
} yield r
result.compile.last

How can I run inter-dependent queries alongside a non-DB operation in the same transaction using slick

Given the data model (<- indicating a foreign key dependency)
TableA <- TableB <- TableC
^ v
-----------------
I need to execute an api DELETE operation that soft-deletes a row in TableC. This delete must also trigger a call to another service (requiring values from TableA and TableB) if there are no more undeleted TableC entries that reference that row's parent in TableB. If the external call fails, I want to rollback the soft-delete. I want to do all of this in an idiomatic fashion (I'm effectively brand new to scala/slick), and use transactions for the rollback
Based on what I've read, I need to be using for comprehension to assemble the queries, but I'm having issues getting the database operations to gel nicely with the external service call. My original plan was:
val select = for {
tableCRow <- tableBDao.TableQueryC.filter(_.id === idParam)
tableBRow <- tableBDao.TableQueryB if tableCRow.tableBForeignKey === tableBRow.id
tableARow <- TableADao.TableQueryA if tableCRow.tableAForeignKey === tableARow.id
count <- tableBDao.TableQueryC.filter(_.tableBForeignKey === tableBRow.id).map(_.id).countDefined
_ <- tableBDao.softDeleteRow(idParam)
_ <- if (count > 1) DBIO.successful(httpRequestService.deleteOtherResource(tableARow.someValue, tableBRow.someValue))
} yield ()
db.run(select.result)
But this had problems because I couldn't pass Slick's Rep[T] values to my httpRequestService method. I then tried to break it down into two portions - SELECT first, then DELETE, like so:
val select = for {
tableCRow <- tableBDao.TableQueryC.filter(_.id === idParam)
tableBRow <- tableBDao.TableQueryB if tableCRow.tableBForeignKey === tableBRow.id
tableARow <- TableADao.TableQueryA if tableCRow.tableAForeignKey === tableARow.id
count <- tableBDao.TableQueryC.filter(_.tableBForeignKey === tableBRow.id).map(_.id).countDefined
} yield (tableBRow.date.formatted("yyyy-MM-DD"), tableARow.externalServiceId, count)
val result: Future[Option[(String, Long, Integer)]] = db.run(select.result.headOption)
result.map {
case None => throw new IllegalArgumentException("exception message")
case Some(data) =>
val delete = for {
_ <- tableBDao.softDeleteRow(idParam)
_ <- if (data._3 > 1) DBIO.successful(httpRequestService.cancelSchedulerJob(data._2, data._1))
} yield numRows
db.run(delete.transactionally)
}
But, despite this actually passing IntelliJ IDEA checks, it won't compile as my count query returns a Rep[Int], which lacks a map function. Additionally, each of my table(A|B|C)Row maps raises an error because they're expecting slick.lifted.Query[Nothing,Nothing,Seq] and they're getting slick.lifted.Query[Nothing,T,Seq]. Finally, the db.run statement doesn't want to use headOption, and apparently returns Any which doesn't support map
halp
Solved this by finally understanding how slick puts things together in a for-comprehension. I had to pull the count out of the query and into a followup groupby->map function, which accumulated my list of things I wanted to count and THEN counted them as opposed to counting as part of the query. This fixed all the rest of the problems too, as the count query was throwing off the expected return types of everything else.
basically, the solution looked like (thing1 was for a join):
val select = (for {
thing1 <- query1
thing2 <- query2
thing3 <- query3
listToCount <- query4 selecting everything I wanted to count
} yield (thing2, thing3, listToCount))
.groupBy({
case (thing2, thing3, listToCount) =>
(thing2, thing3)
})
.map({
case ((thing2, thing3), list) =>
(thing2.deliveryDate, thing3.schedulerJobId, list.map(_._3).length)
})

Conditionally running Slick statements in a for comprehension

Given the following Slick code:
val doUpdate = true
val table1 = TableQuery[Table1DB]
val table2 = TableQuery[Table2DB]
val table3 = TableQuery[Table3DB]
val action = (for {
_ <- table1.filter(_.id === id).update(table1Record)
_ <- table2.filter(_.id === id).update(table2Record)
_ <- table3.filter(_.id === id).update(table3Record)
} yield ()).transactionally
What I need is to update table2 and table3 ONLY if the variable doUpdate is true. This is my attempt:
val action = (for {
_ <- table1.filter(_.id === id).update(table1Record)
_ <- table2.filter(_.id === id).update(table2Record)
if (doUpdate == true)
_ <- table3.filter(_.id === id).update(table3Record)
if (doUpdate == true)
} yield ()).transactionally
This compiles, but when I run it I get the following error Action.withFilter failed. How to add the condition?
As you notices if you use if inside a for it will call withFilter underneath. In case of pipelines (DBIO, Futures, Tasks) that filtering boils down to whether or not continue with a pipeline (happy path) or fail it (generating Failure or similar).
If all you want is to update, then you can go with the if road and recover the Future. However, it would we ugly, since depending on how you write the case it might also recover from errors you would like to be informed about.
Alternative is to simply use if-else to provide a different DBIOAction:
(for {
_ <- {
if (doUpdate) conditionalAction.map(_ => ())
else DBIOAction.successful(())
}
} yield ()).transactionally
In your case it could look something like:
val action = (for {
_ <- table1.filter(_.id === id).update(table1Record)
_ <- {
if (doUpdate) DBIOAction.seq(
table2.filter(_.id === id).update(table2Record),
table3.filter(_.id === id).update(table3Record)
)
else DBIOAction.successful(())
}
} yield ()).transactionally

In Slick 3.0, how to simplify nested `db.run`?

I'm using Slick 3.0, and following is my codes:
def registerMember(newMember: TeamMember): Future[Long] = {
db.run(
teamProfileTable.filter(u => u.ID === newMember.ID).result.headOption
).flatMap {
case None => Future(-1)
case _ => db.run(
(teamProfileTable returning teamProfileTable.map(_.staffID)) += newMember.toTeamRecord
)
}
}
This may look ok. But when there are more layers of callback, the codes may become hard to read. I tried to simplify the codes using for-expression or andThen.. But due to the pattern matching part, I can only use flatMap to implement this..
Does anyone have ideas about how to refactor this?
I think a for comprehension should be okay here, you just need conditional handling of the Option in the result of the first Future. Something like this should work (note I did not compile check this):
def registerMember(newMember: TeamMember): Future[Long] = {
for{
r1Opt <- db.run(teamProfileTable.filter(u => u.ID === newMember.ID).result.headOption
r2 <- r1Opt.fold(Future.successful(-1L))(r1 => db.run((teamProfileTable returning teamProfileTable.map(_.staffID)) += newMember.toTeamRecord)
} yield r2
}
You can see on the right side of the fold that I have access to the result of the first Future if it was a Some (as r1).
I would even take this a step further and create separate methods for the steps of the for comprehension to clean things up, like so:
def registerMember(newMember: TeamMember): Future[Long] = {
def findMember =
db.run(teamProfileTable.filter(u => u.ID === newMember.ID).result.headOption
def addMember(r1Opt:Option[TeamMember]) = {
r1Opt.fold(Future.successful(-1L)){r1 =>
db.run((teamProfileTable returning teamProfileTable.map(_.staffID)) +=
newMember.toTeamRecord)
}
}
for{
r1Opt <- findMember
r2 <- addMember(r1Opt)
} yield r2
}
Another approach to simplify nested db.runs in Slick 3.0 when the query spans two tables could be to join the queries into a single query. Joining and Zipping. However, the OP seems to have the somewhat rarer case of nested queries on the same table so this approach may not be helpful in that particular case.
val query = slickLoginInfos join slickUserLoginInfos on
((l,ul) => l.id === ul.loginInfoId)
db.run((for { (l, ul) <- query } yield (ul)).result.headOption)

SLICK: How to use query result in another query?

I'd like to perform something like the following:
I'd like to return a list of users sorted first by who the user is "following", second by some additional point score.
The following code below which I wrote however doesn't work because the funder is the lifted Slick type and therefore is never found in the List.
//The following represents the query for only funders who we are following
val following_funders: List[User] = (
for {
funder <- all_funders
f <- follows if f.followerId === id //get all the current users follower objects
if f.followeeId === funder.id
} yield funder
).list
val all_funders_sorted = for {
funder <- all_funders
following_funder = following_funders contains funder
} yield (funder, following_funder)
//sort the funders by whether or not they are following the funder and then map it to only the funders (i.e. remove the boolean)
all_funders_sorted.sortBy(_._2.desc).sortBy(_._1.score.desc).map( x => x._1 )
All help appreciated!
You need to work with ids (i.e. primary keys) in Slick. That's how objects are uniquely identified on the db side. You do not need to execute the first query. You can use it as a component of your second without executing it first using the in operator:
//The following represents the query for only funders who we are following
val following_funders_ids = (
for {
funder <- all_funders
f <- follows if f.followerId === id //get all the current users follower objects
if f.followeeId === funder.id
} yield funder.id
val all_funders_sorted = for {
funder <- all_funders
following_funder = funder.id in following_funders_ids
} yield (funder, following_funder)
//sort the funders by whether or not they are following the funder and then map it to only the funders (i.e. remove the boolean)
all_funders_sorted.sortBy(_._1.impactPoints.desc).sortBy(_._2.desc).map( x => x._1 )
Be aware that your sort order was wrong, if you first want to sort by following. Slick translates .sortBy(_.a).sortBy(_.b) to ORDER BY B,A because that's how Scala collections work:
scala> List( (1,"b"), (2,"a") ).sortBy(_._1).sortBy(_._2)
res0: List[(Int, String)] = List((2,a), (1,b))
Ended up figuring it out the following way by using 'inSet'
//The following represents the query for only funders who we are following
val following_funders_ids: List[Long] = (
for {
funder <- all_funders
f <- follows if f.followerId === id //get all the current users follower objects
if f.followeeId === funder.id
} yield funder.id
).list
val all_funders_sorted = for {
funder <- all_funders
following_funder = funder.id inSet following_funders_ids
} yield (funder, following_funder)
//sort the funders by whether or not they are following the funder and then map it to only the funders (i.e. remove the boolean)
all_funders_sorted.sortBy(_._2.desc).sortBy(_._1.impactPoints.desc).map( x => x._1 )