Doobie. Compose .update.withGeneratedKeys() and .update.run - scala

Referencing to this question.
I want to insert some entity by some condition. It can either be inserted or not. If the condition is true the entity is inserted. I want to insert some other data in various tables. It looks like this:
val q = sql"insert into some_table (some_field) select 42 where ...(some condition)"
val inserts = List(
sql"insert ...",
sql"insert ...",
sql"insert ..."
)
for {
id <- q.update.withGeneratedKeys[Long]("id")
_ <- inserts.reduce(_ ++ _).update.run
} yield id
The problem is this does not compile because the first insert is a fs2.Stream but the second one is not.
I was trying to replace _ <- inserts.reduce... with _ = inserts.reduce. The app can compile but inserts in the second line does not occur.
UPD
My possible way to solve this problem:
...
for {
idOpt <- q.update.withGeneratedKeys[Long]("id").compile.last
_ <- idOpt.fold(0.pure[ConnectionIO])(_ => inserts.reduce(_ ++ _).update.run)
} yield idOpt
This works, but IMHO this is not pretty. Is there a better way to do it?

One way to perform your batch inserts - if you have similar data - is to use updateMany - see doc:
import doobie._
type PersonInfo = (String, Option[Short])
def insertMany(ps: List[PersonInfo]): ConnectionIO[Int] = {
val sql = "insert into person (name, age) values (?, ?)"
Update[PersonInfo](sql).updateMany(ps)
}
// Some rows to insert
val data = List[PersonInfo](
("Frank", Some(12)),
("Daddy", None))
Also, if you remove.compile.last, you can use the fact that if your resulting Stream q.update.withGeneratedKeys[Long]("id") is empty, you 'exit early' the for-comprehension.
So all in all, here is what you could do:
import fs2.Stream
val result =
// Now the for-comprehension operates on a Stream instead of an Option
for {
r <- q.update.withGeneratedKeys[Long]("id")
_ <- Stream.eval(insertMany(data)) // insertMany(data) is defined like in the snippet above
} yield r
result.compile.last

Related

How can I run inter-dependent queries alongside a non-DB operation in the same transaction using slick

Given the data model (<- indicating a foreign key dependency)
TableA <- TableB <- TableC
^ v
-----------------
I need to execute an api DELETE operation that soft-deletes a row in TableC. This delete must also trigger a call to another service (requiring values from TableA and TableB) if there are no more undeleted TableC entries that reference that row's parent in TableB. If the external call fails, I want to rollback the soft-delete. I want to do all of this in an idiomatic fashion (I'm effectively brand new to scala/slick), and use transactions for the rollback
Based on what I've read, I need to be using for comprehension to assemble the queries, but I'm having issues getting the database operations to gel nicely with the external service call. My original plan was:
val select = for {
tableCRow <- tableBDao.TableQueryC.filter(_.id === idParam)
tableBRow <- tableBDao.TableQueryB if tableCRow.tableBForeignKey === tableBRow.id
tableARow <- TableADao.TableQueryA if tableCRow.tableAForeignKey === tableARow.id
count <- tableBDao.TableQueryC.filter(_.tableBForeignKey === tableBRow.id).map(_.id).countDefined
_ <- tableBDao.softDeleteRow(idParam)
_ <- if (count > 1) DBIO.successful(httpRequestService.deleteOtherResource(tableARow.someValue, tableBRow.someValue))
} yield ()
db.run(select.result)
But this had problems because I couldn't pass Slick's Rep[T] values to my httpRequestService method. I then tried to break it down into two portions - SELECT first, then DELETE, like so:
val select = for {
tableCRow <- tableBDao.TableQueryC.filter(_.id === idParam)
tableBRow <- tableBDao.TableQueryB if tableCRow.tableBForeignKey === tableBRow.id
tableARow <- TableADao.TableQueryA if tableCRow.tableAForeignKey === tableARow.id
count <- tableBDao.TableQueryC.filter(_.tableBForeignKey === tableBRow.id).map(_.id).countDefined
} yield (tableBRow.date.formatted("yyyy-MM-DD"), tableARow.externalServiceId, count)
val result: Future[Option[(String, Long, Integer)]] = db.run(select.result.headOption)
result.map {
case None => throw new IllegalArgumentException("exception message")
case Some(data) =>
val delete = for {
_ <- tableBDao.softDeleteRow(idParam)
_ <- if (data._3 > 1) DBIO.successful(httpRequestService.cancelSchedulerJob(data._2, data._1))
} yield numRows
db.run(delete.transactionally)
}
But, despite this actually passing IntelliJ IDEA checks, it won't compile as my count query returns a Rep[Int], which lacks a map function. Additionally, each of my table(A|B|C)Row maps raises an error because they're expecting slick.lifted.Query[Nothing,Nothing,Seq] and they're getting slick.lifted.Query[Nothing,T,Seq]. Finally, the db.run statement doesn't want to use headOption, and apparently returns Any which doesn't support map
halp
Solved this by finally understanding how slick puts things together in a for-comprehension. I had to pull the count out of the query and into a followup groupby->map function, which accumulated my list of things I wanted to count and THEN counted them as opposed to counting as part of the query. This fixed all the rest of the problems too, as the count query was throwing off the expected return types of everything else.
basically, the solution looked like (thing1 was for a join):
val select = (for {
thing1 <- query1
thing2 <- query2
thing3 <- query3
listToCount <- query4 selecting everything I wanted to count
} yield (thing2, thing3, listToCount))
.groupBy({
case (thing2, thing3, listToCount) =>
(thing2, thing3)
})
.map({
case ((thing2, thing3), list) =>
(thing2.deliveryDate, thing3.schedulerJobId, list.map(_._3).length)
})

Do Aggregation with Slick

My database structure looks like this:
id | content
I what to get the entry with max id (not just id).
I read the answer How to make aggregations with slick, but I found there is no first method in the statement: Query(Coffees.map(_.price).max).first. How to do that now?
What if I need the content of the item with the max id?
To retrieve another column, you could do something like the following. The below example calculates the max of one column, finds the row with that maximum value, and returns the value of another column in that row:
val coffees = TableQuery[Coffees]
val mostExpensiveCoffeeQuery =
for {
maxPrice <- coffees.map(_.price).max.result
c <- maxPrice match {
case Some(p) => coffees.filter(_.price === p).result
case None => DBIO.successful(Seq())
}
} yield c.headOption.map(_.name)
val mostExpensiveCoffee = db.run(mostExpensiveCoffeeQuery)
// Future[Option[String]]
Alternatively, to return a full Coffees object:
val mostExpensiveCoffeeQuery =
for {
...
} yield c.headOption
val mostExpensiveCoffee = db.run(mostExpensiveCoffeeQuery)
// Future[Option[Coffees]]

How to mix select and delete in a Slick transaction

Why does it not work to combine a SELECT and a DELETE statement in a Slick query? as in:
val query = (for {
item <- SomeTable
_ <- OtherTable.filter(_.id === item.id).delete
} yield ()).transactionally
"Cannot resolve symbol 'transactionally'"
(without .transactionally, it is a Query[Nothing, Nothing, Seq], if that helps)
while the two actions work separately:
val query = (for {
item <- SomeTable
} yield ()).transactionally
,
val query = (for {
_ <- OtherTable.filter(_.id === 2).delete
} yield ()).transactionally
OK so this is a classic example of mixing DBIO with Query.
In your first case:
val query = (for {
item <- SomeTable // this is `Query`
_ <- OtherTable.filter(_.id === item.id).delete // this is `DBIO`
} yield ()).transactionally
Obviously for DML you can use only actions (Query is for DQL - being simply SELECT).
So first thing is - change your code to use only DBIOs. Below example is incorrect.
val query = (for {
item <- SomeTable.result // this is `DBIO` now
_ <- OtherTable.filter(_.id === item.id).delete // but this won't work !!
} yield ()).transactionally
OK, we are nearly there - the problem is that it doesn't compile. What you need to do is to be aware that now this part:
item <- SomeTable.result
returns Seq of your SomeTable case class (which among other things contains your id).
So let's take into account:
val query = (for {
items <- SomeTable.result // I changed the name to `items` to reflect it's plural nature
_ <- OtherTable.filter(_.id.inset(items.map(_.id))).delete // I needed to change it to generate `IN` query
} yield ()).transactionally

Slick 3.0 delete and returning values

If I want a returning value when inserting a new row, I can do something like
val insertQuery = myTable returning myTable.map(_.id) += SomeData(someString)
How can I achieve the same effect when deleting?
I tried
val deleteQuery = myTable filter (_.id ===id) returning myTable.map(_.someColumn) delete
But apparently this does not compile.
I can resort to the for comprehension but I wonder if there is a shorter way.
The best way I know of to do this is to do something like:
val query = db.filter(....)
val action = for {
results <- query.result
_ <- query.delete
} yield results
db.run(action.withTransactionIsolation(TransactionIsolation.RepeatableRead))
Wish it was shorter.

How do you change lifted types back to Scala types when using Slick lifted embedding?

How do you 'un-lift' a value inside a query in Slick when using lifted embedding? I was hoping a 'get', 'toLong' or something like that may do the trick, but no such luck.
The following code does not compile:
val userById = for {
uid <- Parameters[Long]
u <- Users if u.id === uid
} yield u
val userFirstNameById = for {
uid <- Parameters[Long]
u <- userById(uid)
---------------^
// type mismatch; found : scala.slick.lifted.Column[Long] required: Long
} yield u.name
You can't, for 2 reasons:
1) with val this is happening at compile time, there is no Long
value uid. userById(uid) binds a Long uid to the compile time
generated prepared statement, and then .list, .first, etc. invoke
the query.
2) the other issue is as soon as you Parameterize a query,
composition is no longer possible -- it's a limitation dating back to
ScalaQuery.
Your best bet is to delay Parameterization until the final composed query:
val forFooBars = for{
f <- Foos
b <- Bars if f.id is b.fooID
} yield(f,b)
val allByStatus = for{ id ~ active <- Parameters[(Long,Boolean)]
(f,b) <- forFooBars if (f.id is id) && (b.active is active)
} yield(f,b)
def findAllByActive(id: Long, isActive: Boolean) = allByStatus(id, isActive).list
At any rate, in your example you could just as well do:
val byID = Users.createFinderBy(_.id)
The only way that I know to get this kind of thing to work is wrap the query val in a def and pass in a runtime variable, which means Slick has to re-generate the sql on every request, and no prepared statement is sent to underlying DBMS. In some cases you have to do this, like passing in a List(1,2,3) for inList.
def whenNothingElseWorks(id: Long) = {
val userFirstNameById = for {u <- userById(id.bind)} yield u.name
}