I am new to Quill and I am using it to write a query on my PostgreSQL DB.
I have a query where I need to groupBy a column depending on a condition.
val offers = // .. Some query ..
val q = quote(
offers
.groupBy(o => if (someCondition) o.column1 else o.column2)
.map { case (p, o) => (p, o.size) }
)
However, my test is failing with this error
org.postgresql.util.PSQLException: ERROR: column "x22.column1" must
appear in the GROUP BY clause or be used in an aggregate function
I can't really understand the error message, and I am not sure if such a query is possible in quill, so any help would be appreciated.
The test works fine without the condition, if only e.g. o.column1 is specified.
Related
Given the data model (<- indicating a foreign key dependency)
TableA <- TableB <- TableC
^ v
-----------------
I need to execute an api DELETE operation that soft-deletes a row in TableC. This delete must also trigger a call to another service (requiring values from TableA and TableB) if there are no more undeleted TableC entries that reference that row's parent in TableB. If the external call fails, I want to rollback the soft-delete. I want to do all of this in an idiomatic fashion (I'm effectively brand new to scala/slick), and use transactions for the rollback
Based on what I've read, I need to be using for comprehension to assemble the queries, but I'm having issues getting the database operations to gel nicely with the external service call. My original plan was:
val select = for {
tableCRow <- tableBDao.TableQueryC.filter(_.id === idParam)
tableBRow <- tableBDao.TableQueryB if tableCRow.tableBForeignKey === tableBRow.id
tableARow <- TableADao.TableQueryA if tableCRow.tableAForeignKey === tableARow.id
count <- tableBDao.TableQueryC.filter(_.tableBForeignKey === tableBRow.id).map(_.id).countDefined
_ <- tableBDao.softDeleteRow(idParam)
_ <- if (count > 1) DBIO.successful(httpRequestService.deleteOtherResource(tableARow.someValue, tableBRow.someValue))
} yield ()
db.run(select.result)
But this had problems because I couldn't pass Slick's Rep[T] values to my httpRequestService method. I then tried to break it down into two portions - SELECT first, then DELETE, like so:
val select = for {
tableCRow <- tableBDao.TableQueryC.filter(_.id === idParam)
tableBRow <- tableBDao.TableQueryB if tableCRow.tableBForeignKey === tableBRow.id
tableARow <- TableADao.TableQueryA if tableCRow.tableAForeignKey === tableARow.id
count <- tableBDao.TableQueryC.filter(_.tableBForeignKey === tableBRow.id).map(_.id).countDefined
} yield (tableBRow.date.formatted("yyyy-MM-DD"), tableARow.externalServiceId, count)
val result: Future[Option[(String, Long, Integer)]] = db.run(select.result.headOption)
result.map {
case None => throw new IllegalArgumentException("exception message")
case Some(data) =>
val delete = for {
_ <- tableBDao.softDeleteRow(idParam)
_ <- if (data._3 > 1) DBIO.successful(httpRequestService.cancelSchedulerJob(data._2, data._1))
} yield numRows
db.run(delete.transactionally)
}
But, despite this actually passing IntelliJ IDEA checks, it won't compile as my count query returns a Rep[Int], which lacks a map function. Additionally, each of my table(A|B|C)Row maps raises an error because they're expecting slick.lifted.Query[Nothing,Nothing,Seq] and they're getting slick.lifted.Query[Nothing,T,Seq]. Finally, the db.run statement doesn't want to use headOption, and apparently returns Any which doesn't support map
halp
Solved this by finally understanding how slick puts things together in a for-comprehension. I had to pull the count out of the query and into a followup groupby->map function, which accumulated my list of things I wanted to count and THEN counted them as opposed to counting as part of the query. This fixed all the rest of the problems too, as the count query was throwing off the expected return types of everything else.
basically, the solution looked like (thing1 was for a join):
val select = (for {
thing1 <- query1
thing2 <- query2
thing3 <- query3
listToCount <- query4 selecting everything I wanted to count
} yield (thing2, thing3, listToCount))
.groupBy({
case (thing2, thing3, listToCount) =>
(thing2, thing3)
})
.map({
case ((thing2, thing3), list) =>
(thing2.deliveryDate, thing3.schedulerJobId, list.map(_._3).length)
})
I have the following column in my db, thats a Boolean, but also accepts NULL, so true, false, and NULL are all valid:
def rtb = column[Option[Boolean]]("rtb")
and have the following optional input from the client that I'd like to filter on:
rtbFromClient: Option[Boolean] = ...
I have the following (based on this answer on how to do queries in slick: https://stackoverflow.com/a/40888918/5300930):
val query = userTable.
filter(row =>
if (rtbFromClient.isDefined)
row.rtb.get === rtbFromClient.get
else
LiteralColumn(true)
)
but am getting this error when the code runs:
Caught exception while computing default value for Rep[Option[_]].getOrElse -- This cannot be done lazily when the value is needed on the database side
I thought it may be because row.rtb.get was throwing exception on call to get because the value in the db was null, so tried changing it to row.rtb.getOrElse(null) and row.rtb.getOrElse(None) but neither of these worked either)
Also tried the following:
if (rtbFromClient.isDefined) {
val query = query.filter(_.rtb.isDefined).filter(_.rtb.get === rtbFromClient.get)
}
But this also throws the same error at runtime:
Caught exception while computing default value for Rep[Option[_]].getOrElse -- This cannot be done lazily when the value is needed on the database side
To summarise:
I have an Option[Boolean] column in my db that can contain true, false or NULL (the actual mysql type is a tinyint(1), which is mapped to a slick Option[Boolean])
I have an optional filter provided by the user, rtbFromClient that I'd like to filter on if present. If present, this will be either true or false
I have other optional filters (not shown here) that have similar behavior so I'd like to be able to combine them easily
I had the same issue. My solution is (tested with Slick 3.3.x):
val query = usersTable.filterOpt(rtbFromClient)(_.rtb === _)
Situation 1 (when rtbFromClient is empty) corresponds the following SQL:
select * from users;
Situation 2 (rtbFromClient is defined):
select * from users where rtb = ?;
Fist check if the column is null and then go ahead with another comparison. Ensure that rtbFromClient is not an option.
val query = userTable.filter(row => !row.rtb.isEmpty &&
row.rtb === rtbFromClient)
The first condition ensures that nulls are filtered and the second condition checks the value in case the column value is not null.
In case you have optional value, then below code helps.
def query(value: Option[Boolean]) = value match {
case Some(userGivenRtbFromClient) =>
userTable.filter(row => !row.rtbFromClient.isNull &&
row.rtbFromClient === userGivenRtbFromClient)
case None => userTable.filter(row => row.rtbFromClient.isNull)
}
The Much cleaner version is here.
rtbFromClient: Option[Boolean]
rtbFromClient User given optional value to compare with slick column.
userTable.filter(row => rtbFromClient.map(value => row.rtb === Some(value): Option[Boolean]).getOrElse(row.rtb.isEmpty))
var query = userTable.drop(page.skip).take(page.top)
if (rtbFromClient.isDefined) {
query = query.filter(row => row.rtb.isDefined && row.rtb === rtbFromClient)
}
// More if blocks with optional filters here
...
dbProvider.db.run(query.result)
try this
val query = userTable.filter(row => rtbFromClient.map(x => Option(x) === row.rtb).getOrElse(row.rtb.isEmpty.?))
I was facing the same issue. After playing around I realised one can filter by the option directly, so if we have the column
def rtb = column[Option[Boolean]]("rtb")
and we're trying to filter by
rtbFromClient: Option[Boolean] = ...
then we can do:
val query: DBIO[Seq[TableElementType]] =
if (rtbFromClient.isDefined)
userTable.filter(_.rtb === rtbFromClient).result
else
userTable.result
I am starting to work with slick and scala and it seems that I still do not have the basics iron out:
I am using slick 3.0.0 and scala 2.11.7 and I am connecting to an Oracle database.
I want to get the number of rows in my table so I went and did a search and found the following:
1) This one tells me that .run does not exist:
mytable.length.run
2)This one tells me that there is a type mismatch found: slick.lifted.Rep[Int] and expected String:
var q = for(u <- mytable) yield u.id
print(q.size)
3)This one compiles and runs but prints Rep(Pure $#309962262):
var q = for{row <- mytable} yield row
println(Query(q.length))
So I am not sure if it is because I do not understand how this works but I was imagining that the following should happen:
A) constructQuery
b) "run" query.
So the other queries that I am using are as follow:
val db = Database.forConfig("Oracle")
try{
val f: Future[Unit] = {
val query: StreamingDBIO[Seq[String], String] = participants.map(_.id).result // A)"Construct query"
val stremQuery: DatabasePublisher[String] = db.stream(query) //B) "Run query"
streamQuery.foreach(println)
}
Await.result(f, Duration.Inf)
}
finally db.close
What am I missing? is number 3 not giving me what I want because is not under a db.stream/db.run/db.something command? Or am I just lost =)
Thanks in advance
Tona
Querying a database in slick basically consists of these three steps:
Create a Query
Convert Query to an Action
Execute the Action on a database
So your example would look something like this (types are optional and added for clarity):
val query = mytable.length // length is an aggregation
val action = query.result
val result: Future[Int] = db.run(action)
// Access result in a non blocking way (recommended):
result.map(count: Int => ...)
// or for completeness use Await (not recommended):
val count: Int = Await.result(result, Duration.Inf)
Further reading:
Slick documentation, Queries
Futures and Promises in scala
have a table where formDefinitionId is nullable column in Postgres DB (9.3) . Slick version is 2.1.0
we have a this filter on a query
table.filter(_.formDefinitionId === request.formDefinitionId)
request.formDefinitionId where request is nothing but a Scala case class with formDefinitionId as an Option[Int]
case class Request(formDefinitionId : Option[Int])
now when request.formDefinitionId is None slick generates following query
(x2."form_definition_id" = null)
vs
(x2."form_definition_id” IS NULL) - this is what Postgres expects
Work around is to create a different filter based on request.formDefinitionId
e.g.
request.formDefinitionId match {
case Some(fid) => table.filter(_.formDefinitionId === fid)
case None => table.filter(_.formDefinitionId.isNull)
}
however creating such condition for every property which is Option isn't feasible - there exists numerous Option properties across many tables. Is there a better/generic way to achieve this? I would imagine this is a common scenario with people using Slick
Didn't test (I don't have a Slick setup here).
But as = null returns nothing, shouldn't something like this work?
table.filter(r => r.formDefinitionId.isNull || r.formDefinitionId === fid)
You can use the following solution with Slick 3.3.x:
table
// case 1: both are empty
.filterIf(request.formDefinitionId.isEmpty)(_.formDefinitionId.isEmpty)
// case 2: both are defined and equal
.filterOpt(request.formDefinitionId)(_.formDefinitionId === _)
case 1 corresponds the following SQL:
select * from table1 when form_definition_id is null;
case 2:
select * from table1 when form_definition_id = ?;
I am a newer to slick and now also confused with this problem. Maybe it's toolate to you.I wanted to do some search with different column(Option[text]). Here is my code ,It is worked to me.
val someone1 = cou.filter(_.id === id_S)
val someone2 = cou.filter(_.first_name === first)
val someone3 = cou.filter(_.last_name === last)
val someone = (someone1) union (someone2) union (someone3)
dbConfig.db.run(someone.result).map(_.toList)
I use Anorm to do database queries. When I do an executeUpdate(), how should I do proper error handling? it has return type MayErr[IntegrityConstraintViolation,Int], is this a Set or a Map?
There is an example, but I don't understand how I should handle the return value:
val result = SQL("delete from City where id = 99").executeUpdate().fold(
e => "Oops, there was an error" ,
c => c + " rows were updated!"
)
How do I check if the query failed? (using result), and how do I get the numer of affected rows if the query was successful?
At the moment I use this code:
SQL(
"""
INSERT INTO users (firstname, lastname) VALUES ({firstname}, {lastname})
"""
).on("firstname" -> user.firstName, "lastname" -> user.lastName)
.executeUpdate().fold(
e => "Oops, therw was an error",
c => c + " rows were updated!"
)
But I don't know how my error-handling code should look like. Is there any example on how to use the return value of type MayErr[IntegrityConstraintViolation,Int]?
It looks like MayErr is wrapping Either. So it's neither a Map nor a Set, but rather an object that can contain one of two differently typed objects.
Take a look at this question, and you'll see some ways of processing an Either object, which in this case contains either an IntegrityConstraintViolation or an Int. Referring to http://scala.playframework.org/.../Scala$MayErr.html, it looks like you can grab an Either object by referring to the value member e. There seems to be an implicit conversion available too, so you can just treat a MayErr[IntegrityConstraintViolation,Int] as an Either[IntegrityConstraintViolation,Int] without further ceremony.
You could obviously do a
val updateResult = ....executeUpdate()
val success = updateResult.fold(e => false, c => true)
It looks like you can also call
val success = updateResult.isRight
More generally, you can access the wrapped Either with
updateResult.e match {
case Left(error) => ... do something with error ...
case Right(updateCount) => ...do something with updateCount...
}
Maybe someone more familiar with Play would explain why scala.Either is wrapped in MayErr?