How to create Anorm query to skip updating None values in DB (Scala) - scala

I am using Anorm (2.5.1) in my Play+Scala application (2.5.x, 2.11.11). I keep facing the issue quite often where if the case class argument value is None, I don't want that parameter value to be inserted/updated in SQL DB. For example:
case class EditableUser(
user_pk: String,
country: Option[String],
country_phone_code: Option[Int],
phonenumber: Option[String],
emailid: Option[String],
format_all: Option[String]
)
....
val eUser: EditableUser = EditableUser("PK0001", None, None, None, Some("xyz#email.com"), Some("yes"))
...
SQL"""
update #$USR SET
COUNTRY=${eUser.country},
COUNTRY_PHONE_CODE=${eUser.country_phone_code},
PHONENUMBER=${eUser.phonenumber},
EMAILID=${emailid},
FORMAT_ALL=${format_all}
where (lower(USER_PK)=lower(${eUser.user_pk}))
""".execute()
Here when the value is None, Anorm will insert 'null' into corresponding column in SQL DB. Instead I want to write the query in such a way that Anorm skips updating those values which are None i.e. does not overwrite.

You should use boundStatements/preparedStatement and while setting values for the query don’t set the values for the columns which are none.
For example
SQL(
"""
select * from Country c
join CountryLanguage l on l.CountryCode = c.Code
where c.code = {countryCode};
"""
).on("countryCode" -> "FRA")
Or in your case:
import play.api.db.DB
import anorm._
val stat = DB.withConnection(implicit c =>
SQL("SELECT name, email FROM user WHERE id={id}").on("id" -> 42)
)
While writing you your query you check if the value you are going to put in on(x->something) is not None if it’s nice don’t put it hence you will not update the values which are none.

Without the ability (or library) to access the attribute names themselves, it would still be possible, if slightly clunky in some circles, to build the update statement dynamically depending on the values that are present in the case class:
case class Foo(name:String, age:Option[Int], heightCm:Option[Int])
...
def phrase(k:String,v:Option[Int]):String=if (v.isDefined) s", $k={$k}" else ""
def update(foo:Foo) : Either[String, Foo] = DB.withConnection { implicit c =>
def stmt(foo:Foo) = "update foo set "+
//-- non option fields
"name={name}" +
//-- option fields
phrase("age", foo.age) +
phrase("heightCm", foo.heightCm)
SQL(stmt(foo))
.on('name -> name, 'age -> age, 'heightCm -> heightCm)
.executeUpdate()
The symbols that are not present in the actual submitted SQL can still be specified in the on. Catering for other data types also needed.

Related

How to update table query in Slick

How can I convert Query[MappedProjection[Example, (Option[String], Int, UUID, UUID)], Example, Seq] to Query[Examples, Example, Seq]?
Details
I am trying to drop a column from an existing table(Examples in this case) and move the data to another table (Examples2 in this case). I don't want to change all the existing code base, so I plan to join these two tables and map the results to Example.
import slick.lifted.Tag
import slick.driver.PostgresDriver.api._
import java.util.UUID
case class Example(
field1: Option[String] = None,
field2: Int,
someForeignId: UUID,
id: UUID,
)
object Example
class Examples(tag: Tag) extends Table[Example](tag, "entityNotes") {
def field1 = column[Option[String]]("field1")
def field2 = column[Int]("field2")
def someForeignId = column[UUID]("someForeignId")
def id = column[UUID]("id", O.PrimaryKey)
def someForeignKey = foreignKey(
"someForeignIdToExamples2",
someForeignId,
Examples2.query,
)(
_.id.?
)
def * =
(
field1.?,
field2,
someForeignId,
id,
) <> ((Example.apply _).tupled, Example.unapply)
}
object Examples{
val query = TableQuery[Examples]
}
Basically, all the functions in the codebase call Examples.query. If I update that query by joining two tables, the problem will be solved (of course with a performance shortcoming because of one extra join for each call).
To use the query with the existing code base, we need to keep the type the same. For example, we we can use filter as follows:
val query_ = TableQuery[Examples]
val query: Query[Examples, Example, Seq] = query_.filter(_.field2 > 5)
Everything will work without a problem since we keep the type of the query as it is supposed to be.
However, I cannot do that with a join if I want to use data from the second table.
val query_ = TableQuery[Examples]
val query = query
.join(Examples2.query_)
.on(_.someForeignId === _.id)
.map({
case (e, e2) =>
((
e2.value.?,
e1.field2,
e2.id
e.id,
) <> ((Example.apply _).tupled, Example.unapply))
})
This is where I got stuck. Its type is Query[MappedProjection[Example, (Option[String], Int, UUID, UUID)], Example, Seq].
Can anyone help? Btw, we don't have to use map. This is just what I got so far.

Slick: could not find implicit value for parameter e: slick.jdbc.SetParameter[Option[java.util.UUID]]

I'm workihng with Play! 2.5, Slick 3 and PostgreSQL 9.6.
I'm trying to use a simple plain SQL query with an optional UUID but I get this error:
Slick: could not find implicit value for parameter e: slick.jdbc.SetParameter[Option[java.util.UUID]]
My case class looks like
final case class GaClientId(ip: String, userId: Option[UUID], clientId: String)
And my query like this:
db.run(
sql"""
INSERT INTO ga_client_id(ip, user_id, clientId) VALUES (
${gaClientId.ip}, ${gaClientId.userId}, $gaClientId.userId.orNull)
ON CONFLICT DO NOTHING;""")
I tried to add this:
implicit val getUUIDContent: GetResult[GaClientId] =
GetResult(r => GaClientId(r.<<, r.nextStringOption().map(UUID.fromString), r.<<))
but without result.
How can I do this?
As the error implies you should specify SetParameter not GetParameter. It should be as follows:
implicit val uuidSetter = SetParameter[Option[UUID]] {
case (Some(uuid), params) => params.setString(uuid.toString)
case (None, params) => params.setNull(Types.VARCHAR)
}
And then you should probably add asUpdate to your update SQL:
db.run((sql"""
INSERT INTO ga_client_id(ip, user_id, clientId) VALUES (
${gaClientId.ip}, ${gaClientId.userId}, ${gaClientId.clientId})
ON CONFLICT DO NOTHING;
""").asUpdate)
Based on the answer of #Paul Doelga, you can additionally try
db.run((sql"""
INSERT INTO ga_client_id(ip, user_id, clientId) VALUES (
${gaClientId.ip}, ${gaClientId.userId}::uuid, ${gaClientId.clientId})
ON CONFLICT DO NOTHING;
""").asUpdate)
I originally got the answer from
https://github.com/skinny-framework/skinny-framework/issues/301
I guess the main reason of the failure is that Postgres need a parameter in type UUID, but JDBC doesn't support it, so you have to convert it to String, and convert back to UUID in SQL.

strategy for loading related entities with slick 2

I am using play 2.3 with slick 2.1
I have two related entities - Message and User (a simplified example domain). Messages are written by users.
A recommended way (the only way?) of expressing such a relation is by using explicit userId in Message
My classes and table mappings look like this:
case class Message (
text: String,
userId: Int,
date: Timestamp = new Timestamp(new Date().getTime()),
id: Option[Int] = None) {}
case class User (
userName: String,
displayName: String,
passHash: String,
creationDate: Timestamp = new Timestamp(new Date().getTime()),
lastVisitDate: Option[Timestamp] = None,
// etc
id: Option[Int] = None){}
class MessageTable(tag: Tag) extends Table[Message](tag, "messages") {
def id = column[Int]("id", O.PrimaryKey, O.AutoInc)
def text = column[String]("text")
def userId = column[Int]("user_id")
def date = column[Timestamp]("creation_date")
def * = (title, text, userId, date, id.?) <> (Post.tupled, Post.unapply
def author = foreignKey("message_user_fk", userId, TableQuery[UserTable])(_.id)
}
class UserTable(tag: Tag) extends Table[User](tag, "USER") {
def id = column[Int]("id", O.PrimaryKey, O.AutoInc)
def username = column[String]("username")
def passHash = column[String]("password")
def displayname = column[String]("display_name")
def * = (username, passHash,created, lastvisit, ..., id.?) <> (User.tupled, User.unapply)
}
And a convenient helper object:
object db {
object users extends TableQuery(new UserTable(_)) {
// helper methods, compiled queries
}
object messages extends TableQuery(new MessageTable(_)) {
// helper methods, compiled queries
}
}
Now, this is all perfect internally, but if I want to display the actual message, I want the message class to be able to return it's author name when used in templates.
Here are my considerations:
I don't want (and I wasn't able to anyway) to pass around implicit slick Session where it doesn't belong - like templating engine and model classes
I'd like to avoid requesting additional data for messages one-by-one in this particular case
I am more familiar with Hibernate than Slick; in Hibernate I'd use join-fetching. With Slick, the best idea I came up with is to use another data holder class for display:
class LoadedMessage (user:User, message:Message) {
// getters
def text = message.text
def date = message.date
def author = user.displayName
}
object LoadedMessage {
def apply( u:User , m:Message ) = new LoadedMessage(u, m)
}
and populate it with results of join query:
val messageList: List[LoadedMessage] = (
for (
u <- db.users;
m <- db.messages if u.id === m.userId
) yield (u, m))
.sortBy({ case (u, m) => m.date })
.drop(n)
.take(amount)
.list.map { case (u, m) => LoadedMessage(u, m) }
and then pass it wherever. My concern is with this extra class - not very DRY, so unnecessary conversion (and doesn't seem I can make it implicit), much boilerplate.
What is the common approach?
Is there a way to cut on extra classes and actually make a model able to return it's associations?
Following my comment:
How to handle join results is a matter of personal taste in my opinion, your approach is what I would also use, you have an ad hoc data structure which encapsulate your data and can be easily passed around (for example in views) and accessed.
Two other approaches which comes to mind are
query for fields instead of objects, it's less legible and I usually hate working with tuples (or triples in this case) because I find the notation _._1 way less legible than MyClass.myField. This means doing something like this:
val messageList = (
for (
u <- db.users;
m <- db.messages if u.id === m.userId
) yield (u.displayName, m.date, m.text))
.sortBy({ case (name, date, text) => date })
.drop(n)
.take(amount)
.list()
Which will return a triple and then can be passed in your view, something that is possible but I would never do.
Another option is to pass the tuple of objects, very similar to your approach except for the last map
val messageList: List[(User, Message)] = (
for (
u <- db.users;
m <- db.messages if u.id === m.userId
) yield (u, m))
.sortBy({ case (u, m) => m.date })
.drop(n)
.take(amount)
.list()
Here you can pass in the view something like this:
#(messagesAndAuthors: List[(User, Message)])
and then access the data using tuples and classes access functionality, but again your template would be a collection of _1s and again that is horrible to read, plus you have to do something like messagesAndAuthors._1.name just to get one value.
In the end I prefer passing variables as clean as I can in my views (and I guess this is one of the few universally accepted principle in computer science) and hide the logic used in models, for this using an ad hoc case class is the way to go in my opinion. Case classes are a very powerful tool and it's nice to have it, to you it may looks not DRY and more verbose than other approaches (like Hibernate you talked about) but think that when a developer will read you code it will be as easy as it can get in this moment and that extra small class will save hours of head banging.
For the Session trouble see this related question, at the moment is not possible to avoid passing it around (as far as I know) but you should be able to contain this situation and avoid passing session in your template, I usually create a session in my entry point (most of the time a controller) and pass it to the model.
Note: the code is untested, there may be some mistakes.

Access database column names from a Table?

Let's say I have a table:
object Suppliers extends Table[(Int, String, String, String)]("SUPPLIERS") {
def id = column[Int]("SUP_ID", O.PrimaryKey)
def name = column[String]("SUP_NAME")
def state = column[String]("STATE")
def zip = column[String]("ZIP")
def * = id ~ name ~ state ~ zip
}
Table's database name
The table's database name can be accessed by going: Suppliers.tableName
This is supported by the Scaladoc on AbstractTable.
For example, the above table's database name is "SUPPLIERS".
Columns' database names
Looking through AbstractTable, getLinearizedNodes and indexes looked promising. No column names in their string representations though.
I assume that * means "all the columns I'm usually interested in." * is a MappedProjection, which has this signature:
final case class MappedProjection[T, P <: Product](
child: Node,
f: (P) ⇒ T,
g: (T) ⇒ Option[P])(proj: Projection[P])
extends ColumnBase[T] with UnaryNode with Product with Serializable
*.getLinearizedNodes contains a huge sequence of numbers, and I realized that at this point I'm just doing a brute force inspection of everything in the API for possibly finding the column names in the String.
Has anybody also encountered this problem before, or could anybody give me a better understanding of how MappedProjection works?
It requires you to rely on Slick internals, which may change between versions, but it is possible. Here is how it works for Slick 1.0.1: You have to go via the FieldSymbol. Then you can extract the information you want like how columnInfo(driver: JdbcDriver, column: FieldSymbol): ColumnInfo does it.
To get a FieldSymbol from a Column you can use fieldSym(node: Node): Option[FieldSymbol] and fieldSym(column: Column[_]): FieldSymbol.
To get the (qualified) column names you can simply do the following:
Suppliers.id.toString
Suppliers.name.toString
Suppliers.state.toString
Suppliers.zip.toString
It's not explicitly stated anywhere that the toString will yield the column name, so your question is a valid one.
Now, if you want to programmatically get all the column names, then that's a bit harder. You could try using reflection to get all the methods that return a Column[_] and call toString on them, but it wouldn't be elegant. Or you could hack a bit and get a select * SQL statement from a query like this:
val selectStatement = DB withSession {
Query(Suppliers).selectStatement
}
And then parse our the column names.
This is the best I could do. If someone knows a better way then please share - I'm interested too ;)
Code is based on Lightbend Activator "slick-http-app".
slick version: 3.1.1
Added this method to the BaseDal:
def getColumns(): mutable.Map[String, Type] = {
val columns = mutable.Map.empty[String, Type]
def selectType(t: Any): Option[Any] = t match {
case t: TableExpansion => Some(t.columns)
case t: Select => Some(t.field)
case _ => None
}
def selectArray(t:Any): Option[ConstArray[Node]] = t match {
case t: TypeMapping => Some(t.child.children)
case _ => None
}
def selectFieldSymbol(t:Any): Option[FieldSymbol] = t match {
case t: FieldSymbol => Some(t)
case _ => None
}
val t = selectType(tableQ.toNode)
val c = selectArray(t.get)
for (se <- c.get) {
val col = selectType(se)
val fs = selectFieldSymbol(col.get)
columns += (fs.get.name -> fs.get.tpe)
}
columns
}
this method gets the column names (real names in DB) + types form the TableQ
used imports are:
import slick.ast._
import slick.util.ConstArray

Scala slick query where in list

I am attempting to learn to use Slick to query MySQL. I have the following type of query working to get a single Visit object:
Q.query[(Int,Int), Visit]("""
select * from visit where vistor = ? and location_code = ?
""").firstOption(visitorId,locationCode)
What I would like to know is how can I change the above to query to get a List[Visit] for a collection of Locations...something like this:
val locationCodes = List("loc1","loc2","loc3"...)
Q.query[(Int,Int,List[String]), Visit]("""
select * from visit where vistor = ? and location_code in (?,?,?...)
""").list(visitorId,locationCodes)
Is this possible with Slick?
As the other answer suggests, this is cumbersome to do with static queries. The static query interface requires you to describe the bind parameters as a Product. (Int, Int, String*)
is not valid scala, and using (Int,Int,List[String]) needs some kludges as well. Furthermore, having to ensure that locationCodes.size is always equal to the number of (?, ?...) you have in your query is brittle.
In practice, this is not too much of a problem because you want to be using the query monad instead, which is the type-safe and recommended way to use Slick.
val visitorId: Int = // whatever
val locationCodes = List("loc1","loc2","loc3"...)
// your query, with bind params.
val q = for {
v <- Visits
if v.visitor is visitorId.bind
if v.location_code inSetBind locationCodes
} yield v
// have a look at the generated query.
println(q.selectStatement)
// run the query
q.list
This is assuming you have your tables set up like this:
case class Visitor(visitor: Int, ... location_code: String)
object Visitors extends Table[Visitor]("visitor") {
def visitor = column[Int]("visitor")
def location_code = column[String]("location_code")
// .. etc
def * = visitor ~ .. ~ location_code <> (Visitor, Visitor.unapply _)
}
Note that you can always wrap your query in a method.
def byIdAndLocations(visitorId: Int, locationCodes: List[String]) =
for {
v <- Visits
if v.visitor is visitorId.bind
if v.location_code inSetBind locationCodes
} yield v
}
byIdAndLocations(visitorId, List("loc1", "loc2", ..)) list
It doesn't work because the StaticQuery object (Q) expects to implicitly set the parameters in the query string, using the type parameters of the query method to create a sort of setter object (of type scala.slick.jdbc.SetParameter[T]).
The role of SetParameter[T] is to set a query parameter to a value of type T, where the required types are taken from the query[...] type parameters.
From what I see there's no such object defined for T = List[A] for a generic A, and it seems a sensible choice, since you can't actually write a sql query with a dynamic list of parameters for the IN (?, ?, ?,...) clause
I did an experiment by providing such an implicit value through the following code
import scala.slick.jdbc.{SetParameter, StaticQuery => Q}
def seqParam[A](implicit pconv: SetParameter[A]): SetParameter[Seq[A]] = SetParameter {
case (seq, pp) =>
for (a <- seq) {
pconv.apply(a, pp)
}
}
implicit val listSP: SetParameter[List[String]] = seqParam[String]
with this in scope, you should be able to execute your code
val locationCodes = List("loc1","loc2","loc3"...)
Q.query[(Int,Int,List[String]), Visit]("""
select * from visit where vistor = ? and location_code in (?,?,?...)
""").list(visitorId,locationCodes)
But you must always manually guarantee that the locationCodes size is the same as the number of ? in your IN clause
In the end I believe that a cleaner workaround could be created using macros, to generalize on the sequence type. But I'm not sure it would be a wise choice for the framework, given the aforementioned issues with the dynamic nature of the sequence size.
You can generate in clause automaticly like this:
def find(id: List[Long])(implicit options: QueryOptions) = {
val in = ("?," * id.size).dropRight(1)
Q.query[List[Long], FullCard](s"""
select
o.id, o.name
from
organization o
where
o.id in ($in)
limit
?
offset
?
""").list(id ::: options.limits)
}
And use implicit SetParameter as pagoda_5b says
def seqParam[A](implicit pconv: SetParameter[A]): SetParameter[Seq[A]] = SetParameter {
case (seq, pp) =>
for (a <- seq) {
pconv.apply(a, pp)
}
}
implicit def setLongList = seqParam[Long]
If you have a complex query and the for comprehension mentioned above is not an option, you can do something like the following in Slick 3. But you need to make sure you validate the data in your list query parameter yourself to prevent SQL injection:
val locationCodes = "'" + List("loc1","loc2","loc3").mkString("','") + "'"
sql"""
select * from visit where visitor = $visitor
and location_code in (#$locationCodes)
"""
The # in front of the variable reference disables the type validation and allows you to solve this without supplying a function for the implicit conversion of the list query parameter.