Enum in Plain SQL when using Slick 3.1 - postgresql

I'm using Slick 3.1.0 and Slick-pg 0.10.0. I have an enum as:
object UserProviders extends Enumeration {
type Provider = Value
val Google, Facebook = Value
}
Following the test case, it works fine with the column mapper simply adding the following implicit mapper into my customized driver.
implicit val userProviderMapper = createEnumJdbcType("UserProvider", UserProviders, quoteName = true)
However, when using plain SQL, I encountered the following compilation error:
could not find implicit value for parameter e: slick.jdbc.SetParameter[Option[models.UserProviders.Provider]]
I could not find any document about this. How can I write plain SQL with enum in slick? Thanks.

You need to have an implicit of type SetParameter[T] in scope which tells slick how to set parameters from some custom type T that it doesn't already know about. For example:
implicit val setInstant: SetParameter[Instant] = SetParameter { (instant, pp) =>
pp.setTimestamp(new Timestamp(instant.toEpochMilli))
}
The type of pp is PositionedParameters.
You might also come across the need to tell slick how to extract a query result into some custom type T that it doesn't already know about. For this, you need an implicit GetResult[T] in scope. For example:
implicit def getInstant(implicit get: GetResult[Long]): GetResult[Instant] =
get andThen (Instant.ofEpochMilli(_))

Related

Passing parameters to scala slick query [duplicate]

There is a similar question here but it doesn't actually answer the question.
Is it possible to use IN clause in plain sql Slick?
Note that this is actually part of a larger and more complex query, so I do need to use plain sql instead of slick's lifted embedding. Something like the following will be good:
val ids = List(2,4,9)
sql"SELECT * FROM coffee WHERE id IN ($ids)"
The sql prefix unlocks a StringContext where you can set SQL parameters. There is no SQL parameter for a list, so you can easily end up opening yourself up to SQL injection here if you're not careful. There are some good (and some dangerous) suggestions about dealing with this problem with SQLServer on this question. You have a few options:
Your best bet is probably to use the #$ operator together with mkString to interpolate dynamic SQL:
val sql = sql"""SELECT * FROM coffee WHERE id IN (#${ids.mkString(",")})"""
This doesn't properly use parameters and therefore might be open to SQL-injection and other problems.
Another option is to use regular string interpolation and mkString to build the statement:
val query = s"""SELECT * FROM coffee WHERE id IN (${ids.mkString(",")})"""
StaticQuery.queryNA[Coffee](query)
This is essentially the same approach as using #$, but might be more flexible in the general case.
If SQL-injection vulnerability is a major concern (e.g. if the elements of ids are user provided), you can build a query with a parameter for each element of ids. Then you'll need to provide a custom SetParameter instance so that slick can turn the List into parameters:
implicit val setStringListParameter = new SetParameter[List[String]]{
def apply(v1: List[String], v2: PositionedParameters): Unit = {
v1.foreach(v2.setString)
}
}
val idsInClause = List.fill(ids.length)("?").mkString("(", ",", ")")
val query = s"""SELECT * FROM coffee WHERE id IN ($idsInClause)"""
Q.query[List[String], String](query).apply(ids).list(s)
Since your ids are Ints, this is probably less of a concern, but if you prefer this method, you would just need to change the setStringListParameter to use Int instead of String:
val ids = List(610113193610210035L, 220702198208189710L)
implicit object SetListLong extends SetParameter[List[Long]] {
def apply(vList: List[Long], pp: PositionedParameters) {
vList.foreach(pp.setLong)
}
}
val select = sql"""
select idnum from idnum_0
where idnum in ($ids#${",?" * (ids.size - 1)})
""".as[Long]
#Ben Reich is right.
this is another sample code, test on slick 3.1.0.
($ids#${",?" * (ids.size - 1)})
Although this is not universal answer and may not be what the author wanted, I still want to point this out to whoever views this question.
Some DB backends support array types, and there are extensions to Slick that allow setting these array types in the interpolations.
For example, Postgres has the syntax where column = any(array), and with slick-pg you can use this syntax like so:
def query(ids: Seq[Long]) = db.run(sql"select * from table where ids = any($ids)".as[Long])
This brings a much cleaner syntax, which is friendlier to the statement compiler cache and also safe from SQL injections and overall danger of creating a malformed SQL with the #$var interpolation syntax.
I have written a small extension to Slick that addresses exactly this problem: https://github.com/rtkaczyk/inslick
For the given example the solution would be:
import accode.inslick.syntax._
val ids = List(2,4,9)
sqli"SELECT * FROM coffee WHERE id IN *$ids"
Additionally InSlick works with iterables of tuples or case classes. It's available for all Slick 3.x versions and Scala versions 2.11 - 2.13. We've been using it in production for several months at the company I work for.
The interpolation is safe from SQL injection. It utilises a macro which rewrites the query in a way similar to trydofor's answer
Ran into essentially this same issue in Slick 3.3.3 when trying to use a Seq[Long] in an IN query for MySQL. Kept getting a compilation error from Slick of:
could not find implicit value for parameter e: slick.jdbc.SetParameter[Seq[Long]]
The original question would have been getting something like:
could not find implicit value for parameter e: slick.jdbc.SetParameter[List[Int]]
Slick 3.3.X+ can handle binding the parameters for the IN query, as long as we provide the implicit definition of how Slick should do so for the types we're using. This means adding the implicit val definition somewhere at the class level. So, like:
class MyClass {
// THIS IS THIS KEY LINE TO ENABLE SLICK TO BIND THE PARAMS
implicit val setListInt = SetParameter[List[Int]]((inputList, params) => inputList.foreach(params.setInt))
def queryByHardcodedIds() = {
val ids: List[Int] = List(2,4,9)
sql"SELECT * FROM coffee WHERE id IN ($ids)" // SLICK CAN AUTO-HANDLE BINDING NOW
}
}
Similar for the case of Seq[Long] & others. Just make sure your types/binding aligns to what you need Slick to handle:
implicit val setSeqLong = SetParameter[Seq[Long]]((inputList, params) => inputList.foreach(params.setLong))
// ^^Note the `SetParameter[Seq[Long]]` & `.setLong` for type alignment

Slick "===" compiling only in for comprehension

I am experiencing a strange behaviour with slick and I would like some help figuring out why it happens. The issue is that I have a query that reads as follows:
db.withSession { implicit session =>
tables.users.where(_.id === UserId(1)).firstOption
}
This does not compile producing an error as follows:
inferred type arguments [Boolean] do not conform to method where's type parameter bounds [T <: scala.slick.lifted.Column[_]]
But if I rewrite the code as:
db.withSession { implicit session =>
(for {
u <- tables.users if u.id === UserId(1)
} yield u).firstOption
}
It compiles and works fine.
The table is defined as follows:
class Users(tag: Tag) extends Table[User](tag, "users") {
def id = column[UserId]("id", O.PrimaryKey, O.AutoInc, O.NotNull)
}
And I have an implicit conversion to map the UserId type:
implicit lazy val userIdColumnType = MappedColumnType.base[UserId, Int](_.value, UserId(_))
It looks like a type inference problem, but I can't really understand why it should happen.
Anyone has any on why this should behave differently in the two scenario I reported?
EDIT: After some investigation I found that when using where the implicit conversion for the userIdColumnType has to be in scope, while with the for comprehension it is not needed. Is there a good explanation for this?
You are using === from ScalaTest. It returns a Boolean. Slick's === returns a Column[Boolean]. The methods filter and where prevent using Boolean (at least in the latest version of Slick), to protect you from accidentally using == or also from using ScalaTest's === in your case, which does a local comparison of the underlying values instead of an equality comparison in the database, which is what you actualy want. For comprehensions are desugared to withFilter and can sometimes generate a Boolean value, so unfortunately we cannot disallow Boolean for comprehensions.
To fix this you need to make sure, that Slick's === is picked in queries. Maybe you can affect this with the import order or scope. Or if you are unlucky you can't and they are incompatible.
I am not sure how the userIdColumnType interacts here at the moment.
I fixed this by importing my driver api and making sure it was in scope
i.e. as I'm using a postgresql driver,
import PostgresProfile.api._

Trouble with ReactiveMongo's BSON macros and generics

The following code fails for me:
object Message {
def parse[T](bsonDoc: BSONDocument): Try[T] = {
implicit val bsonHandler = Macros.handler[T]
bsonDoc.seeAsTry[T]
}
}
Message.parse[messages.ClientHello](data)
The error is:
No apply function found for T
implicit val bsonHandler = Macros.handler[T]
^
However, if I hardcode a type (one of my case classes), it's fine:
object Message {
def parse(bsonDoc: BSONDocument): Try[ClientHello] = {
implicit val bsonHandler = Macros.handler[ClientHello]
bsonDoc.seeAsTry[ClientHello]
}
}
Message.parse(data)
So I presume this is a problem using generics. Incidentally, I have to import messages.ClientHello. If I just use messages.ClientHello I get:
not found: value ClientHello
implicit val bsonHandler = Macros.handler[messages.ClientHello]
^
How can I achieve what I'm trying to do, which is to have a single method that will take a BSON document and return an instance of the appropriate case class?
1) Macro applications get expanded immediately when encountered (well, modulo some fine details of type inference that are irrelevant here). This means that when you write handler[T], handler will try to expand with T as a type parameter. This won't lead to anything good, hence the error. To make this work, you need to turn Message.parse into a macro itself.
2) This happens because ReactiveMongo macros are unhygienic. Specifically, https://github.com/ReactiveMongo/ReactiveMongo/blob/v0.10.0/macros/src/main/scala/macros.scala#L142 isn't going to work correctly in situations like yours, because it uses simple name of the class, not a fully qualified name. I think the best way to make the macro work correctly would be using Ident(companion), not Ident(companion.name) - that would ensure that this identifier binds to the companion, not to something in scope having the same name.

How to implement a generic REST api for tables in Play2 with squeryl and spray-json

I'm trying to implement a controller in Play2 which exposes a simple REST-style api for my db-tables. I'm using squeryl for database access and spray-json for converting objects to/from json
My idea is to have a single generic controller to do all the work, so I've set up the following routes in conf/routes:
GET /:tableName controllers.Crud.getAll(tableName)
GET /:tableName/:primaryKey controllers.Crud.getSingle(tableName, primaryKey)
.. and the following controller:
object Crud extends Controller {
def getAll(tableName: String) = Action {..}
def getSingle(tableName: String, primaryKey: Long) = Action {..}
}
(Yes, missing create/update/delete, but let's get read to work first)
I've mapped tables to case classes by extended squeryl's Schema:
object MyDB extends Schema {
val accountsTable = table[Account]("accounts")
val customersTable = table[Customer]("customers")
}
And I've told spray-json about my case classes so it knows how to convert them.
object MyJsonProtocol extends DefaultJsonProtocol {
implicit val accountFormat = jsonFormat8(Account)
implicit val customerFormat = jsonFormat4(Customer)
}
So far so good, it actually works pretty well as long as I'm using the table-instances directly. The problem surfaces when I'm trying to generify the code so that I end up with excatly one controller for accessing all tables: I'm stuck with some piece of code that doesn't compile and I am not sure what's the next step.
It seems to be a type issue with spray-json which occurs when I'm trying to convert the list of objects to json in my getAll function.
Here is my generic attempt:
def getAll(tableName: String) = Action {
val json = inTransaction {
// lookup table based on url
val table = MyDB.tables.find( t => t.name == tableName).get
// execute select all and convert to json
from(table)(t =>
select(t)
).toList.toJson // causes compile error
}
// convert json to string and set correct content type
Ok(json.compactPrint).as(JSON)
}
Compile error:
[error] /Users/code/api/app/controllers/Crud.scala:29:
Cannot find JsonWriter or JsonFormat type class for List[_$2]
[error] ).toList.toJson
[error] ^
[error] one error found
I'm guessing the problem could be that the json-library needs to know at compile-time which model type I'm throwing at it, but I'm not sure (notice the List[_$2] in that compile error). I have tried the following changes to the code which compile and return results:
Remove the generic table-lookup (MyDB.tables.find(.....).get) and instead use the specific table instance e.g. MyDB.accountsTable. Proves that JSON serialization for work . However this is not generic, will require a unique controller and route config per table in db.
Convert the list of objects from db query to a string before calling toJson. I.e: toList.toJson --> toList.toString.toJson. Proves that generic lookup of tables work But not a proper json response since it is a string-serialized list of objects..
Thoughts anyone?
Your guess is correct. MyDb.tables is a Seq[Table[_]], in other words it could hold any type of table. There is no way for the compiler to figure out the type of the table you locate using the find method, and it seems like the type is needed for the JSON conversion. There are ways to get around that, but you'd need to some type of access to the model class.

How to dump all tables using Squeryl

I'm trying to write a quick data browser for a database using Squeryl but I have difficulty iterating over all the tables in a generic way. Based on the Squeryl SchoolDb example i tried the following:
def browseTable(name: String) = {
SchoolDb.tables.find(_.name == name) map { t=>
val fields = t.posoMetaData.fieldsMetaData
val rows = from (t) (s => select(s))
// Print the columns
println(fields.map(_.columnName).mkString("\t"))
rows map { row =>
println(fields.map(f => f.get(row)).mkstring("\t"))
}
}
The compiler is not very happy with this attempt (Missing type type for 'row') and I can sort-of understand its dilemma. Explicitly declaring the parametr as Any just changes the comilation error to "No implicit view available from Any => org.squeryl.dsl.ast.TypedExpressionNode[_]" on 'f.get(row)'
How Can I either fix this issue or change the models (maybe adding a trait of some sort) to enable generic access to all data in all tables?
The compiler complains because f.get method expects an AnyRef parameter. AFAIK, in Scala everything can be safely cast to AnyRef - the compiler will force necessary boxing if needed. So I think this should work: f.get(row.asInstanceOf[AnyRef])
EDIT: I just tested this and it works.