I have a table called Materials. I used slicks schema auto generation to create the TableQuery classes for me.
I can't figure out how to just print the results of a simple query.
Materials.map(_.name)
I've tried
val m = Materials.map(_.name).toString()
println(m)
and get the result
Rep(Bind)
if I try
Materials.map(_.name).forEach(m => println(m))
I get a compile error
value forEach is not a member of slick.lifted.Query[slick.lifted.Rep[Option[String]],Option[String],Seq]
To clarify I'm using just slick 3.1.0 not play slick
You have written a Query, but it needs converted into an Action by calling its result method
val query = materials.map(_.name)
val action = query.result
val results: Future[ Seq[Option[ String ] ]] = db.run( action)
results.foreach( println )
The db object needs to be initialized depending the Slick version that you are using . e.g Slick or Play Slick
I assume that you have this
val materials = TableQuery[Materials]
You can evaluate function with side effects using map:
Materials.map(println(_.name))
Related
There is a similar question here but it doesn't actually answer the question.
Is it possible to use IN clause in plain sql Slick?
Note that this is actually part of a larger and more complex query, so I do need to use plain sql instead of slick's lifted embedding. Something like the following will be good:
val ids = List(2,4,9)
sql"SELECT * FROM coffee WHERE id IN ($ids)"
The sql prefix unlocks a StringContext where you can set SQL parameters. There is no SQL parameter for a list, so you can easily end up opening yourself up to SQL injection here if you're not careful. There are some good (and some dangerous) suggestions about dealing with this problem with SQLServer on this question. You have a few options:
Your best bet is probably to use the #$ operator together with mkString to interpolate dynamic SQL:
val sql = sql"""SELECT * FROM coffee WHERE id IN (#${ids.mkString(",")})"""
This doesn't properly use parameters and therefore might be open to SQL-injection and other problems.
Another option is to use regular string interpolation and mkString to build the statement:
val query = s"""SELECT * FROM coffee WHERE id IN (${ids.mkString(",")})"""
StaticQuery.queryNA[Coffee](query)
This is essentially the same approach as using #$, but might be more flexible in the general case.
If SQL-injection vulnerability is a major concern (e.g. if the elements of ids are user provided), you can build a query with a parameter for each element of ids. Then you'll need to provide a custom SetParameter instance so that slick can turn the List into parameters:
implicit val setStringListParameter = new SetParameter[List[String]]{
def apply(v1: List[String], v2: PositionedParameters): Unit = {
v1.foreach(v2.setString)
}
}
val idsInClause = List.fill(ids.length)("?").mkString("(", ",", ")")
val query = s"""SELECT * FROM coffee WHERE id IN ($idsInClause)"""
Q.query[List[String], String](query).apply(ids).list(s)
Since your ids are Ints, this is probably less of a concern, but if you prefer this method, you would just need to change the setStringListParameter to use Int instead of String:
val ids = List(610113193610210035L, 220702198208189710L)
implicit object SetListLong extends SetParameter[List[Long]] {
def apply(vList: List[Long], pp: PositionedParameters) {
vList.foreach(pp.setLong)
}
}
val select = sql"""
select idnum from idnum_0
where idnum in ($ids#${",?" * (ids.size - 1)})
""".as[Long]
#Ben Reich is right.
this is another sample code, test on slick 3.1.0.
($ids#${",?" * (ids.size - 1)})
Although this is not universal answer and may not be what the author wanted, I still want to point this out to whoever views this question.
Some DB backends support array types, and there are extensions to Slick that allow setting these array types in the interpolations.
For example, Postgres has the syntax where column = any(array), and with slick-pg you can use this syntax like so:
def query(ids: Seq[Long]) = db.run(sql"select * from table where ids = any($ids)".as[Long])
This brings a much cleaner syntax, which is friendlier to the statement compiler cache and also safe from SQL injections and overall danger of creating a malformed SQL with the #$var interpolation syntax.
I have written a small extension to Slick that addresses exactly this problem: https://github.com/rtkaczyk/inslick
For the given example the solution would be:
import accode.inslick.syntax._
val ids = List(2,4,9)
sqli"SELECT * FROM coffee WHERE id IN *$ids"
Additionally InSlick works with iterables of tuples or case classes. It's available for all Slick 3.x versions and Scala versions 2.11 - 2.13. We've been using it in production for several months at the company I work for.
The interpolation is safe from SQL injection. It utilises a macro which rewrites the query in a way similar to trydofor's answer
Ran into essentially this same issue in Slick 3.3.3 when trying to use a Seq[Long] in an IN query for MySQL. Kept getting a compilation error from Slick of:
could not find implicit value for parameter e: slick.jdbc.SetParameter[Seq[Long]]
The original question would have been getting something like:
could not find implicit value for parameter e: slick.jdbc.SetParameter[List[Int]]
Slick 3.3.X+ can handle binding the parameters for the IN query, as long as we provide the implicit definition of how Slick should do so for the types we're using. This means adding the implicit val definition somewhere at the class level. So, like:
class MyClass {
// THIS IS THIS KEY LINE TO ENABLE SLICK TO BIND THE PARAMS
implicit val setListInt = SetParameter[List[Int]]((inputList, params) => inputList.foreach(params.setInt))
def queryByHardcodedIds() = {
val ids: List[Int] = List(2,4,9)
sql"SELECT * FROM coffee WHERE id IN ($ids)" // SLICK CAN AUTO-HANDLE BINDING NOW
}
}
Similar for the case of Seq[Long] & others. Just make sure your types/binding aligns to what you need Slick to handle:
implicit val setSeqLong = SetParameter[Seq[Long]]((inputList, params) => inputList.foreach(params.setLong))
// ^^Note the `SetParameter[Seq[Long]]` & `.setLong` for type alignment
I'm using Slick 3.1.0 and Slick-pg 0.10.0. I have an enum as:
object UserProviders extends Enumeration {
type Provider = Value
val Google, Facebook = Value
}
Following the test case, it works fine with the column mapper simply adding the following implicit mapper into my customized driver.
implicit val userProviderMapper = createEnumJdbcType("UserProvider", UserProviders, quoteName = true)
However, when using plain SQL, I encountered the following compilation error:
could not find implicit value for parameter e: slick.jdbc.SetParameter[Option[models.UserProviders.Provider]]
I could not find any document about this. How can I write plain SQL with enum in slick? Thanks.
You need to have an implicit of type SetParameter[T] in scope which tells slick how to set parameters from some custom type T that it doesn't already know about. For example:
implicit val setInstant: SetParameter[Instant] = SetParameter { (instant, pp) =>
pp.setTimestamp(new Timestamp(instant.toEpochMilli))
}
The type of pp is PositionedParameters.
You might also come across the need to tell slick how to extract a query result into some custom type T that it doesn't already know about. For this, you need an implicit GetResult[T] in scope. For example:
implicit def getInstant(implicit get: GetResult[Long]): GetResult[Instant] =
get andThen (Instant.ofEpochMilli(_))
I'm trying to apply a series of optional filtering operations to a query by using a list of the operations and folding over the list.
val table = TableQuery[Fizz]
val filters = List(filter1(option1)_, filter2(option2)_, filter3(option3)_)
val filteredQuery = filters.foldLeft(table){(q, filter) => filter(q)}
The partially applied filter functions have a signature of
Query[Fizz, FizzRow, Seq] => Query[Fizz, FizzRow, Seq]
Basically, in each function, I am optionally applying the filtering if the filter parameter option* is present. However, the compiler does not like the fact that I am passing in a TableQuery to a function that takes Query, even though TableQuery is a subtype of Query. Is there a way to convert a TableQuery to Query? Or a better way to go about chaining filter functions on a query?
The compiler error in question is
type mismatch;
found :scala.slick.lifted.Query[generated.Tables.Farm,generated.Tables.FarmRow,Seq]
required: scala.slick.lifted.TableQuery[generated.Tables.Farm]
I can get it to compile by using table.drop(0) instead of table but obviously that seems like a poor workaround. I see that there's a to method on TableQuery that converts it to a Query but it also takes an implicit ctc: TypedCollectionTypeConstructor[D].
An example of one of the filterX functions listed above:
def filterCharacteristics(characteristics: Option[List[Int]])(table: Query[Farm,FarmRow,Seq]) = {
characteristics.map(chars =>
(for {
(fc, f) <- Farmcharacteristic.filter(_.characteristicId inSet chars) join table on (_.farmId === _.farmId)
} yield f)).getOrElse(table)
}
I think you can try another approach. Instead of using a fold, you can use a collect to get only the Some values.
Then you can apply a filter to each of the options you have:
val table = TableQuery[Fizz]
val filteredQueries = List(Some(option1), Some(option2), Some(option3)) collect {
case Some(option) => option
} map { currentOption =>
table.filter(currentOption)
}
// We need to get the last value or the TableQuery
val lastValue = filteredQueries reverse headOption
// Or we have Some(Query) or None, In case it is a None, we will use table
lastValue.getOrElse(table)
I am trying to run a query like .filter(_.id === 1).firstOption but the compiler complains there is no symbol firstOption. Was this removed in slick 3? What can I use instead?
To limit the number of results before calling result, use take(num). For example like this:
val result: Future[Option[Whatever]] = db.run((query.filter(_.id === 1).take(1)).result).map(_.headOption)
According to the official docs, the above statement boils down using headOption on the result method.
val result: Future[Option[Whatever]] = db.run((query.filter(_.id === 1)).result.headOption)
query.result returns an object of type DBIOAction. An action in slick is something that can be executed on a database. The actual execution is done by passing the action to db.run() or db.stream(). You can find a more detailed explanation here: http://slick.typesafe.com/doc/3.0.0/api/index.html#slick.dbio.DBIOAction
In a DataFrame object in Apache Spark (I'm using the Scala interface), if I'm iterating over its Row objects, is there any way to extract values by name? I can see how to do some really awkward stuff:
def foo(r: Row) = {
val ix = (0 until r.schema.length).map( i => r.schema(i).name -> i).toMap
val field1 = r.getString(ix("field1"))
val field2 = r.getLong(ix("field2"))
...
}
dataframe.map(foo)
I figure there must be a better way - this is pretty verbose, it requires creating this extra structure, and it also requires knowing the types explicitly, which if incorrect, will produce a runtime exception rather than a compile-time error.
You can use "getAs" from org.apache.spark.sql.Row
r.getAs("field1")
r.getAs("field2")
Know more about getAs(java.lang.String fieldName)
This is not supported at this time in the Scala API. The closest you have is this JIRA titled "Support converting DataFrames to typed RDDs"