scala slick or and query - scala

my question may sound very banal but I still didn't resolve it.
I have the Products Table implemented like
class ProductsTable(tag: Tag) extends Table[Product](tag, "PRODUCTS") {
def id = column[Int]("PRODUCT_ID", O.PrimaryKey, O.AutoInc)
def title = column[String]("NAME")
def description = column[String]("DESCRIPTION")
def style = column[String]("STYLE")
def price = column[Int]("PRICE")
def category_id = column[Int]("CATEGORY_ID")
def size_id = column[Int]("SIZE_ID")
def brand_id = column[Int]("BRAND_ID")
def * = (id.?, title, description, style, price, category_id, size_id, brand_id) <>(Product.tupled, Product.unapply _)
}
and its representation in
val Products = TableQuery[ProductsTable]
How can I implement query equivalent to SQl query:
select * from products where( category_id = 1 or category_id = 2 or category_id = 3 ) and (price between min and max)

Try something like this:
val query = Products filter { p => (p.category_id inSet List(1,2,3)) && p.price > min && p.price < max }
val result = db.run(query.result)
You can use println(query.result.statements) to see what query looks like.
EDIT:
Answer for additional question. You can make a function for your query that accepts optional min and max values:
def getProductsQuery(maybeMin: Option[Int] = None, maybeMax: Option[Int] = None) = {
val initialQuery = val query = Products filter { p => (p.category_id inSet List(1,2,3)) }
val queryWithMin = maybeMin match {
case Some(min) => initialQuery filter { _.price > min }
case None => initialQuery
}
val queryWithMax = maybeMax match {
case Some(max) => queryWithMin filter { _.price < max }
case None => queryWithMin
}
queryWithMax
}
And then you could do any of these:
val q1 = getProductsQuery() // without min or max
val q2 = getProductsQuery(maybeMin = Option(3)) // only min
val q3 = getProductsQuery(maybeMax = Option(10)) // only max
val q4 = getProductsQuery(maybeMin = Option(3), maybeMax = Option(10)) // both
and run any of these as needed...

Related

Scala slick sort by column string name

I'm trying to implement such method in slick:
def paged(page: Long, sorting: String, order: String) = {
infos.sortBy(_.???).drop((page - 1)*INFOS_PER_PAGE).take(INFOS_PER_PAGE).result
}
But I don't know what to put in sortBy and how to cast String to column.
I tried simple function like this:
class PaymentInfoTable(tag: Tag) extends Table[PaymentInfo](tag, "payment_info") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def date = column[DateTime]("date")
def paymentType = column[String]("payment_type")
def category = column[String]("category")
def subCategory = column[String]("sub_category")
def value = column[BigDecimal]("value")
def comment = column[Option[String]]("comment")
def outsidePaypal = column[Boolean]("outside_paypal")
def outsidePayer = column[Option[String]]("outside_payer")
def sorting(col: String) = {
if(col == "id") {
id
} else if (col == "date") {
date
} else if (col == "paymentType") {
paymentType
} else if (col == "category") {
category
} else if (col == "subCategory") {
subCategory
} else if (col == "value") {
value
} else if (col == "comment") {
comment
} else if (col == "outsidePaypal") {
outsidePaypal
} else if (col == "outsidePayer") {
outsidePayer
} else {
id
}
}
but then I can't do
sortBy(_.sorting(sorting).asc)
It says
value asc is not a member of slick.lifted.Rep[_1]
And I have imported
import slick.jdbc.MySQLProfile.api._
How can I sort by string column name?
Basically what you can do is to add a map with the string/property of the table
class ColorsTable(tag: Tag) extends Table[Color](tag, "color") with DynamicSortBySupport.ColumnSelector {
def id = column[Int]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def * = (id, name) <> ((Color.apply _).tupled, Color.unapply)
val select = Map(
"id" -> (this.id),
"name" -> (this.name)
)
}
Then you only need to access that map
case ((sortColumn, sortOrder), queryToSort) =>
val sortOrderRep: Rep[_] => Ordered = ColumnOrdered(_, Ordering(sortOrder))
val sortColumnRep: A => Rep[_] = _.select(sortColumn)
queryToSort.sortBy(sortColumnRep)(sortOrderRep)
You can find more info in this, like, but not same, question Dynamic order by in scala slick with several columns
If you want something more simple, then just create an Order column with your column
if(col == "id") {
ColumnOrdered(id, Ordering(sortOrder))
}

Evaluating multiple filters in a single pass

I have below rdd created and I need to perform a series of filters on the same dataset to derive different counters and aggregates.
Is there a way I can apply these filters and compute aggregates in a single pass, avoiding spark to go over the same dataset multiple times?
val res = df.rdd.map(row => {
// ............... Generate data here for each row.......
})
res.persist(StorageLevel.MEMORY_AND_DISK)
val all = res.count()
val stats1 = res.filter(row => row.getInt(1) > 0)
val stats1Count = stats1.count()
val stats1Agg = stats1.map(r => r.getInt(1)).mean()
val stats2 = res.filter(row => row.getInt(2) > 0)
val stats2Count = stats2.count()
val stats2Agg = stats2.map(r => r.getInt(2)).mean()
You can use aggregate:
case class Stats(count: Int = 0, sum: Int = 0) {
def mean = sum/count
def +(s: Stats): Stats = Stats(count + s.count, sum + s.sum)
def <- (n: Int) = if(n > 0) copy(count + 1, sum + n) else this
}
val (stats1, stats2) = res.aggregate(Stats() -> Stats()) (
{ (s, row) => (s._1 <- row.getInt(1), s._2 <- row.getInt(2)) },
{ _ + _ }
)
val (stat1Count, stats1Agg, stats2Count, stats2Agg) = (stats1.count, stats1.mean, stats2.count, stats2.mean)

Slick sorting joda date

I am using slick-joda-mapper 2.2, slick 3.1 and play 2.5
It maps Joda date without any problems.
import com.github.tototoshi.slick.PostgresJodaSupport._
trait QuoteMapping {
self: HasDatabaseConfigProvider[JdbcProfile] =>
import driver.api._
val quotes = TableQuery[Quotes]
class Quotes(tag: Tag) extends Table[Quote](tag, "QUOTE") {
def id = column[Long]("ID", O.PrimaryKey, O.AutoInc)
def startDate = column[LocalDate]("START_DATE") // All OK
def endDate = column[LocalDate]("END_DATE") // All OK
def pricePerDay = column[Double]("PRICE_PER_DAY")
def totalPrice = column[Double]("TOTAL_PRICE")
def * = (id.?, startDate, endDate, pricePerDay, totalPrice) <>(Quote.tupled, Quote.unapply)
}
}
Is it possible to sort joda date in slick?
/** Return a page of Quote */
def list(page: Int = 0, pageSize: Int = 10, orderBy: Int = 1, filter: String = "%"): Future[Page[(Quote)]] = {
val offset = pageSize * page
var query =
quotes.filter(_.id.asColumnOf[String] like filter.toLowerCase)
.drop(offset)
.take(pageSize)
orderBy match {
case 1 => query = query.sortBy(_.id.asc)
case 2 => query = query.sortBy(_.startDate.asc) // no .asc method
case 3 => query = query.sortBy(_.endDate.asc) // no .asc method
case 4 => query = query.sortBy(_.pricePerDay.asc)
case 5 => query = query.sortBy(_.totalPrice.asc)
case -1 => query = query.sortBy(_.id.desc)
case -2 => query = query.sortBy(_.startDate.desc) // no .desc method
case -3 => query = query.sortBy(_.endDate.desc) // no .desc method
case -4 => query = query.sortBy(_.pricePerDay.desc)
case -5 => query = query.sortBy(_.totalPrice.desc)
}
for {
totalRows <- count(filter)
list = query.result
result <- db.run(list)
} yield Page(result, page, offset, totalRows, pageSize)
}
Since the corresponding issue is still open, I think you can't.
I just had bad import instead:
org.joda.time.LocalDate
it was:
java.time.LocalDate
All works as expected, I hope it helps someone

Insert if not exists in Slick 3.0.0

I'm trying to insert if not exists, I found this post for 1.0.1, 2.0.
I found snippet using transactionally in the docs of 3.0.0
val a = (for {
ns <- coffees.filter(_.name.startsWith("ESPRESSO")).map(_.name).result
_ <- DBIO.seq(ns.map(n => coffees.filter(_.name === n).delete): _*)
} yield ()).transactionally
val f: Future[Unit] = db.run(a)
I'm struggling to write the logic from insert if not exists with this structure. I'm new to Slick and have little experience with Scala. This is my attempt to do insert if not exists outside the transaction...
val result: Future[Boolean] = db.run(products.filter(_.name==="foo").exists.result)
result.map { exists =>
if (!exists) {
products += Product(
None,
productName,
productPrice
)
}
}
But how do I put this in the transactionally block? This is the furthest I can go:
val a = (for {
exists <- products.filter(_.name==="foo").exists.result
//???
// _ <- DBIO.seq(ns.map(n => coffees.filter(_.name === n).delete): _*)
} yield ()).transactionally
Thanks in advance
It is possible to use a single insert ... if not exists query. This avoids multiple database round-trips and race conditions (transactions may not be enough depending on isolation level).
def insertIfNotExists(name: String) = users.forceInsertQuery {
val exists = (for (u <- users if u.name === name.bind) yield u).exists
val insert = (name.bind, None) <> (User.apply _ tupled, User.unapply)
for (u <- Query(insert) if !exists) yield u
}
Await.result(db.run(DBIO.seq(
// create the schema
users.schema.create,
users += User("Bob"),
users += User("Bob"),
insertIfNotExists("Bob"),
insertIfNotExists("Fred"),
insertIfNotExists("Fred"),
// print the users (select * from USERS)
users.result.map(println)
)), Duration.Inf)
Output:
Vector(User(Bob,Some(1)), User(Bob,Some(2)), User(Fred,Some(3)))
Generated SQL:
insert into "USERS" ("NAME","ID") select ?, null where not exists(select x2."NAME", x2."ID" from "USERS" x2 where x2."NAME" = ?)
Here's the full example on github
This is the version I came up with:
val a = (
products.filter(_.name==="foo").exists.result.flatMap { exists =>
if (!exists) {
products += Product(
None,
productName,
productPrice
)
} else {
DBIO.successful(None) // no-op
}
}
).transactionally
It's is a bit lacking though, for example it would be useful to return the inserted or existing object.
For completeness, here the table definition:
case class DBProduct(id: Int, uuid: String, name: String, price: BigDecimal)
class Products(tag: Tag) extends Table[DBProduct](tag, "product") {
def id = column[Int]("id", O.PrimaryKey, O.AutoInc) // This is the primary key column
def uuid = column[String]("uuid")
def name = column[String]("name")
def price = column[BigDecimal]("price", O.SqlType("decimal(10, 4)"))
def * = (id, uuid, name, price) <> (DBProduct.tupled, DBProduct.unapply)
}
val products = TableQuery[Products]
I'm using a mapped table, the solution works also for tuples, with minor changes.
Note also that it's not necessary to define the id as optional, according to the documentation it's ignored in insert operations:
When you include an AutoInc column in an insert operation, it is silently ignored, so that the database can generate the proper value
And here the method:
def insertIfNotExists(productInput: ProductInput): Future[DBProduct] = {
val productAction = (
products.filter(_.uuid===productInput.uuid).result.headOption.flatMap {
case Some(product) =>
mylog("product was there: " + product)
DBIO.successful(product)
case None =>
mylog("inserting product")
val productId =
(products returning products.map(_.id)) += DBProduct(
0,
productInput.uuid,
productInput.name,
productInput.price
)
val product = productId.map { id => DBProduct(
id,
productInput.uuid,
productInput.name,
productInput.price
)
}
product
}
).transactionally
db.run(productAction)
}
(Thanks Matthew Pocock from Google group thread, for orienting me to this solution).
I've run into the solution that looks more complete. Section 3.1.7 More Control over Inserts of the Essential Slick book has the example.
At the end you get smth like:
val entity = UserEntity(UUID.random, "jay", "jay#localhost")
val exists =
users
.filter(
u =>
u.name === entity.name.bind
&& u.email === entity.email.bind
)
.exists
val selectExpression = Query(
(
entity.id.bind,
entity.name.bind,
entity.email.bind
)
).filterNot(_ => exists)
val action = usersDecisions
.map(u => (u.id, u.name, u.email))
.forceInsertQuery(selectExpression)
exec(action)
// res17: Int = 1
exec(action)
// res18: Int = 0
according to the slick 3.0 manual insert query section (http://slick.typesafe.com/doc/3.0.0/queries.html), the inserted values can be returned with id as below:
def insertIfNotExists(productInput: ProductInput): Future[DBProduct] = {
val productAction = (
products.filter(_.uuid===productInput.uuid).result.headOption.flatMap {
case Some(product) =>
mylog("product was there: " + product)
DBIO.successful(product)
case None =>
mylog("inserting product")
(products returning products.map(_.id)
into ((prod,id) => prod.copy(id=id))) += DBProduct(
0,
productInput.uuid,
productInput.name,
productInput.price
)
}
).transactionally
db.run(productAction)
}

How can I filter with inSetBind for multiple columns in Slick?

I have the following table definition (simplified):
class Houses(tag: Tag) extends Table[HouseRow](tag, "HOUSE") {
def houseId = column[Long]("HOUSE_ID", O.NotNull, O.PrimaryKey, O.AutoInc)
def houseName = column[String]("HOUSE_NAME", O.NotNull)
def houseType = column[String]("HOUSE_TYPE", O.NotNull)
def uniqueHouseName = index("UQ_HOUSE_NAME_HOUSE_TYPE", (houseName, houseType), true)
def * = (houseId, houseName, houseType) <> (HouseRow.tupled, HouseRow.unapply)
}
val houses = TableQuery[Houses]
I'd like to select houses that match on a set of the uniqueHouseName index as follows.
case class HouseKey(houseName: String, houseType: String)
val houseKeys: Seq(HouseKey("name1", "type1"), HouseKey("name2", "type2"))
A naive inSetBind filter will match on for eg. HouseRow(ID, "name1", "type2") which is incorrect.
In MySql I would do something like:
SELECT * FROM HOUSE h
WHERE(h.HOUSE_TYPE, d.HOUSE_NAME) IN
(
SELECT 'type1' as HOUSE_TYPE, 'name1' as HOUSE_NAME
UNION
SELECT 'type2', 'name2'
);
Like #cvogt version, but doesn't blow up on empty list:
val filteredHouses =
houses.filter(h =>
houseKeys.map(hk => h.houseName === hk.houseName &&
h.houseType === hk.houseType)
.reduceOption(_ || _).getOrElse(false: Rep[Boolean])
)
Tested in slick 3.1.0
Adapting tuxdna's answer to allow arbitrary seqs. This query can however not be precompiled to SQL at the moment and has a runtime overhead.
val filteredHouses =
houses.filter(h =>
houseKeys.map(hk => h.houseName === hk.houseName && h.houseType === hk.houseType)
.reduce(_ || _)
)
This is not complete answer, but for only two pairs of values you could do this:
val filteredHouses = for {
h <- houses
if (h.houseName === "name1" && h.houseType === "type1") || (
h.houseName === "name2" && h.houseType === "type2")
} yield h