Related
I am working on a Library data model where each book can have multiple authors and vice versa (many to many).
I want to pass a list of books to a html view page that each book includes a list of its author(s).
To do that I have defined the following tables for book and authors:
private class BookTable(tag: Tag) extends Table[Book](tag, "book") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def publishDate = column[Date]("publish_date")
def memberId = column[Option[Long]]("member_id")
def member = foreignKey("member_fk",memberId,members)(_.id)
type Data = (Long, String, Date, Option[Long])
def constructBook: Data => Book = {
case (id, name, publishDate, memberId) =>
Book(id, name, publishDate, memberId)
}
def extractBook: PartialFunction[Book, Data] = {
case Book(id, name, publishDate, memberId, _) =>
(id, name, publishDate, memberId)
}
def * = (id, name, publishDate, memberId) <> (constructBook, extractBook.lift)
}
private class AuthorBookTable (tag: Tag) extends Table[AuthorBook](tag, "author_book") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def authorId = column[Long]("author_id")
def bookId = column[Long]("book_id")
def memberId = column[Option[Long]]("member_id")
def author = foreignKey("author_fk",authorId,authors)(_.id)
def book = foreignKey("book_fk",bookId,books)(_.id)
def * = (id, authorId, bookId) <> ((AuthorBook.apply _).tupled, AuthorBook.unapply)
}
private class AuthorTable (tag: Tag) extends Table[Author](tag, "author") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def * = (id, name) <> ((Author.apply _).tupled, Author.unapply)
}
The book case class is as below:
case class Book(id: Long, name: String, publishDate: Date, memberId: Option[Long] = None, authors: Seq[Author]= Seq.empty)
{
def updateAuthors(authorss: Seq[Author]) = {
this.copy(authors=authorss)
}
}
In controller I use the below:
def getBooks = Action.async { implicit request =>
repo.getBooks.map { books =>
val booksWithAuthors=books.map( b=> {val updateB=b.updateAuthors( repo.getBookAuthors(b.id))
updateB})
Ok(Json.toJson(booksWithAuthors))
}
}
My question is about the getBookAuthors implementation shown below:
implicit def waitForFuture[A](f:Future[A]) = {
def res: A = Await.result(f, Duration.Inf)
res
}
def getBookAuthors(id: Long): Seq[Author] = {
val result=db.run {
val innerJoin = for {
(ab, a) <- authorBooks join authors on (_.authorId === _.id)
} yield (a, ab.bookId)
innerJoin.filter(_._2 === id).sortBy(_._1.name).map(_._1).result
}
waitForFuture(result)
}
My concern is that the getBookAuthors function is blocking and I am not sure if it's the best practice. Please advise if there is a better way to do this.
As you are saying, blocking methods are pretty bad in this context and you will lost the advantages of using a non-blocking library as Slick.
the getBookAuthors would be written as follows, returning a Future[Seq[Author]] thats needs to be managed in the caller
def getBookAuthors(id: Long): Future[Seq[Author]] =
db.run {
val innerJoin = for {
(ab, a) <- authorBooks join authors on (_.authorId === _.id)
} yield (a, ab.bookId)
innerJoin.filter(_._2 === id).sortBy(_._1.name).map(_._1).result
}
So the caller should be rewritten as:
def getBooks = Action.async { implicit request =>
repo.getBooks.flatMap { books =>
Future.sequence(
books.map { b =>
repo.getBookAuthors(b.id).map(authors => b.updateAuthors(authors))
}
).map { booksWithAuthors =>
Ok(Json.toJson(booksWithAuthors))
}
}
}
This means that, once you will have the books: Seq[Book] you will map over it to integrate the authors and this will end with a Seq[Future[Book]].
Then it can be transformed into a Future[Seq[Book]] (with authors) with the Future.sequence method.
Finally you need to flatMap on the outer Future to move from Future[Future[Seq[Book]]] to a simpler Future[Seq[Book]]
This second snippet can be refactored in a more clean way taking advantage of the for-comprehension that is a syntactic sugar for the flatMap
private def addAuthorsToBooks(books: Seq[Book]): Future[Seq[Book]] =
Future.sequence(
books.map { b =>
repo.getBookAuthors(b.id).map(authors => b.updateAuthors(authors))
}
)
def getBooks = Action.async { implicit request =>
for {
books <- repo.getBooks
booksWithAuthors <- addAuthorsToBooks(books)
} yield Ok(Json.toJson(booksWithAuthors))
}
I am using play 2.6.6 , scala 2.12.3 and slick 3.0.0.
I had following case class structure initially where there was a nested case class:
case class Device(id: Int, deviceUser: Option[DeviceUser] =None)
case class DeviceUser(name: Option[String] = None)
So, I had created following projection for Device class:
class DevicesTable(tag: Tag) extends Table[Device](tag, "DEVICES") {
def id = column[Int]("ID", O.PrimaryKey)
def name = column[Option[String]]("NAME")
def deviceUser = name.<>[Option[DeviceUser]](
{
(param: Option[String]) => {
param match {
case Some(name) => Some(DeviceUser(Some(name)))
case None => None
}
}
},
{
(t: Option[DeviceUser]) =>
{
t match {
case Some(user) => Some(user.name)
case None => None
}
}
}
)
def * = (id, deviceUser).<>(Device.tupled, Device.unapply)
}
The above setup was working fine. I could easily store and retrieve data using the above projection. But now, my requirement has changed and I need to store list of nested case class. So, the class structure is now as follow :
case class Device(id: Int, deviceUser: Option[List[DeviceUser]] =None)
case class DeviceUser(name: Option[String] = None)
Is there some way where I could define projection for the field deviceUser: Option[List[DeviceUser]] ?
Update : I am looking for more of a non-relational approach here.
Since, no body has suggested a solution so far, I am sharing the approach that I am using right now. It works but of course is not the best solution. Specially, I want to avoid using Await here and would like to develop a generic implicit parser.
ALso, I had to create a separate DeviceUsersTable.
case class DeviceUser(id: Int,name: Option[String] = None)
class DeviceUserRepo #Inject()(protected val dbConfigProvider: DatabaseConfigProvider) {
val dbConfig = dbConfigProvider.get[JdbcProfile]
val db = dbConfig.db
import dbConfig.profile.api._
val DeviceUsers = TableQuery[DeviceUserTable]
private def _findById(id: Int): DBIO[Option[DeviceUser]] =
DeviceUsers.filter(_.id === id).result.headOption
def findById(id: Int): Future[Option[DeviceUser]] =
db.run(_findById(id))
def all: Future[List[DeviceUser]] =
db.run(DeviceUsers.to[List].result)
def create(deviceUser: DeviceUser): Future[Int] = {
db.run(DeviceUsers returning DeviceUsers.map(_.id) += deviceUser)
}
class DeviceUserTable(tag: Tag) extends Table[DeviceUser](tag, "DEVICE_USERS") {
def id = column[Int]("ID", O.PrimaryKey)
def name = column[Option[String]]("NAME")
def * = (id, name).<>(DeviceUser.tupled, DeviceUser.unapply)
}
}
And the original DevicesTable now looks like this :
class DevicesTable(tag: Tag) extends Table[Device](tag, "DEVICES") {
implicit val deviceUserConverter = MappedColumnType.base[Option[List[DeviceUser]], String](
deviceUsersOpt => {
deviceUsersOpt match {
case Some(users:List[DeviceUser]) =>val listOfId = users.map{
k => val res = deviceUserRepo.create(k)
Await.result(res, 10 seconds)
}
listOfId.mkString(",")
case None => ""
}
},
str =>{
val listOfIds = (str split "," map Integer.parseInt).toList.filterNot(k => k.equals(""))
if(listOfIds.nonEmpty){
val users = listOfIds.map{ k =>
val res = deviceUserRepo.findById(k)
Await.result(res, 10 seconds)
}
Some(users.flatten)
} else {
None
}
}
)
def id = column[Int]("ID", O.PrimaryKey)
def deviceUser = column[Option[List[DeviceUser]]]("DEVICE_USERS")
def * = (id, deviceUser).<>(Device.tupled, Device.unapply)
}
Having a table with the columns
class Data(tag: Tag) extends Table[DataRow](tag, "data") {
def id = column[Int]("id", O.PrimaryKey)
def name = column[String]("name")
def state = column[State]("state")
def price = column[Int]("price")
def * = (id.?, name, state, price) <> ((DataRow.apply _).tupled, DataRow.unapply)
}
I'd like to write a function that would select a single row, and update the columns where the supplied values are not null.
def update(id: Int, name: Option[String], state: Option[State], price: Option[Int])
eg.
update(1, None, None, Some(5)) would update only the price of the data row 1, leaving the name and state intact
update(1, Some("foo"), None, Some(6)) would update the name and price, but leave its state intact.
I guess some smart mapping could be used, but I'm having a hard time expressing it, not sure how it could spit out different length tuples depending on the inputs (wether their value is defined), since they are more or less "unrelated" classes.
def update(id: Int, name: Option[String], state: Option[State], price: Option[Int]) = {
table.fiter(_.id == id). ???? .update(name, state, price)
}
I solved it in the following way.
The implementation below works only if it is a Product object.
Execute the update statement except for None for the Option type and null for the object type.
package slick.extensions
import slick.ast._
import slick.dbio.{ Effect, NoStream }
import slick.driver.JdbcDriver
import slick.jdbc._
import slick.lifted._
import slick.relational.{ CompiledMapping, ProductResultConverter, ResultConverter, TypeMappingResultConverter }
import slick.util.{ ProductWrapper, SQLBuilder }
import scala.language.{ existentials, higherKinds, implicitConversions }
trait PatchActionExtensionMethodsSupport { driver: JdbcDriver =>
trait PatchActionImplicits {
implicit def queryPatchActionExtensionMethods[U <: Product, C[_]](
q: Query[_, U, C]
): PatchActionExtensionMethodsImpl[U] =
createPatchActionExtensionMethods(updateCompiler.run(q.toNode).tree, ())
}
///////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////// Patch Actions
///////////////////////////////////////////////////////////////////////////////////////////////
type PatchActionExtensionMethods[T <: Product] = PatchActionExtensionMethodsImpl[T]
def createPatchActionExtensionMethods[T <: Product](tree: Node, param: Any): PatchActionExtensionMethods[T] =
new PatchActionExtensionMethodsImpl[T](tree, param)
class PatchActionExtensionMethodsImpl[T <: Product](tree: Node, param: Any) {
protected[this] val ResultSetMapping(_, CompiledStatement(_, sres: SQLBuilder.Result, _),
CompiledMapping(_converter, _)) = tree
protected[this] val converter = _converter.asInstanceOf[ResultConverter[JdbcResultConverterDomain, Product]]
protected[this] val TypeMappingResultConverter(childConverter, toBase, toMapped) = converter
protected[this] val ProductResultConverter(elementConverters # _ *) =
childConverter.asInstanceOf[ResultConverter[JdbcResultConverterDomain, Product]]
private[this] val updateQuerySplitRegExp = """(.*)(?<=set )((?:(?= where)|.)+)(.*)?""".r
private[this] val updateQuerySetterRegExp = """[^\s]+\s*=\s*\?""".r
/** An Action that updates the data selected by this query. */
def patch(value: T): DriverAction[Int, NoStream, Effect.Write] = {
val (seq, converters) = value.productIterator.zipWithIndex.toIndexedSeq
.zip(elementConverters)
.filter {
case ((Some(_), _), _) => true
case ((None, _), _) => false
case ((null, _), _) => false
case ((_, _), _) => true
}
.unzip
val (products, indexes) = seq.unzip
val newConverters = converters.zipWithIndex
.map(c => (c._1, c._2 + 1))
.map {
case (c: BaseResultConverter[_], idx) => new BaseResultConverter(c.ti, c.name, idx)
case (c: OptionResultConverter[_], idx) => new OptionResultConverter(c.ti, idx)
case (c: DefaultingResultConverter[_], idx) => new DefaultingResultConverter(c.ti, c.default, idx)
case (c: IsDefinedResultConverter[_], idx) => new IsDefinedResultConverter(c.ti, idx)
}
val productResultConverter =
ProductResultConverter(newConverters: _*).asInstanceOf[ResultConverter[JdbcResultConverterDomain, Any]]
val newConverter = TypeMappingResultConverter(productResultConverter, (p: Product) => p, (a: Any) => toMapped(a))
val newValue: Product = new ProductWrapper(products)
val newSql = sres.sql match {
case updateQuerySplitRegExp(prefix, setter, suffix) =>
val buffer = StringBuilder.newBuilder
buffer.append(prefix)
buffer.append(
updateQuerySetterRegExp
.findAllIn(setter)
.zipWithIndex
.filter(s => indexes.contains(s._2))
.map(_._1)
.mkString(", ")
)
buffer.append(suffix)
buffer.toString()
}
new SimpleJdbcDriverAction[Int]("patch", Vector(newSql)) {
def run(ctx: Backend#Context, sql: Vector[String]): Int =
ctx.session.withPreparedStatement(sql.head) { st =>
st.clearParameters
newConverter.set(newValue, st)
sres.setter(st, newConverter.width + 1, param)
st.executeUpdate
}
}
}
}
}
Example
// Model
case class User(
id: Option[Int] = None,
name: Option[String] = None,
username: Option[String] = None,
password: Option[String] = None
)
// Table
class Users(tag: Tag) extends Table[User](tag, "users") {
def id = column[Int]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def username = column[String]("username")
def password = column[String]("password")
override def * = (id.?, name.?, username.?, password.?) <>(User.tupled, User.unapply)
}
// TableQuery
object Users extends TableQuery(new Users(_))
// CustomDriver
trait CustomDriver extends PostgresDriver with PatchActionExtensionMethodsSupport {
override val api: API = new API {}
trait API extends super.API with PatchActionImplicits
}
// Insert
Users += User(Some(1), Some("Test"), Some("test"), Some("1234"))
// User patch
Users.filter(_.id === 1).patch(User(name = Some("Change Name"), username = Some("")))
https://gist.github.com/bad79s/1edf9ea83ba08c46add03815059acfca
Building on JonasAnso's answer, converting that to slick v3.0+, and putting it into a transaction:
def partialUpdate(id: Int, name: Option[String], login: Option[String]): Future[Int] = {
val selectQ = users.filter(_.id === id)
val query = selectQ.result.head.flatMap { data =>
selectQ.update(data.patch(name, login))
}
db.run(query)
}
As I commented the question is similar to an existing one, but you don't seem to have any extra requirements.
The simplest approach is just SELECT + UPDATE. For example you add a patch function in your DataRow class defining how you want to update your model
def patch(name: Option[String], state: Option[State], price: Option[Int]): Data {
this.copy(name = name.getOrElse(this.name), ...)
}
And you add a partialUpdate method in your repo class
class DataRepo {
private val Datas = TableQuery[Data]
val db = ???
def partialUpdate(id: Int, name: Option[String], state: Option[State], price: Option[Int]): Future[Int] = {
val query = Datas.filter(_.id === id)
for {
data <- db.run(query.result.head)
result <- db.run(query.update(data.patch(name, state, price)))
} yield result
}
}
As you see the main problem of this solution is that there are 2 SQL statements, SELECT and UPDATE.
Other solution is to use plain SQL (http://slick.typesafe.com/doc/3.0.0/sql.html) but of course this gives other problems.
I've created a many-to-many collection using Slick 3.0, but I'm struggling to retrieve data in the way I want.
There is a many-to-many relationship between Events and Interests. Here are my tables:
case class EventDao(title: String,
id: Option[Int] = None)
class EventsTable(tag: Tag)
extends Table[EventDao](tag, "events") {
def id = column[Int]("event_id", O.PrimaryKey, O.AutoInc)
def title = column[String]("title")
def * = (
title,
id.?) <> (EventDao.tupled, EventDao.unapply)
def interests = EventInterestQueries.query.filter(_.eventId === id)
.flatMap(_.interestFk)
}
object EventQueries {
lazy val query = TableQuery[EventsTable]
val findById = Compiled { k: Rep[Int] =>
query.filter(_.id === k)
}
}
Here's EventsInterests:
case class EventInterestDao(event: Int, interest: Int)
class EventsInterestsTable(tag: Tag)
extends Table[EventInterestDao](tag, "events_interests") {
def eventId = column[Int]("event_id")
def interestId = column[Int]("interest_id")
def * = (
eventId,
interestId) <> (EventInterestDao.tupled, EventInterestDao.unapply)
def eventFk = foreignKey("event_fk", eventId, EventQueries.query)(e => e.id)
def interestFk = foreignKey("interest_fk", interestId, InterestQueries.query)(i => i.id)
}
object EventInterestQueries {
lazy val query = TableQuery[EventsInterestsTable]
}
And finally Interests:
case class InterestDao(name: String,
id: Option[Int] = None)
class InterestsTable(tag: Tag)
extends Table[InterestDao](tag, "interests") {
def id = column[Int]("interest_id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def name_idx = index("idx_name", name, unique = true)
def * = (
name,
id.?) <> (InterestDao.tupled, InterestDao.unapply)
def events = EventInterestQueries.query.filter(_.interestId === id)
.flatMap(_.eventFk)
}
object InterestQueries {
lazy val query = TableQuery[InterestsTable]
val findById = Compiled { k: Rep[Int] =>
query.filter(_.id === k)
}
}
I can query and retrieve tuples of (event.name, interest) with the following:
val eventInterestQuery = for {
event <- EventQueries.query
interest <- event.interests
} yield (event.title, interest.name)
Await.result(db.run(eventInterestQuery.result).map(println), Duration.Inf)
So this is what I currently have.
What I want is to be able to populate a case class like:
case class EventDao(title: String,
interests: Seq[InterestDao],
id: Option[Int] = None)
The trouble is that if I update my case class like this, it messes up my def * projection in EventsTable. Also, I'll have to rename the EventsTable.interests filter to something like EventsTable.interestIds which is a bit ugly but I could live with if necessary.
Also, I can't find a way of writing a for query that yields (event.name, Seq(interest.name)). Anyway, that's just a stepping stone to me being able to yield a (EventDao, Seq(InterestDao)) tuple which is what I really want to return.
Does anyone know how I can achieve these things? I also want to be able to 'take' a certain number of Interests, so for some queries all would be returned, but for others only the first 3 would be.
So after reading this page and chatting on the mailing list, I finally got it working:
val eventInterestQuery = for {
event <- EventQueries.query
interest <- event.interests
} yield (event, interest)
Await.result(db.run(eventInterestQuery.result
// convert the interests to a sequence.
.map {
_.groupBy(_._1)
.map {
case (k,v) => (k, v.map(_._2))
}.toSeq
}
), Duration.Inf)
The only issue with groupBy is you lose order. You could fold the result. I've written this helper for my current project:
def foldOneToMany[A, B](in: Seq[(A, Option[B])], eq: (A, B) => Boolean)
(f: (A, B) => A): Seq[A] =
in.foldLeft(List.empty[A]) {
case (head :: tail, (_, Some(rel))) if eq(head, rel) =>
f(head, rel) :: tail
case (r, (el, Some(rel))) => f(el, rel) :: r
case (r, (el, None)) => el :: r
}.reverse
It could do with some love. Now it takes in a function A,B => Boolean to determine if B belongs to A and a function A,B => A that adds B to A.
Virtualeyes also has a point. In Postgres you could use array_agg function to use a little less bandwidth from the db.
I am using SLICK 1.0.0-RC2. I have defined the following two tables Directorate and ServiceArea where Directorate has a one to many relationship with ServiceArea
case class Directorate(dirCode: String, name: String)
object Directorates extends Table[Directorate]("DIRECTORATES") {
def dirCode = column[String]("DIRECTORATE_CODE", O.PrimaryKey)
def name = column[String]("NAME")
def * = dirCode ~ name <> (Directorate, Directorate.unapply _)
}
case class ServiceArea(areaCode: String, dirCode: String, name: String)
object ServiceAreas extends Table[ServiceArea]("SERVICE_AREAS") {
def areaCode = column[String]("AREAE_CODE", O.PrimaryKey)
def dirCode = column[String]("DIRECTORATE_CODE")
def name = column[String]("NAME")
def directorate = foreignKey("DIR_FK", dirCode, Directorates)(_.dirCode)
def * = areaCode ~ dirCode ~ name <> (ServiceArea, ServiceArea.unapply _)
}
To make the Directorate case class useful in my Play application form I am trying to redefine the Directorate case class to have a Seq of ServiceAreas that are related to that Directorate.
case class Directorate(dirCode: String, name: String, serviceAreas: Seq[ServiceArea])
My problem is now with the Directorate table projection. I have attempted to create a method in Directorates:
def serviceAreas = (for { a <- ServiceAreas
if (a.dirCode === dirCode)
} yield (a)).list map {
case t: ServiceArea => t
}
so that I can try something like
def * = dirCode ~ name ~ serviceAreas <> (Directorate, Directorate.unapply _)
but this cannot not work as serviceAreas only goes one way.
It seems reasonable to me that for the Directorate case class to be a useful domain object that it should be able contain the related ServiceAreas.
I'm wondering how I should traverse the inverse relationship so that Directorate table projection will work.
I'm sure there is a more elegant solution, but this should do the trick:
import scala.slick.driver.H2Driver.simple._
import Database.threadLocalSession
object SlickExperiments2 {
Database.forURL("jdbc:h2:mem:test1", driver = "org.h2.Driver") withSession {
(Directorates.ddl ++ ServiceAreas.ddl).create
case class Directorate(dirCode: String, name: String) {
def serviceAreas: Seq[ServiceArea] = (for {
a <- ServiceAreas
if (a.dirCode === dirCode)
} yield (a)).list
}
object Directorates extends Table[Directorate]("DIRECTORATES") {
def dirCode = column[String]("DIRECTORATE_CODE", O.PrimaryKey)
def name = column[String]("NAME")
def * = dirCode ~ name <> (Directorate, Directorate.unapply _)
}
case class ServiceArea(areaCode: String, dirCode: String, name: String)
object ServiceAreas extends Table[ServiceArea]("SERVICE_AREAS") {
def areaCode = column[String]("AREAE_CODE", O.PrimaryKey)
def dirCode = column[String]("DIRECTORATE_CODE")
def name = column[String]("NAME")
def directorate = foreignKey("DIR_FK", dirCode, Directorates)(_.dirCode)
def * = areaCode ~ dirCode ~ name <> (ServiceArea, ServiceArea.unapply _)
}
Directorates.insert(Directorate("Dircode", "Dirname"))
ServiceAreas.insertAll(ServiceArea("a", "Dircode", "A"), ServiceArea("b", "Dircode", "B"))
val sa = (for{
d <- Directorates
} yield d).list map { case t: Directorate => t.serviceAreas}
println(sa)
}
//> List(List(ServiceArea(a,Dircode,A), ServiceArea(b,Dircode,B)))
}