Slick - many to many relationship - scala

I am working on a Library data model where each book can have multiple authors and vice versa (many to many).
I want to pass a list of books to a html view page that each book includes a list of its author(s).
To do that I have defined the following tables for book and authors:
private class BookTable(tag: Tag) extends Table[Book](tag, "book") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def publishDate = column[Date]("publish_date")
def memberId = column[Option[Long]]("member_id")
def member = foreignKey("member_fk",memberId,members)(_.id)
type Data = (Long, String, Date, Option[Long])
def constructBook: Data => Book = {
case (id, name, publishDate, memberId) =>
Book(id, name, publishDate, memberId)
}
def extractBook: PartialFunction[Book, Data] = {
case Book(id, name, publishDate, memberId, _) =>
(id, name, publishDate, memberId)
}
def * = (id, name, publishDate, memberId) <> (constructBook, extractBook.lift)
}
private class AuthorBookTable (tag: Tag) extends Table[AuthorBook](tag, "author_book") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def authorId = column[Long]("author_id")
def bookId = column[Long]("book_id")
def memberId = column[Option[Long]]("member_id")
def author = foreignKey("author_fk",authorId,authors)(_.id)
def book = foreignKey("book_fk",bookId,books)(_.id)
def * = (id, authorId, bookId) <> ((AuthorBook.apply _).tupled, AuthorBook.unapply)
}
private class AuthorTable (tag: Tag) extends Table[Author](tag, "author") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def * = (id, name) <> ((Author.apply _).tupled, Author.unapply)
}
The book case class is as below:
case class Book(id: Long, name: String, publishDate: Date, memberId: Option[Long] = None, authors: Seq[Author]= Seq.empty)
{
def updateAuthors(authorss: Seq[Author]) = {
this.copy(authors=authorss)
}
}
In controller I use the below:
def getBooks = Action.async { implicit request =>
repo.getBooks.map { books =>
val booksWithAuthors=books.map( b=> {val updateB=b.updateAuthors( repo.getBookAuthors(b.id))
updateB})
Ok(Json.toJson(booksWithAuthors))
}
}
My question is about the getBookAuthors implementation shown below:
implicit def waitForFuture[A](f:Future[A]) = {
def res: A = Await.result(f, Duration.Inf)
res
}
def getBookAuthors(id: Long): Seq[Author] = {
val result=db.run {
val innerJoin = for {
(ab, a) <- authorBooks join authors on (_.authorId === _.id)
} yield (a, ab.bookId)
innerJoin.filter(_._2 === id).sortBy(_._1.name).map(_._1).result
}
waitForFuture(result)
}
My concern is that the getBookAuthors function is blocking and I am not sure if it's the best practice. Please advise if there is a better way to do this.

As you are saying, blocking methods are pretty bad in this context and you will lost the advantages of using a non-blocking library as Slick.
the getBookAuthors would be written as follows, returning a Future[Seq[Author]] thats needs to be managed in the caller
def getBookAuthors(id: Long): Future[Seq[Author]] =
db.run {
val innerJoin = for {
(ab, a) <- authorBooks join authors on (_.authorId === _.id)
} yield (a, ab.bookId)
innerJoin.filter(_._2 === id).sortBy(_._1.name).map(_._1).result
}
So the caller should be rewritten as:
def getBooks = Action.async { implicit request =>
repo.getBooks.flatMap { books =>
Future.sequence(
books.map { b =>
repo.getBookAuthors(b.id).map(authors => b.updateAuthors(authors))
}
).map { booksWithAuthors =>
Ok(Json.toJson(booksWithAuthors))
}
}
}
This means that, once you will have the books: Seq[Book] you will map over it to integrate the authors and this will end with a Seq[Future[Book]].
Then it can be transformed into a Future[Seq[Book]] (with authors) with the Future.sequence method.
Finally you need to flatMap on the outer Future to move from Future[Future[Seq[Book]]] to a simpler Future[Seq[Book]]
This second snippet can be refactored in a more clean way taking advantage of the for-comprehension that is a syntactic sugar for the flatMap
private def addAuthorsToBooks(books: Seq[Book]): Future[Seq[Book]] =
Future.sequence(
books.map { b =>
repo.getBookAuthors(b.id).map(authors => b.updateAuthors(authors))
}
)
def getBooks = Action.async { implicit request =>
for {
books <- repo.getBooks
booksWithAuthors <- addAuthorsToBooks(books)
} yield Ok(Json.toJson(booksWithAuthors))
}

Related

Slick 3.0.0 - update row with only non-null values

Having a table with the columns
class Data(tag: Tag) extends Table[DataRow](tag, "data") {
def id = column[Int]("id", O.PrimaryKey)
def name = column[String]("name")
def state = column[State]("state")
def price = column[Int]("price")
def * = (id.?, name, state, price) <> ((DataRow.apply _).tupled, DataRow.unapply)
}
I'd like to write a function that would select a single row, and update the columns where the supplied values are not null.
def update(id: Int, name: Option[String], state: Option[State], price: Option[Int])
eg.
update(1, None, None, Some(5)) would update only the price of the data row 1, leaving the name and state intact
update(1, Some("foo"), None, Some(6)) would update the name and price, but leave its state intact.
I guess some smart mapping could be used, but I'm having a hard time expressing it, not sure how it could spit out different length tuples depending on the inputs (wether their value is defined), since they are more or less "unrelated" classes.
def update(id: Int, name: Option[String], state: Option[State], price: Option[Int]) = {
table.fiter(_.id == id). ???? .update(name, state, price)
}
I solved it in the following way.
The implementation below works only if it is a Product object.
Execute the update statement except for None for the Option type and null for the object type.
package slick.extensions
import slick.ast._
import slick.dbio.{ Effect, NoStream }
import slick.driver.JdbcDriver
import slick.jdbc._
import slick.lifted._
import slick.relational.{ CompiledMapping, ProductResultConverter, ResultConverter, TypeMappingResultConverter }
import slick.util.{ ProductWrapper, SQLBuilder }
import scala.language.{ existentials, higherKinds, implicitConversions }
trait PatchActionExtensionMethodsSupport { driver: JdbcDriver =>
trait PatchActionImplicits {
implicit def queryPatchActionExtensionMethods[U <: Product, C[_]](
q: Query[_, U, C]
): PatchActionExtensionMethodsImpl[U] =
createPatchActionExtensionMethods(updateCompiler.run(q.toNode).tree, ())
}
///////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////// Patch Actions
///////////////////////////////////////////////////////////////////////////////////////////////
type PatchActionExtensionMethods[T <: Product] = PatchActionExtensionMethodsImpl[T]
def createPatchActionExtensionMethods[T <: Product](tree: Node, param: Any): PatchActionExtensionMethods[T] =
new PatchActionExtensionMethodsImpl[T](tree, param)
class PatchActionExtensionMethodsImpl[T <: Product](tree: Node, param: Any) {
protected[this] val ResultSetMapping(_, CompiledStatement(_, sres: SQLBuilder.Result, _),
CompiledMapping(_converter, _)) = tree
protected[this] val converter = _converter.asInstanceOf[ResultConverter[JdbcResultConverterDomain, Product]]
protected[this] val TypeMappingResultConverter(childConverter, toBase, toMapped) = converter
protected[this] val ProductResultConverter(elementConverters # _ *) =
childConverter.asInstanceOf[ResultConverter[JdbcResultConverterDomain, Product]]
private[this] val updateQuerySplitRegExp = """(.*)(?<=set )((?:(?= where)|.)+)(.*)?""".r
private[this] val updateQuerySetterRegExp = """[^\s]+\s*=\s*\?""".r
/** An Action that updates the data selected by this query. */
def patch(value: T): DriverAction[Int, NoStream, Effect.Write] = {
val (seq, converters) = value.productIterator.zipWithIndex.toIndexedSeq
.zip(elementConverters)
.filter {
case ((Some(_), _), _) => true
case ((None, _), _) => false
case ((null, _), _) => false
case ((_, _), _) => true
}
.unzip
val (products, indexes) = seq.unzip
val newConverters = converters.zipWithIndex
.map(c => (c._1, c._2 + 1))
.map {
case (c: BaseResultConverter[_], idx) => new BaseResultConverter(c.ti, c.name, idx)
case (c: OptionResultConverter[_], idx) => new OptionResultConverter(c.ti, idx)
case (c: DefaultingResultConverter[_], idx) => new DefaultingResultConverter(c.ti, c.default, idx)
case (c: IsDefinedResultConverter[_], idx) => new IsDefinedResultConverter(c.ti, idx)
}
val productResultConverter =
ProductResultConverter(newConverters: _*).asInstanceOf[ResultConverter[JdbcResultConverterDomain, Any]]
val newConverter = TypeMappingResultConverter(productResultConverter, (p: Product) => p, (a: Any) => toMapped(a))
val newValue: Product = new ProductWrapper(products)
val newSql = sres.sql match {
case updateQuerySplitRegExp(prefix, setter, suffix) =>
val buffer = StringBuilder.newBuilder
buffer.append(prefix)
buffer.append(
updateQuerySetterRegExp
.findAllIn(setter)
.zipWithIndex
.filter(s => indexes.contains(s._2))
.map(_._1)
.mkString(", ")
)
buffer.append(suffix)
buffer.toString()
}
new SimpleJdbcDriverAction[Int]("patch", Vector(newSql)) {
def run(ctx: Backend#Context, sql: Vector[String]): Int =
ctx.session.withPreparedStatement(sql.head) { st =>
st.clearParameters
newConverter.set(newValue, st)
sres.setter(st, newConverter.width + 1, param)
st.executeUpdate
}
}
}
}
}
Example
// Model
case class User(
id: Option[Int] = None,
name: Option[String] = None,
username: Option[String] = None,
password: Option[String] = None
)
// Table
class Users(tag: Tag) extends Table[User](tag, "users") {
def id = column[Int]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def username = column[String]("username")
def password = column[String]("password")
override def * = (id.?, name.?, username.?, password.?) <>(User.tupled, User.unapply)
}
// TableQuery
object Users extends TableQuery(new Users(_))
// CustomDriver
trait CustomDriver extends PostgresDriver with PatchActionExtensionMethodsSupport {
override val api: API = new API {}
trait API extends super.API with PatchActionImplicits
}
// Insert
Users += User(Some(1), Some("Test"), Some("test"), Some("1234"))
// User patch
Users.filter(_.id === 1).patch(User(name = Some("Change Name"), username = Some("")))
https://gist.github.com/bad79s/1edf9ea83ba08c46add03815059acfca
Building on JonasAnso's answer, converting that to slick v3.0+, and putting it into a transaction:
def partialUpdate(id: Int, name: Option[String], login: Option[String]): Future[Int] = {
val selectQ = users.filter(_.id === id)
val query = selectQ.result.head.flatMap { data =>
selectQ.update(data.patch(name, login))
}
db.run(query)
}
As I commented the question is similar to an existing one, but you don't seem to have any extra requirements.
The simplest approach is just SELECT + UPDATE. For example you add a patch function in your DataRow class defining how you want to update your model
def patch(name: Option[String], state: Option[State], price: Option[Int]): Data {
this.copy(name = name.getOrElse(this.name), ...)
}
And you add a partialUpdate method in your repo class
class DataRepo {
private val Datas = TableQuery[Data]
val db = ???
def partialUpdate(id: Int, name: Option[String], state: Option[State], price: Option[Int]): Future[Int] = {
val query = Datas.filter(_.id === id)
for {
data <- db.run(query.result.head)
result <- db.run(query.update(data.patch(name, state, price)))
} yield result
}
}
As you see the main problem of this solution is that there are 2 SQL statements, SELECT and UPDATE.
Other solution is to use plain SQL (http://slick.typesafe.com/doc/3.0.0/sql.html) but of course this gives other problems.

Slick join two tables and get result of both

I have a Many to Many relationship setup like this:
Person <-> PersonField <-> Field
Now I want to query not only all the fields of a Person (I can do that), but a joined version of PersonField with Field of a Person. (I want to query/retrieve the Information in the Pivot/Intermediate Table "PersonField" as well!)
Person:
case class Person(id: Long, name: String)
{
def fields =
{
person <- Persons.all.filter(_.id === this.id)
field <- person.fields
} yield field
}
class Persons(tag: Tag) extends Table[Person](tag, "persons")
{
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def * = (id, name) <> (Person.tupled, Person.unapply)
def fields = PersonFields.all.filter(_.personID === id).flatMap(_.fieldFK)
}
object Persons
{
lazy val all = TableQuery[Persons]
}
Field:
case class Field(id: Long, name: String, description: Option[String])
class Fields(tag: Tag) extends Table[Field](tag, "fields")
{
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def description = column[Option[String]]("description")
def * = (id, name, description) <> (Field.tupled, Field.unapply)
}
object Fields
{
lazy val all = TableQuery[Fields]
}
PersonField:
case class PersonField(id: Long, personID: Long, fieldID: Long, value: String)
// TODO add constraint to make (personID, fieldID) unique
class PersonFields(tag: Tag) extends Table[PersonField](tag, "person_field")
{
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def personID = column[Long]("person_id")
def fieldID = column[Long]("field_id")
def value = column[String]("value")
def * = (id, personID, fieldID, value) <> (PersonField.tupled, PersonField.unapply)
def personFK = foreignKey("person_fk", personID, Persons.all)(_.id)
def fieldFK = foreignKey("field_fk", fieldID, Fields.all)(_.id)
}
object PersonFields
{
lazy val all = TableQuery[PersonFields]
}
Now to query all the fields of a Person I have a little helper-class:
def getFields(p: Person): Future[Seq[Field]] =
{
val query = p.fields
db.run(query.result)
}
So I can do
val personX ...
personX.onSuccess
{
case p: Person =>
{
val fields = helper.getFields(p)
fields.onSuccess
{
case f: Seq[Field] => f foreach println
}
}
}
Now each field of personX gets printed to the console. Works like a charm.
The thing is, I want to get the PersonField as well (with the Field)!
So I tried the following changes (among others that didn't work, which I can't remember)
In Person:
def fields =
{
for
{
person <- Persons.all.filter(_.id === this.id)
field <- person.fields join Fields.all on (_.fieldID === _.id)
} yield field
}
In PersonS
def fields = PersonFields.all.filter(_.personID === id) // No flatMap here!
then getFields(p: Person) looks like this:
def getFields(p: Person): Future[Seq[(PersonField, Field)]]
but
personX.onSuccess
{
case p: Person =>
{
val fields = helper.getFields(p)
fields.onSuccess
{
case f: Seq[(PersonField, Field)] => f map(f => println(f._1)}
}
}
}
gives me nothing, so I guess my join must be wrong. But what exactly am I doing wrong?
You can join all three, then yield the result
for {
((personField, person), field) <- PersonFields.all join Persons.all on (_.personId === _.id) join Fields.all on (_._1.fieldId === _.id)
if person.id === this.id
} yield (personField, person, field)
(I am not sure I got exactly what you were trying to get out of the query, so you can just edit the yield part )

in Slick 3.0, how to I get from a query to a case class?

I am trying to use Slick for database in a Scala application, and running into some issues (or my misunderstandings) of how to query (find) and convert the result to a case class.
I am not mapping the case class, but the actual values, with the intent of creating the case class on the fly. so, my table is:
object Tables {
class Names(tag: Tag) extends Table[Name](tag, "NAMES") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def first = column[String]("first")
def middle = column[String]("last")
def last = column[String]("last")
def * = (id.?, first, middle.?, last) <> ((Name.apply _).tupled, Name.unapply)
}
object NamesQueries {
lazy val query = TableQuery[Names]
val findById = Compiled { k: Rep[Long] =>
query.filter(_.id === k)
}
}
}
and here is the query:
object NamesDAO {
def insertName(name: Name) {
NamesQueries.query += name.copy(id = None)
}
def findName(nameId: Long) = {
val q = NamesQueries.findById(nameId) // AppliedCompiledFunction[Long, Query[Tables.Names, Tables.Names.TableElementType, Seq],Seq[Tables.Names.TableElementType]]
val resultSeq = Database.forConfig("schoolme").run(q.result) // Future[Seq[Tables.Names.TableElementType]]
val result = resultSeq.map { r => // val result: Future[(Option[Long], String, Option[String], String) => Name]
val rr = r.map{ name => // val rr: Seq[(Option[Long], String, Option[String], String) => Name]
Name.apply _
}
rr.head
}
result
}
}
however, the findName method seems to return Future((Option[Long], String, Option[String], String) => Name) instead of a Future(Name). What am i doing wrong? Is it just a matter of just using asInstanceOf[Name]?
EDIT: expanded findName to smaller chunks with comments for each one, as sap1ens suggested.
well, i'll be damned.
following sap1ens comment above, I broke findName to multiple steps (and edited the question). but after that, i went back and gave my val an explicit type, and that worked. see here:
def findName(nameId: Long) = {
val q = NamesQueries.findById(nameId)
val resultSeq: Future[Seq[Name]] = Database.forConfig("schoolme").run(q.result)
val result = resultSeq.map { r =>
val rr = r.map{ name =>
name
}
rr.head
}
result
}
so, type inference was the (/my) culprit this time. remember, remember.

Slick 3.0 many-to-many query with the join as an iterable

I've created a many-to-many collection using Slick 3.0, but I'm struggling to retrieve data in the way I want.
There is a many-to-many relationship between Events and Interests. Here are my tables:
case class EventDao(title: String,
id: Option[Int] = None)
class EventsTable(tag: Tag)
extends Table[EventDao](tag, "events") {
def id = column[Int]("event_id", O.PrimaryKey, O.AutoInc)
def title = column[String]("title")
def * = (
title,
id.?) <> (EventDao.tupled, EventDao.unapply)
def interests = EventInterestQueries.query.filter(_.eventId === id)
.flatMap(_.interestFk)
}
object EventQueries {
lazy val query = TableQuery[EventsTable]
val findById = Compiled { k: Rep[Int] =>
query.filter(_.id === k)
}
}
Here's EventsInterests:
case class EventInterestDao(event: Int, interest: Int)
class EventsInterestsTable(tag: Tag)
extends Table[EventInterestDao](tag, "events_interests") {
def eventId = column[Int]("event_id")
def interestId = column[Int]("interest_id")
def * = (
eventId,
interestId) <> (EventInterestDao.tupled, EventInterestDao.unapply)
def eventFk = foreignKey("event_fk", eventId, EventQueries.query)(e => e.id)
def interestFk = foreignKey("interest_fk", interestId, InterestQueries.query)(i => i.id)
}
object EventInterestQueries {
lazy val query = TableQuery[EventsInterestsTable]
}
And finally Interests:
case class InterestDao(name: String,
id: Option[Int] = None)
class InterestsTable(tag: Tag)
extends Table[InterestDao](tag, "interests") {
def id = column[Int]("interest_id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def name_idx = index("idx_name", name, unique = true)
def * = (
name,
id.?) <> (InterestDao.tupled, InterestDao.unapply)
def events = EventInterestQueries.query.filter(_.interestId === id)
.flatMap(_.eventFk)
}
object InterestQueries {
lazy val query = TableQuery[InterestsTable]
val findById = Compiled { k: Rep[Int] =>
query.filter(_.id === k)
}
}
I can query and retrieve tuples of (event.name, interest) with the following:
val eventInterestQuery = for {
event <- EventQueries.query
interest <- event.interests
} yield (event.title, interest.name)
Await.result(db.run(eventInterestQuery.result).map(println), Duration.Inf)
So this is what I currently have.
What I want is to be able to populate a case class like:
case class EventDao(title: String,
interests: Seq[InterestDao],
id: Option[Int] = None)
The trouble is that if I update my case class like this, it messes up my def * projection in EventsTable. Also, I'll have to rename the EventsTable.interests filter to something like EventsTable.interestIds which is a bit ugly but I could live with if necessary.
Also, I can't find a way of writing a for query that yields (event.name, Seq(interest.name)). Anyway, that's just a stepping stone to me being able to yield a (EventDao, Seq(InterestDao)) tuple which is what I really want to return.
Does anyone know how I can achieve these things? I also want to be able to 'take' a certain number of Interests, so for some queries all would be returned, but for others only the first 3 would be.
So after reading this page and chatting on the mailing list, I finally got it working:
val eventInterestQuery = for {
event <- EventQueries.query
interest <- event.interests
} yield (event, interest)
Await.result(db.run(eventInterestQuery.result
// convert the interests to a sequence.
.map {
_.groupBy(_._1)
.map {
case (k,v) => (k, v.map(_._2))
}.toSeq
}
), Duration.Inf)
The only issue with groupBy is you lose order. You could fold the result. I've written this helper for my current project:
def foldOneToMany[A, B](in: Seq[(A, Option[B])], eq: (A, B) => Boolean)
(f: (A, B) => A): Seq[A] =
in.foldLeft(List.empty[A]) {
case (head :: tail, (_, Some(rel))) if eq(head, rel) =>
f(head, rel) :: tail
case (r, (el, Some(rel))) => f(el, rel) :: r
case (r, (el, None)) => el :: r
}.reverse
It could do with some love. Now it takes in a function A,B => Boolean to determine if B belongs to A and a function A,B => A that adds B to A.
Virtualeyes also has a point. In Postgres you could use array_agg function to use a little less bandwidth from the db.

How to parametrize Scala Slick queries by WHERE clause conditions?

Assume these two simple queries:
def findById(id: Long): Option[Account] = database.withSession { implicit s: Session =>
val query = for (a <- Accounts if a.id === id) yield a.*
query.list.headOption
}
def findByUID(uid: String): Option[Account] = database.withSession { implicit s: Session =>
val query = for (a <- Accounts if a.uid === uid) yield a.*
query.list.headOption
}
I would like to rewrite it to remove the boilerplate duplication to something like this:
def findBy(criteria: ??? => Boolean): Option[Account] = database.withSession {
implicit s: Session =>
val query = for (a <- Accounts if criteria(a)) yield a.*
query.list.headOption
}
def findById(id: Long) = findBy(_.id === id)
def findByUID(uid: Long) = findBy(_.uid === uid)
I don't know how to achieve it for there are several implicit conversions involved in the for comprehension I haven't untangled yet. More specifically: what would be the type of ??? => Boolean in the findBy method?
EDIT
These are Account and Accounts classes:
case class Account(id: Option[Long], uid: String, nick: String)
object Accounts extends Table[Account]("account") {
def id = column[Option[Long]]("id")
def uid = column[String]("uid")
def nick = column[String]("nick")
def * = id.? ~ uid ~ nick <> (Account, Account.unapply _)
}
I have this helper Table:
abstract class MyTable[T](_schemaName: Option[String], _tableName: String) extends Table[T](_schemaName, _tableName) {
import scala.slick.lifted._
def equalBy[B: BaseTypeMapper]
(proj:this.type => Column[B]):B => Query[this.type,T] = { (str:B) =>
Query[this.type,T,this.type](this) where { x => proj(x) === str} }
}
Now you can do:
val q=someTable.equalBy(_.someColumn)
q(someValue)