I want to create a query that returns all groups with users count (including empty groups)
SELECT g.id, count(relation.user_id) FROM groups g
FULL JOIN users2groups relation ON g.id=r.group_id
GROUP BY g.id;
for this model:
class Users(tag: Tag) extends Table[User](tag, "users") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
...
}
class Groups(tag: Tag) extends Table[Group](tag, "groups") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
...
}
val users = TableQuery[Users]
val groups = TableQuery[Groups]
/** relation for users and group */
class Users2Groups(tag: Tag) extends Table[(Long,Long)](tag, "users2groups") {
def userId = column[Long]("user_id")
def groupId = column[Long]("group_id")
def user = foreignKey("user_fk", userId, users)(_.id)
def group = foreignKey("group_fk", groupId, groups)(_.id)
def * = (userId, groupId)
def ? = (userId.?, groupId.?)
def pk = primaryKey("pk_user2group", (userId, userId))
}
This is my solution with slick:
val query = for {
(g, rel) <-
groups leftJoin
users2groups on (_.id === _.groupId)
} yield (g, rel.groupId.?)
val result = query.groupBy(_._1.id).map(e => (e._1, e._2.length)).list
result foreach println
But it doesn't work correctly. It returns the incorrect amount of users for empty groups (returns users count = 1 instead of 0).
My environment: scala-2.11.2, slick-2.1.0, PostgreSQL
I don't see what's wrong. Could well be a Slick bug. Please report one here: https://github.com/slick/slick
Related
I am working on a Library data model where each book can have multiple authors and vice versa (many to many).
I want to pass a list of books to a html view page that each book includes a list of its author(s).
To do that I have defined the following tables for book and authors:
private class BookTable(tag: Tag) extends Table[Book](tag, "book") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def publishDate = column[Date]("publish_date")
def memberId = column[Option[Long]]("member_id")
def member = foreignKey("member_fk",memberId,members)(_.id)
type Data = (Long, String, Date, Option[Long])
def constructBook: Data => Book = {
case (id, name, publishDate, memberId) =>
Book(id, name, publishDate, memberId)
}
def extractBook: PartialFunction[Book, Data] = {
case Book(id, name, publishDate, memberId, _) =>
(id, name, publishDate, memberId)
}
def * = (id, name, publishDate, memberId) <> (constructBook, extractBook.lift)
}
private class AuthorBookTable (tag: Tag) extends Table[AuthorBook](tag, "author_book") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def authorId = column[Long]("author_id")
def bookId = column[Long]("book_id")
def memberId = column[Option[Long]]("member_id")
def author = foreignKey("author_fk",authorId,authors)(_.id)
def book = foreignKey("book_fk",bookId,books)(_.id)
def * = (id, authorId, bookId) <> ((AuthorBook.apply _).tupled, AuthorBook.unapply)
}
private class AuthorTable (tag: Tag) extends Table[Author](tag, "author") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def * = (id, name) <> ((Author.apply _).tupled, Author.unapply)
}
The book case class is as below:
case class Book(id: Long, name: String, publishDate: Date, memberId: Option[Long] = None, authors: Seq[Author]= Seq.empty)
{
def updateAuthors(authorss: Seq[Author]) = {
this.copy(authors=authorss)
}
}
In controller I use the below:
def getBooks = Action.async { implicit request =>
repo.getBooks.map { books =>
val booksWithAuthors=books.map( b=> {val updateB=b.updateAuthors( repo.getBookAuthors(b.id))
updateB})
Ok(Json.toJson(booksWithAuthors))
}
}
My question is about the getBookAuthors implementation shown below:
implicit def waitForFuture[A](f:Future[A]) = {
def res: A = Await.result(f, Duration.Inf)
res
}
def getBookAuthors(id: Long): Seq[Author] = {
val result=db.run {
val innerJoin = for {
(ab, a) <- authorBooks join authors on (_.authorId === _.id)
} yield (a, ab.bookId)
innerJoin.filter(_._2 === id).sortBy(_._1.name).map(_._1).result
}
waitForFuture(result)
}
My concern is that the getBookAuthors function is blocking and I am not sure if it's the best practice. Please advise if there is a better way to do this.
As you are saying, blocking methods are pretty bad in this context and you will lost the advantages of using a non-blocking library as Slick.
the getBookAuthors would be written as follows, returning a Future[Seq[Author]] thats needs to be managed in the caller
def getBookAuthors(id: Long): Future[Seq[Author]] =
db.run {
val innerJoin = for {
(ab, a) <- authorBooks join authors on (_.authorId === _.id)
} yield (a, ab.bookId)
innerJoin.filter(_._2 === id).sortBy(_._1.name).map(_._1).result
}
So the caller should be rewritten as:
def getBooks = Action.async { implicit request =>
repo.getBooks.flatMap { books =>
Future.sequence(
books.map { b =>
repo.getBookAuthors(b.id).map(authors => b.updateAuthors(authors))
}
).map { booksWithAuthors =>
Ok(Json.toJson(booksWithAuthors))
}
}
}
This means that, once you will have the books: Seq[Book] you will map over it to integrate the authors and this will end with a Seq[Future[Book]].
Then it can be transformed into a Future[Seq[Book]] (with authors) with the Future.sequence method.
Finally you need to flatMap on the outer Future to move from Future[Future[Seq[Book]]] to a simpler Future[Seq[Book]]
This second snippet can be refactored in a more clean way taking advantage of the for-comprehension that is a syntactic sugar for the flatMap
private def addAuthorsToBooks(books: Seq[Book]): Future[Seq[Book]] =
Future.sequence(
books.map { b =>
repo.getBookAuthors(b.id).map(authors => b.updateAuthors(authors))
}
)
def getBooks = Action.async { implicit request =>
for {
books <- repo.getBooks
booksWithAuthors <- addAuthorsToBooks(books)
} yield Ok(Json.toJson(booksWithAuthors))
}
I have two models:
case class User(uid: Option[Int], email: String, password: String, created_at: Timestamp, updated_at: Timestamp)
case class UserProfile(firstname: String, lastname: String, gender: Int, user_id: Long),
And a DAO with user table defined:
package dao
import java.sql.Timestamp
import scala.concurrent.{Await, Future}
import javax.inject.Inject
import models.{User, UserProfile}
import play.api.db.slick.DatabaseConfigProvider
import play.api.db.slick.HasDatabaseConfigProvider
import play.api.libs.concurrent.Execution.Implicits.defaultContext
import slick.driver.JdbcProfile
import slick.profile.SqlProfile.ColumnOption.SqlType
import scala.concurrent.duration._
import com.github.t3hnar.bcrypt._
class UsersDAO #Inject()(protected val dbConfigProvider: DatabaseConfigProvider) extends HasDatabaseConfigProvider[JdbcProfile] {
import driver.api._
private val Users = TableQuery[UsersTable]
private val UsersProfile = TableQuery[UserProfileTable]
def all(): Future[Seq[User]] = db.run(Users.result)
def insert(user: User): Future[Int] = {
println("coming inside insert of user dao")
println(user)
// insertUP(user)
val hashPassword = user.password.bcrypt
val updatedUser = user.copy(password = hashPassword)
val query = db.run((Users returning Users.map(_.uid)) += updatedUser)
// val uid = Await.result(query, 30 seconds)
// println(s"UID ---------> $uid")
query
}
def findByEmail(email: String): Option[User] = {
val query = for {
u <- Users if u.email === email
} yield u
val f: Future[Option[User]] = db.run(query.result).map(_.headOption)
val result = Await.result(f, 30 seconds)
println(result.isDefined)
result
}
def authenticate(username: String, password: String): Future[Option[User]] = {
val query = db.run(Users.filter(_.email === username).result.map(_.headOption.filter(user => password.isBcrypted(user.password)))).map(_.headOption)
query
}
private class UsersTable(tag: Tag) extends Table[User](tag, "users") {
def uid = column[Int]("uid", O.PrimaryKey, O.AutoInc, O.SqlType("INT"))
def email = column[String]("email")
def password = column[String]("password")
def created_at = column[Timestamp]("created_at", SqlType("timestamp not null default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP"))
def updated_at = column[Timestamp]("updated_at", SqlType("timestamp not null default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP"))
def idx = index("email_UNIQUE", email, unique = true)
def * = (uid.?, email, password, created_at, updated_at) <> (User.tupled, User.unapply _)
}
private class UserProfileTable(tag: Tag) extends Table[UserProfile](tag, "user_profile"){
def firstname = column[String]("firstname")
def lastname = column[String]("lastname")
def gender = column[Int]("gender")
def user_id = column[Int]("user_id")
def * = (firstname, lastname, gender, user_id) <> (UserProfile.tupled, UserProfile.unapply)
def fk_user_id = foreignKey("fk_user_id", user_id, Users)(_.uid)
}
}
In the insert function, how could I add uid to user_id field of user profile table in the same function or call?
Edit 1
Tried the solution suggested by Pawel, but getting exception:
failed slick.SlickException: This DBMS allows only a single AutoInc column to be returned from an INSERT
Edit - 2
Now, after trying some solutions, the insert function is like this:
def insert(user: User): Future[Int] = {
val hashPassword = user.password.bcrypt
val updatedUser = user.copy(password = hashPassword)
val insertUser = (Users returning Users.map(_.uid)) += updatedUser
def insertUserProfile(updatedUserProfile: UserProfile) = (UsersProfile returning UsersProfile.map(_.user_id)) += updatedUserProfile
val insertUserThenProfile = for {
createdUserId <- insertUser
createdUserProfileId <- insertUserProfile(UserProfile("First name", "Last name", gender = 0, user_id = createdUserId))
} yield createdUserProfileId
db.run(insertUserThenProfile.transactionally)
}
But still getting the error:
failed slick.SlickException: This DBMS allows only a single AutoInc column to be returned from an INSERT
Solution
Pawels solution should work fine, but some DBMS gives exception for not returning AUtoInc field, and for trying to return something else.
You can see the note in documentation:
http://slick.lightbend.com/doc/3.0.0/queries.html
Note
Many database systems only allow a single column to be returned which must be the table’s auto-incrementing primary key. If you ask for other columns a SlickException is thrown at runtime (unless the database actually supports it).
So, Now, my models are like this:
case class User(uid: Option[Int], email: String, password: String, created_at: Timestamp, updated_at: Timestamp)
case class UserProfile(upid: Option[Int], firstname: String, lastname: String, gender: Int, user_id: Int)
And the Table class:
private class UserProfileTable(tag: Tag) extends Table[UserProfile](tag, "user_profile"){
def upid= column[Int]("upid", O.PrimaryKey, O.AutoInc, O.SqlType("INT"), O.Default(0))
def firstname = column[String]("firstname")
def lastname = column[String]("lastname")
def gender = column[Int]("gender")
def user_id = column[Int]("user_id")
def * = (upid.?, firstname, lastname, gender, user_id) <> (UserProfile.tupled, UserProfile.unapply)
def fk_user_id = foreignKey("fk_user_id", user_id, Users)(_.uid)
}
And finally the insert method:
def insert(user: User): Future[Int] = {
val hashPassword = user.password.bcrypt
val updatedUser = user.copy(password = hashPassword)
val insertUser = (Users returning Users.map(_.uid)) += updatedUser
def insertUserProfile(updatedUserProfile: UserProfile) = (UsersProfile returning UsersProfile.map(_.upid)) += updatedUserProfile
val insertUserThenProfile = for {
createdUserId <- insertUser
createdUserProfileId <- insertUserProfile(UserProfile(Some(0), "First name", "Last name", gender = 0, user_id = createdUserId))
} yield createdUserProfileId
db.run(insertUserThenProfile.transactionally)
}
I don't know how are you planning to provide values for UserProfile (it's up to you, maybe an additional parameter in the insert method), but I would try something like this:
def insert(user: User): Future[UserProfile] = {
val hashPassword = user.password.bcrypt
val updatedUser = user.copy(password = hashPassword)
val insertUser = (Users returning Users.map(_.uid)) += updatedUser
def insertUserProfile(updatedUserProfile: UserProfile) = (UsersProfile returning UsersProfile.map(_.user_id)) += updatedUserProfile
val insertUserThenProfile = for {
createdUserId <- insertUser
createdUserProfileId <- insertUserProfile(UserProfile("First name", "Last name", gender = 0, user_id = createdUserId))
} yield createdUserProfileId
db.run(insertUserThenProfile.transactionally)
}
Make a query to insert a new Member and return the member with its id
val memberWithId = (queryMember returning queryMember.map(_.id) into ((c, id) => c.copy(id = id))) += registerMember.copy(password = pwHash)
Then implement the query as a transaction. Insert Member, insert User Profile(you can use the id from the success of the first statement)
val transaction = (for {
m: Member <- memberWithId
p: UsersProfile <- queryProfile returning queryProfile += MemberProfile(m.id, firstName, ...)
} yield m).transactionally
Run it as a Try so that you can more easily parse errors rather than crashing.
db.run(transaction.asTry)
I have a Many to Many relationship setup like this:
Person <-> PersonField <-> Field
Now I want to query not only all the fields of a Person (I can do that), but a joined version of PersonField with Field of a Person. (I want to query/retrieve the Information in the Pivot/Intermediate Table "PersonField" as well!)
Person:
case class Person(id: Long, name: String)
{
def fields =
{
person <- Persons.all.filter(_.id === this.id)
field <- person.fields
} yield field
}
class Persons(tag: Tag) extends Table[Person](tag, "persons")
{
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def * = (id, name) <> (Person.tupled, Person.unapply)
def fields = PersonFields.all.filter(_.personID === id).flatMap(_.fieldFK)
}
object Persons
{
lazy val all = TableQuery[Persons]
}
Field:
case class Field(id: Long, name: String, description: Option[String])
class Fields(tag: Tag) extends Table[Field](tag, "fields")
{
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def description = column[Option[String]]("description")
def * = (id, name, description) <> (Field.tupled, Field.unapply)
}
object Fields
{
lazy val all = TableQuery[Fields]
}
PersonField:
case class PersonField(id: Long, personID: Long, fieldID: Long, value: String)
// TODO add constraint to make (personID, fieldID) unique
class PersonFields(tag: Tag) extends Table[PersonField](tag, "person_field")
{
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def personID = column[Long]("person_id")
def fieldID = column[Long]("field_id")
def value = column[String]("value")
def * = (id, personID, fieldID, value) <> (PersonField.tupled, PersonField.unapply)
def personFK = foreignKey("person_fk", personID, Persons.all)(_.id)
def fieldFK = foreignKey("field_fk", fieldID, Fields.all)(_.id)
}
object PersonFields
{
lazy val all = TableQuery[PersonFields]
}
Now to query all the fields of a Person I have a little helper-class:
def getFields(p: Person): Future[Seq[Field]] =
{
val query = p.fields
db.run(query.result)
}
So I can do
val personX ...
personX.onSuccess
{
case p: Person =>
{
val fields = helper.getFields(p)
fields.onSuccess
{
case f: Seq[Field] => f foreach println
}
}
}
Now each field of personX gets printed to the console. Works like a charm.
The thing is, I want to get the PersonField as well (with the Field)!
So I tried the following changes (among others that didn't work, which I can't remember)
In Person:
def fields =
{
for
{
person <- Persons.all.filter(_.id === this.id)
field <- person.fields join Fields.all on (_.fieldID === _.id)
} yield field
}
In PersonS
def fields = PersonFields.all.filter(_.personID === id) // No flatMap here!
then getFields(p: Person) looks like this:
def getFields(p: Person): Future[Seq[(PersonField, Field)]]
but
personX.onSuccess
{
case p: Person =>
{
val fields = helper.getFields(p)
fields.onSuccess
{
case f: Seq[(PersonField, Field)] => f map(f => println(f._1)}
}
}
}
gives me nothing, so I guess my join must be wrong. But what exactly am I doing wrong?
You can join all three, then yield the result
for {
((personField, person), field) <- PersonFields.all join Persons.all on (_.personId === _.id) join Fields.all on (_._1.fieldId === _.id)
if person.id === this.id
} yield (personField, person, field)
(I am not sure I got exactly what you were trying to get out of the query, so you can just edit the yield part )
I'm having an issue composing a chained insert query using slick. I want to insert a row, have the primary key returned, then insert a new row with a foreign key to the row I just inserted.
I've tried using flatmap to chain the result of the first insert into the second insert, however the following code seems to only perform the first insert into the "users" table, and not the second insert into "lists"
Sample code is below, thank you in advance for the help!
package com.example.jdbc
import slick.driver.PostgresDriver.api._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success}
object DatabaseControllerMin {
lazy val db = Database.forConfig("postgres")
lazy val recreateTables = DBIO.seq(
// drop the old tables
(users.schema ++ lists.schema).drop,
// Create the tables, including primary and foreign keys
(users.schema ++ lists.schema).create
)
class Users(tag: Tag) extends Table[(Int, String)](tag, "users") {
def id = column[Int]("user_id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def * = (id, name)
}
lazy val users = TableQuery[Users]
class Lists(tag: Tag) extends Table[(Int, String, Int)](tag, "lists") {
def id = column[Int]("list_id", O.PrimaryKey, O.AutoInc)
def name = column[String]("name")
def userId = column[Int]("user_id")
def * = (id, name, userId)
def user = foreignKey("users", userId, users)(_.id)
}
lazy val lists = TableQuery[Lists]
def recreateDatabase = {
try {
val q = recreateTables andThen {
users returning users.map(_.id) into ((user, id) => user.copy(_1 = id)) +=(0, "Tyler")
} flatMap {
user => {
lists +=(0, s"${user._2}'s List", user._1)
}
}
db.run(q)
} finally db.close
}
def main(args: Array[String]) {
recreateDatabase onComplete {
case Success(x) => {
println("Success!!!")
}
case Failure(t) => {
println("Failure")
t.printStackTrace
}
}
}
}
I am finding it difficult to figure out the best wy to create an instance of a class (a DTO class) which can be passed as json to the calling client.
I have the following class structure.
object Suppliers extends Table[(Int, String, String, String, String, String)]("SUPPLIERS") {
def id = column[Int]("SUP_ID", O.PrimaryKey) // This is the primary key column
def name = column[String]("SUP_NAME")
def street = column[String]("STREET")
def city = column[String]("CITY")
def state = column[String]("STATE")
def zip = column[String]("ZIP")
def * = id ~ name ~ street ~ city ~ state ~ zip
}
object Coffees extends Table[(Int,String, Double,Int, Int)]("COFFEES") {
def id = column[Int]("Id",O.PrimaryKey)
def name = column[String]("COF_NAME")
def price = column[Double]("PRICE")
def sales = column[Int]("SALES")
def total = column[Int]("TOTAL")
def * = id ~ name ~ price ~ sales ~ total
}
object CoffeeSuppliers extends Table[(Int,Int,Int)]("CoffeeSuppliers") {
def id = column[Int]("Id",O.PrimaryKey)
def supID = column[Int]("Sup_ID")
def coffeeID = column[Int]("Coffee_ID")
def supplier = foreignKey("SUP_FK", supID, Suppliers)(_.id)
def coffees = foreignKey("COF_FK", coffeeID,Coffees)(_.id)
def * = id ~ supID ~ coffeeID
}
I am using this simple join query to retrieve Supplier with id 101 and all the coffees he supplies.
val q3 = for {
((cs,c),s) <- CoffeeSuppliers innerJoin
Coffees on (_.coffeeID === _.id) innerJoin
Suppliers on (_._1.supID === _.id) if cs.supID === 101
} yield (cs,c,s)
The query is working fine and I am able to retrieve the data.
But, I want to construct a DTO class out of the query result. The Class structure is as fallows
case class CoffeeDTO(
id:Option[Int] = Some(0),
name:String[String] = "",
price:Double= 0.0
)
case class SupplierDTO (
id:Option[Int] = Some(0),
name:String = "",
coffees:List[CoffeeDTO] = Nil
)
How to create an instance of SupplierDTO and assign the values from the query result?
How about something like this:
q3.map{ case (cs,c,s) => ((s.id.?,s.name),(c.id.?,c.name,c.price)) } // remove not needed columns and make ids Options
.list // run query
.groupBy( _._1 ) // group by supplier
.map{ case (s,scPairs) => SupplierDTO( s._1, // map each group to a supplier with associated coffees
s._2,
scPairs.map(_._2) // map group of pairs to only coffee tuples
.map(CoffeeDTP.tupled) // create coffee objects
)}
.head // take just the one supplier out of the list