I have a ConnectionIO[Option[Int]] and map over the Option to produce a ConnectionIO[Option[String]] with a query the Some[Int] otherwise keep the Nones. I was able to do this with aforcomprehension and amatch`:
def findWidgetByOwner(name: String): ConnectionIO[Option[String]] = for {
opt <- sql"SELECT owner_id FROM owners WHERE name = $name".query[Int].option
widget <- opt match {
case None => None.pure[ConnectionIO]
case Some(id) => sql"SELECT widget_name FROM widgets WHERE owner_id = $id".query[String].option
}
} yield widget
I know i'm getting tripped up by the ConnectionIO container, but I can't find a cleaner mapping approach that to transform ConnectionIO[Option[Int]] to ConnectionIO[Option[String]].
It would be cleaner to join using SQL instead of scala:
def findWidgetByOwner(name: String): ConnectionIO[Option[String]] =
sql"""
SELECT widgets.widget_name FROM widgets WHERE owner_id = $id
INNER JOIN owners ON widgets.owner_id = owners.owner_id
WHERE owners.name = $name
""".query[Int].option
But if you want to clean up the original, some incantation of sequence would probably work (not tested):
import cats._, cats.data._, cats.implicits._
...
widget <- opt.map {id =>
sql"SELECT widget_name FROM widgets WHERE owner_id = $id".query[String].unique
}.sequence
Note: You have to change query[String].option to .query[String].unique otherwise widget becomes an Option[Option[String]] which if the widget query can be null, might be desirable, but requires a .flatten at the end.
Related
How can I convert Query[MappedProjection[Example, (Option[String], Int, UUID, UUID)], Example, Seq] to Query[Examples, Example, Seq]?
Details
I am trying to drop a column from an existing table(Examples in this case) and move the data to another table (Examples2 in this case). I don't want to change all the existing code base, so I plan to join these two tables and map the results to Example.
import slick.lifted.Tag
import slick.driver.PostgresDriver.api._
import java.util.UUID
case class Example(
field1: Option[String] = None,
field2: Int,
someForeignId: UUID,
id: UUID,
)
object Example
class Examples(tag: Tag) extends Table[Example](tag, "entityNotes") {
def field1 = column[Option[String]]("field1")
def field2 = column[Int]("field2")
def someForeignId = column[UUID]("someForeignId")
def id = column[UUID]("id", O.PrimaryKey)
def someForeignKey = foreignKey(
"someForeignIdToExamples2",
someForeignId,
Examples2.query,
)(
_.id.?
)
def * =
(
field1.?,
field2,
someForeignId,
id,
) <> ((Example.apply _).tupled, Example.unapply)
}
object Examples{
val query = TableQuery[Examples]
}
Basically, all the functions in the codebase call Examples.query. If I update that query by joining two tables, the problem will be solved (of course with a performance shortcoming because of one extra join for each call).
To use the query with the existing code base, we need to keep the type the same. For example, we we can use filter as follows:
val query_ = TableQuery[Examples]
val query: Query[Examples, Example, Seq] = query_.filter(_.field2 > 5)
Everything will work without a problem since we keep the type of the query as it is supposed to be.
However, I cannot do that with a join if I want to use data from the second table.
val query_ = TableQuery[Examples]
val query = query
.join(Examples2.query_)
.on(_.someForeignId === _.id)
.map({
case (e, e2) =>
((
e2.value.?,
e1.field2,
e2.id
e.id,
) <> ((Example.apply _).tupled, Example.unapply))
})
This is where I got stuck. Its type is Query[MappedProjection[Example, (Option[String], Int, UUID, UUID)], Example, Seq].
Can anyone help? Btw, we don't have to use map. This is just what I got so far.
Referencing to this question.
I want to insert some entity by some condition. It can either be inserted or not. If the condition is true the entity is inserted. I want to insert some other data in various tables. It looks like this:
val q = sql"insert into some_table (some_field) select 42 where ...(some condition)"
val inserts = List(
sql"insert ...",
sql"insert ...",
sql"insert ..."
)
for {
id <- q.update.withGeneratedKeys[Long]("id")
_ <- inserts.reduce(_ ++ _).update.run
} yield id
The problem is this does not compile because the first insert is a fs2.Stream but the second one is not.
I was trying to replace _ <- inserts.reduce... with _ = inserts.reduce. The app can compile but inserts in the second line does not occur.
UPD
My possible way to solve this problem:
...
for {
idOpt <- q.update.withGeneratedKeys[Long]("id").compile.last
_ <- idOpt.fold(0.pure[ConnectionIO])(_ => inserts.reduce(_ ++ _).update.run)
} yield idOpt
This works, but IMHO this is not pretty. Is there a better way to do it?
One way to perform your batch inserts - if you have similar data - is to use updateMany - see doc:
import doobie._
type PersonInfo = (String, Option[Short])
def insertMany(ps: List[PersonInfo]): ConnectionIO[Int] = {
val sql = "insert into person (name, age) values (?, ?)"
Update[PersonInfo](sql).updateMany(ps)
}
// Some rows to insert
val data = List[PersonInfo](
("Frank", Some(12)),
("Daddy", None))
Also, if you remove.compile.last, you can use the fact that if your resulting Stream q.update.withGeneratedKeys[Long]("id") is empty, you 'exit early' the for-comprehension.
So all in all, here is what you could do:
import fs2.Stream
val result =
// Now the for-comprehension operates on a Stream instead of an Option
for {
r <- q.update.withGeneratedKeys[Long]("id")
_ <- Stream.eval(insertMany(data)) // insertMany(data) is defined like in the snippet above
} yield r
result.compile.last
Let's say I have a table:
object Suppliers extends Table[(Int, String, String, String)]("SUPPLIERS") {
def id = column[Int]("SUP_ID", O.PrimaryKey)
def name = column[String]("SUP_NAME")
def state = column[String]("STATE")
def zip = column[String]("ZIP")
def * = id ~ name ~ state ~ zip
}
Table's database name
The table's database name can be accessed by going: Suppliers.tableName
This is supported by the Scaladoc on AbstractTable.
For example, the above table's database name is "SUPPLIERS".
Columns' database names
Looking through AbstractTable, getLinearizedNodes and indexes looked promising. No column names in their string representations though.
I assume that * means "all the columns I'm usually interested in." * is a MappedProjection, which has this signature:
final case class MappedProjection[T, P <: Product](
child: Node,
f: (P) ⇒ T,
g: (T) ⇒ Option[P])(proj: Projection[P])
extends ColumnBase[T] with UnaryNode with Product with Serializable
*.getLinearizedNodes contains a huge sequence of numbers, and I realized that at this point I'm just doing a brute force inspection of everything in the API for possibly finding the column names in the String.
Has anybody also encountered this problem before, or could anybody give me a better understanding of how MappedProjection works?
It requires you to rely on Slick internals, which may change between versions, but it is possible. Here is how it works for Slick 1.0.1: You have to go via the FieldSymbol. Then you can extract the information you want like how columnInfo(driver: JdbcDriver, column: FieldSymbol): ColumnInfo does it.
To get a FieldSymbol from a Column you can use fieldSym(node: Node): Option[FieldSymbol] and fieldSym(column: Column[_]): FieldSymbol.
To get the (qualified) column names you can simply do the following:
Suppliers.id.toString
Suppliers.name.toString
Suppliers.state.toString
Suppliers.zip.toString
It's not explicitly stated anywhere that the toString will yield the column name, so your question is a valid one.
Now, if you want to programmatically get all the column names, then that's a bit harder. You could try using reflection to get all the methods that return a Column[_] and call toString on them, but it wouldn't be elegant. Or you could hack a bit and get a select * SQL statement from a query like this:
val selectStatement = DB withSession {
Query(Suppliers).selectStatement
}
And then parse our the column names.
This is the best I could do. If someone knows a better way then please share - I'm interested too ;)
Code is based on Lightbend Activator "slick-http-app".
slick version: 3.1.1
Added this method to the BaseDal:
def getColumns(): mutable.Map[String, Type] = {
val columns = mutable.Map.empty[String, Type]
def selectType(t: Any): Option[Any] = t match {
case t: TableExpansion => Some(t.columns)
case t: Select => Some(t.field)
case _ => None
}
def selectArray(t:Any): Option[ConstArray[Node]] = t match {
case t: TypeMapping => Some(t.child.children)
case _ => None
}
def selectFieldSymbol(t:Any): Option[FieldSymbol] = t match {
case t: FieldSymbol => Some(t)
case _ => None
}
val t = selectType(tableQ.toNode)
val c = selectArray(t.get)
for (se <- c.get) {
val col = selectType(se)
val fs = selectFieldSymbol(col.get)
columns += (fs.get.name -> fs.get.tpe)
}
columns
}
this method gets the column names (real names in DB) + types form the TableQ
used imports are:
import slick.ast._
import slick.util.ConstArray
My table data forms a tree structure where one row can reference a parent row in the same table.
What I am trying to achieve, using Slick, is to write a query that will return a row and all it's children. Also, I would like to do the same, but write a query that will return a child and all it's ancestors.
In other words:
findDown(1) should return
List(Group(1, 0, "1"), Group(3, 1, "3 (Child of 1)"))
findUp(5) should return
List(Group(5, 2, "5 (Child of 2)"), Group(2, 0, "2"))
Here is a fully functional worksheet (except for the missing solutions ;-).
package com.exp.worksheets
import scala.slick.driver.H2Driver.simple._
object ParentChildTreeLookup {
implicit val session = Database.forURL("jdbc:h2:mem:test1;", driver = "org.h2.Driver").createSession()
session.withTransaction {
Groups.ddl.create
}
Groups.insertAll(
Group(1, 0, "1"),
Group(2, 0, "2"),
Group(3, 1, "3 (Child of 1)"),
Group(4, 3, "4 (Child of 3)"),
Group(5, 2, "5 (Child of 2)"),
Group(6, 2, "6 (Child of 2)"))
case class Group(
id: Long = -1,
id_parent: Long = -1,
label: String = "")
object Groups extends Table[Group]("GROUPS") {
def id = column[Long]("ID", O.PrimaryKey, O.AutoInc)
def id_parent = column[Long]("ID_PARENT")
def label = column[String]("LABEL")
def * = id ~ id_parent ~ label <> (Group, Group.unapply _)
def autoInc = id_parent ~ label returning id into {
case ((_, _), id) => id
}
def findDown(groupId: Long)(implicit session: Session) = { ??? }
def findUp(groupId: Long)(implicit session: Session) = { ??? }
}
}
A really bad, and static attempt at findDown may be something like:
private def groupsById = for {
group_id <- Parameters[Long]
g <- Groups; if g.id === group_id
} yield g
private def childrenByParentId = for {
parent_id <- Parameters[Long]
g <- Groups; if g.id_parent === parent_id
} yield g
def findDown(groupId: Long)(implicit session: Session) = { groupsById(groupId).list union childrenByParentId(groupId).list }
But, I'm looking for a way for Slick to recursively search the same table using the id and id_parent link. Any other good ways to solve the problem is really welcome. Keep in mind though, that it would be best to minimise the number of database round-trips.
You could try calling SQL from slick. The SQL call to go up the hierarchy would look something like this (This is for SQL Server):
WITH org_name AS
(
SELECT DISTINCT
parent.id AS parent_id,
parentname.label as parent_label,
child.id AS child_id,
childname.ConceptName as child_label
FROM
Group parent RIGHT OUTER JOIN
Group child ON child.parent_id = parent.id
),
jn AS
(
SELECT
parent_id,
parent_label,
child_id,
child_label
FROM
org_name
WHERE
parent_id = 5
UNION ALL
SELECT
C.parent_id,
C.parent_label,
C.child_id,
C.child_label
FROM
jn AS p JOIN
org_name AS C ON C.child_id = p.parent_id
)
SELECT DISTINCT
jn.parent_id,
jn.parent_label,
jn.child_id,
jn.child_label
FROM
jn
ORDER BY
1;
If you want to go down the hierarchy change the line:
org_name AS C ON C.child_id = p.parent_id
to
org_name AS C ON C.parent_id = p.child_id
In plain SQL this would be tricky. You would have multiple options:
Use a stored procedure to collect the correct records (recursively). Then convert those records into a tree using code
Select all the records and convert those into a tree using code
Use more advanced technique as described here (from Optimized SQL for tree structures) and here. Then convert those records into a tree using code
Depending on the way you want to do it in SQL you need to build a Slick query. The concept of Leaky Abstractions is very evident here.
So getting the tree structure requires two steps:
Get the correct (or all records)
Build (using regular code) a tree from those records
Since you are using Slick I don't think it's an option, but another database type might be a better fit for your data model. Check out NoSQL for the differences between the different types.
With a class and table definition looking like this:
case class Group(
id: Long = -1,
id_parent: Long = -1,
label: String = "",
description: String = "")
object Groups extends Table[Group]("GROUPS") {
def id = column[Long]("ID", O.PrimaryKey, O.AutoInc)
def id_parent = column[Long]("ID_PARENT")
def label = column[String]("LABEL")
def description = column[String]("DESC")
def * = id ~ id_parent ~ label ~ design <> (Group, Group.unapply _)
def autoInc = id_parent ~ label ~ design returning id into {
case ((_, _, _), id) => id
}
}
To update a record, I can do this:
def updateGroup(id: Long) = Groups.where(_.id === id)
def updateGroup(g: Group)(implicit session: Session) = updateGroup(g.id).update(g)
But I can't get updates to work using for expressions:
val findGById = for {
id <- Parameters[Long]
g <- Groups; if g.id === id
} yield g
def updateGroupX(g: Group)(implicit session: Session) = findGById(g.id).update(g)
----------------------------------------------------------------------------^
Error: value update is not a member of scala.slick.jdbc.MutatingUnitInvoker[com.exp.Group]
I'm obviously missing something in the documentation.
The update method is supplied by the type UpdateInvoker. An instance of that type can be implicitly created from a Query by the methods productQueryToUpdateInvoker and/or tableQueryToUpdateInvoker (found in the BasicProfile), if they are in scope.
Now the type of your findById method is not a Query but a BasicQueryTemplate[Long, Group]. Looking at the docs, I can find no way from a BasicQueryTemplate (which is a subtype of StatementInvoker) to an UpdateInvoker, neither implicit nor explicit. Thinking about it, that makes kinda sense to me, since I understand a query template (invoker) to be something that has already been "compiled" from an abstract syntax tree (Query) to a prepared statement rather early, before parameterization, whereas an update invoker can only be built from an abstract syntax tree, i.e. a Query object, because it needs to analyze the query and extract its parameters/columns. At least that's the way it appears to work at present.
With that in mind, a possible solution unfolds:
def findGById(id: Long) = for {
g <- Groups; if g.id === id
} yield g
def updateGroupX(g: Group)(implicit session: Session) = findGById(g.id).update(g)
Where findById(id: Long) has the type Query[Groups, Group] which is converted by productQueryToUpdateInvoker to an UpdateInvoker[Group] on which the update method can finally be called.
Hope this helped.
Refer to http://madnessoftechnology.blogspot.ru/2013/01/database-record-updates-with-slick-in.html
I stuck with the updating today, and this blog post helped me much. Also refer to the first comment under the post.