I have two tables that are identical, each in a different database. Also, the tables have different names.
I have hardcoded the name of the table in the class BankDB:
class BankDB(tag: Tag) extends Table[Bank](tag, "banks1") {
def sk = column[Int]("sk", O.PrimaryKey)
def name = column[String]("name")
// other columns
What I need is, depending on the database name, to set the name of the table, like so:
val tableName = if (dbName == "DB1") "banks1" else "banks2"
And then use tableName in TableQuery to have Slick point to the correct table:
val db = Database.forConfig(dbName)
try {
val banks = TableQuery[BankDB](tableName) // <== this doesn't work
val future = db.run(banks.filter(_.sk === sk).result)
val result = Await.result(future, Duration.Inf)
result
} finally db.close
Is this possible? or I need to define to classes, one for each database?
class BankDB(tag: Tag,tableName: String) extends Table[Bank](tag, tableName)
def getTableQuery(tableName: String) =
TableQuery[BankDB]((t: slick.lifted.Tag) => new BankDB(t, tableName))
Related
For a scala project, I'm using play-slick with play-slick-evolutions both version 5.0.0 and I generate my db classes with slick-codegen version 3.3.3.
I have a table with a primary key column and some columns with default values. I want to insert one row without mentioning the primary key column nor any columns with default values. Ideally, this action should return the new primary key of the created row.
My problem is that the generated code from slick-codegen seems to only allow to insert full rows because it uses an own case class for the rows. This is how the generated code looks (without the comments):
case class SalesOrderRow(idSalesOrder: Int, fkCustomer: Int, createdAt: java.sql.Timestamp, createdBy: Option[String] = None)
implicit def GetResultSalesOrderRow(implicit e0: GR[Int], e1: GR[java.sql.Timestamp], e2: GR[Option[String]]): GR[SalesOrderRow] = GR{
prs => import prs._
SalesOrderRow.tupled((<<[Int], <<[Int], <<[java.sql.Timestamp], <<?[String]))
}
class SalesOrder(_tableTag: Tag) extends profile.api.Table[SalesOrderRow](_tableTag, Some("test"), "sales_order") {
def * = (idSalesOrder, fkCustomer, createdAt, createdBy) <> (SalesOrderRow.tupled, SalesOrderRow.unapply)
def ? = ((Rep.Some(idSalesOrder), Rep.Some(fkCustomer), Rep.Some(createdAt), createdBy)).shaped.<>({r=>import r._; _1.map(_=> SalesOrderRow.tupled((_1.get, _2.get, _3.get, _4)))}, (_:Any) => throw new Exception("Inserting into ? projection not supported."))
val idSalesOrder: Rep[Int] = column[Int]("id_sales_order", O.AutoInc, O.PrimaryKey)
val fkCustomer: Rep[Int] = column[Int]("fk_customer")
val createdAt: Rep[java.sql.Timestamp] = column[java.sql.Timestamp]("created_at")
val createdBy: Rep[Option[String]] = column[Option[String]]("created_by", O.Length(20,varying=true), O.Default(None))
}
lazy val SalesOrder = new TableQuery(tag => new SalesOrder(tag))
With this generated code I could now insert a row with mentioning the full column:
val insertActionsNotSoNice =
DBIO.seq(
salesOrders += Tables.SalesOrderRow(0, 3, new Timestamp(System.currentTimeMillis()), Some("这个不好"))
)
But I want to omit the primary key at the beginning and the timestamp parameter that has a default value. But can't do something like this.
val insertActionsNotCompiling =
DBIO.seq(
salesOrders.map(so => (so.fkCustomer, so.createdBy) += (3, Some("这个好")))
)
I found many examples with something like the latter approach in the slick documentation and in the web but it was always the case that their classes used for the database did have a tuple instead of an own row class. Like...
class Coffees(tag: Tag) extends Table[(String, Int, Double, Int, Int)](tag, "COFFEES")
instead of
class Coffees(_tableTag: Tag) extends profile.api.Table[CoffeesRow](_tableTag, Some("test"), "coffee")
Do I have to throw the slick-codegen out of my project and write all the classes myself to fit my needs or is my composition of libraries with play-slick wrong? Or is there a simple trick to omit columns when inserting what isn't documented?
After some days of trying several things, I found one solution.
You can map over a query to select only the columns you want to instert and then use the returning if you want to return the id of the inserted row.
import dao.Tables.profile.api._
// ...
val salesOrders = TableQuery[SalesOrder]
val insertStatement = salesOrders.map(so => (so.fkCustomer, so.createdBy)) returning salesOrders.map(_.idSalesOrder) into ((_, id) => id)
Then you can write something like:
val newSalesOrderIdFuture = db.run(insertStatement += (5, Some("有效")))
So I omitted the primary key column (id_sales_order) and a timestamp column that defaults to now() in the database (created_at) in my insert statement.
class Employee(tag: Tag) extends Table[table_types.user](tag, "EMPLOYEE") {
def employeeID = column[Int]("EMPLOYEE_ID")
def empName = column[String]("NAME")
def startDate = column[String]("START_DATE")
def * = (employeeID, empName, startDate)
}
object employeeHandle {
def insert(emp:Employee):Future[Any] = {
val dao = new SlickPostgresDAO
val db = dao.db
val insertdb = DBIO.seq(employee += (emp))
db.run(insertdb)
}
}
Insert into database a million employee records
object Hello extends App {
val employees = List[*1 million employee list*]
for(employee<-employees) {
employeeHandle.insert(employee)
*Code to connect to rest api to confirm entry*
}
}
However when I run the above code I soon run out of connections to Postgres. How can I do it in parallel (in a non blocking way) but at the same time ensure I don't run out of connections to postgres.
I think you don't need to do it in parallel; I don't see how it can solve it. Instead you could simply create connection before you start that loop and pass it to employeeHandle.insert(db, employee).
Something like (I don't know scala):
object Hello extends App {
val dao = new SlickPostgresDAO
val db = dao.db
val employees = List[*1 million employee list*]
for(employee<-employees) {
employeeHandle.insert(db, employee)
*Code to connect to rest api to confirm entry*
}
}
Almost all examples of slick insert I have come across uses blocking to fullfil the results. It would be nice to have one that doesn't.
My take on it:
object Hello extends App {
val employees = List[*1 million employee list*]
val groupedList = employees.grouped(10).toList
insertTests()
def insertTests(l: List[List[Employee]] = groupedList): Unit = {
val ins = l.head
val futures = ins.map { no => employeeHandle.insert(employee)}
val seq = Future.sequence(futures)
Await.result(seq, Duration.Inf)
if(l.nonEmpty) insertTests(l.tail)
}
}
Also the connection parameter in insert handle should be outside
object employeeHandle {
val dao = new SlickPostgresDAO
val db = dao.db
def insert(emp:Employee):Future[Any] = {
val insertdb = DBIO.seq(employee += (emp))
db.run(insertdb)
}
}
I'm new to Slick and struggling to find a good canonical example for the following.
I'd like to insert a row into two tables. The first table has a primary key which auto-increments. The second table is related to the first via its primary key.
So I'd like to:
Start a transaction
Insert a row into table 1, which generates a key
Insert a row into table 2, with a foreign key generated in the previous step
End transaction (rollback steps 2 & 3 if either fail)
Would appreciate a canonical example for the above logic, and any related suggestions on my definitions below (I'm very new to Slick!). Thanks!
Insert logic for table 1
private def insertAndReturn(entry: Entry) =
entries returning entries.map(_.id)
into ((_, newId) => entry.copy(id = newId))
def insert(entry: Entry): Future[Entry] =
db.run(insertAndReturn(entry) += entry)
(similar for table 2)
Table 1
class EntryTable(tag: Tag) extends Table[Entry](tag, "tblEntry") {
def id = column[EntryId]("entryID", O.PrimaryKey, O.AutoInc)
...
def * = (id, ...).shaped <> (Entry.tupled, Entry.unapply)
}
Table 2
class UsernameChangeTable(tag: Tag) extends Table[UserNameChange](tag, "tblUserNameChange") {
def entryId = column[EntryId]("entryID")
...
def entry = foreignKey("ENTRY_FK", entryId, entryDao.entries)(
_.id, onUpdate = Restrict, onDelete = Cascade
)
I'm using a MySQL database and Slick 3.1.0.
All that you have to do is
val tx =
insertAndReturn(entry).flatMap { id =>
insertUserNameChange(UserNameChange(id, ...))
}.transactionally
db.run(tx)
Note that insertUserNameChange is the function which inserts the UserNameChange instance into the database. It needs the EntryId which you get back from the previous insertion action.
Compose actions using flatMap and use transactionally to run the whole query in a transaction.
Your Slick tables look fine.
Here is a canonical example implementing this functionality
package models
import scala.concurrent.{Future, Await}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.Duration
import slick.backend.DatabasePublisher
import slick.driver.H2Driver.api._
case class Supplier1(id:Int,name:String)
class Suppliers1(tag:Tag) extends Table[Supplier1](tag,"SUPPLIERS") {
def id:Rep[Int] = column[Int]("SUP_ID",O.PrimaryKey,O.AutoInc)
def name:Rep[String] = column[String]("NAME")
def * = (id,name) <>
(Supplier1.tupled,Supplier1.unapply)
}
case class Coffee1(id:Int,name:String,suppId:Int)
class Coffees1(tag:Tag) extends Table[Coffee1](tag,"COFFEES"){
def id:Rep[Int] = column[Int]("C_ID",O.PrimaryKey,O.AutoInc)
def name:Rep[String] = column[String]("COFFEE_NAME")
def suppId:Rep[Int] = column[Int]("SUP_ID")
def * = (id,name,suppId) <> (Coffee1.tupled,Coffee1.unapply)
def supplier = foreignKey("supp_fk", suppId, TableQuery[Suppliers])(_.id)
}
object HelloSlick1 extends App{
val db = Database.forConfig("h2mem1")
val suppliers = TableQuery[Suppliers1]
val coffees = TableQuery[Coffees1]
val setUpF = (suppliers.schema ++ coffees.schema).create
val insertSupplier = suppliers returning suppliers.map(_.id)
//val tx = (insertSupplier += Supplier1(0,"SUPP 1")).flatMap(id=>(coffees += Coffee1(0,"COF",id))).transactionally
val tx = for{
supId <- insertSupplier += Supplier1(0,"SUPP 1")
cId <- coffees += Coffee1(0,"COF",supId)
} yield ()
tx.transactionally
def exec[T](action: DBIO[T]): T =
Await.result(db.run(action), Duration.Inf)
exec(setUpF)
exec(tx)
exec(suppliers.result.map(println))
exec(coffees.result.map(println))
}
How can I create queries for postgresql view using slick 3?
I didn't find an answer in the slick documentation.
The question relates to my another question. I got right answer but I don't know how to implement it using slick.
There is only rudimentary support for views in Slick 3, that doesn't guarantee full compile-time safety and compositionality, the latter especially matters considering most views strongly depend on data in other tables.
You can describe a view as a Table and separate schema manipulation statements, which you must use instead of standard table schema extension methods like create and drop. Here is an example for your registries-n-rows case subject to the REGISTRY and ROWS table are already present in the database:
case class RegRn(id: Int, name: String, count: Long)
trait View{
val viewName = "REG_RN"
val registryTableName = "REGISTRY"
val rowsTableName = "ROWS"
val profile: JdbcProfile
import profile.api._
class RegRns(tag: Tag) extends Table[RegRn](tag, viewName) {
def id = column[Int] ("REGISTRY_ID")
def name = column[String]("NAME", O.SqlType("VARCHAR"))
def count = column[Long] ("CT", O.SqlType("VARCHAR"))
override def * = (id, name, count) <> (RegRn.tupled, RegRn.unapply)
...
}
val regRns = TableQuery[RegRns]
val createViewSchema = sqlu"""CREATE VIEW #$viewName AS
SELECT R.*, COALESCE(N.ct, 0) AS CT
FROM #$registryTableName R
LEFT JOIN (
SELECT REGISTRY_ID, count(*) AS CT
FROM #$rowsTableName
GROUP BY REGISTRY_ID
) N ON R.REGISTRY_ID=N.REGISTRY_ID"""
val dropViewSchema = sqlu"DROP VIEW #$viewName"
...
}
You can now create a view with db.run(createViewSchema), drop it with db.run(dropViewSchema) and of course call MTable.getTables("REG_RN") to expectedly find its tableType is "VIEW". Queries are the same as for other tables, e.g.
db run regRns.result.head. You can even insert values into a view as you do for a normal Slick table if the rules allow (not your case due to COALESCE and the subquery).
As I mentioned everything will become a mess when you want to compose existing Tables to create a view. You will have to always keep their names and definitions in sync, as it is not possible now to write anything that would at least guarantee the shape of the view conforms to combined shape of the underlying tables for example. Well, there is no way apart from ugly ones like this:
trait View{
val profile: JdbcProfile
import profile.api._
val registryTableName = "REGISTRY"
val registryId = "REGISTRY_ID"
val regitsryName = "NAME"
class Registries(tag: Tag) extends Table[Registry](tag, registryTableName) {
def id = column[Int] (registryId)
def name = column[String](regitsryName, O.SqlType("VARCHAR"))
override def * = (id, name) <> (Registry.tupled, Registry.unapply)
...
}
val rowsTableName = "ROWS"
val rowsId = "ROW_ID"
val rowsRow = "ROW"
class Rows(tag: Tag) extends Table[Row](tag, rowsTableName) {
def id = column[String](rowsId, O.SqlType("VARCHAR"))
def rid = column[Int] (registryId)
def r = column[String]("rowsRow", O.SqlType("VARCHAR"))
override def * = (id, rid, r) <> (Row.tupled, Row.unapply)
...
}
val viewName = "REG_RN"
class RegRns(tag: Tag) extends Table[RegRn](tag, viewName) {
def id = column[Int] ("REGISTRY_ID")
def name = column[String]("NAME", O.SqlType("VARCHAR"))
def count = column[Long] ("CT", O.SqlType("VARCHAR"))
override def * = (id, name, count) <> (RegRn.tupled, RegRn.unapply)
...
}
val registries = TableQuery[Registries]
val rows = TableQuery[Rows]
val regRns = TableQuery[RegRns]
val createViewSchema = sqlu"""CREATE VIEW #$viewName AS
SELECT R.*, COALESCE(N.ct, 0) AS CT
FROM #$registryTableName R
LEFT JOIN (
SELECT #$registryId, count(*) AS CT
FROM #$rowsTableName
GROUP BY #$registryId
) N ON R.#$registryId=N.#$registryId"""
val dropViewSchema = sqlu"DROP VIEW #$viewName"
...
}
What about appending the query text after the view preamble:
val yourAwesomeQryComposition : TableQuery = ...
val qryText = yourAwesomeQryComposition.map(reg => (reg.id, ....)).result.statements.head
val createViewSchema = sqlu"""CREATE VIEW #$viewName AS #${qryText}"""
I have methods in my Play app that query database tables with over hundred columns. I can't define case class for each such query, because it would be just ridiculously big and would have to be changed with each alter of the table on the database.
I'm using this approach, where result of the query looks like this:
Map(columnName1 -> columnVal1, columnName2 -> columnVal2, ...)
Example of the code:
implicit val getListStringResult = GetResult[List[Any]] (
r => (1 to r.numColumns).map(_ => r.nextObject).toList
)
def getSomething(): Map[String, Any] = DB.withSession {
val columns = MTable.getTables(None, None, None, None).list.filter(_.name.name == "myTable").head.getColumns.list.map(_.column)
val result = sql"""SELECT * FROM myTable LIMIT 1""".as[List[Any]].firstOption.map(columns zip _ toMap).get
}
This is not a problem when query only runs on a single database and single table. I need to be able to use multiple tables and databases in my query like this:
def getSomething(): Map[String, Any] = DB.withSession {
//The line below is no longer valid because of multiple tables/databases
val columns = MTable.getTables(None, None, None, None).list.filter(_.name.name == "table1").head.getColumns.list.map(_.column)
val result = sql"""
SELECT *
FROM db1.table1
LEFT JOIN db2.table2 ON db2.table2.col1 = db1.table1.col1
LIMIT 1
""".as[List[Any]].firstOption.map(columns zip _ toMap).get
}
The same approach can no longer be used to retrieve column names. This problem doesn't exist when using something like PHP PDO or Java JDBCTemplate - these retrieve column names without any extra effort needed.
My question is: how do I achieve this with Slick?
import scala.slick.jdbc.{GetResult,PositionedResult}
object ResultMap extends GetResult[Map[String,Any]] {
def apply(pr: PositionedResult) = {
val rs = pr.rs // <- jdbc result set
val md = rs.getMetaData();
val res = (1 to pr.numColumns).map{ i=> md.getColumnName(i) -> rs.getObject(i) }.toMap
pr.nextRow // <- use Slick's advance method to avoid endless loop
res
}
}
val result = sql"select * from ...".as(ResultMap).firstOption
Another variant that produces map with not null columns (keys in lowercase):
private implicit val getMap = GetResult[Map[String, Any]](r => {
val metadata = r.rs.getMetaData
(1 to r.numColumns).flatMap(i => {
val columnName = metadata.getColumnName(i).toLowerCase
val columnValue = r.nextObjectOption
columnValue.map(columnName -> _)
}).toMap
})