Updating table with enum - scala

Trying to insert the information into DB that looks like this:
(UUID, EnumType)
with following logic:
var t = TestTable.query.map(t=> (t.id, t.enumType)) ++= toAdd.map(idTest, enumTest)))
but compiler throws an error for TestTable.query.map(t=> (t.id, t.enumType)) it's interpriting it as type Iteratable[Nothing], am I missing something?
Test table looks like this:
object TestTable {
val query = TableQuery[TestTable]
}
class TestTable(tag: slick.lifted.Tag) extends Table[TestTable](tag, "test_table") {
val id = column[UUID]("id")
val enumType = column[EnumType]("enumType")
override val * = (id, testType) <> (
(TestTable.apply _).tupled,
TestTable.unapply
)

Suppose you have following data structure:
object Color extends Enumeration {
val Blue = Value("Blue")
val Red = Value("Red")
val Green = Value("Green")
}
case class MyType(id: UUID, color: Color.Value)
Define slick schema as following:
class TestTable(tag: slick.lifted.Tag) extends Table[MyType](tag, "test_table") {
val id = column[UUID]("id")
val color = column[Color.Value]("color")
override val * = (id, color) <> ((MyType.apply _).tupled, MyType.unapply)
}
object TestTable {
lazy val query = TableQuery[TestTable]
}
To map enum to SQL data type slick requires implicit MappedColumnType:
implicit val colorTypeColumnMapper: JdbcType[Color.Value] = MappedColumnType.base[Color.Value, String](
e => e.toString,
s => Color.withName(s)
)
Now you can insert values into DB in this way:
val singleInsertAction = TestTable.query += MyType(UUID.randomUUID(), Color.Blue)
val batchInsertAction = TestTable.query ++= Seq(
MyType(UUID.randomUUID(), Color.Blue),
MyType(UUID.randomUUID(), Color.Red),
MyType(UUID.randomUUID(), Color.Green)
)

Related

how to insert usert defined type in cassandra by using lagom scala framework

I am using Lagom(scala) framework and i could find any way to save scala case class object in cassandra with has complex Type. so how to i insert cassandra UDT in Lagom scala. and can any one explain hoe to use BoundStatement.setUDTValue() method.
I have tried to do by using com.datastax.driver.mapping.annotations.UDT.
but does not work for me. I have also tried com.datastax.driver.core
Session Interface. but again it does not.
case class LeadProperties(
name: String,
label: String,
description: String,
groupName: String,
fieldDataType: String,
options: Seq[OptionalData]
)
object LeadProperties{
implicit val format: Format[LeadProperties] = Json.format[LeadProperties]
}
#UDT(keyspace = "leadpropertieskeyspace", name="optiontabletype")
case class OptionalData(label: String)
object OptionalData {
implicit val format: Format[OptionalData] = Json.format[OptionalData]
}
my query:----
val optiontabletype= """
|CREATE TYPE IF NOT EXISTS optiontabletype(
|value text
|);
""".stripMargin
val createLeadPropertiesTable: String = """
|CREATE TABLE IF NOT EXISTS leadpropertiestable(
|name text Primary Key,
|label text,
|description text,
|groupname text,
|fielddatatype text,
|options List<frozen<optiontabletype>>
);
""".stripMargin
def createLeadProperties(obj: LeadProperties): Future[List[BoundStatement]] = {
val bindCreateLeadProperties: BoundStatement = createLeadProperties.bind()
bindCreateLeadProperties.setString("name", obj.name)
bindCreateLeadProperties.setString("label", obj.label)
bindCreateLeadProperties.setString("description", obj.description)
bindCreateLeadProperties.setString("groupname", obj.groupName)
bindCreateLeadProperties.setString("fielddatatype", obj.fieldDataType)
here is the problem I am not getting any method for cassandra Udt.
Future.successful(List(bindCreateLeadProperties))
}
override def buildHandler(): ReadSideProcessor.ReadSideHandler[PropertiesEvent] = {
readSide.builder[PropertiesEvent]("PropertiesOffset")
.setGlobalPrepare(() => PropertiesRepository.createTable)
.setPrepare(_ => PropertiesRepository.prepareStatements)
.setEventHandler[PropertiesCreated](ese ⇒
PropertiesRepository.createLeadProperties(ese.event.obj))
.build()
}
I was faced with the same issue and solve it following way:
Define type and table:
def createTable(): Future[Done] = {
session.executeCreateTable("CREATE TYPE IF NOT EXISTS optiontabletype(filed1 text, field2 text)")
.flatMap(_ => session.executeCreateTable(
"CREATE TABLE IF NOT EXISTS leadpropertiestable ( " +
"id TEXT, options list<frozen <optiontabletype>>, PRIMARY KEY (id))"
))
}
Call this method in buildHandler() like this:
override def buildHandler(): ReadSideProcessor.ReadSideHandler[FacilityEvent] =
readSide.builder[PropertiesEvent]("PropertiesOffset")
.setPrepare(_ => prepare())
.setGlobalPrepare(() => {
createTable()
})
.setEventHandler[PropertiesCreated](processPropertiesCreated)
.build()
Then in processPropertiesCreated() I used it like:
private val writePromise = Promise[PreparedStatement] // initialized in prepare
private def writeF: Future[PreparedStatement] = writePromise.future
private def processPropertiesCreated(eventElement: EventStreamElement[PropertiesCreated]): Future[List[BoundStatement]] = {
writeF.map { ps =>
val userType = ps.getVariables.getType("options").getTypeArguments.get(0).asInstanceOf[UserType]
val newValue = userType.newValue().setString("filed1", "1").setString("filed2", "2")
val bindWriteTitle = ps.bind()
bindWriteTitle.setString("id", eventElement.event.id)
bindWriteTitle.setList("options", eventElement.event.keys.map(_ => newValue).toList.asJava) // todo need to convert, now only stub
List(bindWriteTitle)
}
}
And read it like this:
def toFacility(r: Row): LeadPropertiesTable = {
LeadPropertiesTable(
id = r.getString(fId),
options = r.getList("options", classOf[UDTValue]).asScala.map(udt => OptiontableType(field1 = udt.getString("field1"), field2 = udt.getString("field2"))
)
}
My prepare() function:
private def prepare(): Future[Done] = {
val f = session.prepare("INSERT INTO leadpropertiestable (id, options) VALUES (?, ?)")
writePromise.completeWith(f)
f.map(_ => Done)
}
This is not a very well written code, but I think will help to proceed work.

Slick - Inserting a row into two tables linked with an auto-incrementing key?

I'm new to Slick and struggling to find a good canonical example for the following.
I'd like to insert a row into two tables. The first table has a primary key which auto-increments. The second table is related to the first via its primary key.
So I'd like to:
Start a transaction
Insert a row into table 1, which generates a key
Insert a row into table 2, with a foreign key generated in the previous step
End transaction (rollback steps 2 & 3 if either fail)
Would appreciate a canonical example for the above logic, and any related suggestions on my definitions below (I'm very new to Slick!). Thanks!
Insert logic for table 1
private def insertAndReturn(entry: Entry) =
entries returning entries.map(_.id)
into ((_, newId) => entry.copy(id = newId))
def insert(entry: Entry): Future[Entry] =
db.run(insertAndReturn(entry) += entry)
(similar for table 2)
Table 1
class EntryTable(tag: Tag) extends Table[Entry](tag, "tblEntry") {
def id = column[EntryId]("entryID", O.PrimaryKey, O.AutoInc)
...
def * = (id, ...).shaped <> (Entry.tupled, Entry.unapply)
}
Table 2
class UsernameChangeTable(tag: Tag) extends Table[UserNameChange](tag, "tblUserNameChange") {
def entryId = column[EntryId]("entryID")
...
def entry = foreignKey("ENTRY_FK", entryId, entryDao.entries)(
_.id, onUpdate = Restrict, onDelete = Cascade
)
I'm using a MySQL database and Slick 3.1.0.
All that you have to do is
val tx =
insertAndReturn(entry).flatMap { id =>
insertUserNameChange(UserNameChange(id, ...))
}.transactionally
db.run(tx)
Note that insertUserNameChange is the function which inserts the UserNameChange instance into the database. It needs the EntryId which you get back from the previous insertion action.
Compose actions using flatMap and use transactionally to run the whole query in a transaction.
Your Slick tables look fine.
Here is a canonical example implementing this functionality
package models
import scala.concurrent.{Future, Await}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.Duration
import slick.backend.DatabasePublisher
import slick.driver.H2Driver.api._
case class Supplier1(id:Int,name:String)
class Suppliers1(tag:Tag) extends Table[Supplier1](tag,"SUPPLIERS") {
def id:Rep[Int] = column[Int]("SUP_ID",O.PrimaryKey,O.AutoInc)
def name:Rep[String] = column[String]("NAME")
def * = (id,name) <>
(Supplier1.tupled,Supplier1.unapply)
}
case class Coffee1(id:Int,name:String,suppId:Int)
class Coffees1(tag:Tag) extends Table[Coffee1](tag,"COFFEES"){
def id:Rep[Int] = column[Int]("C_ID",O.PrimaryKey,O.AutoInc)
def name:Rep[String] = column[String]("COFFEE_NAME")
def suppId:Rep[Int] = column[Int]("SUP_ID")
def * = (id,name,suppId) <> (Coffee1.tupled,Coffee1.unapply)
def supplier = foreignKey("supp_fk", suppId, TableQuery[Suppliers])(_.id)
}
object HelloSlick1 extends App{
val db = Database.forConfig("h2mem1")
val suppliers = TableQuery[Suppliers1]
val coffees = TableQuery[Coffees1]
val setUpF = (suppliers.schema ++ coffees.schema).create
val insertSupplier = suppliers returning suppliers.map(_.id)
//val tx = (insertSupplier += Supplier1(0,"SUPP 1")).flatMap(id=>(coffees += Coffee1(0,"COF",id))).transactionally
val tx = for{
supId <- insertSupplier += Supplier1(0,"SUPP 1")
cId <- coffees += Coffee1(0,"COF",supId)
} yield ()
tx.transactionally
def exec[T](action: DBIO[T]): T =
Await.result(db.run(action), Duration.Inf)
exec(setUpF)
exec(tx)
exec(suppliers.result.map(println))
exec(coffees.result.map(println))
}

Creating table view using slick

How can I create queries for postgresql view using slick 3?
I didn't find an answer in the slick documentation.
The question relates to my another question. I got right answer but I don't know how to implement it using slick.
There is only rudimentary support for views in Slick 3, that doesn't guarantee full compile-time safety and compositionality, the latter especially matters considering most views strongly depend on data in other tables.
You can describe a view as a Table and separate schema manipulation statements, which you must use instead of standard table schema extension methods like create and drop. Here is an example for your registries-n-rows case subject to the REGISTRY and ROWS table are already present in the database:
case class RegRn(id: Int, name: String, count: Long)
trait View{
val viewName = "REG_RN"
val registryTableName = "REGISTRY"
val rowsTableName = "ROWS"
val profile: JdbcProfile
import profile.api._
class RegRns(tag: Tag) extends Table[RegRn](tag, viewName) {
def id = column[Int] ("REGISTRY_ID")
def name = column[String]("NAME", O.SqlType("VARCHAR"))
def count = column[Long] ("CT", O.SqlType("VARCHAR"))
override def * = (id, name, count) <> (RegRn.tupled, RegRn.unapply)
...
}
val regRns = TableQuery[RegRns]
val createViewSchema = sqlu"""CREATE VIEW #$viewName AS
SELECT R.*, COALESCE(N.ct, 0) AS CT
FROM #$registryTableName R
LEFT JOIN (
SELECT REGISTRY_ID, count(*) AS CT
FROM #$rowsTableName
GROUP BY REGISTRY_ID
) N ON R.REGISTRY_ID=N.REGISTRY_ID"""
val dropViewSchema = sqlu"DROP VIEW #$viewName"
...
}
You can now create a view with db.run(createViewSchema), drop it with db.run(dropViewSchema) and of course call MTable.getTables("REG_RN") to expectedly find its tableType is "VIEW". Queries are the same as for other tables, e.g.
db run regRns.result.head. You can even insert values into a view as you do for a normal Slick table if the rules allow (not your case due to COALESCE and the subquery).
As I mentioned everything will become a mess when you want to compose existing Tables to create a view. You will have to always keep their names and definitions in sync, as it is not possible now to write anything that would at least guarantee the shape of the view conforms to combined shape of the underlying tables for example. Well, there is no way apart from ugly ones like this:
trait View{
val profile: JdbcProfile
import profile.api._
val registryTableName = "REGISTRY"
val registryId = "REGISTRY_ID"
val regitsryName = "NAME"
class Registries(tag: Tag) extends Table[Registry](tag, registryTableName) {
def id = column[Int] (registryId)
def name = column[String](regitsryName, O.SqlType("VARCHAR"))
override def * = (id, name) <> (Registry.tupled, Registry.unapply)
...
}
val rowsTableName = "ROWS"
val rowsId = "ROW_ID"
val rowsRow = "ROW"
class Rows(tag: Tag) extends Table[Row](tag, rowsTableName) {
def id = column[String](rowsId, O.SqlType("VARCHAR"))
def rid = column[Int] (registryId)
def r = column[String]("rowsRow", O.SqlType("VARCHAR"))
override def * = (id, rid, r) <> (Row.tupled, Row.unapply)
...
}
val viewName = "REG_RN"
class RegRns(tag: Tag) extends Table[RegRn](tag, viewName) {
def id = column[Int] ("REGISTRY_ID")
def name = column[String]("NAME", O.SqlType("VARCHAR"))
def count = column[Long] ("CT", O.SqlType("VARCHAR"))
override def * = (id, name, count) <> (RegRn.tupled, RegRn.unapply)
...
}
val registries = TableQuery[Registries]
val rows = TableQuery[Rows]
val regRns = TableQuery[RegRns]
val createViewSchema = sqlu"""CREATE VIEW #$viewName AS
SELECT R.*, COALESCE(N.ct, 0) AS CT
FROM #$registryTableName R
LEFT JOIN (
SELECT #$registryId, count(*) AS CT
FROM #$rowsTableName
GROUP BY #$registryId
) N ON R.#$registryId=N.#$registryId"""
val dropViewSchema = sqlu"DROP VIEW #$viewName"
...
}
What about appending the query text after the view preamble:
val yourAwesomeQryComposition : TableQuery = ...
val qryText = yourAwesomeQryComposition.map(reg => (reg.id, ....)).result.statements.head
val createViewSchema = sqlu"""CREATE VIEW #$viewName AS #${qryText}"""

Chisel: Access to Module Parameters from Tester

How does one access the parameters used to construct a Module from inside the Tester that is testing it?
In the test below I am passing the parameters explicitly both to the Module and to the Tester. I would prefer not to have to pass them to the Tester but instead extract them from the module that was also passed in.
Also I am new to scala/chisel so any tips on bad techniques I'm using would be appreciated :).
import Chisel._
import math.pow
class TestA(dataWidth: Int, arrayLength: Int) extends Module {
val dataType = Bits(INPUT, width = dataWidth)
val arrayType = Vec(gen = dataType, n = arrayLength)
val io = new Bundle {
val i_valid = Bool(INPUT)
val i_data = dataType
val i_array = arrayType
val o_valid = Bool(OUTPUT)
val o_data = dataType.flip
val o_array = arrayType.flip
}
io.o_valid := io.i_valid
io.o_data := io.i_data
io.o_array := io.i_array
}
class TestATests(c: TestA, dataWidth: Int, arrayLength: Int) extends Tester(c) {
val maxData = pow(2, dataWidth).toInt
for (t <- 0 until 16) {
val i_valid = rnd.nextInt(2)
val i_data = rnd.nextInt(maxData)
val i_array = List.fill(arrayLength)(rnd.nextInt(maxData))
poke(c.io.i_valid, i_valid)
poke(c.io.i_data, i_data)
(c.io.i_array, i_array).zipped foreach {
(element,value) => poke(element, value)
}
expect(c.io.o_valid, i_valid)
expect(c.io.o_data, i_data)
(c.io.o_array, i_array).zipped foreach {
(element,value) => poke(element, value)
}
step(1)
}
}
object TestAObject {
def main(args: Array[String]): Unit = {
val tutArgs = args.slice(0, args.length)
val dataWidth = 5
val arrayLength = 6
chiselMainTest(tutArgs, () => Module(
new TestA(dataWidth=dataWidth, arrayLength=arrayLength))){
c => new TestATests(c, dataWidth=dataWidth, arrayLength=arrayLength)
}
}
}
If you make the arguments dataWidth and arrayLength members of TestA you can just reference them. In Scala this can be accomplished by inserting val into the argument list:
class TestA(val dataWidth: Int, val arrayLength: Int) extends Module ...
Then you can reference them from the test as members with c.dataWidth or c.arrayLength

Slick code generation for only a single schema

Is there a way to have Slick's code generation generate code for only a single schema? Say, public? I have extensions that create a whole ton of tables (eg postgis, pg_jobman) that make the code that slick generates gigantic.
Use this code with appropriate values and schema name,
object CodeGenerator {
def outputDir :String =""
def pkg:String =""
def schemaList:String = "schema1, schema2"
def url:String = "dburl"
def fileName:String =""
val user = "dbUsername"
val password = "dbPassword"
val slickDriver="scala.slick.driver.PostgresDriver"
val JdbcDriver = "org.postgresql.Driver"
val container = "Tables"
def generate() = {
val driver: JdbcProfile = buildJdbcProfile
val schemas = createSchemaList
var model = createModel(driver,schemas)
val codegen = new SourceCodeGenerator(model){
// customize Scala table name (table class, table values, ...)
override def tableName = dbTableName => dbTableName match {
case _ => dbTableName+"Table"
}
override def code = {
//imports is copied right out of
//scala.slick.model.codegen.AbstractSourceCodeGenerator
val imports = {
"import scala.slick.model.ForeignKeyAction\n" +
(if (tables.exists(_.hlistEnabled)) {
"import scala.slick.collection.heterogenous._\n" +
"import scala.slick.collection.heterogenous.syntax._\n"
} else ""
) +
(if (tables.exists(_.PlainSqlMapper.enabled)) {
"import scala.slick.jdbc.{GetResult => GR}\n" +
"// NOTE: GetResult mappers for plain SQL are only generated for tables where Slick knows how to map the types of all columns.\n"
} else ""
) + "\n\n" //+ tables.map(t => s"implicit val ${t.model.name.table}Format = Json.format[${t.model.name.table}]").mkString("\n")+"\n\n"
}
val bySchema = tables.groupBy(t => {
t.model.name.schema
})
val schemaFor = (schema: Option[String]) => {
bySchema(schema).sortBy(_.model.name.table).map(
_.code.mkString("\n")
).mkString("\n\n")
}
}
val joins = tables.flatMap( _.foreignKeys.map{ foreignKey =>
import foreignKey._
val fkt = referencingTable.TableClass.name
val pkt = referencedTable.TableClass.name
val columns = referencingColumns.map(_.name) zip
referencedColumns.map(_.name)
s"implicit def autojoin${fkt + name.toString} = (left:${fkt} ,right:${pkt}) => " +
columns.map{
case (lcol,rcol) =>
"left."+lcol + " === " + "right."+rcol
}.mkString(" && ")
})
override def entityName = dbTableName => dbTableName match {
case _ => dbTableName
}
override def Table = new Table(_) {
table =>
// customize table value (TableQuery) name (uses tableName as a basis)
override def TableValue = new TableValue {
override def rawName = super.rawName.uncapitalize
}
// override generator responsible for columns
override def Column = new Column(_){
// customize Scala column names
override def rawName = (table.model.name.table,this.model.name) match {
case _ => super.rawName
}
}
}
}
println(outputDir+"\\"+fileName)
(new File(outputDir)).mkdirs()
val fw = new FileWriter(outputDir+File.separator+fileName)
fw.write(codegen.packageCode(slickDriver, pkg, container))
fw.close()
}
def createModel(driver: JdbcProfile, schemas:Set[Option[String]]): Model = {
driver.simple.Database
.forURL(url, user = user, password = password, driver = JdbcDriver)
.withSession { implicit session =>
val filteredTables = driver.defaultTables.filter(
(t: MTable) => schemas.contains(t.name.schema)
)
PostgresDriver.createModel(Some(filteredTables))
}
}
def createSchemaList: Set[Option[String]] = {
schemaList.split(",").map({
case "" => None
case (name: String) => Some(name)
}).toSet
}
def buildJdbcProfile: JdbcProfile = {
val module = currentMirror.staticModule(slickDriver)
val reflectedModule = currentMirror.reflectModule(module)
val driver = reflectedModule.instance.asInstanceOf[JdbcProfile]
driver
}
}
I encountered the same problem and I found this question. The answer by S.Karthik sent me in the right direction. However, the code in the answer is slightly outdated. And I think a bit over-complicated. So I crafted my own solution:
import slick.codegen.SourceCodeGenerator
import slick.driver.JdbcProfile
import slick.model.Model
import scala.concurrent.duration.Duration
import scala.concurrent.{Await, ExecutionContext}
val slickDriver = "slick.driver.PostgresDriver"
val jdbcDriver = "org.postgresql.Driver"
val url = "jdbc:postgresql://localhost:5432/mydb"
val outputFolder = "/path/to/src/test/scala"
val pkg = "com.mycompany"
val user = "user"
val password = "password"
object MySourceCodeGenerator {
def run(slickDriver: String, jdbcDriver: String, url: String, outputDir: String,
pkg: String, user: Option[String], password: Option[String]): Unit = {
val driver: JdbcProfile =
Class.forName(slickDriver + "$").getField("MODULE$").get(null).asInstanceOf[JdbcProfile]
val dbFactory = driver.api.Database
val db = dbFactory.forURL(url, driver = jdbcDriver, user = user.orNull,
password = password.orNull, keepAliveConnection = true)
try {
// **1**
val allSchemas = Await.result(db.run(
driver.createModel(None, ignoreInvalidDefaults = false)(ExecutionContext.global).withPinnedSession), Duration.Inf)
// **2**
val publicSchema = new Model(allSchemas.tables.filter(_.name.schema.isEmpty), allSchemas.options)
// **3**
new SourceCodeGenerator(publicSchema).writeToFile(slickDriver, outputDir, pkg)
} finally db.close
}
}
MySourceCodeGenerator.run(slickDriver, jdbcDriver, url, outputFolder, pkg, Some(user), Some(password))
I'll explain what's going on here:
I copied the run function from the SourceCodeGenerator class that's in the slick-codegen library. (I used version slick-codegen_2.10-3.1.1.)
// **1**: In the origninal code, the generated Model was referenced in a val called m. I renamed that to allSchemas.
// **2**: I created a new Model (publicSchema), using the options from the original model, and using a filtered version of the tables set from the original model. It turns out tables from the public schema don't get a schema name in the model. Hence the isEmpty. Should you need tables from one or more other schemas, you can easily create a different filter expression.
// **3**: I create a SourceCodeGenerator with the created publicSchema model.
Of course, it would even be better if the Slick codegenerator could incorporate an option to select one or more schemas.