Related
My use case has case classes something like
case class Address(name:String,pincode:String){
override def toString =name +"=" +pincode
}
case class Department(name:String){
override def toString =name
}
case class emp(address:Address,department:Department)
I want to create a DSL like below.Can anyone share the links about how to create a DSL and any suggestions to achieve the below.
emp.withAddress("abc","12222").withDepartment("HR")
Update:
Actual use case class may have more fields close to 20. I want to avoid redudancy of code
I created a DSL using reflection so that we don't need to add every field to it.
Disclamer: This DSL is extremely weakly typed and I did it just for fun. I don't really think this is a good approach in Scala.
scala> create an Employee where "homeAddress" is Address("a", "b") and "department" is Department("c") and that_s it
res0: Employee = Employee(a=b,null,c)
scala> create an Employee where "workAddress" is Address("w", "x") and "homeAddress" is Address("y", "z") and that_s it
res1: Employee = Employee(y=z,w=x,null)
scala> create a Customer where "address" is Address("a", "b") and "age" is 900 and that_s it
res0: Customer = Customer(a=b,900)
The last example is the equivalent of writing:
create.a(Customer).where("address").is(Address("a", "b")).and("age").is(900).and(that_s).it
A way of writing DSLs in Scala and avoid parentheses and the dot is by following this pattern:
object.method(parameter).method(parameter)...
Here is the source:
// DSL
object create {
def an(t: Employee.type) = new ModelDSL(Employee(null, null, null))
def a(t: Customer.type) = new ModelDSL(Customer(null, 0))
}
object that_s
class ModelDSL[T](model: T) {
def where(field: String): ValueDSL[ModelDSL2[T], Any] = new ValueDSL(value => {
val f = model.getClass.getDeclaredField(field)
f.setAccessible(true)
f.set(model, value)
new ModelDSL2[T](model)
})
def and(t: that_s.type) = new { def it = model }
}
class ModelDSL2[T](model: T) {
def and(field: String) = new ModelDSL(model).where(field)
def and(t: that_s.type) = new { def it = model }
}
class ValueDSL[T, V](callback: V => T) {
def is(value: V): T = callback(value)
}
// Models
case class Employee(homeAddress: Address, workAddress: Address, department: Department)
case class Customer(address: Address, age: Int)
case class Address(name: String, pincode: String) {
override def toString = name + "=" + pincode
}
case class Department(name: String) {
override def toString = name
}
I really don't think you need the builder pattern in Scala. Just give your case class reasonable defaults and use the copy method.
i.e.:
employee.copy(address = Address("abc","12222"),
department = Department("HR"))
You could also use an immutable builder:
case class EmployeeBuilder(address:Address = Address("", ""),department:Department = Department("")) {
def build = emp(address, department)
def withAddress(address: Address) = copy(address = address)
def withDepartment(department: Department) = copy(department = department)
}
object EmployeeBuilder {
def withAddress(address: Address) = EmployeeBuilder().copy(address = address)
def withDepartment(department: Department) = EmployeeBuilder().copy(department = department)
}
You could do
object emp {
def builder = new Builder(None, None)
case class Builder(address: Option[Address], department: Option[Department]) {
def withDepartment(name:String) = {
val dept = Department(name)
this.copy(department = Some(dept))
}
def withAddress(name:String, pincode:String) = {
val addr = Address(name, pincode)
this.copy(address = Some(addr))
}
def build = (address, department) match {
case (Some(a), Some(d)) => new emp(a, d)
case (None, _) => throw new IllegalStateException("Address not provided")
case _ => throw new IllegalStateException("Department not provided")
}
}
}
and use it as emp.builder.withAddress("abc","12222").withDepartment("HR").build().
You don't need optional fields, copy, or the builder pattern (exactly), if you are willing to have the build always take the arguments in a particular order:
case class emp(address:Address,department:Department, id: Long)
object emp {
def withAddress(name: String, pincode: String): WithDepartment =
new WithDepartment(Address(name, pincode))
final class WithDepartment(private val address: Address)
extends AnyVal {
def withDepartment(name: String): WithId =
new WithId(address, Department(name))
}
final class WithId(address: Address, department: Department) {
def withId(id: Long): emp = emp(address, department, id)
}
}
emp.withAddress("abc","12222").withDepartment("HR").withId(1)
The idea here is that each emp parameter gets its own class which provides a method to get you to the next class, until the final one gives you an emp object. It's like currying but at the type level. As you can see I've added an extra parameter just as an example of how to extend the pattern past the first two parameters.
The nice thing about this approach is that, even if you're part-way through the build, the type you have so far will guide you to the next step. So if you have a WithDepartment so far, you know that the next argument you need to supply is a department name.
If you want to avoid modifying the origin classes you can use implicit class, e.g.
implicit class EmpExtensions(emp: emp) {
def withAddress(name: String, pincode: String) {
//code omitted
}
// code omitted
}
then import EmpExtensions wherever you need these methods
I can't set up this generic trait whose parametrized forms can be consumed by a common class/object/method. I have tried different +|-|_ combinations :-)
Update: The first comment below shows that this can work if the Wrapper is also parametrized. Can a non-parametrized Wrapper do the job? Can an object Wrapper do the job? Can some magic combination of +|-|_ and all that give me the same desired result with a non-parametrized Wrapper or object?
case class OldStuff(name: String)
case class NewStuff(id: Int)
trait Poster[T] {
def translate(i: Int):T
}
class NewPoster extends Poster[NewStuff] {
def translate(i: Int):NewStuff = new NewStuff(3)
}
class OldPoster extends Poster[OldStuff] {
def translate(i: Int):OldStuff = new OldStuff("A" * 3)
}
val old = new OldPoster()
// so far so good
class Wrapper{
var poster: Poster[_] = null
def setter(p: Poster[_]) = {poster = p }
def prepare_input[A]( ) = {
val i: Int = 5
println(poster.translate(i))
}
}
val w= new Wrapper()
val old = new OldPoster()
w.setter(old)
scala> w.setter(old)
<console>:58: error: type mismatch;
found : OldPoster
required: Poster[_]
w.setter(old)
First, I don't see such error with Scala 2.11.
Then, would be better to avoid erasure by Poster[_]:
class Wrapper[T] {
var poster: Poster[T] = null
def setter(p: Poster[T]) = {poster = p }
def prepare_input() = {
val i: Int = 5
println(poster.translate(i))
}
}
val w= new Wrapper[OldStuff]()
val old = new OldPoster()
w.setter(old)
Finally, not using mutability would make the code more predictable against concurrency.
class Wrapper[T](poster: Poster[T]) {
def prepare_input() = {
val i: Int = 5
println(poster.translate(i))
}
}
val w = new Wrapper(new OldPoster())
I'm new to Slick. I'm creating a test suite for a Java application with Scala, ScalaTest and Slick. I'm using slick to prepare data before the test and to do assertions on the data after the test. The database used has some tables with more than 22 columns. I use slick-codegen to generate my schema code.
For tables with more than 22 columns, slick-codegen does not generate a case class, but a HList-based custom type and a companion ‘constructor’ method. As I understand it, this is because the limitation that tuples and case classes can only have 22 fields. The way the code is generated, the fields of a Row-object can only be accessed by index.
I have a couple of questions about this:
For what I understand, the 22 fields restriction for case classes is already fixed in Scala 2.11, right?
If that's the case, would it be possible to customize slick-codegen to generate case classes for all tables? I looked into this: I managed to set override def hlistEnabled = false in an overridden SourceCodeGenerator. But this results in Cannot generate tuple for > 22 columns, please set hlistEnable=true or override compound. So I don’t get the point of being able to disbale HList. May be the catch is in the ‘or override compound’ part, but I don't understand what that means.
Searching the internet on slick and 22 columns, I came across some solutions based on nested tuples. Would it be possible to customize the codegen to use this approach?
If generating code with case classes with > 22 fields is not a viable option, I think it would be possible to generate an ordinary class, which has an ‘accessor’ function for each column, thus providing a ‘mapping’ from index-based access to name-based access. I’d be happy to implement the generation for this myself, but I think I need some pointers where to start. I think it should be able to override the standard codegen for this. I already use an overridden SourceCodeGenerator for some custom data types. But apart from this use case, the documentation of the code generator does not help me that much.
I would really appreciate some help here. Thanks in advance!
I ended up further customizing slick-codegen. First, I'll answer my own questions, then I'll post my solution.
Answers to questions
The 22 arity limit might be lifted for case classes, it is not for tuples. And slick-codegen also generates some tuples, which I was not fully aware of when I asked the question.
Not relevant, see answer 1. (This might become relevant if the 22 arity limit gets lifted for tuples as well.)
I chose not to investigate this further, so this question remains unanswered for now.
This is the approach I took, eventually.
Solution: the generated code
So, I ended up generating "ordinary" classes for tables with more than 22 columns. Let me give an example of what I generate now. (Generator code follows below.) (This example has less than 22 columns, for brevity and readability reasons.)
case class BigAssTableRow(val id: Long, val name: String, val age: Option[Int] = None)
type BigAssTableRowList = HCons[Long,HCons[String,HCons[Option[Int]]], HNil]
object BigAssTableRow {
def apply(hList: BigAssTableRowList) = new BigAssTableRow(hlist.head, hList.tail.head, hList.tail.tail.head)
def unapply(row: BigAssTableRow) = Some(row.id :: row.name :: row.age)
}
implicit def GetResultBoekingenRow(implicit e0: GR[Long], e1: GR[String], e2: GR[Optional[Int]]) = GR{
prs => import prs._
BigAssTableRow.apply(<<[Long] :: <<[String] :: <<?[Int] :: HNil)
}
class BigAssTable(_tableTag: Tag) extends Table[BigAssTableRow](_tableTag, "big_ass") {
def * = id :: name :: age :: :: HNil <> (BigAssTableRow.apply, BigAssTableRow.unapply)
val id: Rep[Long] = column[Long]("id", O.PrimaryKey)
val name: Rep[String] = column[String]("name", O.Length(255,varying=true))
val age: Rep[Option[Int]] = column[Option[Int]]("age", O.Default(None))
}
lazy val BigAssTable = new TableQuery(tag => new BigAssTable(tag))
The hardest part was finding out how the * mapping works in Slick. There's not much documentation, but I found this Stackoverflow answer rather enlightening.
I created the BigAssTableRow object to make the use of HList transparent for the client code. Note that the apply function in the object overloads the apply from the case class. So I can still create entities by calling BigAssTableRow(id: 1L, name: "Foo"), while the * projection can still use the apply function that takes an HList.
So, I can now do things like this:
// I left out the driver import as well as the scala.concurrent imports
// for the Execution context.
val collection = TableQuery[BigAssTable]
val row = BigAssTableRow(id: 1L, name: "Qwerty") // Note that I leave out the optional age
Await.result(db.run(collection += row), Duration.Inf)
Await.result(db.run(collection.filter(_.id === 1L).result), Duration.Inf)
For this code it's totally transparent wether tuples or HLists are used under the hood.
Solution: how this is generated
I'll just post my entire generator code here. It's not perfect; please let me know if you have suggestions for improvement! Huge parts are just copied from the slick.codegen.AbstractSourceCodeGenerator and related classes and then slightly changed. There are also some things that are not directly related to this question, such as the addition of the java.time.* data types and the filtering of specific tables. I left them in, because they might be of use. Also note that this example is for a Postgres database.
import slick.codegen.SourceCodeGenerator
import slick.driver.{JdbcProfile, PostgresDriver}
import slick.jdbc.meta.MTable
import slick.model.Column
import scala.concurrent.Await
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.Duration
object MySlickCodeGenerator {
val slickDriver = "slick.driver.PostgresDriver"
val jdbcDriver = "org.postgresql.Driver"
val url = "jdbc:postgresql://localhost:5432/dbname"
val outputFolder = "/path/to/project/src/test/scala"
val pkg = "my.package"
val user = "user"
val password = "password"
val driver: JdbcProfile = Class.forName(slickDriver + "$").getField("MODULE$").get(null).asInstanceOf[JdbcProfile]
val dbFactory = driver.api.Database
val db = dbFactory.forURL(url, driver = jdbcDriver, user = user, password = password, keepAliveConnection = true)
// The schema is generated using Liquibase, which creates these tables that I don't want to use
def excludedTables = Array("databasechangelog", "databasechangeloglock")
def tableFilter(table: MTable): Boolean = {
!excludedTables.contains(table.name.name) && schemaFilter(table.name.schema)
}
// There's also an 'audit' schema in the database, I don't want to use that one
def schemaFilter(schema: Option[String]): Boolean = {
schema match {
case Some("public") => true
case None => true
case _ => false
}
}
// Fetch data model
val modelAction = PostgresDriver.defaultTables
.map(_.filter(tableFilter))
.flatMap(PostgresDriver.createModelBuilder(_, ignoreInvalidDefaults = false).buildModel)
val modelFuture = db.run(modelAction)
// customize code generator
val codegenFuture = modelFuture.map(model => new SourceCodeGenerator(model) {
// add custom import for added data types
override def code = "import my.package.Java8DateTypes._" + "\n" + super.code
override def Table = new Table(_) {
table =>
// Use different factory and extractor functions for tables with > 22 columns
override def factory = if(columns.size == 1) TableClass.elementType else if(columns.size <= 22) s"${TableClass.elementType}.tupled" else s"${EntityType.name}.apply"
override def extractor = if(columns.size <= 22) s"${TableClass.elementType}.unapply" else s"${EntityType.name}.unapply"
override def EntityType = new EntityTypeDef {
override def code = {
val args = columns.map(c =>
c.default.map( v =>
s"${c.name}: ${c.exposedType} = $v"
).getOrElse(
s"${c.name}: ${c.exposedType}"
)
)
val callArgs = columns.map(c => s"${c.name}")
val types = columns.map(c => c.exposedType)
if(classEnabled){
val prns = (parents.take(1).map(" extends "+_) ++ parents.drop(1).map(" with "+_)).mkString("")
s"""case class $name(${args.mkString(", ")})$prns"""
} else {
s"""
/** Constructor for $name providing default values if available in the database schema. */
case class $name(${args.map(arg => {s"val $arg"}).mkString(", ")})
type ${name}List = ${compoundType(types)}
object $name {
def apply(hList: ${name}List): $name = new $name(${callArgs.zipWithIndex.map(pair => s"hList${tails(pair._2)}.head").mkString(", ")})
def unapply(row: $name) = Some(${compoundValue(callArgs.map(a => s"row.$a"))})
}
""".trim
}
}
}
override def PlainSqlMapper = new PlainSqlMapperDef {
override def code = {
val positional = compoundValue(columnsPositional.map(c => if (c.fakeNullable || c.model.nullable) s"<<?[${c.rawType}]" else s"<<[${c.rawType}]"))
val dependencies = columns.map(_.exposedType).distinct.zipWithIndex.map{ case (t,i) => s"""e$i: GR[$t]"""}.mkString(", ")
val rearranged = compoundValue(desiredColumnOrder.map(i => if(columns.size > 22) s"r($i)" else tuple(i)))
def result(args: String) = s"$factory($args)"
val body =
if(autoIncLastAsOption && columns.size > 1){
s"""
val r = $positional
import r._
${result(rearranged)} // putting AutoInc last
""".trim
} else {
result(positional)
}
s"""
implicit def $name(implicit $dependencies): GR[${TableClass.elementType}] = GR{
prs => import prs._
${indent(body)}
}
""".trim
}
}
override def TableClass = new TableClassDef {
override def star = {
val struct = compoundValue(columns.map(c=>if(c.fakeNullable)s"Rep.Some(${c.name})" else s"${c.name}"))
val rhs = s"$struct <> ($factory, $extractor)"
s"def * = $rhs"
}
}
def tails(n: Int) = {
List.fill(n)(".tail").mkString("")
}
// override column generator to add additional types
override def Column = new Column(_) {
override def rawType = {
typeMapper(model).getOrElse(super.rawType)
}
}
}
})
def typeMapper(column: Column): Option[String] = {
column.tpe match {
case "java.sql.Date" => Some("java.time.LocalDate")
case "java.sql.Timestamp" => Some("java.time.LocalDateTime")
case _ => None
}
}
def doCodeGen() = {
def generator = Await.result(codegenFuture, Duration.Inf)
generator.writeToFile(slickDriver, outputFolder, pkg, "Tables", "Tables.scala")
}
def main(args: Array[String]) {
doCodeGen()
db.close()
}
}
Update 2019-02-15: *with the release of Slick 3.3.0, as answered by #Marcus there's built-in support for code generation of tables with > 22 columns.
As of Slick 3.2.0, the simplest solution for >22 param case class is to define the default projection in the * method using mapTo instead of the <> operator (per documented unit test):
case class BigCase(id: Int,
p1i1: Int, p1i2: Int, p1i3: Int, p1i4: Int, p1i5: Int, p1i6: Int,
p2i1: Int, p2i2: Int, p2i3: Int, p2i4: Int, p2i5: Int, p2i6: Int,
p3i1: Int, p3i2: Int, p3i3: Int, p3i4: Int, p3i5: Int, p3i6: Int,
p4i1: Int, p4i2: Int, p4i3: Int, p4i4: Int, p4i5: Int, p4i6: Int)
class bigCaseTable(tag: Tag) extends Table[BigCase](tag, "t_wide") {
def id = column[Int]("id", O.PrimaryKey)
def p1i1 = column[Int]("p1i1")
def p1i2 = column[Int]("p1i2")
def p1i3 = column[Int]("p1i3")
def p1i4 = column[Int]("p1i4")
def p1i5 = column[Int]("p1i5")
def p1i6 = column[Int]("p1i6")
def p2i1 = column[Int]("p2i1")
def p2i2 = column[Int]("p2i2")
def p2i3 = column[Int]("p2i3")
def p2i4 = column[Int]("p2i4")
def p2i5 = column[Int]("p2i5")
def p2i6 = column[Int]("p2i6")
def p3i1 = column[Int]("p3i1")
def p3i2 = column[Int]("p3i2")
def p3i3 = column[Int]("p3i3")
def p3i4 = column[Int]("p3i4")
def p3i5 = column[Int]("p3i5")
def p3i6 = column[Int]("p3i6")
def p4i1 = column[Int]("p4i1")
def p4i2 = column[Int]("p4i2")
def p4i3 = column[Int]("p4i3")
def p4i4 = column[Int]("p4i4")
def p4i5 = column[Int]("p4i5")
def p4i6 = column[Int]("p4i6")
// HList-based wide case class mapping
def m3 = (
id ::
p1i1 :: p1i2 :: p1i3 :: p1i4 :: p1i5 :: p1i6 ::
p2i1 :: p2i2 :: p2i3 :: p2i4 :: p2i5 :: p2i6 ::
p3i1 :: p3i2 :: p3i3 :: p3i4 :: p3i5 :: p3i6 ::
p4i1 :: p4i2 :: p4i3 :: p4i4 :: p4i5 :: p4i6 :: HNil
).mapTo[BigCase]
def * = m3
}
EDIT
So, if you then want the slick-codegen to produce huge tables using the mapTo method described above, you override the relevant parts to the code generator and add in a mapTo statement:
package your.package
import slick.codegen.SourceCodeGenerator
import slick.{model => m}
class HugeTableCodegen(model: m.Model) extends SourceCodeGenerator(model) with GeneratorHelpers[String, String, String]{
override def Table = new Table(_) {
table =>
// always defines types using case classes
override def EntityType = new EntityTypeDef{
override def classEnabled = true
}
// allow compound statements using HNil, but not for when "def *()" is being defined, instead use mapTo statement
override def compoundValue(values: Seq[String]): String = {
// values.size>22 assumes that this must be for the "*" operator and NOT a primary/foreign key
if(hlistEnabled && values.size > 22) values.mkString("(", " :: ", s" :: HNil).mapTo[${StringExtensions(model.name.table).toCamelCase}Row]")
else if(hlistEnabled) values.mkString(" :: ") + " :: HNil"
else if (values.size == 1) values.head
else s"""(${values.mkString(", ")})"""
}
// should always be case classes, so no need to handle hlistEnabled here any longer
override def compoundType(types: Seq[String]): String = {
if (types.size == 1) types.head
else s"""(${types.mkString(", ")})"""
}
}
}
You then structure the codegen code in a separate project as documented so that it generates the source at compile time. You can pass your classname as an argument to the SourceCodeGenerator you're extending:
lazy val generateSlickSchema = taskKey[Seq[File]]("Generates Schema definitions for SQL tables")
generateSlickSchema := {
val managedSourceFolder = sourceManaged.value / "main" / "scala"
val packagePath = "your.sql.table.package"
(runner in Compile).value.run(
"slick.codegen.SourceCodeGenerator", (dependencyClasspath in Compile).value.files,
Array(
"env.db.connectorProfile",
"slick.db.driver",
"slick.db.url",
managedSourceFolder.getPath,
packagePath,
"slick.db.user",
"slick.db.password",
"true",
"your.package.HugeTableCodegen"
),
streams.value.log
)
Seq(managedSourceFolder / s"${packagePath.replace(".","/")}/Tables.scala")
}
There are few options available as you have already found out - nested tuples, conversion from Slick HList to Shapeless HList and then to case classes and so on.
I found all those options too complicated for the task and went with customised Slick Codegen to generate simple wrapper class with accessors.
Have a look at this gist.
class MyCodegenCustomisations(model: Model) extends slick.codegen.SourceCodeGenerator(model){
import ColumnDetection._
override def Table = new Table(_){
table =>
val columnIndexByName = columns.map(_.name).zipWithIndex.toMap
def getColumnIndex(columnName: String): Option[Int] = {
columnIndexByName.get(columnName)
}
private def getWrapperCode: Seq[String] = {
if (columns.length <= 22) {
//do not generate wrapper for tables which get case class generated by Slick
Seq.empty[String]
} else {
val lines =
columns.map{c =>
getColumnIndex(c.name) match {
case Some(colIndex) =>
//lazy val firstname: Option[String] = row.productElement(1).asInstanceOf[Option[String]]
val colType = c.exposedType
val line = s"lazy val ${c.name}: $colType = values($colIndex).asInstanceOf[$colType]"
line
case None => ""
}
}
Seq("",
"/*",
"case class Wrapper(private val row: Row) {",
"// addressing HList by index is very slow, let's convert it to vector",
"private lazy val values = row.toList.toVector",
""
) ++ lines ++ Seq("}", "*/", "")
}
}
override def code: Seq[String] = {
val originalCode = super.code
originalCode ++ this.getWrapperCode
}
}
}
This issue is solved in Slick 3.3:
https://github.com/slick/slick/pull/1889/
This solution provides def * and def ? and also supports plain SQL.
I tried the example mentioned in Luanne's article The essence of Spring Data Neo4j 4 in Scala. The code can be found in the neo4j-ogm-scala repository.
package neo4j.ogm.scala.domain
import org.neo4j.ogm.annotation.GraphId;
import scala.beans.BeanProperty
import org.neo4j.ogm.annotation.NodeEntity
import org.neo4j.ogm.annotation.Relationship
import org.neo4j.ogm.session.Session;
import org.neo4j.ogm.session.SessionFactory;
abstract class Entity {
#GraphId
#BeanProperty
var id: Long = _
override def equals(o: Any): Boolean = o match {
case other: Entity => other.id.equals(this.id)
case _ => false
}
override def hashCode: Int = id.hashCode()
}
#NodeEntity
class Category extends Entity {
var name: String = _
def this(name: String) {
this()
this.name = name
}
}
#NodeEntity
class Ingredient extends Entity {
var name: String = _
#Relationship(`type` = "HAS_CATEGORY", direction = "OUTGOING")
var category: Category = _
#Relationship(`type` = "PAIRS_WITH", direction = "UNDIRECTED")
var pairings: Set[Pairing] = Set()
def addPairing(pairing: Pairing): Unit = {
pairing.first.pairings +(pairing)
pairing.second.pairings +(pairing)
}
def this(name: String, category: Category) {
this()
this.name = name
this.category = category
}
}
#RelationshipEntity(`type` = "PAIRS_WITH")
class Pairing extends Entity {
#StartNode
var first: Ingredient = _
#EndNode
var second: Ingredient = _
def this(first: Ingredient, second: Ingredient) {
this()
this.first = first
this.second = second
}
}
object Neo4jSessionFactory {
val sessionFactory = new SessionFactory("neo4j.ogm.scala.domain")
def getNeo4jSession(): Session = {
System.setProperty("username", "neo4j")
System.setProperty("password", "neo4j")
sessionFactory.openSession("http://localhost:7474")
}
}
object Main extends App {
val spices = new Category("Spices")
val turmeric = new Ingredient("Turmeric", spices)
val cumin = new Ingredient("Cumin", spices)
val pairing = new Pairing(turmeric, cumin)
cumin.addPairing(pairing)
val session = Neo4jSessionFactory.getNeo4jSession()
val tx: Transaction = session.beginTransaction()
try {
session.save(spices)
session.save(turmeric)
session.save(cumin)
session.save(pairing)
tx.commit()
} catch {
case e: Exception => // tx.rollback()
} finally {
// tx.commit()
}
}
The problem is that nothing gets saved to Neo4j. Can you please point out the problem in my code?
Thanks,
Manoj.
Scala’s Long is an instance of a Value class. Value classes were explicitly introduced to avoid allocating runtime objects. At the JVM level therefore Scala's Long is equivalent to Java’s primitive long which is why it has the primitive type signature J. It cannot be therefore be null, and should not be used as a graphId. Although Scala mostly will do auto-boxing between its own Long and Java’s Long class, this doesn’t apply to declarations, only to operations on those objects.
The #GraphId isn't being picked up on your entities. I have zero knowledge of Scala but it looks like the scala long isn't liked much by the OGM; var id: java.lang.Long = _ works fine.
There is some data that I have pulled from a remote API, for which I use a Future-style interface. The data is structured as a linked-list. A relevant example data container is shown below.
case class Data(information: Int) {
def hasNext: Boolean = ??? // Implemented
def next: Future[Data] = ??? // Implemented
}
Now I'm interested in adding some functionality to the data class, such as map, foreach, reduce, etc. To do so I want to implement some form of IterableLike such that it inherets these methods.
Given below is the trait Data may extend, such that it gets this property.
trait AsyncIterable[+T]
extends IterableLike[Future[T], AsyncIterable[T]]
{
def hasNext : Boolean
def next : Future[T]
// How to implement?
override def iterator: Iterator[Future[T]] = ???
override protected[this] def newBuilder: mutable.Builder[Future[T], AsyncIterable[T]] = ???
override def seq: TraversableOnce[Future[T]] = ???
}
It should be a non-blocking implementation, which when acted on, starts requesting the next data from the remote data source.
It is then possible to do cool stuff such as
case class Data(information: Int) extends AsyncIterable[Data]
val data = Data(1) // And more, of course
// Asynchronously print all the information.
data.foreach(data => println(data.information))
It is also acceptable for the interface to be different. But the result should in some way represent asynchronous iteration over the collection. Preferably in a way that is familiar to developers, as it will be part of an (open source) library.
In production I would use one of following:
Akka Streams
Reactive Extensions
For private tests I would implement something similar to following.
(Explanations are below)
I have modified a little bit your Data:
abstract class AsyncIterator[T] extends Iterator[Future[T]] {
def hasNext: Boolean
def next(): Future[T]
}
For it we can implement this Iterable:
class AsyncIterable[T](sourceIterator: AsyncIterator[T])
extends IterableLike[Future[T], AsyncIterable[T]]
{
private def stream(): Stream[Future[T]] =
if(sourceIterator.hasNext) {sourceIterator.next #:: stream()} else {Stream.empty}
val asStream = stream()
override def iterator = asStream.iterator
override def seq = asStream.seq
override protected[this] def newBuilder = throw new UnsupportedOperationException()
}
And if see it in action using following code:
object Example extends App {
val source = "Hello World!";
val iterator1 = new DelayedIterator[Char](100L, source.toCharArray)
new AsyncIterable(iterator1).foreach(_.foreach(print)) //prints 1 char per 100 ms
pause(2000L)
val iterator2 = new DelayedIterator[String](100L, source.toCharArray.map(_.toString))
new AsyncIterable(iterator2).reduceLeft((fl: Future[String], fr) =>
for(l <- fl; r <- fr) yield {println(s"$l+$r"); l + r}) //prints 1 line per 100 ms
pause(2000L)
def pause(duration: Long) = {println("->"); Thread.sleep(duration); println("\n<-")}
}
class DelayedIterator[T](delay: Long, data: Seq[T]) extends AsyncIterator[T] {
private val dataIterator = data.iterator
private var nextTime = System.currentTimeMillis() + delay
override def hasNext = dataIterator.hasNext
override def next = {
val thisTime = math.max(System.currentTimeMillis(), nextTime)
val thisValue = dataIterator.next()
nextTime = thisTime + delay
Future {
val now = System.currentTimeMillis()
if(thisTime > now) Thread.sleep(thisTime - now) //Your implementation will be better
thisValue
}
}
}
Explanation
AsyncIterable uses Stream because it's calculated lazily and it's simple.
Pros:
simplicity
multiple calls to iterator and seq methods return same iterable with all items.
Cons:
could lead to memory overflow because stream keeps all prevously obtained values.
first value is eagerly gotten during creation of AsyncIterable
DelayedIterator is very simplistic implementation of AsyncIterator, don't blame me for quick and dirty code here.
It's still strange for me to see synchronous hasNext and asynchronous next()
Using Twitter Spool I've implemented a working example.
To implement spool I modified the example in the documentation.
import com.twitter.concurrent.Spool
import com.twitter.util.{Await, Return, Promise}
import scala.concurrent.{ExecutionContext, Future}
trait AsyncIterable[+T <: AsyncIterable[T]] { self : T =>
def hasNext : Boolean
def next : Future[T]
def spool(implicit ec: ExecutionContext) : Spool[T] = {
def fill(currentPage: Future[T], rest: Promise[Spool[T]]) {
currentPage foreach { cPage =>
if(hasNext) {
val nextSpool = new Promise[Spool[T]]
rest() = Return(cPage *:: nextSpool)
fill(next, nextSpool)
} else {
val emptySpool = new Promise[Spool[T]]
emptySpool() = Return(Spool.empty[T])
rest() = Return(cPage *:: emptySpool)
}
}
}
val rest = new Promise[Spool[T]]
if(hasNext) {
fill(next, rest)
} else {
rest() = Return(Spool.empty[T])
}
self *:: rest
}
}
Data is the same as before, and now we can use it.
// Cool stuff
implicit val ec = scala.concurrent.ExecutionContext.global
val data = Data(1) // And others
// Print all the information asynchronously
val fut = data.spool.foreach(data => println(data.information))
Await.ready(fut)
It will trow an exception on the second element, because the implementation of next was not provided.