I have a class that I am trying to deserialize using the json4s CustomSerializer functionality. I need to do this due to the inability of json4s to deserialize mutable collections.
This is the basic structure of the class I want to deserialize (don't worry about why the class is structured like this):
case class FeatureValue(timestamp:Double)
object FeatureValue{
implicit def ordering[F <: FeatureValue] = new Ordering[F] {
override def compare(a: F, b: F): Int = {
a.timestamp.compareTo(b.timestamp)
}
}
}
class Point {
val features = new HashMap[String, SortedSet[FeatureValue]]
def add(name:String, value:FeatureValue):Unit = {
val oldValue:SortedSet[FeatureValue] = features.getOrElseUpdate(name, SortedSet[FeatureValue]())
oldValue += value
}
}
Json4s serializes this just fine. A serialized instance might look like the following:
{"features":
{
"CODE0":[{"timestamp":4.8828914447482E8}],
"CODE1":[{"timestamp":4.8828914541333E8}],
"CODE2":[{"timestamp":4.8828915127325E8},{"timestamp":4.8828910097466E8}]
}
}
I've tried writing a custom deserializer, but I don't know how to deal with the list tails. In a normal matcher you can just call your own function recursively, but in this case the function is anonymous and being called through the json4s API. I cannot find any examples that deal with this and I can't figure it out.
Currently I can match only a single hash key, and a single FeatureValue instance in its value. Here is the CustomSerializer as it stands:
import org.json4s.{FieldSerializer, DefaultFormats, Extraction, CustomSerializer}
import org.json4s.JsonAST._
class PointSerializer extends CustomSerializer[Point](format => (
{
case JObject(JField("features", JObject(Nil)) :: Nil) => new Point
case JObject(List(("features", JObject(List(
(feature:String, JArray(List(JObject(List(("timestamp",JDouble(ts)))))))))
))) => {
val point = new Point
point.add(feature, FeatureValue(ts))
point
}
},
{
// don't need to customize this, it works fine
case x: Point => Extraction.decompose(x)(DefaultFormats + FieldSerializer[Point]())
}
))
If I try to change to using the :: separated list format, so far I have gotten compiler errors. Even if I didn't get compiler errors, I am not sure what I would do with them.
You can get the list of json features in your pattern match and then map over this list to get the Features and their codes.
class PointSerializer extends CustomSerializer[Point](format => (
{
case JObject(List(("features", JObject(featuresJson)))) =>
val features = featuresJson.flatMap {
case (code:String, JArray(timestamps)) =>
timestamps.map { case JObject(List(("timestamp",JDouble(ts)))) =>
code -> FeatureValue(ts)
}
}
val point = new Point
features.foreach((point.add _).tupled)
point
}, {
case x: Point => Extraction.decompose(x)(DefaultFormats + FieldSerializer[Point]())
}
))
Which deserializes your json as follows :
import org.json4s.native.Serialization.{read, write}
implicit val formats = Serialization.formats(NoTypeHints) + new PointSerializer
val json = """
{"features":
{
"CODE0":[{"timestamp":4.8828914447482E8}],
"CODE1":[{"timestamp":4.8828914541333E8}],
"CODE2":[{"timestamp":4.8828915127325E8},{"timestamp":4.8828910097466E8}]
}
}
"""
val point0 = read[Point]("""{"features": {}}""")
val point1 = read[Point](json)
point0.features // Map()
point1.features
// Map(
// CODE0 -> TreeSet(FeatureValue(4.8828914447482E8)),
// CODE2 -> TreeSet(FeatureValue(4.8828910097466E8), FeatureValue(4.8828915127325E8)),
// CODE1 -> TreeSet(FeatureValue(4.8828914541333E8))
// )
Related
I am trying to build a multi level Validator object that is fairly generic. The idea being you have levels of validations if Level 1 passes then you do Level 2, etc. but I am struggling with one specific area: creating a function call but not executing it until a later point.
Data:
case class FooData(alpha: String, beta: String) extends AllData
case class BarData(gamma: Int, delta: Int) extends AllData
ValidationError:
case class ValidationError(code: String, message: String)
Validator:
object Validator {
def validate(validations: List[List[Validation]]): List[ValidationError] = {
validations match {
case head :: nil => // Execute the functions and get the results back
// Recursively work down the levels (below syntax may be incorrect)
case head :: tail => validate(head) ... // if no errors then validate(tail) etc.
...
}
}
}
Sample Validator:
object CorrectNameFormatValidator extends Validation {
def validate(str: String): Seq[ValidationError] = {
...
}
}
How I wish to use it:
object App {
def main(args: Array[String]): Unit = {
val fooData = FooData(alpha = "first", beta = "second")
val levelOneValidations = List(
CorrectNameFormatValidator(fooData.alpha),
CorrectNameFormatValidator(fooData.beta),
SomeOtherValidator(fooData.beta)
)
// I don't want these to execute as function calls here
val levelTwoValidations = List(
SomeLevelTwoValidator (fooData.alpha),
SomeLevelTwoValidator(fooData.beta),
SomeOtherLevelValidator(fooData.beta),
SomeOtherLevelValidator(fooData.alpha)
)
val validationLevels = List(levelOneValidations, levelTwoValidations)
Validator.validate(validationLevels)
}
}
Am I doing something really convoluted when I don't need to be or am I just missing a component?
Essentially I want to define when a function will be called and with which parameters but I don't want the call to happen until I say within the Validator. Is this something that's possible?
You can use lazy val or def when defining levelOneValidation, levelTwoValidations and validationLevel:
object App {
def main(args: Array[String]): Unit = {
val fooData = FooData(alpha = "first", beta = "second")
lazy val levelOneValidations = List(
CorrectNameFormatValidator(fooData.alpha),
CorrectNameFormatValidator(fooData.beta),
SomeOtherValidator(fooData.beta)
)
// I don't want these to execute as function calls here
lazy val levelTwoValidations = List(
SomeLevelTwoValidator (fooData.alpha),
SomeLevelTwoValidator(fooData.beta),
SomeOtherLevelValidator(fooData.beta),
SomeOtherLevelValidator(fooData.alpha)
)
lazy val validationLevels = List(levelOneValidations, levelTwoValidations)
Validator.validate(validationLevels)
}
}
You also need to change validate method to get the validations ByName and not
ByValue using : => :
object Validator {
def validate(validations: => List[List[Validation]]): List[ValidationError] = {
validations match {
case head :: nil => // Execute the functions and get the results back
// Recursively work down the levels (below syntax may be incorrect)
case head :: tail => validate(head) ... // if no errors then validate(tail) etc.
...
}
}
}
Anyway, I think you can implement it differently by just using some OOP design patterns, like Chain of Responsibility.
I'm trying to figure out if a member field in any given case class is also a case class. Taken from this answer, given an instance or an object, I can pass it along and determine if it's a case class:
def isCaseClass(v: Any): Boolean = {
import reflect.runtime.universe._
val typeMirror = runtimeMirror(v.getClass.getClassLoader)
val instanceMirror = typeMirror.reflect(v)
val symbol = instanceMirror.symbol
symbol.isCaseClass
}
However, what I'd like, is to take a case class, extract all of its member fields, and find out which ones are case classes themselves. Something in this manner:
def innerCaseClasses[A](parentCaseClass:A): List[Class[_]] = {
val nestedCaseClasses = ListBuffer[Class[_]]()
val fields = parentCaseClass.getClass.getDeclaredFields
fields.foreach(field => {
if (??? /*field is case class */ ) {
nestedCaseClasses += field.getType
}
})
nestedCaseClasses.toList
}
I thought maybe I could extract the fields, their classes, and use reflection to instantiate a new instance of that member field as its own class. I'm not 100% how to do that, and it seems like perhaps there's an easier way. Is there?
Ah! I've figured it out (simplified the function which tells the determination):
import reflect.runtime.universe._
case class MyThing(str:String, num:Int)
case class WithMyThing(name:String, aThing:MyThing)
val childThing = MyThing("Neat" , 293923)
val parentCaseClass = WithMyThing("Nate", childThing)
def isCaseClass(v: Any): Boolean = {
val typeMirror = runtimeMirror(v.getClass.getClassLoader)
val instanceMirror = typeMirror.reflect(v)
val symbol = instanceMirror.symbol
symbol.isCaseClass
}
def innerCaseClasses[A](parentCaseClass:A): Unit = {
val fields = parentCaseClass.asInstanceOf[Product].productIterator
fields.foreach(field => {
println(s"Field: ${field.getClass.getSimpleName} isCaseClass? " + isCaseClass(field))
})
}
innerCaseClasses(parentCaseClass)
printout:
Field: String isCaseClass? false
Field: MyThing isCaseClass? true
For my project, I would like to make a tree model; let's say it's about files and directories. But files can be in multiple directories at the same time, so more like the same way you add tags to email in gmail.
I want to build a model for competences (say java, scala, angular, etc) and put them in categories. In this case java and scala are languages, agila and scrum are ways of working, angular is a framework / toolkit and so forth. But then we want to group stuff flexibly, ie play, java and scala are in a 'backend' category and angular, jquery, etc are in a frontend category.
I figured I would have a table competences like so:
case class Competence (name: String, categories: Option[Category])
and the categories as follows:
case class Category ( name: String, parent: Option[Category] )
This will compile, but SORM will generate an error (from activator console):
scala> import models.DB
import models.DB
scala> import models.Category
import models.Category
scala> import models.Competence
import models.Competence
scala> val cat1 = new Category ( "A", None )
cat1: models.Category = Category(A,None)
scala> val sav1 = DB.save ( cat1 )
sorm.Instance$ValidationException: Entity 'models.Category' recurses at 'models.Category'
at sorm.Instance$Initialization$$anonfun$2.apply(Instance.scala:216)
at sorm.Instance$Initialization$$anonfun$2.apply(Instance.scala:216)
at scala.Option.map(Option.scala:146)
at sorm.Instance$Initialization.<init>(Instance.scala:216)
at sorm.Instance.<init>(Instance.scala:38)
at models.DB$.<init>(DB.scala:5)
at models.DB$.<clinit>(DB.scala)
... 42 elided
Although I want the beautiful simplicity of sorm, will I need to switch to Slick for my project to implement this? I had the idea that link tables would be implicitly generated by sorm. Or could I simply work around the problem by making a:
case class Taxonomy ( child: Category, parent: Category )
and then do parsing / formatting work on the JS side? It seems to make the simplicity of using sorm disappear somewhat.
To give some idea, what I want is to make a ajaxy page where a user can add new competences in a list on the left, and then link/unlink them to whatever category tag in the tree he likes.
I encountered the same question. I needed to define an interation between two operant, which can be chained(recursive). Like:
case class InteractionModel(
val leftOperantId: Int,
val operation: String ,
val rightOperantId: Int,
val next: InteractionModel)
My working around: change this case class into Json(String) and persist it as String, when retreiving it, convert it from Json. And since it's String, do not register it as sorm Entity.
import spray.json._
case class InteractionModel(
val leftOperantId: Int,
val operation: String ,
val rightOperantId: Int,
val next: InteractionModel) extends Jsonable {
def toJSON: String = {
val js = this.toJson(InteractionModel.MyJsonProtocol.ImJsonFormat)
js.compactPrint
}
}
//define protocol
object InteractionModel {
def fromJSON(in: String): InteractionModel = {
in.parseJson.convertTo[InteractionModel](InteractionModel.MyJsonProtocol.ImJsonFormat)
}
val none = new InteractionModel((-1), "", (-1), null) {
override def toJSON = "{}"
}
object MyJsonProtocol extends DefaultJsonProtocol {
implicit object ImJsonFormat extends RootJsonFormat[InteractionModel] {
def write(im: InteractionModel) = {
def recWrite(i: InteractionModel): JsObject = {
val next = i.next match {
case null => JsNull
case iNext => recWrite(i.next)
}
JsObject(
"leftOperantId" -> JsNumber(i.leftOperantId),
"operation" -> JsString(i.operation.toString),
"rightOperantId" -> JsNumber(i.rightOperantId),
"next" -> next)
}
recWrite(im)
}
def read(value: JsValue) = {
def recRead(v: JsValue): InteractionModel = {
v.asJsObject.getFields("leftOperantId", "operation", "rightOperantId", "next") match {
case Seq(JsNumber(left), JsString(operation), JsNumber(right), nextJs) =>
val next = nextJs match {
case JsNull => null
case js => recRead(js)
}
InteractionModel(left.toInt, operation, right.toInt, next)
case s => InteractionModel.none
}
}
recRead(value)
}
}
}
}
I'm trying to write a custom SPickler / Unpickler pair to work around some the current limitations of scala-pickling.
The data type I'm trying to pickle is a case class, where some of the fields already have their own SPickler and Unpickler instances.
I'd like to use these instances in my custom pickler, but I don't know how.
Here's an example of what I mean:
// Here's a class for which I want a custom SPickler / Unpickler.
// One of its fields can already be pickled, so I'd like to reuse that logic.
case class MyClass[A: SPickler: Unpickler: FastTypeTag](myString: String, a: A)
// Here's my custom pickler.
class MyClassPickler[A: SPickler: Unpickler: FastTypeTag](
implicit val format: PickleFormat) extends SPickler[MyClass[A]] with Unpickler[MyClass[A]] {
override def pickle(
picklee: MyClass[A],
builder: PBuilder) {
builder.beginEntry(picklee)
// Here we save `myString` in some custom way.
builder.putField(
"mySpecialPickler",
b => b.hintTag(FastTypeTag.ScalaString).beginEntry(
picklee.myString).endEntry())
// Now we need to save `a`, which has an implicit SPickler.
// But how do we use it?
builder.endEntry()
}
override def unpickle(
tag: => FastTypeTag[_],
reader: PReader): MyClass[A] = {
reader.beginEntry()
// First we read the string.
val myString = reader.readField("mySpecialPickler").unpickle[String]
// Now we need to read `a`, which has an implicit Unpickler.
// But how do we use it?
val a: A = ???
reader.endEntry()
MyClass(myString, a)
}
}
I would really appreciate a working example.
Thanks!
Here is a working example:
case class MyClass[A](myString: String, a: A)
Note that the type parameter of MyClass does not need context bounds. Only the custom pickler class needs the corresponding implicits:
class MyClassPickler[A](implicit val format: PickleFormat, aTypeTag: FastTypeTag[A],
aPickler: SPickler[A], aUnpickler: Unpickler[A])
extends SPickler[MyClass[A]] with Unpickler[MyClass[A]] {
private val stringUnpickler = implicitly[Unpickler[String]]
override def pickle(picklee: MyClass[A], builder: PBuilder) = {
builder.beginEntry(picklee)
builder.putField("myString",
b => b.hintTag(FastTypeTag.ScalaString).beginEntry(picklee.myString).endEntry()
)
builder.putField("a",
b => {
b.hintTag(aTypeTag)
aPickler.pickle(picklee.a, b)
}
)
builder.endEntry()
}
override def unpickle(tag: => FastTypeTag[_], reader: PReader): MyClass[A] = {
reader.hintTag(FastTypeTag.ScalaString)
val tag = reader.beginEntry()
val myStringUnpickled = stringUnpickler.unpickle(tag, reader).asInstanceOf[String]
reader.endEntry()
reader.hintTag(aTypeTag)
val aTag = reader.beginEntry()
val aUnpickled = aUnpickler.unpickle(aTag, reader).asInstanceOf[A]
reader.endEntry()
MyClass(myStringUnpickled, aUnpickled)
}
}
In addition to the custom pickler class, we also need an implicit def which returns a pickler instance specialized for concrete type arguments:
implicit def myClassPickler[A: SPickler: Unpickler: FastTypeTag](implicit pf: PickleFormat) =
new MyClassPickler
Consider the Favorites table object below, we want to write a query to find Favorites by their type (defined below). We have also defined a Typemapper, to map a FavoriteType to a String for the database
import scala.slick.driver.PostgresDriver.simple._
//Other imports have been omitted in this question
object Favorites extends Table[Favorite]("favorites") {
// Convert the favoriteTypes to strings for the database
implicit val favoriteMapping: TypeMapper[FavorietType] = MappedTypeMapper.base[FavorietType, String](
favType => FavorietType.values.find(_ == favType).get.mapping,
mapping => FavorietType.values.find(_.mapping == mapping).get
)
def favoriteType = column[FavoriteType]("type")
//other columns here
This is the query I want to write (however it does not compile)
def queryByFavoriteType(ftype : FavoriteType)(implicit s: Session) = {
for(
f <- Favorieten if f.favoriteType === ftype
) yield f
}
}
Here I have defined de different FavoriteType objects (this is outside the Favorieten Object)
sealed case class FavorietType(mapping: String) {
override def toString = mapping.capitalize
}
object FavoriteType {
object Exam extends FavoriteType("examen")
object Topic extends FavoriteType("onderwerp")
object Paper extends FavoriteType("profielwerkstuk")
val values = Seq(Exam , Topic , Paper )
}
The problem I have here is that the query does not compile:
value === is not a member of scala.slick.lifted.Column[models.gebruiker.FavorietType]
It appears that === can not be used to compare a User-defined type, is this true? Is there an alternative way to do this?
Edit
Related issue: before I had my TypeMapper without explicit type, it was defined as implicit val favoriteMapping = MappedTypeMapper.base[FavorietType, String]( ...
When I would write a query that would compare a FavoriteType.Exam (for example) such as
def queryByFavoriteExam()(implicit s: Session) = {
for(f <- Favorieten if f.favorietType === FavorietType.Exam) yield f
}
This would result in the error could not find implicit value for evidence parameter of type scala.slick.lifted.TypeMapper[models.gebruiker.FavorietType.Exam.type]
The solution for this is the same as the one presented below
When in doubt with Slick, go check out the unit tests. After reading their docs on mapping in a custom type and then looking at their unit tests, I got your query code to compile by changing it to:
def queryByFavoriteType(ftype : FavoriteType)(implicit s: Session) = {
for(f <- Favorites if f.favoriteType === (ftype:FavoriteType)) yield f
}
Also, I had imported the H2Driver just to get things to compile (import scala.slick.driver.H2Driver.simple._). I was assuming that you also had imported whatever driver it is that you need for your db.
EDIT
My full code example is as follows:
import scala.slick.driver.PostgresDriver.simple._
import scala.slick.session.Session
sealed case class FavoriteType(mapping: String) {
override def toString = mapping.capitalize
}
case class Favorite(ft:FavoriteType, foo:String)
object FavoriteType {
object Exam extends FavoriteType("examen")
object Topic extends FavoriteType("onderwerp")
object Paper extends FavoriteType("profielwerkstuk")
val values = Seq(Exam , Topic , Paper )
}
object Favorites extends Table[Favorite]("favorites") {
// Convert the favoriteTypes to strings for the database
implicit val favoriteMapping = MappedTypeMapper.base[FavoriteType, String](
{favType => FavoriteType.values.find(_ == favType).get.mapping},
{mapping => FavoriteType.values.find(_.mapping == mapping).get}
)
def favoriteType = column[FavoriteType]("type")
def foo = column[String]("foo")
def * = favoriteType ~ foo <> (Favorite.apply _, Favorite.unapply _)
def queryByFavoriteType(ftype : FavoriteType)(implicit s: Session) = {
for(f <- Favorites if f.favoriteType === (ftype:FavoriteType)) yield f
}
}