How to deserialize a scala tree with JSON4S - scala

Serialization works fine but I have nothing for deserialization. I found interesting solution for abstract class here How to serialize sealed abstract class with Json4s in Scala? but it doesn't deal with trees.
This the code of my test with a standard JSON4S :
import org.json4s._
import org.json4s.native.JsonMethods._
import org.json4s.native.Serialization.{ read, write }
import org.json4s.native.Serialization
abstract class Tree
case class Node(nameN: String, trees: List[Tree]) extends Tree
case class Leaf(nameL: String) extends Tree
object Tree extends App {
implicit val formats = Serialization.formats(NoTypeHints)
// object creation to test the serialization
val root =
Node(
"Grand Pavois project",
List(
Node(
"studies",
List(
Leaf("preliminary studies"),
Leaf("detailled studies")
)
),
Node(
"realization",
List(
Leaf("ground"),
Leaf("building"),
Leaf("roof")
)
),
Node(
"delivery",
List(
Leaf("quality inspection"),
Leaf("customer delivery")
)
)
)
)
val serialized = write(root) // object creation and serialization
println(s"serialized: $serialized") // print the result, this is OK
// and now what about deserialization?
// string creation for deserialization
// ( it is the same as serialized above, I do like that to trace for the demo)
val rootString = """
{
"nameN": "Grand Pavois project",
"trees": [
{
"nameN": "studies",
"trees": [
{
"nameL": "preliminary studies"
},
{
"nameL": "detailled studies"
}
]
},
{
"nameN": "realization",
"trees": [
{
"nameL": "ground"
},
{
"nameL": "building"
},
{
"nameL": "roof"
}
]
},
{
"nameN": "delivery",
"trees": [
{
"nameL": "quality inspection"
},
{
"nameL": "customer delivery"
}
]
}
]
}
"""
//standard deserialization below that produce an error :
// "Parsed JSON values do not match with class constructor"
val rootFromString = read[Tree](rootString)
}
Now I guess the solution is with a custom deserializer probably a recusive one but how to define it? That is the question. Thanks for your help.

This solution doesn't use a custom deserializer, but instead creates a type that matches both Node and Leaf and then converts to the appropriate type later.
case class JsTree(nameN: Option[String], nameL: Option[String], trees: Option[List[JsTree]])
def toTree(node: JsTree): Tree = node match {
case JsTree(Some(name), None, Some(trees)) =>
Node(name, trees.map(toTree))
case JsTree(None, Some(name), None) =>
Leaf(name)
case _ =>
throw new IllegalArgumentException
}
val rootFromString = toTree(read[JsTree](rootString))
The JsTree class will match both Node and Leaf values because it has option fields that match all the fields in both classes. The toTree method recursively converts the JsTree to the appropriate Tree subclass based on which fields are actually present.
Update: Custom serializer
Here is the solution using a custom serializer:
import org.json4s.JsonDSL._
class TreeSerializer extends CustomSerializer[Tree](format => ({
case obj: JObject =>
implicit val formats: Formats = format
if ((obj \ "trees") == JNothing) {
Leaf(
(obj \ "nameL").extract[String]
)
} else {
Node(
(obj \ "nameN").extract[String],
(obj \ "trees").extract[List[Tree]]
)
}
}, {
case node: Node =>
JObject("nameN" -> JString(node.nameN), "trees" -> node.trees.map(Extraction.decompose))
case leaf: Leaf =>
"nameL" -> leaf.nameL
}))
Use it like this:
implicit val formats: Formats = DefaultFormats + new TreeSerializer
read[Tree](rootString)

Related

Scala Play Read: How to Flatten Json containing Array of arrays to model

I am looking for a way to define a Reads which allows me to map a JSON containing the following structure:
{
"offers": [
[
{
"id": "1234",
(etc.)
}
]
]
}
to model such case class TransportOffer(offers: List[Offer])
Unfortunately I haven't been able to do this yet. This is my code:
implicit val transportOfferReads: Reads[TransportOffer] = (
(JsPath \ "offers").read[List[List[Offer]]].flatMap(r => r.flatten)
)(TransportOffer.apply _)
In this case the flattening is not possible, as flatMap expects another Reads. How would I wrap the flattened List into another Reads?
Or is there a simpler way?
I'll present 3 options:
Flattening in a short reads:
case class Offer(id: String)
object Offer {
implicit val format: OFormat[Offer] = Json.format[Offer]
}
case class TransportOffer(offers: List[Offer])
object TransportOffer {
implicit val transportOfferReads: Reads[TransportOffer] =
(JsPath \ "offers").read[List[List[Offer]]].map(x => TransportOffer(x.flatten))
}
Then calling:
Json.parse(jsonString).validate[TransportOffer].foreach(println)
outputs:
TransportOffer(List(Offer(1234)))
Code run at Scastie
Explicitly writing Reads:
implicit val transportOfferReads: Reads[TransportOffer] = (json: JsValue) => {
json \ "offers" match {
case JsUndefined() =>
JsError("offers undefined")
case JsDefined(value) =>
value.validate[List[List[Offer]]].map(x => TransportOffer(x.flatten))
}
Code run at Scastie.
First transform the json, into the model you'd like. For that define a transformer:
val jsonTransformer = (__ \ "offers").json
.update(__.read[JsArray].map{o => {
JsArray(o.value.flatMap(_.asOpt[JsArray].map(_.value)).flatten)
}})
Then, assuming we have the case classes and their formatters:
case class Offer(id: String)
object Offer {
implicit val format: OFormat[Offer] = Json.format[Offer]
}
case class TransportOffer(offers: List[Offer])
object TransportOffer {
implicit val format: OFormat[TransportOffer] = Json.format[TransportOffer]
}
We can call:
Json.parse(jsonString).transform(jsonTransformer) match {
case JsSuccess(value, _) =>
value.validate[TransportOffer].foreach(println)
case JsError(errors) =>
println(errors)
???
}
Output is:
TransportOffer(List(Offer(1234)))
Code run at Scastie.

How to parse dynamic JSON with Circe

I'm trying to parse JSON where same field can be either array or object. Same as, specific field can be either string or number. Please consider examples below.
Empty object
{
"technicalData": {}
}
Collection with field being either string or number
{
"technicalData": [
{
"techValueString": "0.173"
},
{
"techValueString": 0.173
}
]
}
How can I do it with Circe mapping to Scala classes accepting Nil when data is {}?
case class Response(technicalData: Seq[TechnicalData])
case class TechnicalData(techValueString: String)
Thanks.
Here is less verbose solution applying Circe's Decoders
case class Response(technicalData: Seq[TechnicalData])
case class TechnicalData(techValueString: String)
class StringToResponse() extends (String => Response) {
implicit val responseDecoder: Decoder[Response] = Decoder.instance { c =>
for {
technicalData <- c.downField("technicalData").focus match {
case None => Right(Nil)
case Some(seq) => seq.asArray match {
case None => Right(Nil)
case Some(_) => c.get[Seq[TechnicalData]]("technicalData")
}
}
} yield {
Response(technicalData)
}
}
implicit val technicalDataDecoder: Decoder[TechnicalData] = (
Decoder.instance(_.get[String]("techValueString")).or(
Decoder.instance(_.get[Double]("techValueString").map(_.toString))
)
) mapN TechnicalData
override def apply(body: String): Response = {
decode[Response](body) match {
case Right(response) => response
case Left(e) => throw new RuntimeException(e)
}
}
}
Hope this will help someone who would come across similar problem.
This is a really verbose way of resolving your problem but I hope it has the advantage of letting you identify, or even rectify, every limit cases, which you might need:
import io.circe._
import io.circe.parser.parse
case class Response(technicalData: Seq[TechnicalData])
case class TechnicalData(techValueString: String)
val stringAsJson1 = """{
"technicalData": {}
}"""
val stringAsJson2 = """{
"technicalData": [
{
"techValueString": "0.173"
},
{
"techValueString": 0.173
}
]
}"""
def manageTechnicalDataAsArray(jsonArray: Vector[io.circe.Json]): Response = {
Response(
jsonArray.map(cell => {
val value = cell.asObject
.getOrElse(throw new Exception("technicalData as a array should have each cell as an object"))
.apply("techValueString")
.getOrElse(throw new Exception("techValueString should be a key of any cell under technicalData array"))
TechnicalData(value.asNumber
.map(_.toString)
.getOrElse(
value.asString
.getOrElse(throw new Exception("techValueString value should be either string or number"))
)
)
}
)
)
}
def manageTechnicalDataAsObject(jsonObject: io.circe.JsonObject): Response = {
jsonObject.toIterable match {
case empty if empty.isEmpty => Response(Nil)
case _ => throw new Exception("technicalData when object should be empty")
}
}
def parseResponse(jsonAsString: String): Response = {
parse(jsonAsString).getOrElse(Json.Null)
.asObject
.map(_("technicalData")
.getOrElse(throw new Exception("the json should contain a technicalData key"))
.arrayOrObject(throw new Exception("technicalData should contain either an objet or array"),
manageTechnicalDataAsArray,
manageTechnicalDataAsObject
)
).getOrElse(throw new Exception("the json should contain an object at top"))
}
println(parseResponse(stringAsJson1))
println(parseResponse(stringAsJson2))
I might come with a shorter version soon but less indicative on limit cases. You can explore them with tweaked version of a good json of yours.
Hope it helps.
EDIT: Here is a shorter and cleaner solution than above, which come after #Sergey Terentyev well found one. Well, it might be less readeable somehow, but it tends to do the same thing with more or less way to handle limit cases:
// Structure part
case class TechnicalData(techValueString: String)
object TechnicalData {
def apply[T](input: T) = new TechnicalData(input.toString)
}
case class Response(technicalData: Seq[TechnicalData])
// Decoding part
import io.circe.{Decoder, parser, JsonObject, JsonNumber}
import io.circe.Decoder.{decodeString, decodeJsonNumber}
def tDDGenerator[C : Decoder]: Decoder[TechnicalData] = Decoder.forProduct1("techValueString")(TechnicalData.apply[C])
implicit val technicalDataDecoder: Decoder[TechnicalData] = tDDGenerator[String].or(tDDGenerator[JsonNumber])
implicit val responseDecoder: Decoder[Response] = Decoder[JsonObject]
.emap(_("technicalData").map(o => Right(o.as[Seq[TechnicalData]].fold(_ => Nil, identity)))
.getOrElse(Right(Nil))
.map(Response.apply))
// Test part
val inputStrings = Seq(
"""{
| "technicalData": [
| {
| "techValueString": "0.173"
| },
| {
| "techValueString": 0.173
| }
| ]
|}
""".stripMargin,
"""{
| "technicalData": {}
|}
""".stripMargin
)
inputStrings.foreach(parser.decode[Response](_).fold(println,println))

Building Reads converter and case class at runtime in Play Framework

I have a file that contains the following array of JSON objects:
[
{
"type": "home",
"number": 1111
},
{
"type": "office",
"number": 2222
},
{
"type": "mobile",
"number": 3333
}
]
In Play Framework 2.x I would define an implicit Reads converter to read the file and convert it to a Scala structure:
implicit val implicitRead : Reads[MyClass] = (
(JsPath \ "type").read[String] and
(JsPath \ "number").read[Int]
) (MyClass.apply _)
the Scala case class defined as:
case class MyClass (myType: String, myNumber: Int)
and parsing the JSON with:
val json = // file record content
json.validate[MyClass] match {
case s: JsSuccess[MyClass] => {
val myObject: MyClass = s.get
// do something with myObject
}
case e: JsError => {
// error handling flow
}
Now, my problem is that I know the structure of the JSON file only at runtime, not at compilation time. Is it possible to build both the implicit Reads converter and the case class at runtime?
Use case classes directly with play-json:
Change the case class to:
case class MyClass (`type`: String, number: Int)
Add the json-formatter to the companion object:
object MyClass {
implicit val format = Json.format[MyClass]
}
The validate function looks now:
val myClass = // file record content
json.validate[Seq[MyClass]] match {
case JsSuccess(myClasses, _) => myClasses
case e: JsError => // handle error case
}
That's all you need. If you are not happy with the parameter names, you can use a Wrapper case class.

How to feed JSON to CASE CLASS directly using functional programming in scala?

{
"cars": [{
"amount": 120.00,
"name": "Car1"
}, {
"amount": 245.00,
"name": "Car2"
}]
}
I am reading above JSON as following in my Controller
val body: JsObject = request.body.asInstanceOf[JsObject]
I am having following CASE CLASS
case class BIC(name: String, amount: Double)
I want to create List[BIC] objects by reading data from JSON [e.g. body] using Functional style
Use Play JSON.
Example:
case class Wrapper(cars: List[Bic])
case class BIC(name: String, amount: Double)
Then in your controller:
implicit val wrapperFormats = Json.format[Wrapper]
implicit val bICFormats = Json.format[BIC]
def postCars(): Action[JsValue] = Action(json.parse) { implicit request =>
request.body.validate[Wrapper] match {
case JsSuccess(obj, _) => {
//do something with obj.
}
case JsError(err) => {
BadRequest(
JsObject(
"error" -> err.toString
)
)
}
}
}
Please note that I am returning Action[JsValue] this is so JQuery will run success when using AJAX.
I hope this helps,
Rhys
another reference:
https://www.playframework.com/documentation/2.5.x/ScalaJsonCombinators
First, define two case classes for your model like this :
object Models {
case class Bic(name : String, amount : Double)
object Bic {
implicit val BicFormat = Json.format[Bic]
}
case class Cars(bics : List[Bic])
object Cars {
implicit val CarsFormat = Json.format[Cars]
}
}
You're using the Play Framework so you can use the JSON library.
In your controller, if you want to read the bics, you can do it like that :
def getCars = Action(parse.json) { request =>
request.body.validate[Cars] map { cars =>
// treat your cars ..
}
}

Nested document with reactive mongo and Scala

I'm trying to store a nested document in MongoDB through Scala. The document looks like:
Project {
"_id": ObjectId("528547370cf6e41449003512"),
"highLevelCode": NumberLong(3),
"description": [
{"_id": ObjectId("528547370cf6e41449003521"),
"lang": "en",
"desc": "desc in English"},
{"_id ": ObjectId("528547370cf6e41449003522"),
"lang": "fr",
"desc": "desc en francais"}],
"budget": NumberLong(12345)
}
Basically I want to store nested descriptions, which could be of multiple languages in the Project document.
The code I wrote is:
import reactivemongo.bson._
import reactivemongo.bson.handlers.{BSONWriter, BSONReader}
import reactivemongo.bson.BSONLong
import reactivemongo.bson.BSONString
case class LocaleText(
id: Option[BSONObjectID],
lang: String,
textDesc: String
)
object LocaleText {
implicit object LocaleTextBSONReader extends BSONReader[LocaleText] {
def fromBSON(document: BSONDocument): LocaleText = {
val doc = document.toTraversable
LocaleText(
doc.getAs[BSONObjectID]("_id"),
doc.getAs[BSONString]("lang").map(_.value).get,
doc.getAs[BSONString]("textDesc").map(_.value).get
)
}
}
implicit object LocaleTextBSONWriter extends BSONWriter[LocaleText] {
def toBSON(localText: LocaleText) = {
BSONDocument(
"_id" -> localText.id.getOrElse(BSONObjectID.generate),
"lang" -> BSONString(localText.lang),
"textDesc" -> BSONString(localText.textDesc)
)
}
}
}
case class Project(
id: Option[BSONObjectID],
description: List[LocaleText],
budget: Option[Long]
)
object Project {
implicit object ProjectReader extends BSONReader[Project]{
def fromBSON(doc: BSONDocument): Project = {
val document = doc.toTraversable
Project(
document.getAs[BSONObjectID]("_id"),
document.getAs[BSONArray]("description").map { values =>
values.values.toList.flatMap { case value =>
value match {
case v: LocaleText => Some(v.asInstanceOf[LocaleText])
case _ => None
}
}
}.getOrElse(List.empty),
document.getAs[BSONLong]("budget").map(_.value)
)
}
}
implicit object ProjectWriter extends BSONWriter[Project]{
def toBSON(project: Project): BSONDocument = {
BSONDocument(
"_id" -> project.id.getOrElse(BSONObjectID.generate),
"description" -> BSONArray(project.description)
).append(Seq(
project.budget.map(b => "budget" -> BSONLong(b))
).flatten:_*)
}
}
}
However, it gave me compilation error like
overloaded method value apply with alternatives: [error] (producer: reactivemongo.bson.Implicits.Producer[(String, reactivemongo.bson.BSONValue)],producers: reactivemongo.bson.Implicits.Producer[(String, reactivemongo.bson.BSONValue)])reactivemongo.bson.AppendableBSONDocument
[error] (els: (String, reactivemongo.bson.BSONValue))reactivemongo.bson.AppendableBSONDocument
[error] cannot be applied to ((String, reactivemongo.bson.BSONObjectID), List[LocaleText])...
Basically Scala doesn't like the line
"description" -> BSONArray(project.description)
However, the following alternative works although I cannot use a List/Array to allow more than two languages:
case class LocaleText(
enDesc: String,
frDesc: String)
case class Project(
id: Option[BSONObjectID],
description: LocaleText)
object Project {
implicit object LocaleTextBSONReader extends BSONReader[LocaleText] {
def fromBSON(document: BSONDocument): LocaleText = {
val doc = document.toTraversable
LocaleText(
doc.getAs[BSONString]("enDesc").map(_.value).get,
doc.getAs[BSONString]("frDesc").map(_.value).get
)
}
}
implicit object LocaleTextBSONWriter extends BSONWriter[LocaleText] {
def toBSON(localText: LocaleText) = {
BSONDocument(
"enDesc" -> BSONString(localText.enDesc),
"frDesc" -> BSONString(localText.frDesc)
)
}
}
implicit object ProjectReader extends BSONReader[Project]{
def fromBSON(doc: BSONDocument): Project = {
val document = doc.toTraversable
Project(
document.getAs[BSONObjectID]("_id"),
document.getAs[BSONString]("iatiId").map(_.value).get,
LocaleTextBSONReader.fromBSON(document.getAs[BSONDocument]("description").get)
}
}
implicit object ProjectWriter extends BSONWriter[Project]{
def toBSON(project: Project): BSONDocument = {
BSONDocument(
"_id" -> project.id.getOrElse(BSONObjectID.generate),
"iatiId" -> BSONString(project.iatiId),
"description" -> LocaleTextBSONWriter.toBSON(project.description)
}
}
How can I convert project.description, which a List of LocaleText to BSONArray for Mongo? I appreciate if you can shed some light on my problem. Thank you very much for your help.
Finally I found the solution to my own question, hope this will help some others who struggle with ReactiveMongo 0.8 as well:
case class LocaleText(
lang: String,
desc: String)
case class Project(
id: Option[BSONObjectID],
descriptions: List[LocaleText])
object Project {
implicit object LocaleTextBSONReader extends BSONReader[LocaleText] {
def fromBSON(document: BSONDocument): LocaleText = {
val doc = document.toTraversable
LocaleText(
doc.getAs[BSONString]("lang").get.value,
doc.getAs[BSONString]("desc").get.value
)
}
}
implicit object LocaleTextBSONWriter extends BSONWriter[LocaleText] {
def toBSON(localText: LocaleText) = {
BSONDocument(
"lang" -> BSONString(localText.lang),
"desc" -> BSONString(localText.desc)
)
}
}
implicit object ProjectReader extends BSONReader[Project]{
def fromBSON(doc: BSONDocument): Project = {
val document = doc.toTraversable
Project(
document.getAs[BSONObjectID]("_id"),
document.getAs[BSONArray]("descriptions").get.toTraversable.toList.map { descText =>
LocaleTextBSONReader.fromBSON(descText.asInstanceOf[TraversableBSONDocument]
}
)
}
}
implicit object ProjectWriter extends BSONWriter[Project]{
def toBSON(project: Project): BSONDocument = {
BSONDocument(
"_id" -> project.id.getOrElse(BSONObjectID.generate),
"descriptions" -> BSONArray(project.descriptions.map {
description => LocaleTextBSONWriter.toBSON(description)
}: _*)
}
}
It might be an issue in the library. I tested your code using the latest version of reactivemongo and it compiled just fine (I needed to adapt your code to fit the new syntax for BSONReaders and BSONWriters but that shouldn't have any influence on the error).
Using reactivemongo 0.10.0 you can even use the newly provided macros:
import reactivemongo.bson._
case class LocaleText(id: Option[BSONObjectID], lang: String, textDesc: String)
object LocaleText {
implicit val localeTextBSONHandler = Macros.handler[LocaleText]
}
case class Project(id: Option[BSONObjectID], description: List[LocaleText], budget: Option[Long])
object Project {
implicit val projectBSONHandler = Macros.handler[Project]
}