serialization issue in a field that can be of type int or of type List[String], it is a field that in each element is a list that always has 2 elements in the list the first element the value field is always a list [ String] and in the second element it is always of type int; the issue is in the value field and it fails to infer it because of this behavior.
body:
json response:
{
"custom_fields": [
{
"id": "d0c016df-e09a-492e-a7a2-cc92e1993627",
"name": "Sprint",
"type": "labels",
"date_created": "1586927852599",
"hide_from_guests": false,
"value": [
"64e19188-8440-4b35-8459-d49348c92e55"
],
"required": false
},
{
"id": "4513e77f-9866-4139-bcc5-458945db0deb",
"name": "Story Points",
"type": "drop_down",
"date_created": "1579556449403",
"hide_from_guests": false,
"value": 1,
"required": false
}
]
}
issue on field:
value
scala cire code serializer:
implicit val decodeCustomFields: Decoder[CustomFieldsItem] = new Decoder[CustomFieldsItem] {
final def apply(c: HCursor): Decoder.Result[CustomFieldsItem] =
for {
id <- c.downField("id").as[Option[String]]
name <- c.downField("name").as[Option[String]]
typeF <- c.downField("type").as[Option[String]]
date_created <- c.downField("date_created").as[Option[String]]
hide_from_guests <- c.downField("hide_from_guests").as[Option[Boolean]]
value <- c.downField("value").as[Option[List[String]]] // here it fails !!
required <- c.downField("required").as[Option[Boolean]]
} yield {
CustomFieldsItem( id, name, typeF, date_created, hide_from_guests, value , required )
}
}
scala unmarshal code for http response, it takes the circe serializer :
private def runRequest(req: HttpRequest): Future[CustomFieldsItem] =
Http()
.singleRequest(req)
.flatMap {
res =>
val ser = Unmarshal(res).to[CustomFieldsItem]
ser
}
message error on unmarshal:
An error has occurred: C[A]: DownField(value),MoveRight,DownArray,DownField(custom_fields),DownArray,DownField(tasks)
How could I improve or fix this serialization?
Make CustomFields be of type (CustomField[List[String]], CustomField[Int]) with a generic typed CustomField case class ?
The issue is that your modelisation doesn't reflect the reality of what you're receiving, and you're trying to fix that afterwards. If it always has 2 elements, it's not a list, it's a Tuple.
Related
Given incoming json like below, how can i decode it the given case class based on condition.
Incoming JSON
{
"config": {
"files": ["welcome"],
"channel": "media"
}
}
Case Classes
case class File(`type`: String, value: String)
case class Config(files: List[File],
channel: String = "BC")
object Config{
implicit val FileDecoder: Decoder[File] = deriveDecoder[File]
implicit val ConfigDecoder: Decoder[Config] = deriveDecoder[Config]
}
case class Inventory(config: Config)
object Inventory {
implicit val InventoryDecoder: Decoder[Inventory] = deriveDecoder[Inventory]
}
I do not have control on incoming files values in json it can be List[String] or List[File] so i need to handle both cases in my decoding logic.
So as we can see above my aim is to check if the incomes files values is List[String] then transform that values to below where type is hardcoded to "audio"
"files": [{
"type": "audio",
"value": "welcome.mp3"
}],
The overall json should look like below before it mapped into case classes for auto decoding.
{
"config": {
"files": [{
"type": "audio",
"value": "welcome.mp3"
}],
"channel": "media"
}
}
what i understood that this can be achieved either by transforming the json before decoding or can also be achieved during the decoding of files.
I tried writing the decoding logic at File level but i am not able to succeed. I am not getting the crux of how to do this.
Tried code
implicit val FileDecoder: Decoder[File] = deriveDecoder[File].prepare { (aCursor: ACursor) =>
{
if(!aCursor.values.contains("type")){
aCursor.values.map( v =>
Json.arr(
Json.fromFields(
Seq(
("type", Json.fromString("audio")),
("value", v.head)
)
)
)
)
}
}
}
We can use a custom Decoder for File to provide a default value to type
final case class File(`type`: String, value: String)
object File {
implicit final val FileDecoder: Decoder[File] =
Decoder.instance { cursor =>
(
cursor.getOrElse[String](k = "type")(fallback = "audio"),
cursor.get[String](k = "value")
).mapN(File.apply)
}.or(
Decoder[String].map(value => File(`type` = "audio", value))
)
}
Which can be used like this:
val data =
"""[
{
"type": "audio",
"value": "welcome.mp3"
},
{
"value": "foo.mp3"
},
"bar.mp3"
]"""
parser.decode[List[File]](data)
// res: Either[io.circe.Error, List[File]] =
// Right(List(
// File("audio", "welcome.mp3"),
// File("audio", "foo.mp3"),
// File("audio", "bar.mp3")
// ))
You can see the code running here.
I am trying to read below json structure using scala code but i am not able to read it as case class object.
"a": {
"source": [
{
"name": "test",
"basePath": "/Users/bounce/anuj/data/test",
"granularity": "None"
},
{
"name": "test1",
"basePath": "/Users/bounce/anuj/data/test1",
"granularity": "None"
}
]
}
}
Below is the code that i am using :
val mapper = new ObjectMapper().registerModule(DefaultScalaModule)
val mapData = mapper.readValue(source, classOf[Map[String,FactConfig]])
#JsonInclude(Include.NON_EMPTY)
case class Source(name: String,
basePath: String,
granularity: String)
case class FactConfig(
source : List[Source])
val d = mapData.get("a").asInstanceOf[FactConfig]
When I m trying to convert it to FactConfig object I am getting below exception
Exception in thread "main" java.lang.ClassCastException: scala.collection.immutable.Map$Map1 cannot be cast to com.bounce.dp.etl.config.FactConfig
Please suggest to me what I am doing wrong.
In both Java and Scala world we normally lose generic type information during compilation. By using classOf operator jackson is not aware what should it do with map keys and values. The workaround is to use com.fasterxml.jackson.core.type.TypeReference, namely
val mapData = mapper.readValue(source, new TypeReference[Map[String, FactConfig]] {})
I have a project in akka and circe. I am currently sending this json in the response body:
{ "a": 33, "things": [{ "id1": ["2"], "id2": [], "id4": [], "id5": [] } ] }
I am trying to extract it to this case class:
case class Body(things: Option[List[Things]])
case class Things(id1: Option[List[String]] = None, id2: Option[List[String]] = None, id3: Option[List[String]] = None, id4:Option[List[String]] = None)
I am doing this using this code :
entity(as[Body]).as(Body.getIds)
This code relies on the below companion object
object Body extends ErrorAccumulatingCirceSupport {
def getIds(body: Body): List[String] =
body.things.get.flatMap(_.id4.getOrElse(List()))
}
However, unless I actually pass in a value in the objects within the things array (e.g. "id2": [ "33"])
It never finishes unmarshalling the json. Grateful for any advice!
I have a json like below,
I have json like below I need to extract the value from them
{
"filed1": "value1",
"message": {
"payload": [{
"type": ["Extra","ID"],
info": {
"value": 8
}
}, {
"type": ["Free"],
info": {
"value": 99
}
}, {
"type": ["Actual"],
info": {
"value": 100
}
}]
},
"code": "0000"
}
{
"filed1": "value1",
"message": {
"payload": [{
"type": ["Extra", "new"],
"value": 9
}]
},
"code": "0001"
}
from the above two types of json .
If the input json has list of type keys then look for type field which has element Extra and get the value inside info
If the input json has one type key then check type field , if it has element Extra and get the direct va;ue
I am trying like below for type but it fails for list of types json, i.e first json input
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.json4s.JsonDSL._
val json = parse(myjsonString, true)
val field = compact(render(json \\ "type"))
val ok = field.contains("[\"Extra\"")
if(ok == true){
println("value " + compact(render(json \\ "value")))
}
You need to use json4s to do the work for you. Specifically, you need to use the extract method on the json to convert it into your particular data structure. Once you have done that you can process it as Scala types.
This is my attempt at matching the structure of your JSON with Scala case classes:
case class PayloadInfo(value: Int)
case class Payload(`type`: List[String], info: PayloadInfo)
case class Message(payload: List[Payload])
case class Full(filed1: String, message: Message, code: String)
implicit val formats = DefaultFormats
val json = parse(myjsonString, true)
val full = json.extract[Full]
val res = full.message.payload.filter(_.`type`.contains("Extra"))
[ The backticks around type are required because it is a reserved word ]
I may have got the types a bit wrong but this should give you the idea of how to go about it.
You can also process the JValues directly but that is going to be much messier.
This might be a novice question, I really apologies for that but I wasn't able to find it any where. Code and output mentioned.
Using Play Framework 2.3.9 (Scala)
// controller
object Products extends Controller {
def list = Action {
Ok(Json.obj("products" -> Product.all))
}
}
// Model
case class Product(id: Long, name: Option[String])
object Product {
implicit val format: Format[Product] = Json.format[Product]
def all = DB.withConnection { implicit c =>
SQL("SELECT * FROM Products;").apply().map { row =>
Product(
id = row[Long]("id"),
name = row[Option[String]]("name")
)
}.toList
}
}
output generated:
{
"products": [{
"id": 1,
"name": "test"
}, {
"id": 2
}]
}
output wanted:
{
"products": [{
"id": 1,
"name": "test"
}, {
"id": 2,
"name": null
}]
}
What type of json value are you after for None? Do you mean a string "None", or the json null value null? At any rate, the formats line:
implicit val format: Format[Product] = Json.format[Product]
creates an automatic json formatter for you, but you can write your own to output the json in any custom format, e.g.:
implicit val productWrites = new Writes[Product] {
def writes(product:Product) = Json.obj(
"id" -> product.id,
"name" -> product.name
)
}
which will produce a value null (JsNull), i.e. "name" : null. for a product like Product(1,None). The following:
implicit val productWrites = new Writes[Product] {
def writes(product:Product) = Json.obj(
"id" -> product.id,
"name" -> JsString(product.name.getOrElse("None"))
)
}
would produce a json string value for name, i.e "name":"None". More info available from the relevant page on the play website.
What you're looking for is an implementation of Writes[Product] like so;
object Product {
implicit val writes = new Writes[Product] {
def writes(p: Product): JsValue = Json.obj(
"id" -> p.id,
"name" -> p.name // will output null and not None (as in your question)
)
}
}