I'm trying to make a transformation to every object in an array in my JSON.
The JSON looks something like this:
{
"foos": [
{
"urls": ["www.google.com", "www.google.com", "www.stackoverflow.com"]
},
{
"urls": ["www.facebook.com"]
},
...
]
}
I'm trying to remove duplicates from the urls array.
I've tried using something like this:
(__ \\ 'urls).json.update(Reads.of[JsArray].map(scopes => JsArray(scopes.value.distinct)))
But keep getting an error about error.path.result.multiple.
I can't seem to find a way to make the change to everything inside of a JSON array.
Also, I can't used case classes here because there are unknown fields on some of the data that I don't want to lose when converting to a case class.
In order to resolve that, we need to define 2 transformers. One acting on the internal object, to remove duplicates, and the other, to aggregate all objects.
The first one is:
val distinctTransformer = (__ \ "urls").json
.update(__.read[JsArray].map{o => {
JsArray(o.value.distinct)
}})
The other one is:
val jsonTransformer = (__ \ "foos").json
.update(__.read[JsArray].map(arr => {
JsArray(arr.value.map(_.transform(distinctTransformer))
.filter(_.isSuccess) // Error handling should be added here
.map(_.get)
)
}))
Then the usage is:
val json = Json.parse(jsonString)
json.transform(jsonTransformer) match {
case JsSuccess(value, _) =>
println(value)
case JsError(errors) =>
println(errors)
}
Code run at Scastie.
Related
I have the below json
"atrr": {
"data": {
"id": "asfasfsaf",
"name": "cal",
"type": "state"
"ref": [
"xyz",
"uhz",
"arz"
]
}
}
I am reading this as below but not getting value k,v
def getData: Map[String, String] = (atrr \ "data").asOpt[Map[String, String]].getOrElse(Map[String, String]())
without ref it works fine.how do I ignore ref[] from json in code that is an object
You can use a custom Reads[Map[String, String]], passed to the .as or .asOpt method. My approach is to use Reads.optionNoError[String] to handle the values inside the main atrr object, so that any non-string field which would have caused an error will instead be treated as a None.
// explicitly construct something that'd normally be constructed implicitly
val customReads: Reads[Map[String, Option[String]] =
Reads.map(Reads.optionNoError[String])
// pass the custom reads explicitly with parens,
// instead of passing the expect type in square brackets
(atrr \ "data").asOpt(customReads)
This results in
Some(
Map(
id -> Some(asfasfsaf),
name -> Some(cal),
type -> Some(state),
ref -> None
)
)
which you can transform how you see fit, for example by doing
.map(_.collect { case (k, Some(v)) => k -> v })
In the following function, I am parsing a json. I want to return JsError which describes all the errors (if any) in the json. But I am unable to figure out how to keep changing the JsError without using a var. I am looking for a more functional way of writing the code.
implicit object PQReads extends Reads[PQ] {
def reads(json:JsValue):JsResult[PQ] = {
val id = (json \ "id").asOpt[UUID]
val d = (json \ "d").asOpt[String]
val h = (json \"h").asOpt[List[String]]
val i = (json \ "i").asOpt[List[String]]
//HERE I WANT TO ADD LOGIC TO CHECK THAT ALL THE OPTIONS ARE DEFINED.
//THE ONES WHICH ARE NONE SHOULD BE INDICATED BACK IN JSERROR
// (EG D AND I FIELDS ARE MISSING). HOW DO I DO THIS WITHOUT
// USING VAR TYPE FOR JSERROR.
//IF ALL THE FIELDS ARE PRESENT THEN I'LL RETURN PQ INSTANCE IN JSSUCCESS
}
I have not used Play for quite some time, but I don't think you have to do the reading manually.
When you have case class PQ(id: UUID, d :String, h: List[String], i: List[String])
,
you can simply write implicit val pqReads = Json.reads[PQ].
See here for the documentation on the automated mapping.
But if you really want to do it yourself, take a look at the reading combinators.
implicit val pqReads: Reads[PQ] = (
(JsPath \ "id").read[UUID] and
(JsPath \ "d").read[String] and
(JsPath \ "h").read[List[String]] and
(JsPath \ "i").read[List[String]]
)(PQ.apply _)
For the conversions,
val validJson = Json.parse("""
{
"id": "586c154d-1e0f-428c-97bc-200dec9328bb",
"d": "d",
"h": ["h", "i"],
"i": []
}
""")
println(validJson.validate[PQ])
val invalidJson = Json.parse("""
{
"id": "586c154d-1e0f-428c-97bc-200dec9328bb",
"d": "d",
"h": 123
}
""")
println(invalidJson.validate[PQ])
both of the implicit vals provide the same result.
JsSuccess(PQ(586c154d-1e0f-428c-97bc-200dec9328bb,d,List(h, i),List()),)
JsError(List((/i,List(JsonValidationError(List(error.path.missing),WrappedArray()))), (/h,List(JsonValidationError(List(error.expected.jsarray),WrappedArray())))))
With play-json I use always case-classes!
I simplified your problem to the essence:
import play.api.libs.json._
Define a case class
case class PQ(id:UUID, d:String, h:List[String], i: List[String])
Add a formatter in the companion object:
object PQ {
implicit val jsonFormat: Format[PQ] = Json.format[PQ]
}
And use validate:
Json.parse(YOUR_JSON).validate[PQ] match {
case JsSuccess(pq, _) => println(pq)
case JSError(errors) => println(s"Handle exception $other")
}
This returns either the PQ or a List with the errors. No need to do anything here.
In short I'm dealing with JSON that has to look like:
{
"data": JSON.stringify(...), // eg. JSON.stringify(null) or JSON.stringify({p: "v"})
...
}
I wish to parse it into a case class resembling something like:
case class Foo(data: JsObject, ...)
And the Reads[Food] I've defined looks like this:
val fooReads: Reads[Foo] = (
(__ \"data").readNullable[String].map(_.filter(s => s != null && !s.isEmpty).map(Json.parse(_).as[JsObject]).getOrElse(JsObject(Seq()))),
...
)
But I keep getting the following error:
ValidationError(error.expected.jsobject,WrappedArray())
I'm pretty sure my Reads[Foo] is now resembling an ugly tumor that doesn't need to exist, so some help would be nice.
This? I did use corrected version of JSON above.You had readNullable above but case class data was not an option so I removed it.
implicit val reads = (__ \ "data").read[JsObject].map { data => Foo(data)}
More of an FYI but Play JSON Combinators don't work with case classes with one field: How to turn json to case class when case class has only one field
Nullable version:
case class Foo(data: Option[JsObject])
implicit val reads:Reads[Foo] = (__ \ "data").readNullable[JsObject].map { data => Foo(data)}
Updated version to read serialized version of JSON string:
implicit val reads:Reads[Foo] = (__ \ "data").readNullable[String].map{
case Some(x) => Foo(data = Json.parse(x).asOpt[JsObject])
case _ => Foo(data=None)
}
I kept readNullable for the String read so library would handle checking if String was empty or null.
The solution I went for, given the following JSON (with more than one property):
{
"data": "null", // JSON.stringify({p: "v"}) or JSON.stringify(null)
"fullName": "JohnDoe"
}
Scala code:
implicit val fooReads: Reads[Foo] = (
(__ \ "data").read[String].map(Json.parse(_).asOpt[JsObject]) and
(__ \ "fullName").read[String]
)
case class Foo(data: Option[JsObject], fullName: String)
Still not handling the case of a missing data property and I'm not sure if there are any disadvantages to using asOpt[JsObject] so will likely need to refine this going forwards.
I'm writing a library to convert JSON responses from an API for backwards compatibility reasons. And what I need to do is take in arbitrary JSON, and change certain field names. I'm using scala and argonaut, but I don't see any way in the docs or examples of changing the FIELD names, only the values.
I don't know of a particularly nice way to do this, but it's not too awful to write a helper that will replace a field in an object and then use that in a cursor with withObject:
def renameField(before: JsonField, after: JsonField)(obj: JsonObject) =
obj(before).map(v => (obj - before) + (after, v)).getOrElse(obj)
Parse.parseOption("""{ "a": { "b": { "c": 1 } } }""").flatMap { json =>
(json.hcursor --\ "a").withFocus(_.withObject(renameField("b", "z"))).undo
}
This will return Some({"a":{"z":{"c":1}}}) as expected.
I ended up folding over the object I need to convert and adding to a map, and then creating a new json object.
val conversionMap = Map("a" -> "b")
Json(
j.objectOrEmpty.toMap.foldLeft(Map.empty[JsonField, Json]) {
case (acc, (key, value)) =>
acc.updated(conversionMap.getOrElse(key, key), j.fieldOrNull(key))
}.toSeq: _*
)
I'm using the Specs2 JSONMatcher to validate that a GET request is being correctly converted from its internal representation (there are some manipulations we do before generating the JSON). What I need to do is, make sure that an element in the JSON array matches the corresponding object from our repository.
What I've tried:
val response = response.entity.asString // Spray's way of getting the JSON response
repository.all.map { obj =>
resp must */ ("id" -> obj.id)
resp must */ ("state" -> generateState(obj)
}
The problem is that the */ matcher just finds that "state": "whatever" (assuming generateState returns "whatever") exists somewhere in the JSON document, not necessarily in the same one matched by the ID
I tried using the indices but the repository.all method doesn't always return the elements in the same order, so there's no way of matching by index.
What I'd like to do is, iterate over the elements of the JSON array and match each one separately. Say an /## operator (or something) that takes matchers for each element:
resp /## { elem =>
val id = elem("id")
val obj = repository.lookup(id)
elem /("state" -> generateState(obj))
}
Does anyone have a way to do this or something similar?
Probably the easiest thing to do for now (until a serious refactoring of JsonMatchers) is to do some parsing and recursively use a JsonMatchers in a Matcher[String]:
"""{'db' : { 'id' : '1', 'state' : 'ok_1'} }""" must /("db" -> stateIsOk)
// a string matcher for the json string in 'db'
def stateIsOk: Matcher[String] = { json: String =>
// you need to provide a way to access the 'id' field
// then you can go on using a json matcher for the state
findId(json) must beSome { id: String =>
val obj = repository.lookup(id)
json must /("state" -> generate(obj))
}
}
// I am using my own parse function here
def findId(json: String): Option[String] =
parse(json).flatMap { a =>
findDeep("id", a).collect { case JSONArray(List(v)) => v.toString }
}
// dummy system
def generate(id: String) = "ok_"+id
case object repository {
def lookup(id: String) = id
}
What I did in the end is use responseAs[JArray], JArray#arr and JObject#values to convert the JSON structures into Lists and Maps, and then used the standard List and Map matchers. Much more flexible.