I have a large json object: myNestedObject
{
"size": 2,
"values": [{
"name": "mullock",
"upstatus": "Green",
"details": {
"key": "rupture farms",
"server": "mudos",
"owner": "magog_cartel",
"type": "NORMAL",
"links": {
"self": [{
"address": "https://mudos.com:port/access"
}]
}
}
},{
"name": "tassadar",
"upstatus": "Orange",
"details": {
"key": "archon",
"server": "protoss",
"owner": "aspp67",
"type": "NORMAL",
"links": {
"self": [{
"address": "https://aiur.com:port/access"
}]
}
}
}],
"limit": 100
}
This is what the case classes look like as follows:
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper
case class Result(size: Int, limit: Int, values: Seq[Values])
case class Values(name: String, upstatus: String, details: ValuesDetails)
case class ValuesDetails(key: String, server: String, owner: String, `type`: String, links: ValuesDetailsLinks)
case class ValuesDetailsLinks(self: Seq[ValuesDetailsLinksAddress])
case class ValuesDetailsLinksAddress(address: String)
object Foo {
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
def test(Json): Unit
val return = mapper.readValue[Result](Json)
myNestedObject.size
is a field that gives the length of the array
myNestedObject.values(i)
Trying to extract the below values into a List[String]
myNestedObject.values(i).name
I've been using a crude for loop to extract and the code works.
val selectNames: List[String] = (
for (i <- 0 to myNestedObject.size-1 toList) yield
myNestedObject.values(i).name
)
Refactoring my code and trying to use a .map (unsuccessfully) to do the same thing, two attempts:
myNestedObject.map(_ => values(i).name)
myNestedObject.(values(i).name).asInstanceOfList
Disclaimer: I'm a complete novice at this.
SOLUTION: the values list can be accessed without specifying index
myNestedObject.values.map(_.name)
I think you could do this:
myNestedObject.values.map(_.name)
Dylan Grald's answer is correct, but you could also use an equivalent for-comprehension
for (x <- myNestedObject.values)
yield (x.name)
This desugars to the version using the map method. For simple cases like this I prefer just calling the map method directly, but I thought I would mention the for version as an alternative.
Related
i have a function that exctract from a json list of elements based on the path i give it.
the func looks like this:
def findAllValuesAtPath(jsValue: JsObject, path: String): List[JsValue] = {
val jsPath = JsPath(path
.split("\\[\\*]\\.")
.flatMap(s => s.split("\\.")
.map(RecursiveSearch)
).toList)
jsPath(jsValue)
}
for example:
{
"person": {
"kids": [
{
"name": "josh",
"age": 5
},
{
"name": "julia",
"age": 13
}
]
}
}
now if i give the path - "person.kids[*].name" I will get list of their names List("josh", "julia") which its what i want.
but if the list of kids were just simple list like:
{
"person": {
"kids": [
"josh",
"julia"
]
}
}
and i will give the path - "person.kids[*]" I will get empty list List() and I want to get this as List("josh", "julia") (without the brakets)
do you see any way to improve my func to handle both cases?
Array example:
[
{
"name": "John"
},
{
"name": "Joseph"
},
{
"name": "Peter"
}
]
I'd like to filter off objects with names which are not starting with Jo:
[
{
"name": "John"
},
{
"name": "Joseph"
}
]
The result might be a String or JValue with json array inside.
I was not able to find a direct JSON query mechanism in json4s hence created a case class.
Mappd the JSON -> filtered it -> wrote it back to JSON
import org.json4s.jackson.JsonMethods.parse
import org.json4s.jackson.Serialization
import org.json4s.native.Serialization.write
import org.json4s.{Formats, ShortTypeHints}
object JsonFIlter {
def main(args: Array[String]): Unit = {
implicit val formats: AnyRef with Formats = Serialization.formats(ShortTypeHints(List(classOf[PersonInfo])))
val parseJson :List[PersonInfo] = parse("""[
| {
| "name": "John"
| },
| {
| "name": "Joseph"
| },
| {
| "name": "Peter"
| }
|]""".stripMargin)
.extract[List[PersonInfo]]
val output = write(parseJson.filter(p => p.name.startsWith("Jo")))
println(output)
}
}
case class PersonInfo(name: String)
I have JSON like this:
"data": {
"project": {
"activityChildren": [
{
"id": 2,
"parents": [
{
"id": 1
}
]
},
]
}
}
I'd like to decode this to List[(Long, List[Long])] with circe-optics. I got as far as:
val activityParents: Map[Long, List[Long]] = root.data.activityChildren.each.json
.getAll(json)
.flatMap { activity =>
root.id.long.getOption(activity).map(_ -> root.parents.each.long.getAll(activity))
}
.toMap
I wonder whether it's possible to define a single lens for this that just turns the JSON into the desired map without explicitly mapping over the intermediate array. If so, how?
I've a nested json data with nested fields that I want to extract and construct a Scala Map.
Heres the sample JSON:
"nested_field": [
{
"airport": "sfo",
"score": 1.0
},
{
"airport": "phx",
"score": 1.0
},
{
"airport": "sjc",
"score": 1.0
}
]
I want to use saveToES() and construct a Scala Map to index the field into ES index with mapping as below:
"nested_field": {
"properties": {
"score": {
"type": "double"
},
"airport": {
"type": "keyword",
"ignore_above": 1024
}
}
}
The json file is read into the dataframe using spark.read.json("example.json"). Whats the right way to construct the Scala Map in this case?
Thanks for any help!
You can do it using the below sample code
import org.json4s.DefaultFormats
import org.json4s.jackson.JsonMethods.parse
case class AirPortScores(airport: String, score: Double)
case class JsonRulesHandler(airports: List[AirPortScores])
val jsonString: String = """{"airports":[{"airport":"sfo","score":1},{"airport":"phx","score":1},{"airport":"sjc","score":1}]}"""
def loadJsonString(JsonString: String): JsonRulesHandler = {
implicit val formats: DefaultFormats.type = org.json4s.DefaultFormats
parse(JsonString).extract[JsonRulesHandler]
}
val parsedJson: JsonRulesHandler = loadJsonString(jsonString)
parsedJson.airports.foreach(println)//you can select parsedJson.airport or scores
//below ouput
AirPortScores(sfo,1.0)
AirPortScores(phx,1.0)
AirPortScores(sjc,1.0)
I am using the facebook graph API and the responses look similar to this:
{
"data": [
{
"id": "311620272349920_311718615673419",
"from": {
"id": "1456046457993048",
"name": "Richard Ettinson"
},
"to": {
"data": [
{
"id": "311620272349920",
"name": "Barbara Fallerman"
}
]
},
"with_tags": {
"data": [
{
"id": "311620272349920",
"name": "Barbara Fallerman"
}
]
},
"message": "I was gong out with her",
"actions": [
{
"name": "Comment",
"link": "https://www.facebook.com/311620272349920/posts/311718615673419"
},
{
"name": "Like",
"link": "https://www.facebook.com/311620272349920/posts/311718615673419"
}
]
}
I managed to for example extract the from field through
val extracted = (json \ "data" \"from").extract[PostFrom]
But I worry that if I use this technique I will need to pass over the Json multiple times to extract all the values I need which could lead to a bad performance.
How exactly could I extract these fields into case classes from the array of non similar objects?
I tried with the following case classes:
abstract class BaseResponse()
case class Data(list:List[Post])
case class Post(id: String, post: PostFrom) extends BaseResponse
case class PostFrom(id: String, name:String)
Which always lead to an empty List, is there a way to get back a Data class which has a list of certain classes which I am interested in? (As example the top level id, from and with_tags)
A possibility I found was to use more case classes instead of inheritance:
case class Root[T](data:Option[T])
case class Post(id: String, from: From, message: String)
case class From(id: String, name:String)
Basically there has to be a root object which takes some kind of graphs response object, additionally it is optional so that it won't throw an exception if there was a problem with the parsing of the response.
I then used it in the following way:
val body = r.entity.asString
val json = parse(r.entity.asString)
val root = json.extract[Root[Post]]
root.data match {
case Some(post) =>
val tagger = Tagger(post.from.id, post.from.name, post.id, post.message)
log.info(s"We received $tagger")
originalSender ! RetrievedTagger(tagger)
case None => originalSender ! NoTaggerFound
}