Working with nested maps from a JSON string - scala

Given a JSON string like this:
{"Locations":
{"list":
[
{"description": "some description", "name": "the name", "id": "dev123"},
{"description": "other description", "name": "other name", "id": "dev59"}
]
}
}
I'd like to return a list of "id"s from a function parsing the above string. JSON.parseFull() (from scala.util.parsing.json) gives me a result of type Option[Any]. Scala REPL shows it as Some(Map(Locations -> Map(list -> List(Map(id -> dev123, ... and as a beginner in Scala I'm puzzled as to which way to approach it.
Scala API docs suggest "to treat it as a collection or monad and use map, flatMap, filter, or foreach". Top-level element is an Option[Any] however that should be Some with a Map that should contain a single key "Locations", that should contain a single key "list" that finally is a List. What would be an idiomatic way in Scala to write a function retrieving the "id"s?

First of all, you should cast json from Any to right type:
val json = anyJson.asInstanceOf[Option[Map[String,List[Map[String,String]]]]]
And then you may extract ids from Option using map method:
val ids = json.map(_("Locations")("list").map(_("id"))).getOrElse(List())

Because Any is everywhere is the returned result, you'll have to cast. Using one of my earlier answers:
class CC[T] { def unapply(a:Any):Option[T] = Some(a.asInstanceOf[T]) }
object M extends CC[Map[String, Any]]
object L extends CC[List[Any]]
object S extends CC[String]
object D extends CC[Double]
object B extends CC[Boolean]
for {
Some(M(map)) <- List(JSON.parseFull(jsonString))
M(locMap) = map("Locations")
L(list) = locMap("list")
description <- list
M(desc) = description
S(id) = desc("id")
} yield id
// res0: List[String] = List(dev123, dev59)

For this type of tasks, you should take a look at Rapture.io. I'm also a scala beginner, but from what I've searched for, this seems to have the friendliest syntax. Here's a short example, taken from a gist:
import rapture.io._
// Let's parse some JSON
val src: Json = Json.parse("""
{
"foo": "Hello world",
"bar": {
"baz": 42
}
}
""")
// We can now access the value bar.baz
val x: Json = src.bar.baz
// And get it as an integer
val y: Int = x.get[Int]
// Alternatively, we can use an extractor to get the values we want:
val json""" { "bar": { "baz": $x }, "foo": $z }""" = src
// Now x = 42 and z = "Hello world".

Is this what you need? (using lift-json)
scala> import net.liftweb.json._
import net.liftweb.json._
scala> implicit val formats = DefaultFormats
formats: net.liftweb.json.DefaultFormats.type = net.liftweb.json.DefaultFormats$#79e379
scala> val jsonString = """{"Locations":
{"list":
[
{"description": "some description", "name": "the name", "id": "dev123"},
{"description": "other description", "name": "other name", "id": "dev59"}
]
}
}"""
jsonString: java.lang.String =
{"Locations":
{"list":
[
{"description": "some description", "name": "the name", "id": "dev123"},
{"description": "other description", "name": "other name", "id": "dev59"}
]
}
}
scala> Serialization.read[Map[String, Map[String, List[Map[String, String]]]]](jsonString)
res43: Map[String,Map[String,List[Map[String,String]]]] = Map(Locations -> Map(list -> List(Map(description -> some desc
ription, name -> the name, id -> dev123), Map(description -> other description, name -> other name, id -> dev59))))
scala> val json = parse(jsonString)
json: net.liftweb.json.package.JValue = JObject(List(JField(Locations,JObject(List(JField(list,JArray(List(JObject(List(
JField(description,JString(some description)), JField(name,JString(the name)), JField(id,JString(dev123)))), JObject(Lis
t(JField(description,JString(other description)), JField(name,JString(other name)), JField(id,JString(dev59))))))))))))
scala> json \\ "id"
res44: net.liftweb.json.JsonAST.JValue = JObject(List(JField(id,JString(dev123)), JField(id,JString(dev59))))
scala> compact(render(res44))
res45: String = {"id":"dev123","id":"dev59"}

In a branch of SON of JSON, this will work. Note that I'm not using the parser. Not that it doesn't exist. It's just that creating an JSON object using the builder methods is easier:
scala> import nl.typeset.sonofjson._
import nl.typeset.sonofjson._
scala> var all = obj(
| locations = arr(
| obj(description = "foo", id = "807",
| obj(description = "bar", id = "23324"
| )
| )
scala> all.locations.map(_.id).as[List[String]]
res2: List[String] = List(23324, 807)
Or use a for comprehension:
scala> (for (location <- all.locations) yield location.id).as[List[String]]
res4: List[String] = List(23324, 807)

Related

How to get the return value from For loop and pass it to .body(StringBody(session => in Gatling using Scala

How to get the return value from For loop and pass it to .body(StringBody(session => in Gatling using Scala
I have created a method with for loop to generate String Array in gatling with scala
def main(args: Array[String]): Unit = {
var AddTest: Array[String] = Array[String]()
for (i <- 0 to 3) {
val TestBulk: String =
s"""{ "name": "Perftest ${Random.alphanumeric.take(6).mkString}",
"testID": "00000000-0000-0000-0000-000000006017",
"typeId": "00000000-0000-0000-0000-000000011001",
"statusId": "00000000-0000-0000-0000-000000005058"};"""
AddTest = TestBulk.split(",")
// val TestBulk2: Nothing = AddTest.mkString(",").replace(';', ',')
// println(TestBulk)
}
}
now I want to pass the return value to .body(StringBody(session =>
.exec(http("PerfTest Bulk Json")
.post("/PerfTest/bulk")
.body(StringBody(session =>
s"""[(Value from the for loop).mkString(",").replace(';', ',')
]""".stripMargin)).asJson
Please help me with the possibilities
Please let me know if
You don't need for loop (or var or split) for this. You also do not have ; anywhere, so last replace is pointless.
val ids = """
"testId": "foo",
"typeId": "bar",
"statusId": "baz"
"""
val data = (1 to 3)
.map { _ => Random.alphanumeric.take(6).mkString }
.map { r => s""""name": "Perftest $r"""" }
.map { s => s"{ $s, $ids }" }
.mkString("[", ",", "]")
exec("foo").post("/bar").body(_ => StringBody(data)).asJson
(I added [ and ] around your generated string to make it look like valid json).
Alternatively, you probably have some library that converts maps and lists to json out-of-the box (I don't know gatling, but there must be something), a bit cleaner way to do this would be with something like this:
val ids = Map(
"testId" -> "foo",
"typeId" -> "bar",
"statusId" -> "baz"
)
val data = (1 to 3)
.map { _ => Random.alphanumeric.take(6).mkString }
.map { r => ids + ("name" -> s"Perftest $r") }
exec("foo").post("/bar").body(_ => StringBody(toJson(data))).asJson
This Worked for me
Thanks to
#dima .
I build this with your suggested method.
import scala.util.Random
import math.Ordered.orderingToOrdered
import math.Ordering.Implicits.infixOrderingOps
import play.api.libs.json._
import play.api.libs.json.Writes
import play.api.libs.json.Json.JsValueWrapper
val data1 = (1 to 2)
.map {r => Json.toJson(Map(
"name" -> Json.toJson(s"Perftest${Random.alphanumeric.take(6).mkString}"),
"domainId"->Json.toJson("343RDFDGF4RGGFG"),
"typeId"->Json.toJson("343RDFDGF4RGGFG"),
"statusId"->Json.toJson("343RDFDGF4RGGFG"),
"excludedFromAutoHyperlinking" ->Json.toJson(true)))}
println(Json.toJson(data1))```

json parsing using circe in scala

I'm trying to make use of circe for json parsing in scala. Can you please help me parse 'pocs' from the data in the case class as well? here is the code:
import io.circe.Decoder
import io.circe.generic.semiauto.deriveDecoder
import io.circe.parser
val json: String =
"""
{
"segmements": [
{
"tableName": "X",
"segmentName": "XX",
"pocs": [
"aa#aa.com",
"bb#bb.com"
]
},
{
"tableName": "Y",
"segmentName": "YY",
"pocs": [
"aa#aa.com",
"bb#bb.com"
]
}
]
}
"""
final case class TableInfo(tableName: String, segmentName: String)
object TableInfo {
implicit final val TableInfoDecoder: Decoder[TableInfo] = deriveDecoder
}
val result = for {
data <- parser.parse(json)
obj <- data.asObject.toRight(left = new Exception("Data was not an object"))
segmements <- obj("segmements").toRight(left = new Exception("Json didn't had the
segments key"))
r <- segmements.as[List[TableInfo]]
} yield r
println(result)
scastie link: https://scastie.scala-lang.org/BalmungSan/eVEvBulOQwGzg5hIJroAoQ/3
Just add parameter typed as collection of String:
final case class TableInfo(tableName: String, segmentName: String, pocs: Seq[String])
scastie

Scala JSON diffs

I have two JSONs with the exact same keys.
val json1 =
"""{
'name': 'Henry',
'age' : 26,
'activities' : {
'school': 'basketball club',
'after-school': 'chess'
}
}"""
val json2 =
"""{
'name': 'David',
'age' : 23,
'activities' : {
'school': 'baseball club',
'after-school': 'programming'
}
}"""
I would like the difference between the two JSONs, for example, such as:
name = Henry, David
age = 23, 26
activities.school= basketball club, baseball club
activities.after-school=chess, programming
It doesn't have to follow the above format but, I would like to get the keys and values that are differing.
You could try diffson, a circe based library: https://github.com/gnieh/diffson
Example:
import diffson._
import diffson.lcs._
import diffson.circe._
import diffson.jsonpatch._
import diffson.jsonpatch.lcsdiff._
import io.circe._
import io.circe.parser._
import cats._
import cats.implicits._
implicit val lcs = new Patience[Json]
val json1 = parse("""{
| "a": 1,
| "b": true,
| "c": ["test", "plop"]
|}""".stripMargin)
val json2 = parse("""{
| "a": 6,
| "c": ["test2", "plop"],
| "d": false
|}""".stripMargin)
val patch =
for {
json1 <- json1
json2 <- json2
} yield diff(json1, json2)
Would return:
[{
"op":"replace",
"path":"/a",
"value":6
},{
"op":"remove",
"path":"/b"
},{
"op":"replace",
"path":"/c/0",
"value":"test2"
},{
"op":"add",
"path":"/d",
"value":false
}]
Basically you can concatenate "add" section values for your exact purpose.
As was already suggested by #dk14 you can use diffson librarry: https://github.com/gnieh/diffson - but JsonPatch structure which it provides might be not very convenient for your use case, so it can be converted into another to get result in desired format.
Please, see some code example below:
import diffson.jsonpatch.{Add, JsonPatch, Remove, Replace}
import diffson.jsonpointer.{Part, Pointer}
import io.circe.Json
// Model representing plain json diff at certain path, that can be rendered at more human readable format
case class JsonPathDiff(path: Pointer, left: Option[Json], right: Option[Json]) {
def readableString: String = {
val pathReadableString: String = {
def partToString(part: Part): String = part.fold(identity, _.toString)
path.parts.toList.map(partToString).mkString(".")
}
def jsonReadableValue(json: Option[Json]): String = json.map(_.toString()).getOrElse("")
val leftValue = jsonReadableValue(left)
val rightValue = jsonReadableValue(right)
s"$pathReadableString = $leftValue , $rightValue"
}
}
// Model representing overall difference between two JSON's
case class JsonDiff(diff: List[JsonPathDiff]) {
def readableString: String = diff.map(_.readableString).mkString("\n")
}
object JsonDiff {
def fromPatch(patch: JsonPatch[Json]): JsonDiff = {
val paths = patch.ops.collect {
case Add(path, value) => JsonPathDiff(path, None, Some(value))
case Remove(path, old) => JsonPathDiff(path, old, None)
case Replace(path, value, old) => JsonPathDiff(path, old, Some(value))
}
JsonDiff(paths)
}
}
def main(args: Array[String]): Unit = {
import diffson._
import diffson.circe._
import diffson.jsonpatch._
import diffson.jsonpatch.lcsdiff.remembering._
import diffson.lcs._
import io.circe._
import io.circe.parser._
val json1 =
s"""{
"name": "Henry",
"age" : 26,
"activities" : {
"school": "basketball club",
"after-school": "chess"
}
}"""
val json2 =
s"""{
"name": "David",
"age" : 23,
"activities" : {
"school": "baseball club",
"after-school": "programming"
}
}"""
implicit val lcs = new Patience[Json]
val patch: Either[ParsingFailure, JsonPatch[Json]] =
for {
json1 <- parse(json1)
json2 <- parse(json2)
} yield diff(json1, json2)
val jsonDiff = JsonDiff.fromPatch(patch.right.get) // Using `get` for sake of example, avoid in real production code
println(jsonDiff.readableString)
}
which will produce next result:
activities.after-school = "chess" , "programming"
activities.school = "basketball club" , "baseball club"
age = 26 , 23
name = "Henry" , "David"
Hope this helps!

Convert JValue to JSON string

I want to convert a jvalue to json string. Here is what my code look like:
import org.json4s._
import org.json4s.JsonDSL._
import org.json4s.native.JsonMethods._
import org.json4s.DefaultFormats._
object Json4sTest {
def main(arg: Array[String]) {
var json = parse("""{"name":"luca", "id": "1q2w3e4r5t", "age": 26, "inner": { "age": 27 }, "url":"http:// www.nosqlnocry.wordpress.com"}""")
// println(json)
val a: List[Map[String, JValue]] = List(Map("inner/age" -> 35, "age" -> 27), Map("name" -> "foo"))
val r = jsonFieldUpdater(json, a)
println(r)
}
def jsonFieldUpdater(json: JValue, list: List[Map[String, JValue]]): JValue =
//
}
}
gets me the result as :
JObject(List((name,JString(foo)), (id,JString(1q2w3e4r5t)), (age,JInt(27)), (inner,JObject(List((age,JInt(35))))), (url,JString(http:// www.nosqlnocry.wordpress.com))))
I am looking for a Json String output as :
{"name":"luca", "id": "1q2w3e4r5t", "age": 27, "inner": { "age": 35 }, "url":"http:// www.nosqlnocry.wordpress.com"}
If you want to print string representation of JValue, you can do it by:
println(compact(render(r)))
This way the String received by method compact(render(r)) looks like this:
{"lotto":{"lotto-id":5,"winning-numbers":[2,45,34,23,7,5,3],"winners":[{"winner-id":23,"numbers":[2,45,34,23,3,5]},{"winner-id":54,"numbers":[52,3,12,11,18,22]}]}}
Or you can use println(pretty(render(r))) to obtain pretty String like this:
{
"lotto":{
"lotto-id":5,
"winning-numbers":[2,45,34,23,7,5,3],
"winners":[{
"winner-id":23,
"numbers":[2,45,34,23,3,5]
},{
"winner-id":54,
"numbers":[52,3,12,11,18,22]
}]
}
}
Examples used from documentation.

Need the best way to find object

Everyone!
I have 3 classes:
case class Foo(id: String, bars: Option[List[Bar]])
case class Bar(id: String, buzzes: Option[List[Buz]])
case class Buz(id: String, name: String)
And the collection:
val col = Option[List[Foo]]
I need to get:
val search: String = "find me"
val (x: Option[Foo], y: Option[Bar], z: Option[Buz]) = where buz.name == search ???
Help please :)
upd:
i have the json
{
"foos": [{
"id": "...",
"bars": [{
"id": "...",
"buzzes": [{
"id": "...",
"name": "find me"
}]
}]
}]
}
and the name in current context will be unique.
my first thought was - transform collection into list of tuples - like this:
{
(foo)(bar)(buz),
(foo)(bar)(buz),
(foo)(bar)(buz)
}
and filter by buz with name == search
But,i don`t know how to :)
A big problem here is that after digging down far enough to find what you're looking for, the result type is going to reflect that morass of types: Option[List[Option[List...etc.
It's going to be easier (not necessarily better) to save off the find as a side effect.
val bz1 = Buz("bz1", "blah")
val bz2 = Buz("bz2", "target")
val bz3 = Buz("bz3", "bliss")
val br1 = Bar("br1", Option(List(bz1,bz3)))
val br2 = Bar("br2", None)
val br3 = Bar("br3", Option(List(bz1,bz2)))
val fo1 = Foo("fo1", Option(List(br1,br2)))
val fo2 = Foo("fo2", None)
val fo3 = Foo("fo3", Option(List(br2,br3)))
val col: Option[List[Foo]] = Option(List(fo1,fo2,fo3))
import collection.mutable.MutableList
val res:MutableList[(String,String,String)] = MutableList()
col.foreach(_.foreach(f =>
f.bars.foreach(_.foreach(br =>
br.buzzes.foreach(_.collect{
case bz if bz.name == "target" => res.+=((f.id,br.id,bz.id))})))))
res // res: MutableList[(String, String, String)] = MutableList((fo3,br3,bz2))