Scala JSON diffs - scala

I have two JSONs with the exact same keys.
val json1 =
"""{
'name': 'Henry',
'age' : 26,
'activities' : {
'school': 'basketball club',
'after-school': 'chess'
}
}"""
val json2 =
"""{
'name': 'David',
'age' : 23,
'activities' : {
'school': 'baseball club',
'after-school': 'programming'
}
}"""
I would like the difference between the two JSONs, for example, such as:
name = Henry, David
age = 23, 26
activities.school= basketball club, baseball club
activities.after-school=chess, programming
It doesn't have to follow the above format but, I would like to get the keys and values that are differing.

You could try diffson, a circe based library: https://github.com/gnieh/diffson
Example:
import diffson._
import diffson.lcs._
import diffson.circe._
import diffson.jsonpatch._
import diffson.jsonpatch.lcsdiff._
import io.circe._
import io.circe.parser._
import cats._
import cats.implicits._
implicit val lcs = new Patience[Json]
val json1 = parse("""{
| "a": 1,
| "b": true,
| "c": ["test", "plop"]
|}""".stripMargin)
val json2 = parse("""{
| "a": 6,
| "c": ["test2", "plop"],
| "d": false
|}""".stripMargin)
val patch =
for {
json1 <- json1
json2 <- json2
} yield diff(json1, json2)
Would return:
[{
"op":"replace",
"path":"/a",
"value":6
},{
"op":"remove",
"path":"/b"
},{
"op":"replace",
"path":"/c/0",
"value":"test2"
},{
"op":"add",
"path":"/d",
"value":false
}]
Basically you can concatenate "add" section values for your exact purpose.

As was already suggested by #dk14 you can use diffson librarry: https://github.com/gnieh/diffson - but JsonPatch structure which it provides might be not very convenient for your use case, so it can be converted into another to get result in desired format.
Please, see some code example below:
import diffson.jsonpatch.{Add, JsonPatch, Remove, Replace}
import diffson.jsonpointer.{Part, Pointer}
import io.circe.Json
// Model representing plain json diff at certain path, that can be rendered at more human readable format
case class JsonPathDiff(path: Pointer, left: Option[Json], right: Option[Json]) {
def readableString: String = {
val pathReadableString: String = {
def partToString(part: Part): String = part.fold(identity, _.toString)
path.parts.toList.map(partToString).mkString(".")
}
def jsonReadableValue(json: Option[Json]): String = json.map(_.toString()).getOrElse("")
val leftValue = jsonReadableValue(left)
val rightValue = jsonReadableValue(right)
s"$pathReadableString = $leftValue , $rightValue"
}
}
// Model representing overall difference between two JSON's
case class JsonDiff(diff: List[JsonPathDiff]) {
def readableString: String = diff.map(_.readableString).mkString("\n")
}
object JsonDiff {
def fromPatch(patch: JsonPatch[Json]): JsonDiff = {
val paths = patch.ops.collect {
case Add(path, value) => JsonPathDiff(path, None, Some(value))
case Remove(path, old) => JsonPathDiff(path, old, None)
case Replace(path, value, old) => JsonPathDiff(path, old, Some(value))
}
JsonDiff(paths)
}
}
def main(args: Array[String]): Unit = {
import diffson._
import diffson.circe._
import diffson.jsonpatch._
import diffson.jsonpatch.lcsdiff.remembering._
import diffson.lcs._
import io.circe._
import io.circe.parser._
val json1 =
s"""{
"name": "Henry",
"age" : 26,
"activities" : {
"school": "basketball club",
"after-school": "chess"
}
}"""
val json2 =
s"""{
"name": "David",
"age" : 23,
"activities" : {
"school": "baseball club",
"after-school": "programming"
}
}"""
implicit val lcs = new Patience[Json]
val patch: Either[ParsingFailure, JsonPatch[Json]] =
for {
json1 <- parse(json1)
json2 <- parse(json2)
} yield diff(json1, json2)
val jsonDiff = JsonDiff.fromPatch(patch.right.get) // Using `get` for sake of example, avoid in real production code
println(jsonDiff.readableString)
}
which will produce next result:
activities.after-school = "chess" , "programming"
activities.school = "basketball club" , "baseball club"
age = 26 , 23
name = "Henry" , "David"
Hope this helps!

Related

How to get the return value from For loop and pass it to .body(StringBody(session => in Gatling using Scala

How to get the return value from For loop and pass it to .body(StringBody(session => in Gatling using Scala
I have created a method with for loop to generate String Array in gatling with scala
def main(args: Array[String]): Unit = {
var AddTest: Array[String] = Array[String]()
for (i <- 0 to 3) {
val TestBulk: String =
s"""{ "name": "Perftest ${Random.alphanumeric.take(6).mkString}",
"testID": "00000000-0000-0000-0000-000000006017",
"typeId": "00000000-0000-0000-0000-000000011001",
"statusId": "00000000-0000-0000-0000-000000005058"};"""
AddTest = TestBulk.split(",")
// val TestBulk2: Nothing = AddTest.mkString(",").replace(';', ',')
// println(TestBulk)
}
}
now I want to pass the return value to .body(StringBody(session =>
.exec(http("PerfTest Bulk Json")
.post("/PerfTest/bulk")
.body(StringBody(session =>
s"""[(Value from the for loop).mkString(",").replace(';', ',')
]""".stripMargin)).asJson
Please help me with the possibilities
Please let me know if
You don't need for loop (or var or split) for this. You also do not have ; anywhere, so last replace is pointless.
val ids = """
"testId": "foo",
"typeId": "bar",
"statusId": "baz"
"""
val data = (1 to 3)
.map { _ => Random.alphanumeric.take(6).mkString }
.map { r => s""""name": "Perftest $r"""" }
.map { s => s"{ $s, $ids }" }
.mkString("[", ",", "]")
exec("foo").post("/bar").body(_ => StringBody(data)).asJson
(I added [ and ] around your generated string to make it look like valid json).
Alternatively, you probably have some library that converts maps and lists to json out-of-the box (I don't know gatling, but there must be something), a bit cleaner way to do this would be with something like this:
val ids = Map(
"testId" -> "foo",
"typeId" -> "bar",
"statusId" -> "baz"
)
val data = (1 to 3)
.map { _ => Random.alphanumeric.take(6).mkString }
.map { r => ids + ("name" -> s"Perftest $r") }
exec("foo").post("/bar").body(_ => StringBody(toJson(data))).asJson
This Worked for me
Thanks to
#dima .
I build this with your suggested method.
import scala.util.Random
import math.Ordered.orderingToOrdered
import math.Ordering.Implicits.infixOrderingOps
import play.api.libs.json._
import play.api.libs.json.Writes
import play.api.libs.json.Json.JsValueWrapper
val data1 = (1 to 2)
.map {r => Json.toJson(Map(
"name" -> Json.toJson(s"Perftest${Random.alphanumeric.take(6).mkString}"),
"domainId"->Json.toJson("343RDFDGF4RGGFG"),
"typeId"->Json.toJson("343RDFDGF4RGGFG"),
"statusId"->Json.toJson("343RDFDGF4RGGFG"),
"excludedFromAutoHyperlinking" ->Json.toJson(true)))}
println(Json.toJson(data1))```

json parsing using circe in scala

I'm trying to make use of circe for json parsing in scala. Can you please help me parse 'pocs' from the data in the case class as well? here is the code:
import io.circe.Decoder
import io.circe.generic.semiauto.deriveDecoder
import io.circe.parser
val json: String =
"""
{
"segmements": [
{
"tableName": "X",
"segmentName": "XX",
"pocs": [
"aa#aa.com",
"bb#bb.com"
]
},
{
"tableName": "Y",
"segmentName": "YY",
"pocs": [
"aa#aa.com",
"bb#bb.com"
]
}
]
}
"""
final case class TableInfo(tableName: String, segmentName: String)
object TableInfo {
implicit final val TableInfoDecoder: Decoder[TableInfo] = deriveDecoder
}
val result = for {
data <- parser.parse(json)
obj <- data.asObject.toRight(left = new Exception("Data was not an object"))
segmements <- obj("segmements").toRight(left = new Exception("Json didn't had the
segments key"))
r <- segmements.as[List[TableInfo]]
} yield r
println(result)
scastie link: https://scastie.scala-lang.org/BalmungSan/eVEvBulOQwGzg5hIJroAoQ/3
Just add parameter typed as collection of String:
final case class TableInfo(tableName: String, segmentName: String, pocs: Seq[String])
scastie

Convert JValue to JSON string

I want to convert a jvalue to json string. Here is what my code look like:
import org.json4s._
import org.json4s.JsonDSL._
import org.json4s.native.JsonMethods._
import org.json4s.DefaultFormats._
object Json4sTest {
def main(arg: Array[String]) {
var json = parse("""{"name":"luca", "id": "1q2w3e4r5t", "age": 26, "inner": { "age": 27 }, "url":"http:// www.nosqlnocry.wordpress.com"}""")
// println(json)
val a: List[Map[String, JValue]] = List(Map("inner/age" -> 35, "age" -> 27), Map("name" -> "foo"))
val r = jsonFieldUpdater(json, a)
println(r)
}
def jsonFieldUpdater(json: JValue, list: List[Map[String, JValue]]): JValue =
//
}
}
gets me the result as :
JObject(List((name,JString(foo)), (id,JString(1q2w3e4r5t)), (age,JInt(27)), (inner,JObject(List((age,JInt(35))))), (url,JString(http:// www.nosqlnocry.wordpress.com))))
I am looking for a Json String output as :
{"name":"luca", "id": "1q2w3e4r5t", "age": 27, "inner": { "age": 35 }, "url":"http:// www.nosqlnocry.wordpress.com"}
If you want to print string representation of JValue, you can do it by:
println(compact(render(r)))
This way the String received by method compact(render(r)) looks like this:
{"lotto":{"lotto-id":5,"winning-numbers":[2,45,34,23,7,5,3],"winners":[{"winner-id":23,"numbers":[2,45,34,23,3,5]},{"winner-id":54,"numbers":[52,3,12,11,18,22]}]}}
Or you can use println(pretty(render(r))) to obtain pretty String like this:
{
"lotto":{
"lotto-id":5,
"winning-numbers":[2,45,34,23,7,5,3],
"winners":[{
"winner-id":23,
"numbers":[2,45,34,23,3,5]
},{
"winner-id":54,
"numbers":[52,3,12,11,18,22]
}]
}
}
Examples used from documentation.

play scala json of map with list

I've two classes, User & Address.
case class User(
id: Pk[Long] = NotAssigned,
name: String = "",
email: String = "",
addresses: Seq[Address])
case class Address(
id: Pk[Long] = NotAssigned,
userId: Long,
city: String)
From my controller I've to send all users along with their addresses, like Map[User, List[Address]]. I could able to extract them using anorm (mysql) but then I need to send them as json. Could you please help on how to implement the writes & reads for the above Map[User, List[Address]], Thanks.
That should help
import anorm._
import play.api.libs.json._
import play.api.libs.functional.syntax._
case class User(
id: Pk[Long] = NotAssigned,
name: String = "",
email: String = "",
addresses: Seq[Address])
case class Address(
id: Pk[Long] = NotAssigned,
userId: Long,
city: String)
// Play does not provide Format[Pk[A]], so you need to define it
implicit def pkReads[A](implicit r: Reads[Option[A]]): Reads[Pk[A]] = r.map { _.map(Id(_)).getOrElse(NotAssigned) }
implicit def pkWrites[A](implicit w: Writes[Option[A]]): Writes[Pk[A]] = Writes(id => w.writes(id.toOption))
implicit val addrFormat = Json.format[Address]
implicit val userFormat = Json.format[User]
Now you can easily serialize a user:
val as = Seq(Address(Id(2), 1, "biim"))
val u = User(Id(1), "jto", "jto#foo.bar", as)
scala> Json.toJson(u)
res6: play.api.libs.json.JsValue = {"id":1,"name":"jto","email":"jto#foo.bar","addresses":[{"id":2,"userId":1,"city":"biim"}]}
As Julien says, you can't just serialize a Map[User, Seq[Address]]. It just does not make sense since User can't be a key in a Json Object.
You can solve this problem by transforming your Map[User, List[Address]] to a List[User], and the JsonWriter will became easy to write.
Something like:
list.map {
case (user, address) => user.copy(addresses = address.toSeq)
}
The "addresses" in User contains the address so you don't really need to send Map[User, List[Address]] back to client. The Json would be an array of serialized user objects and addresses is part of that. If you do want to send back a Map then the type Map[String, List[Address]] makes more sense in Json serialization context. Here is the code to generate Json for List[User]. The output looks like this
[
{
"id": 1,
"name": "John Doe",
"email": "john#email.com",
"addresses": [
{
"id": 1001,
"userId": 1,
"city": "Chicago"
},
{
"id": 1002,
"userId": 1,
"city": "New York"
}
]
},
{
"id": 2,
"name": "Jane Doe",
"email": "jane#email.com",
"addresses": [
{
"id": 1012,
"userId": 1,
"city": "Dallas"
}
]
}
]
Here is the code the would be in you controller. It has implicit Json formatters that are used by Json.toJson.
implicit object PkWrites extends Writes[Pk[Long]] {
def writes(key: Pk[Long]) = Json.toJson(key.toOption)
}
implicit object PkReads extends Reads[Pk[Long]] {
def reads(json: JsValue) = json match {
case l: JsNumber => JsSuccess(Id(l.value.toLong))
case _ => JsSuccess(NotAssigned)
}
}
implicit val AddressWrites: Writes[Address] = (
(JsPath \ "id").write[Pk[Long]] and
(JsPath \ "userId").write[Long] and
(JsPath \ "city").write[String]
)(unlift(Address.unapply))
implicit val AddressReads: Reads[Address] = (
(JsPath \ "id").read[Pk[Long]] and
(JsPath \ "userId").read[Long] and
(JsPath \ "city").read[String]
)(Address.apply _)
implicit val UserWrites: Writes[User] = (
(JsPath \ "id").write[Pk[Long]] and
(JsPath \ "name").write[String] and
(JsPath \ "email").write[String] and
(JsPath \ "addresses").write[List[Address]]
)(unlift(User.unapply))
def makeJson() = Action {
val johnAddr1 = Address(Id(1001), 1, "Chicago")
val johnAddr2 = Address(Id(1002), 1, "New York")
val janeAddr1 = Address(Id(1012), 1, "Dallas")
val john = User(Id(1), "John Doe", "john#email.com", List(johnAddr1, johnAddr2))
val jane = User(Id(2), "Jane Doe", "jane#email.com", List(janeAddr1))
Ok(Json.toJson(List(john, jane)))
// Ok(Json.toJson(map))
}

Working with nested maps from a JSON string

Given a JSON string like this:
{"Locations":
{"list":
[
{"description": "some description", "name": "the name", "id": "dev123"},
{"description": "other description", "name": "other name", "id": "dev59"}
]
}
}
I'd like to return a list of "id"s from a function parsing the above string. JSON.parseFull() (from scala.util.parsing.json) gives me a result of type Option[Any]. Scala REPL shows it as Some(Map(Locations -> Map(list -> List(Map(id -> dev123, ... and as a beginner in Scala I'm puzzled as to which way to approach it.
Scala API docs suggest "to treat it as a collection or monad and use map, flatMap, filter, or foreach". Top-level element is an Option[Any] however that should be Some with a Map that should contain a single key "Locations", that should contain a single key "list" that finally is a List. What would be an idiomatic way in Scala to write a function retrieving the "id"s?
First of all, you should cast json from Any to right type:
val json = anyJson.asInstanceOf[Option[Map[String,List[Map[String,String]]]]]
And then you may extract ids from Option using map method:
val ids = json.map(_("Locations")("list").map(_("id"))).getOrElse(List())
Because Any is everywhere is the returned result, you'll have to cast. Using one of my earlier answers:
class CC[T] { def unapply(a:Any):Option[T] = Some(a.asInstanceOf[T]) }
object M extends CC[Map[String, Any]]
object L extends CC[List[Any]]
object S extends CC[String]
object D extends CC[Double]
object B extends CC[Boolean]
for {
Some(M(map)) <- List(JSON.parseFull(jsonString))
M(locMap) = map("Locations")
L(list) = locMap("list")
description <- list
M(desc) = description
S(id) = desc("id")
} yield id
// res0: List[String] = List(dev123, dev59)
For this type of tasks, you should take a look at Rapture.io. I'm also a scala beginner, but from what I've searched for, this seems to have the friendliest syntax. Here's a short example, taken from a gist:
import rapture.io._
// Let's parse some JSON
val src: Json = Json.parse("""
{
"foo": "Hello world",
"bar": {
"baz": 42
}
}
""")
// We can now access the value bar.baz
val x: Json = src.bar.baz
// And get it as an integer
val y: Int = x.get[Int]
// Alternatively, we can use an extractor to get the values we want:
val json""" { "bar": { "baz": $x }, "foo": $z }""" = src
// Now x = 42 and z = "Hello world".
Is this what you need? (using lift-json)
scala> import net.liftweb.json._
import net.liftweb.json._
scala> implicit val formats = DefaultFormats
formats: net.liftweb.json.DefaultFormats.type = net.liftweb.json.DefaultFormats$#79e379
scala> val jsonString = """{"Locations":
{"list":
[
{"description": "some description", "name": "the name", "id": "dev123"},
{"description": "other description", "name": "other name", "id": "dev59"}
]
}
}"""
jsonString: java.lang.String =
{"Locations":
{"list":
[
{"description": "some description", "name": "the name", "id": "dev123"},
{"description": "other description", "name": "other name", "id": "dev59"}
]
}
}
scala> Serialization.read[Map[String, Map[String, List[Map[String, String]]]]](jsonString)
res43: Map[String,Map[String,List[Map[String,String]]]] = Map(Locations -> Map(list -> List(Map(description -> some desc
ription, name -> the name, id -> dev123), Map(description -> other description, name -> other name, id -> dev59))))
scala> val json = parse(jsonString)
json: net.liftweb.json.package.JValue = JObject(List(JField(Locations,JObject(List(JField(list,JArray(List(JObject(List(
JField(description,JString(some description)), JField(name,JString(the name)), JField(id,JString(dev123)))), JObject(Lis
t(JField(description,JString(other description)), JField(name,JString(other name)), JField(id,JString(dev59))))))))))))
scala> json \\ "id"
res44: net.liftweb.json.JsonAST.JValue = JObject(List(JField(id,JString(dev123)), JField(id,JString(dev59))))
scala> compact(render(res44))
res45: String = {"id":"dev123","id":"dev59"}
In a branch of SON of JSON, this will work. Note that I'm not using the parser. Not that it doesn't exist. It's just that creating an JSON object using the builder methods is easier:
scala> import nl.typeset.sonofjson._
import nl.typeset.sonofjson._
scala> var all = obj(
| locations = arr(
| obj(description = "foo", id = "807",
| obj(description = "bar", id = "23324"
| )
| )
scala> all.locations.map(_.id).as[List[String]]
res2: List[String] = List(23324, 807)
Or use a for comprehension:
scala> (for (location <- all.locations) yield location.id).as[List[String]]
res4: List[String] = List(23324, 807)