I am trying to get value from a nested JSON using json4s in scala.
The parse method works well for the string provided manually but null for the string provided from a file.
Here is the code
import org.json4s.jackson.JsonMethods.{parse, pretty}
import scala.io.Source
object ParseJson4s {
def map_fields(lines: String) = {
val testString = """{"Information":{"Context":"firstContext", "Assets":{"Asset":{"Name":"firstName"}}}}"""
val parseJSON_test = parse(testString)
val parseJSON = parse(lines)
println("Name from method " + pretty(parseJSON_test \ "Information" \ "Assets" \ "Asset" \ "Name"))
println("Name from file " + pretty(parseJSON \ "Information" \ "Assets" \ "Asset" \ "Name"))
}
def main(args: Array[String]): Unit = {
val file = Source.fromFile("testFile.txt").getLines()
file.foreach(map_fields)
}
}
and here is the test file
"""{"Information":{"Context":"firstContext", "Assets":{"Asset":{"Name":"firstName"}}}}"""
"""{"Information":{"Context":"secondContext", "Assets":{"Asset":{"Name":"secondName"}}}}"""
Output:
Name from method "firstName"
Name from file
Name from method "firstName"
Name from file
Thanks
Is """ mandadory for JSON records in your text file? I removed them and it works for me.
Results I've got in console:
Name from file "firstName"
Name from file "secondName"
Source code:
import org.json4s.jackson.JsonMethods.{parse, pretty}
import scala.io.Source
object Json4sTest {
def map_fields(lines: String) = {
val parseJSON = parse(lines)
println("Name from file " + pretty(parseJSON \ "Information" \ "Assets" \ "Asset" \ "Name"))
}
def main(args: Array[String]): Unit = {
val file = Source.fromFile("testFile.txt").getLines()
file.foreach(map_fields)
}
}
testFile.txt:
{"Information":{"Context":"firstContext", "Assets":{"Asset":{"Name":"firstName"}}}}
{"Information":{"Context":"secondContext", "Assets":{"Asset":{"Name":"secondName"}}}}
The reason why it fails is because the lines in the file are not valid JSON strings. A JSON string cannot start with triple quotes or quotes for that matter, unless it's just a simple string.
Triple quote(""") in scala is used to create multi-line strings and strings that contain quotes within them. You need to use them only when you define string literals(hard code strings) in scala.
So, remove the triple quotes from the lines in the file and it should give you proper results.
Try to parse from the file with jsoniter-scala, it will clearly report a position and a cause of the problem.
https://github.com/plokhotnyuk/jsoniter-scala
Related
I have a json string:
val message = "{\"me\":\"a\",
\"version\":\"1.0\",
\"message_metadata\": \"{
\"event_type\":\"UpdateName\",
\"start_date\":\"1515\"}\"
}"
I want to extract the value of the field event_type from this json string.
I have used below code to extract the value:
val mapper = new ObjectMapper
val root = mapper.readTree(message)
val metadata =root.at("/message_metadata").asText()
val root1 = mapper.readTree(metadata)
val event_type =root1.at("/event_type").asText()
print("eventType:" + event_type.toString) //UpdateName
This works fine and I get the value as UpdateName. But I when I want to get the event type in a single line as below:
val mapper = new ObjectMapper
val root = mapper.readTree(message)
val event_type =root.at("/message_metadata/event_type").asText()
print("eventType:" + event_type.toString) //Empty string
Here event type returns a empty sting. This might be because of the message_metadata has Json object as a string value. Is there a way I can get the value of event_type in a single line?
The problem is that your JSON message contains an object who's message_metadata field itself contains JSON, so it must be decoded separately. I'd suggest that you don't put JSON into JSON but only encode the data structure once.
Example:
val message = "{\"me\":\"a\",
\"version\":\"1.0\",
\"message_metadata\": {
\"event_type\":\"UpdateName\",
\"start_date\":\"1515\"
}
}"
You can parse your JSON using case classes and then get your event_type field from there.
case class Json(me: String, version: String, message_metadata: Message)
case class Message(event_type: String, start_date: String)
object Mapping {
def main(args: Array[String]): Unit = {
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper
val objectMapper = new ObjectMapper() with ScalaObjectMapper
objectMapper.registerModule(DefaultScalaModule)
val str = "{\n \"me\": \"a\",\n \"version\": \"1.0\",\n \"message_metadata\": {\n \"event_type\": \"UpdateName\",\n \"start_date\": \"1515\"\n }\n}"
val json = objectMapper.readValue(str, classOf[Json])
//to print event_type
println(json.message_metadata.event_type)
//output: UpdateName
}
}
You can even convert a JSON to Scala Case Class and then get the particular field from the case class.
Please find a working and detailed answer which I have provided using generics here.
I am doing some handson on Scala
I have a json file named simple.json
"employees":[
{"firstName":"John", "lastName":"Doe"},
{"firstName":"Anna", "lastName":"Smith"},
{"firstName":"Peter","lastName":"Jones"}
]
I want read this json file from and keep the values of this values in a scala List
Expected output :
List("John|Doe", "Anna|Smith", "Peter|Jones")
I am not able to proceed with the scala file
package pack1
import scala.io.Source
import org.json.simple.JSONObject;
import org.json.simple.JSONArray;
import org.json.simple.parser.ParseException;
import org.json.simple.parser.JSONParser;
object ss {
def main(args: Array[String]): Unit = {
val mySource=Source.fromFile("C:\\inputfiles\\simple.json").getLines().mkString
println(mySource)
val parser = new JSONParser();
val jArray = parser.parse(mySource);
}
}
How do I proceed with the code to get the expected output
Using play-json
if you are using sbt project. Add following line to dependencies in your project.
build.sbt
libraryDependencies ++= Seq("com.typesafe.play" %% "play-json" % "2.5.4")
Source code
val lines = Source.fromFile("<file-path>").getLines.mkString
val json = Json.parse("{" + lines.replaceAll("""\n""", "").trim + "}")
val list = (json \ "employees").as[List[JsValue]]
.map(name => s"""${(name \ "firstName").as[String]}|${(name \ "lastName").as[String]}""")
Scala REPL
scala> import play.api.libs.json._
scala> val str = """ {"employees" : [ {"firstName":"John", "lastName":"Doe"} ] }""".trim
str: String = {"employees" : [ {"firstName":"John", "lastName":"Doe"} ] }
scala> val json = Json.parse(str)
json: play.api.libs.json.JsValue = {"employees":[{"firstName":"John","lastName":"Doe"}]}
scala> val list = (json \ "employees").as[List[JsValue]].map(name => s"""${(name \ "firstName").as[String]}|${(name \ "lastName").as[String]}""")
list: List[String] = List(John|Doe)
Note that below string is not a valid json
val str = """
"employees":[
{"firstName":"John", "lastName":"Doe"},
{"firstName":"Anna", "lastName":"Smith"},
{"firstName":"Peter","lastName":"Jones"}
] """.stripMargin
This has to be wrapped inside a { } for Json.parse to parse properly.
val validJsonStr = "{" + str + "}"
I have the following Json as var dataObject ={"files": ["code.R", "data.cv", "input.txt"]}.
I am posting this json as a body from the client side and I want to parse the Json and read these files names in the server side in play scala.
Please help
Because you have only one field, you can't use json combinators,
But you can do as follow:
case class Selection(files:List[String])
object Selection{
implicit val selectionReads = (__ \ 'files).read[List[String]].map{ l => Selection(l) }
implicit val selectionWrites = (__ \ 'files).write[List[String]].contramap { (selection: Selection) => selection.files}
//You can replace the above 2 lines with this line - depends on you.
implicit val selectionFormat: Format[Selection] = (__ \ 'files).format[List[String]].inmap(files => Selection(files), (selection: Selection) => selection.files)
}
Make sure you import:
import play.api.libs.functional.syntax._
This is the documentation: https://www.playframework.com/documentation/2.5.x/ScalaJson
And the solution is simple:
import play.api.libs.json._
val json: JsValue = Json.parse("{ "files": ["code.R","data.csv","input.txt"] }")
val files = (json \ "files").get
using net.liftweb.json what is the difference \ and \ operators when parsing json ?
import net.liftweb.json._
val parsed = JsonParser.parse(jsonString)
val name = parsed.\("firstName")
val userId = parsed.\\("userId")
"\\" will extract the value even if it's present within nested json whereas "\" will extract the value only if present as a top-level attribute.
Consider this json
val json = """{"nested1":{"nested2": {"myKey":"myValue"}}}"""
val jsonMsg = parse(json)
In this case
(jsonMsg \ "myKey").values
retruns None
where as
(jsonMsg \\ "myKey").values
returns myValue
I am experimenting with the json4s library (based on lift-json). One of the things I would like to do is to parse a JSON string into an AST, and then manipulate it.
For example, I would like to upsert a field (insert the field into the AST if it does not exist, or update its value if it does).
I have not been able to find how to do it in the documentation. Experimenting with the available methods, I have come up with the following, which works, but feels clumsy.
import org.json4s._
import org.json4s.JsonDSL._
import org.json4s.native.JsonMethods._
object TestJson {
implicit val formats = DefaultFormats
def main(args: Array[String]): Unit = {
val json = """{"foo":1, "bar":{"foo":2}}"""
val ast = parse(json).asInstanceOf[JObject]
println( upsertField(ast, ("foo" -> "3")) )
println( upsertField(ast, ("foobar" -> "3")) )
}
def upsertField(src:JObject, fld:JField): JValue = {
if(src \ fld._1 == JNothing){
src ~ fld
}
else{
src.replace(List(fld._1), fld._2)
}
}
}
I dislike it for many reasons:
Having to explicitly cast the results of parse(json) to JObject
The result of the upsertField function is a JValue, which I will have to recast if I want to manipulate the object further
The upsertField function just feels very unelegant
It does not work for fields that are not at the top level of the hierarchy
Is there a better way to transform the AST?
EDIT: as a workaround to the problem, I have managed to convert my JSON to Scala regular classes, and manipulate them with lenses (Using Lenses on Scala Regular Classes)
There is the merge function which creates or overrides a field. You can also update fields that are not at the root level of the tree.
import org.json4s._
import org.json4s.JsonDSL._
import org.json4s.jackson.JsonMethods._
object mergeJson extends App {
val json =
"""
|{
| "foo":1,
| "bar": {
| "foo": 2
| }
|}
|""".stripMargin
val ast = parse(json)
val updated = ast merge (("foo", 3) ~ ("bar", ("fnord", 5)))
println(pretty(updated))
// {
// "foo" : 3,
// "bar" : {
// "foo" : 2,
// "fnord" : 5
// }
// }
}
Let me also give you the SON of JSON version:
import nl.typeset.sonofjson._
val json = parse("""{ "foo" : 1, "bar" : { "foo" : 2 } }""")
// or, perhaps a little easier
val json = obj(foo = 1, bar = obj(foo = 2))
json.foo = "3"
json.foobar = "3"
When I was implementing some very specific json diff using lift json I used a lot of recursive functions to get to the jpath where I need to modify value, and modified json was constructed when recursion "collapsed". LiftJson is immutable after all. You mentioned lenses as another approach, which is very interesting by itself. But my current favourite is play-json library that is working like a charm in a situation when you need to do json-to-json transformation:
from Mandubian Blog:
val gizmo2gremlin = (
(__ \ 'name).json.put(JsString("gremlin")) and
(__ \ 'description).json.pickBranch(
(__ \ 'size).json.update( of[JsNumber].map{ case JsNumber(size) => JsNumber(size * 3) } ) and
(__ \ 'features).json.put( Json.arr("skinny", "ugly", "evil") ) and
(__ \ 'danger).json.put(JsString("always"))
reduce
) and
(__ \ 'hates).json.copyFrom( (__ \ 'loves).json.pick )
) reduce
Yummy Features: all transformers are combinators that can be mixed together, validation, shapeless support, auto marshaling of case classes with implicit overrides, stand-alone library.