I have a JSON string like
{
"key1": "value1",
"definition": {
// JSON content here
}
}
"definition" key in JSON can contains JSONArray, JSONObject.
For example, it can have
"key2" : ""
or
"key2" : {}
or
"key2" : []
To accommodate this, I have created corresponding Scala class like
import com.google.gson.JsonObject
class JsonData {
var key1: String = _
var definition: JsonObject = _
}
While mapping JSON string to class JsonData, I am getting "definition" in JsonData instance as empty.
Sample code:
import com.fasterxml.jackson.databind.{DeserializationFeature, ObjectMapper}
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper
import com.google.gson.JsonObject
object TestMe {
val mapper = new ObjectMapper with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
def main(args: Array[String]): Unit = {
val jsonString = "{\"key1\": \"value1\",\"definition\": {\"key2\" : \"abc\"}}"
val data = mapper.readValue[JsonData](jsonString)
println(data.definition.getAsJsonObject()) //empty
val jsonString1 = "{\"key1\": \"value1\",\"definition\": {\"key2\" : {\"key3\" : \"\"}}}"
val data1 = mapper.readValue[JsonData](jsonString1)
println(data1.definition.getAsJsonObject()) //empty
val jsonString2 = "{\"key1\": \"value1\",\"definition\": {\"key2\" : [\"a\",\"b\"]}}"
val data2 = mapper.readValue[JsonData](jsonString2)
println(data2.definition.getAsJsonObject()) //empty
}
class JsonData {
var key1: String = _
var definition: JsonObject = _
}
}
How can I read JSON string and map it to class which has one of its attribute type of JsonObject?
Versions:
Scala : 2.11
Jackson-core = 2.6.x;
Gson = 2.6.x;
Jackson-databind = 2.6.x;
Jackson-module-scala = 2.6.5;
I would use com.fasterxml.jackson.databind.JsonNode instead of using Google's Gson JsonObject class. Using Jackson's own classes should make it pretty trivial.
Although you may just map to a Map[String, Any] instead for this kind of flexibility, unless you really need it to still be in Json.
Related
Code of Getting Document
Herr i have a case class User having some Attributes and have index "firstindex", i want to get the document on the bases of "id"
import co.elastic.clients.elasticsearch.core.{GetRequest, GetResponse}
import java.util.concurrent.CompletableFuture
class GetIndex {
def Getting(): Unit = {
val esClient = ElasticSearchConnection.getClient
val request: GetRequest = new GetRequest("firstindex", "1")
val response: CompletableFuture[GetResponse[User]] = esClient.get(request, classOf[User])
println(response)
}
}
Error
constructor GetRequest in class GetRequest cannot be accessed in class GetIndex from class GetIndex
val request: GetRequest = new GetRequest("firstindex", "1")
I'm trying to make use of circe for json parsing in scala. Can you please help me parse 'pocs' from the data in the case class as well? here is the code:
import io.circe.Decoder
import io.circe.generic.semiauto.deriveDecoder
import io.circe.parser
val json: String =
"""
{
"segmements": [
{
"tableName": "X",
"segmentName": "XX",
"pocs": [
"aa#aa.com",
"bb#bb.com"
]
},
{
"tableName": "Y",
"segmentName": "YY",
"pocs": [
"aa#aa.com",
"bb#bb.com"
]
}
]
}
"""
final case class TableInfo(tableName: String, segmentName: String)
object TableInfo {
implicit final val TableInfoDecoder: Decoder[TableInfo] = deriveDecoder
}
val result = for {
data <- parser.parse(json)
obj <- data.asObject.toRight(left = new Exception("Data was not an object"))
segmements <- obj("segmements").toRight(left = new Exception("Json didn't had the
segments key"))
r <- segmements.as[List[TableInfo]]
} yield r
println(result)
scastie link: https://scastie.scala-lang.org/BalmungSan/eVEvBulOQwGzg5hIJroAoQ/3
Just add parameter typed as collection of String:
final case class TableInfo(tableName: String, segmentName: String, pocs: Seq[String])
scastie
I am trying to de-serialize the below json string to scala object using jackson json api
{ "Domain1": { "data-file": "dataFile1", "filter": {
"affected-object": "AffectedObject1", "affected-nd":
"AffectedNd1" } }, "Domain2": { "data-file": "dataFile2",
"filter": { "affected-ci": "AffectedCI2", "affected-net":
"AffectedNet2" } } }
I tried to use case class and tried first using "ClassOf" in ValueType of "readValue" Method but the output is Map of Map Object. Data is not converted into Case Class Object.
case class CrossDomainFilterObj(#JsonProperty("data-file")dataFile: String,
#JsonProperty("filter")filter: Map[String,String])
val jsonString = "{\"Domain1\": {\"data-file\": \"dataFile1\", \"filter\": { \"affected-object\":
\"AffectedObject1\", \"affected-nd\" : \"AffectedNd1\"}},\"Domain2\": {\"data-file\":
\"dataFile2\", \"filter\": { \"affected-ci\":\"AffectedCI2\", \"affected-net\" :
\"AffectedNet2\"}}}"
val mapper = new ObjectMapper
mapper.registerModule(DefaultScalaModule)
val data = mapper.readValue(jsonString, classOf[Map[String, CrossDomainFilterObj]])
println(data)
I am getting output like below
Map(Domain1 -> Map(data-file -> dataFile1, filter -> Map(affected-object ->
AffectedObject1, affected-nd -> AffectedNd1)), Domain2 -> Map(data-file ->
dataFile2, filter -> Map(affected-ci -> AffectedCI2,
affected-net -> AffectedNet2)))
But I am expecting an output like below
Map(Domain1 -> CrossDomainFilterObj(dataFile1, Map(affected-object ->
AffectedObject1, affected-nd -> AffectedNd1)), Domain2 ->
CrossDomainFilterObj(dataFile2, Map(affected-ci ->
AffectedCI2, affected-net -> AffectedNet2)))
Then i tried using TypeReference as ValueType as shown below,
case class CrossDomainFilterObj(#JsonProperty("data-file")dataFile: String,
#JsonProperty("filter")filter: Map[String,String])
val jsonString = "{\"Domain1\": {\"data-file\": \"dataFile1\", \"filter\": { \"affected-object\":
\"AffectedObject1\", \"affected-nd\" : \"AffectedNd1\"}},\"Domain2\": {\"data-file\":
\"dataFile2\", \"filter\": { \"affected-ci\":\"AffectedCI2\", \"affected-net\" :
\"AffectedNet2\"}}}"
val mapper = new ObjectMapper
mapper.registerModule(DefaultScalaModule)
val reference = new TypeReference[Map[String, CrossDomainFilterObj]] {}
val data = mapper.readValue(jsonString, reference)
println(data)
I am getting error like below
dead code following this construct
"val data = mapper.readValue(jsonString, reference)"
Could someone help to identify what I am doing wrong here.
Just make sure you use ScalaObjectMapper:
val mapper = new ObjectMapper() with ScalaObjectMapper
Then this should work:
val data = mapper.readValue[Map[String, CrossDomainFilterObj]](jsonString)
Looking for a good example of polymorphic serialization deserialization using jackson with scala
got an exception :
Exception in thread "main"
Blockquote
org.codehaus.jackson.map.exc.UnrecognizedPropertyException:
Unrecognized field "animals" (Class Zoo), not marked as ignorable
after trying the following code :
import org.codehaus.jackson.annotate.{ JsonTypeInfo, JsonSubTypes }
import org.codehaus.jackson.annotate.JsonSubTypes.Type
#JsonTypeInfo(
use = JsonTypeInfo.Id.NAME,
include= JsonTypeInfo.As.PROPERTY,
property = "type"
)
#JsonSubTypes(Array(
new Type(value= classOf[Cat] , name = "cat"),
new Type(value= classOf[Dog] , name = "dog")
)
)
abstract class Animal {
val name:String = "NoName"
}
class Cat extends Animal{
val favoriteToy = "edi"
}
class Dog extends Animal{
val breed = "German Shepherd"
val color = "brown"
}
class Zoo {
val animals = new scala.collection.mutable.ListBuffer[Animal]
}
import org.codehaus.jackson.map.ObjectMapper
object Foo {
def main (args:Array[String]) {
val mapper = new ObjectMapper()
mapper.setPropertyNamingStrategy(CamelCaseNamingStrategy )
val source = scala.io.Source.fromFile("input.json" )
val input = source.mkString
source.close
val zoo = mapper.readValue(input,classOf[Zoo])
println(mapper.writeValueAsString(zoo))
}
import org.codehaus.jackson.map.introspect.{AnnotatedField, AnnotatedMethod}
import org.codehaus.jackson.map.{MapperConfig, PropertyNamingStrategy}
object CamelCaseNamingStrategy extends PropertyNamingStrategy{
override def nameForGetterMethod (config: MapperConfig[_], method: AnnotatedMethod, defaultName: String) =
{
translate(defaultName)
}
override def nameForSetterMethod (config: MapperConfig[_], method: AnnotatedMethod, defaultName: String) = {
translate(defaultName)
}
override def nameForField (config: MapperConfig[_], field: AnnotatedField, defaultName: String) = {
translate(defaultName)
}
def translate(defaultName:String) = {
val nameChars = defaultName.toCharArray
val nameTranslated = new StringBuilder(nameChars.length*2)
for ( c <- nameChars){
if (Character.isUpperCase(c)){
nameTranslated.append("_")
}
nameTranslated.append( Character.toLowerCase(c))
}
nameTranslated.toString
}
}
file input.json
{
"animals":
[
{"type":"dog","name":"Spike","breed":"mutt","color":"red"},
{"type":"cat","name":"Fluffy","favoriteToy":"spider ring"}
]
}
If you're doing polymorphic deserialization in Scala I'd strongly recommend using case classes and Jackson's scala module.
object Test {
import com.fasterxml.jackson.annotation.JsonSubTypes.Type
import com.fasterxml.jackson.annotation.{JsonSubTypes, JsonTypeInfo}
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper
#JsonTypeInfo(
use = JsonTypeInfo.Id.NAME,
include = JsonTypeInfo.As.PROPERTY,
property = "type"
)
#JsonSubTypes(Array(
new Type(value = classOf[Cat], name = "cat"),
new Type(value = classOf[Dog], name = "dog")
))
trait Animal
case class Dog(name: String, breed: String, leash_color: String) extends Animal
case class Cat(name: String, favorite_toy: String) extends Animal
case class Zoo(animals: Iterable[Animal])
def main(args: Array[String]): Unit = {
val objectMapper = new ObjectMapper with ScalaObjectMapper
objectMapper.registerModule(DefaultScalaModule)
val dogStr = """{"type": "dog", "name": "Spike", "breed": "mutt", "leash_color": "red"}"""
val catStr = """{"type": "cat", "name": "Fluffy", "favorite_toy": "spider ring"}"""
val zooStr = s"""{"animals":[$dogStr, $catStr]}"""
val zoo = objectMapper.readValue[Zoo](zooStr)
println(zoo)
// Prints: Zoo(List(Dog(Spike,mutt,red), Cat(Fluffy,spider ring)))
}
}
Ok, Got it here is a working example with scala based on Deserialize JSON with Jackson into Polymorphic by Programmer Bruce:
import org.codehaus.jackson.annotate.JsonSubTypes.Type
import org.codehaus.jackson.annotate.{JsonSubTypes, JsonTypeInfo}
#JsonTypeInfo(
use = JsonTypeInfo.Id.NAME,
include= JsonTypeInfo.As.PROPERTY,
property = "type"
)
#JsonSubTypes(Array(
new Type(value= classOf[Cat] , name = "cat"),
new Type(value= classOf[Dog] , name = "dog")
)
)
abstract class Animal {
var name:String =""
}
class Dog extends Animal{
var breed= "German Shepherd"
var color = "brown"
}
class Cat extends Animal{
var favoriteToy:String = "nothing"
}
class Zoo {
var animals = new Array[Animal](5)
}
import org.codehaus.jackson.annotate.JsonAutoDetect.Visibility
import org.codehaus.jackson.annotate.JsonMethod
import org.codehaus.jackson.map.{DeserializationConfig, ObjectMapper}
object Foo {
def main (args:Array[String]) {
val mapper = new ObjectMapper().setVisibility(JsonMethod.FIELD,Visibility.ANY)
mapper.configure(DeserializationConfig.Feature.FAIL_ON_UNKNOWN_PROPERTIES,false)
val source = scala.io.Source.fromFile("/input.json" )
val input = source.mkString
println("input " + input)
source.close
val zoo = mapper.readValue(input,classOf[Zoo])
println(mapper.writeValueAsString(zoo))
}
}
file:input.json { "animals": [
{"type":"dog","name":"Spike","breed":"mutt","color":"red"},
{"type":"cat","name":"Fluffy","favoriteToy":"spider ring"}
] }
Below Scala class parses a file using JDOM and populates the values from the file into a Scala immutable Map. Using the + operator on the Map does not seem to have any effect as the Map is always zero.
import java.io.File
import org.jsoup.nodes.Document
import org.jsoup.Jsoup
import org.jsoup.select.Elements
import org.jsoup.nodes.Element
import scala.collection.immutable.TreeMap
class JdkElementDetail() {
var fileLocation: String = _
def this(fileLocation: String) = {
this()
this.fileLocation = fileLocation;
}
def parseFile : Map[String , String] = {
val jdkElementsMap: Map[String, String] = new TreeMap[String , String];
val input: File = new File(fileLocation);
val doc: Document = Jsoup.parse(input, "UTF-8", "http://example.com/");
val e: Elements = doc.getElementsByAttribute("href");
val href: java.util.Iterator[Element] = e.iterator();
while (href.hasNext()) {
var objectName = href.next();
var hrefValue = objectName.attr("href");
var name = objectName.text();
jdkElementsMap + name -> hrefValue
println("size is "+jdkElementsMap.size)
}
jdkElementsMap
}
}
println("size is "+jdkElementsMap.size) always prints "size is 0"
Why is the size always zero, am I not adding to the Map correctly?
Is the only fix for this to convert jdkElementsMap to a var and then use the following?
jdkElementsMap += name -> hrefValue
Removing the while loop here is my updated object:
package com.parse
import java.io.File
import org.jsoup.nodes.Document
import org.jsoup.Jsoup
import org.jsoup.select.Elements
import org.jsoup.nodes.Element
import scala.collection.immutable.TreeMap
import scala.collection.JavaConverters._
class JdkElementDetail() {
var fileLocation: String = _
def this(fileLocation: String) = {
this()
this.fileLocation = fileLocation;
}
def parseFile : Map[String , String] = {
var jdkElementsMap: Map[String, String] = new TreeMap[String , String];
val input: File = new File(fileLocation);
val doc: Document = Jsoup.parse(input, "UTF-8", "http://example.com/");
val elements: Elements = doc.getElementsByAttribute("href");
val elementsScalaIterator = elements.iterator().asScala
elementsScalaIterator.foreach {
keyVal => {
var hrefValue = keyVal.attr("href");
var name = keyVal.text();
println("size is "+jdkElementsMap.size)
jdkElementsMap += name -> hrefValue
}
}
jdkElementsMap
}
}
Immutable data structures -- be they lists or maps -- are just that: immutable. You don't ever change them, you create new data structures based on changes to the old ones.
If you do val x = jdkElementsMap + (name -> hrefValue), then you'll get the new map on x, while jdkElementsMap continues to be the same.
If you change jdkElementsMap into a var, then you could do jdkEleemntsMap = jdkElementsMap + (name -> hrefValue), or just jdkElementsMap += (name -> hrefValue). The latter will also work for mutable maps.
Is that the only way? No, but you have to let go of while loops to achieve the same thing. You could replace these lines:
val href: java.util.Iterator[Element] = e.iterator();
while (href.hasNext()) {
var objectName = href.next();
var hrefValue = objectName.attr("href");
var name = objectName.text();
jdkElementsMap + name -> hrefValue
println("size is "+jdkElementsMap.size)
}
jdkElementsMap
With a fold, such as in:
import scala.collection.JavaConverters.asScalaIteratorConverter
e.iterator().asScala.foldLeft(jdkElementsMap) {
case (accumulator, href) => // href here is not an iterator
val objectName = href
val hrefValue = objectName.attr("href")
val name = objectName.text()
val newAccumulator = accumulator + (name -> hrefValue)
println("size is "+newAccumulator.size)
newAccumulator
}
Or with recursion:
def createMap(hrefIterator: java.util.Iterator[Element],
jdkElementsMap: Map[String, String]): Map[String, String] = {
if (hrefIterator.hasNext()) {
val objectName = hrefIterator.next()
val hrefValue = objectName.attr("href")
val name = objectName.text()
val newMap = jdkElementsMap + name -> hrefValue
println("size is "+newMap.size)
createMap(hrefIterator, newMap)
} else {
jdkElementsMap
}
}
createMap(e.iterator(), new TreeMap[String, String])
Performance-wise, the fold will be rather slower, and the recursion should be very slightly faster.
Mind you, Scala does provide mutable maps, and not just to be able to say it has them: if they fit better you problem, then go ahead and use them! If you want to learn how to use the immutable ones, then the two approaches above are the ones you should learn.
The map is immutable, so any modifications will return the modified map. jdkElementsMap + (name -> hrefValue) returns a new map containing the new pair, but you are discarding the modified map after it is created.
EDIT: It looks like you can convert Java iterables to Scala iterables, so you can then fold over the resulting sequence and accumulate a map:
import scala.collection.JavaConverters._
val e: Elements = doc.getElementsByAttribute("href");
val jdkElementsMap = e.asScala
.foldLeft(new TreeMap[String , String])((map, href) => map + (href.text() -> href.attr("href"))
if you don't care what kind of map you create you can use toMap:
val jdkElementsMap = e.asScala
.map(href => (href.text(), href.attr("href")))
.toMap