I want to print json to string in one line.
case class Data(e: Option[String])
object Data {
implicit val dEncoder = deriveDecoder[Data]
implicit val dDecoder = deriveEncoder[Data]
}
case class Random(a: String,b: String, c: Int,d: Data)
object Random {
implicit val rEncoder = deriveDecoder[Random]
implicit val rDecoder = deriveEncoder[Random]
}
val res = Random("a","b", 1, Data("e"))
when i do res.asJson.toString, i get:
{
"a": "a",
"b": "b",
"c":1,
"d":
{
"e": "e"
}
}
but i want it to be printed in one line without \n as using circe.
**{"a": "a","b": "b", "c": 1,d:{"e": "e"}}**
If you don't mind stripping other unnecessary whitespace characters:
# res.asJson.noSpaces
res9: String = "{\"a\":\"a\",\"b\":\"b\",\"c\":1,\"d\":{\"e\":\"e\"}}"
# println(res.asJson.noSpaces)
{"a":"a","b":"b","c":1,"d":{"e":"e"}}
Related
I'm trying to make use of circe for json parsing in scala. Can you please help me parse 'pocs' from the data in the case class as well? here is the code:
import io.circe.Decoder
import io.circe.generic.semiauto.deriveDecoder
import io.circe.parser
val json: String =
"""
{
"segmements": [
{
"tableName": "X",
"segmentName": "XX",
"pocs": [
"aa#aa.com",
"bb#bb.com"
]
},
{
"tableName": "Y",
"segmentName": "YY",
"pocs": [
"aa#aa.com",
"bb#bb.com"
]
}
]
}
"""
final case class TableInfo(tableName: String, segmentName: String)
object TableInfo {
implicit final val TableInfoDecoder: Decoder[TableInfo] = deriveDecoder
}
val result = for {
data <- parser.parse(json)
obj <- data.asObject.toRight(left = new Exception("Data was not an object"))
segmements <- obj("segmements").toRight(left = new Exception("Json didn't had the
segments key"))
r <- segmements.as[List[TableInfo]]
} yield r
println(result)
scastie link: https://scastie.scala-lang.org/BalmungSan/eVEvBulOQwGzg5hIJroAoQ/3
Just add parameter typed as collection of String:
final case class TableInfo(tableName: String, segmentName: String, pocs: Seq[String])
scastie
Sample Json
"alternateId": [
{
"type": "POPID",
"value": "1-7842-0759-001"
},
{
"type": "CAMID",
"value": "CAMID 0000-0002-7EC1-02FF-O-0000-0000-2"
},
{
"type": "ProgrammeUuid",
"value": "1ddb01e2-6146-4e10-bba9-dde40d0ad886"
}
]
I want to update a existing dataframe with two columns, those two columns are POPID and CAMID . These two values needs to be parsed from json structure
I dont know how to parse this structure , Can you help me on what do i need to change on fetchField method. As per above json POPID is placed first and CAMID is placed second, but in real jsons, it can be placed at one of those 3 places inside alternateId.
val fetchCAMID_udf = udf(fetchCAMID _)
val fetchPOPID_udf = udf(fetchPOPID _)
var updatedDf = //Data frame initialize
updatedDf = updatedDf.withColumn("CAMID", fetchCAMID_udf(col("alternate_id")))
updatedDf = updatedDf.withColumn("POPID", fetchPOPID_udf(col("alternate_id")))
updatedDf .show(10,false)
def fetchCAMID(jsonStr: String): String = {
var CAMID: String = fetchField(jsonStr, "CAMID")
CAMID
}
def fetchPOPID(jsonStr: String): String = {
fetchField(jsonStr, "POPID")
}
def fetchField(jsonStr: String, fieldName: String): String = {
try {
implicit val formats = DefaultFormats
val extractedField = jsonStr match {
case "(unknown)" => jsonStr
case _ => {
val json = JsonMethods.parse(jsonStr)
val resultExtracted = (json \\ fieldName)
val result = resultExtracted match {
case _: JString => resultExtracted.extract[String]
case _: JInt => resultExtracted.extract[Int].toString
case _: JObject => "(unknown)"
}
result
}
}
extractedField
}
catch{
case e: Exception =>{
log.error(s"Fetch field failed. Field name: $fieldName . Json: $jsonStr")
"(unknown)"
}
}
}
Change your fetchField function as the following
def fetchField(jsonStr: String, fieldName: String): String = {
try {
val typeAndValue = (JsonMethods.parse("{"+jsonStr+"}") \ "alternateId" \ "type" \\ classOf[JString]).zip(JsonMethods.parse("{"+jsonStr+"}") \ "alternateId" \ "value" \\ classOf[JString])
typeAndValue.filter(_._1 == fieldName).map(_._2).toList(0)
}catch{
case e: Exception =>{
"(unknown)"
}
}
}
and you get the CAMID and POPID populated
you can read the JSON using Spark and get it using regular spark operations
val df=spark.read.option("multiLine",true).json("test.json")
df.select($"alternateId".getItem(0).as("pop"),$"alternateId".getItem(1).as("cam")).select($"pop.value".as("POPID"),$"cam.value".as("CAMID")).show()
+---------------+--------------------+
| POPID| CAMID|
+---------------+--------------------+
|1-7842-0759-001|CAMID 0000-0002-7...|
+---------------+--------------------+
I want to convert a jvalue to json string. Here is what my code look like:
import org.json4s._
import org.json4s.JsonDSL._
import org.json4s.native.JsonMethods._
import org.json4s.DefaultFormats._
object Json4sTest {
def main(arg: Array[String]) {
var json = parse("""{"name":"luca", "id": "1q2w3e4r5t", "age": 26, "inner": { "age": 27 }, "url":"http:// www.nosqlnocry.wordpress.com"}""")
// println(json)
val a: List[Map[String, JValue]] = List(Map("inner/age" -> 35, "age" -> 27), Map("name" -> "foo"))
val r = jsonFieldUpdater(json, a)
println(r)
}
def jsonFieldUpdater(json: JValue, list: List[Map[String, JValue]]): JValue =
//
}
}
gets me the result as :
JObject(List((name,JString(foo)), (id,JString(1q2w3e4r5t)), (age,JInt(27)), (inner,JObject(List((age,JInt(35))))), (url,JString(http:// www.nosqlnocry.wordpress.com))))
I am looking for a Json String output as :
{"name":"luca", "id": "1q2w3e4r5t", "age": 27, "inner": { "age": 35 }, "url":"http:// www.nosqlnocry.wordpress.com"}
If you want to print string representation of JValue, you can do it by:
println(compact(render(r)))
This way the String received by method compact(render(r)) looks like this:
{"lotto":{"lotto-id":5,"winning-numbers":[2,45,34,23,7,5,3],"winners":[{"winner-id":23,"numbers":[2,45,34,23,3,5]},{"winner-id":54,"numbers":[52,3,12,11,18,22]}]}}
Or you can use println(pretty(render(r))) to obtain pretty String like this:
{
"lotto":{
"lotto-id":5,
"winning-numbers":[2,45,34,23,7,5,3],
"winners":[{
"winner-id":23,
"numbers":[2,45,34,23,3,5]
},{
"winner-id":54,
"numbers":[52,3,12,11,18,22]
}]
}
}
Examples used from documentation.
Everyone!
I have 3 classes:
case class Foo(id: String, bars: Option[List[Bar]])
case class Bar(id: String, buzzes: Option[List[Buz]])
case class Buz(id: String, name: String)
And the collection:
val col = Option[List[Foo]]
I need to get:
val search: String = "find me"
val (x: Option[Foo], y: Option[Bar], z: Option[Buz]) = where buz.name == search ???
Help please :)
upd:
i have the json
{
"foos": [{
"id": "...",
"bars": [{
"id": "...",
"buzzes": [{
"id": "...",
"name": "find me"
}]
}]
}]
}
and the name in current context will be unique.
my first thought was - transform collection into list of tuples - like this:
{
(foo)(bar)(buz),
(foo)(bar)(buz),
(foo)(bar)(buz)
}
and filter by buz with name == search
But,i don`t know how to :)
A big problem here is that after digging down far enough to find what you're looking for, the result type is going to reflect that morass of types: Option[List[Option[List...etc.
It's going to be easier (not necessarily better) to save off the find as a side effect.
val bz1 = Buz("bz1", "blah")
val bz2 = Buz("bz2", "target")
val bz3 = Buz("bz3", "bliss")
val br1 = Bar("br1", Option(List(bz1,bz3)))
val br2 = Bar("br2", None)
val br3 = Bar("br3", Option(List(bz1,bz2)))
val fo1 = Foo("fo1", Option(List(br1,br2)))
val fo2 = Foo("fo2", None)
val fo3 = Foo("fo3", Option(List(br2,br3)))
val col: Option[List[Foo]] = Option(List(fo1,fo2,fo3))
import collection.mutable.MutableList
val res:MutableList[(String,String,String)] = MutableList()
col.foreach(_.foreach(f =>
f.bars.foreach(_.foreach(br =>
br.buzzes.foreach(_.collect{
case bz if bz.name == "target" => res.+=((f.id,br.id,bz.id))})))))
res // res: MutableList[(String, String, String)] = MutableList((fo3,br3,bz2))
Given a JSON string like this:
{"Locations":
{"list":
[
{"description": "some description", "name": "the name", "id": "dev123"},
{"description": "other description", "name": "other name", "id": "dev59"}
]
}
}
I'd like to return a list of "id"s from a function parsing the above string. JSON.parseFull() (from scala.util.parsing.json) gives me a result of type Option[Any]. Scala REPL shows it as Some(Map(Locations -> Map(list -> List(Map(id -> dev123, ... and as a beginner in Scala I'm puzzled as to which way to approach it.
Scala API docs suggest "to treat it as a collection or monad and use map, flatMap, filter, or foreach". Top-level element is an Option[Any] however that should be Some with a Map that should contain a single key "Locations", that should contain a single key "list" that finally is a List. What would be an idiomatic way in Scala to write a function retrieving the "id"s?
First of all, you should cast json from Any to right type:
val json = anyJson.asInstanceOf[Option[Map[String,List[Map[String,String]]]]]
And then you may extract ids from Option using map method:
val ids = json.map(_("Locations")("list").map(_("id"))).getOrElse(List())
Because Any is everywhere is the returned result, you'll have to cast. Using one of my earlier answers:
class CC[T] { def unapply(a:Any):Option[T] = Some(a.asInstanceOf[T]) }
object M extends CC[Map[String, Any]]
object L extends CC[List[Any]]
object S extends CC[String]
object D extends CC[Double]
object B extends CC[Boolean]
for {
Some(M(map)) <- List(JSON.parseFull(jsonString))
M(locMap) = map("Locations")
L(list) = locMap("list")
description <- list
M(desc) = description
S(id) = desc("id")
} yield id
// res0: List[String] = List(dev123, dev59)
For this type of tasks, you should take a look at Rapture.io. I'm also a scala beginner, but from what I've searched for, this seems to have the friendliest syntax. Here's a short example, taken from a gist:
import rapture.io._
// Let's parse some JSON
val src: Json = Json.parse("""
{
"foo": "Hello world",
"bar": {
"baz": 42
}
}
""")
// We can now access the value bar.baz
val x: Json = src.bar.baz
// And get it as an integer
val y: Int = x.get[Int]
// Alternatively, we can use an extractor to get the values we want:
val json""" { "bar": { "baz": $x }, "foo": $z }""" = src
// Now x = 42 and z = "Hello world".
Is this what you need? (using lift-json)
scala> import net.liftweb.json._
import net.liftweb.json._
scala> implicit val formats = DefaultFormats
formats: net.liftweb.json.DefaultFormats.type = net.liftweb.json.DefaultFormats$#79e379
scala> val jsonString = """{"Locations":
{"list":
[
{"description": "some description", "name": "the name", "id": "dev123"},
{"description": "other description", "name": "other name", "id": "dev59"}
]
}
}"""
jsonString: java.lang.String =
{"Locations":
{"list":
[
{"description": "some description", "name": "the name", "id": "dev123"},
{"description": "other description", "name": "other name", "id": "dev59"}
]
}
}
scala> Serialization.read[Map[String, Map[String, List[Map[String, String]]]]](jsonString)
res43: Map[String,Map[String,List[Map[String,String]]]] = Map(Locations -> Map(list -> List(Map(description -> some desc
ription, name -> the name, id -> dev123), Map(description -> other description, name -> other name, id -> dev59))))
scala> val json = parse(jsonString)
json: net.liftweb.json.package.JValue = JObject(List(JField(Locations,JObject(List(JField(list,JArray(List(JObject(List(
JField(description,JString(some description)), JField(name,JString(the name)), JField(id,JString(dev123)))), JObject(Lis
t(JField(description,JString(other description)), JField(name,JString(other name)), JField(id,JString(dev59))))))))))))
scala> json \\ "id"
res44: net.liftweb.json.JsonAST.JValue = JObject(List(JField(id,JString(dev123)), JField(id,JString(dev59))))
scala> compact(render(res44))
res45: String = {"id":"dev123","id":"dev59"}
In a branch of SON of JSON, this will work. Note that I'm not using the parser. Not that it doesn't exist. It's just that creating an JSON object using the builder methods is easier:
scala> import nl.typeset.sonofjson._
import nl.typeset.sonofjson._
scala> var all = obj(
| locations = arr(
| obj(description = "foo", id = "807",
| obj(description = "bar", id = "23324"
| )
| )
scala> all.locations.map(_.id).as[List[String]]
res2: List[String] = List(23324, 807)
Or use a for comprehension:
scala> (for (location <- all.locations) yield location.id).as[List[String]]
res4: List[String] = List(23324, 807)