I'm new in Scala and trying to achieve the following using Scala's foldLeft() method or any other functional solution:
I have the following JSON:
{
"aspects": [
{
"name": "Name",
"values": [
"Phone"
]
},
{
"name": "Color",
"values": [
"Red",
"Black"
]
},
{
"name": "Size",
"values": [
"6",
"10"
]
}
]
}
I want to convert this into the following Seq<String>:
["Name:::Phone", "Color:::Red", "Color:::Black", "Size:::6", "Size:::10"]
I did that using Java style where aspects is an object representing the JSON:
aspects.foreach(pair => {
pair.values.foreach(value => {
valuesList += pair.name + ":::" + value
})
})
What is the best Scala's way to do this?
It depends on the JSON library you are using, but once you have the aspects data in a suitable format the output can be generated is like this:
case class Aspect(name: String, values: Seq[String])
val aspects: Seq[Aspect] = ???
aspects.flatMap(a => a.values.map(a.name + ":::" + _))
I use json4s and jackson and the conversion code is basically just
val aspects = parse(json).extract[Seq[Aspect]]
Check the documentation for details, and check out other JSON libraries which may be more suitable for your application.
Have not parsed the JSON since the object aspects has these values. Just manually constructing the case class and flatmapping it
scala> final case class Aspects(pairs:Seq[Pair])
defined class Aspects
val namepair = Pair("Name",Seq("Phone"))
namepair: Pair = Pair(Name,List(Phone))
val colorpair = Pair("Color",Seq("Red","Black"))
colorpair: Pair = Pair(Color,List(Red, Black))
val sizepair = Pair("Size",Seq("6","10"))
sizepair: Pair = Pair(Size,List(6, 10))
val aspects = Aspects(Seq(namepair,colorpair,sizepair))
aspects: Aspects = Aspects(List(Pair(Name,List(Phone)), Pair(Color,List(Red, Black)), Pair(Size,List(6, 10))))
aspects.pairs.flatMap(pair=>pair.values.map(value=>s"${pair.name}:::$value"))
res1: Seq[String] = List(Name:::Phone, Color:::Red, Color:::Black, Size:::6, Size:::10)
Related
I'm learning Circe and need help navigating json hierarchies.
Given a json defined as follows:
import io.circe._
import io.circe.parser._
val jStr = """
{
"a1": ["c1", "c2"],
"a2": [{"d1": "abc", "d2": "efg"}, {"d1": "hij", "d2": "klm"}]
}
"""
val j = parse(jStr).getOrElse(Json.Null)
val ja = j.hcursor.downField("a2").as[Json].getOrElse("").toString
ja
ja is now: [ { "d1" : "abc", "d2" : "efg" }, { "d1" : "hij", "d2" : "klm" } ]
I can now do the following to this list:
case class Song(id: String, title: String)
implicit val songDecoder: Decoder[Song] = (c: HCursor ) => for {
id <- c.downField("d1").as[String]
title <- c.downField("d2").as[String]
} yield Song(id,title)
io.circe.parser.decode[List[Song]](ja).getOrElse("")
Which returns what I want: List(Song(abc,efg), Song(hij,klm))
My questions are as follows:
How do I add item a1.c1 from the original json (j) to each item retrieved from the array? I want to add it to Song modified as follows: case class Song(id: String, title: String, artist: String)
It seems wrong to turn the json object back into a String for the iterative step of retrieving id and title. Is there a way to do this without turning json into String?
I have the below json
"atrr": {
"data": {
"id": "asfasfsaf",
"name": "cal",
"type": "state"
"ref": [
"xyz",
"uhz",
"arz"
]
}
}
I am reading this as below but not getting value k,v
def getData: Map[String, String] = (atrr \ "data").asOpt[Map[String, String]].getOrElse(Map[String, String]())
without ref it works fine.how do I ignore ref[] from json in code that is an object
You can use a custom Reads[Map[String, String]], passed to the .as or .asOpt method. My approach is to use Reads.optionNoError[String] to handle the values inside the main atrr object, so that any non-string field which would have caused an error will instead be treated as a None.
// explicitly construct something that'd normally be constructed implicitly
val customReads: Reads[Map[String, Option[String]] =
Reads.map(Reads.optionNoError[String])
// pass the custom reads explicitly with parens,
// instead of passing the expect type in square brackets
(atrr \ "data").asOpt(customReads)
This results in
Some(
Map(
id -> Some(asfasfsaf),
name -> Some(cal),
type -> Some(state),
ref -> None
)
)
which you can transform how you see fit, for example by doing
.map(_.collect { case (k, Some(v)) => k -> v })
my json looks like this.
[
{"name": "AABC Register Ltd (Architects accredited in building conservation), with effect from the 2016 to 2017 tax year"},
{"name": "Academic and Research Surgery Society of"},
{"name": "Academic Gaming and Simulation in Education and Training Society for"},
{"name": "Academic Primary Care Society for"}
]
and the code I use to parse this and add it to mongoDB is this
val sourceOrganisations: JsValue = Json.parse(getClass.getResourceAsStream("/json/ApprovedOrganisations.json"))
val organisations: Seq[String] = (sourceOrganisations.as[JsArray] \\ "name").map(jsval => jsval.toString())
println(organisations)
organisations.foreach(organisation => this.insert(Organisation(organisation)))
when querying mongo i get
{ "_id" : ObjectId("5c3dcf309770222486f50b4a"), "name" : "\"Accounting Association American\"" }
but i want this
{ "_id" : ObjectId("5c3dcf309770222486f50b4a"), "name" : "Accounting Association American" }
I have tried looking at why there might be extra quotes but can't find the reason
The problem is because you are doing a toString on JsValue.
When you do a toString on a JsValue which is a JsString you get a String which has the value "value_inside_JsString"
val jsValue: JsValue = JsString("a")
val extraQuotedString = "[[" + jsValue.toString() + "]]"
// extraQuotedString: String = [["a"]]
The correct way is to first cast it to JsString and the getting its value by .value.
val correctString = "[[" + jsValue.as[JsString].value + "]]"
// correctString: String = [[a]]
So,
val json = Json.parse(jsonString)
val organizationsAsJsValues = json match {
case jsArray: JsArray => jsArray \\ "name"
case _ => Seq.empty[JsValue]
}
val organizationAsNames = organizationsAsJsValues.flatMap({
case jsString: JsString => Some(jsString.value)
case _ => None
})
val organizations = organizationAsNames.map(name => Organization(name))
val organisations: Seq[String] = (sourceOrganisations.as[JsArray] \\ "name").map(jsval => jsval.toString().replaceAll("\"",""))
added
replaceAll("\"","") to line to fix it. have no clue why it added the qoutes though
jsval => jsval.toString()
This will give you the JSON representation of jsval. If that was a JsString, it will be the double quoted (and potentially escaped) text content.
You probably want to parse the JSON into a case class (sourceOrganisation = Json.parse(something).as[TheCaseClass]). That way, you can also avoid doing the manual parsing with jsArray \\ "name"
Or at least the final extract the String from the final jsval (if you know that it's a JsString, you could do jsval.as[String]).
I think there should be an easy solution around, but I wasn't able to find it.
I start accessing data from MongoDB with the following in Scala:
val search = MongoDBObject("_id" -> new ObjectId("xxx"))
val fields = MongoDBObject("community.member.name" -> 1, "community.member.age" -> 1)
for (res <- mongoColl.find(search, fields)) {
var memberInfo = res.getAs[BasicDBObject]("community").get
println(memberInfo)
}
and get a BasicDBObject as result:
{
"member" : [
{
"name" : "John Doe",
"age" : "32",
},{
"name" : "Jane Doe",
"age" : "29",
},
...
]
}
I know that I can access values with getAs[String], though this is not working here...
Anyone has an idea? Searching for a solution for several hours...
If you working with complex MongoDB objects, you can use Salat, which provides simple case class serialization.
Sample with your data:
case class Community(members:Seq[Member], _id: ObjectId = new ObjectId)
case class Member(name:String, age:Int)
val mongoColl: MongoCollection = _
val dao = new SalatDAO[Community, ObjectId](mongoColl) {}
val community = Community(Seq(Member("John Doe", 32), Member("Jane Doe", 29)))
dao.save(community)
for {
c <- dao.findOneById(community._id)
m <- c.members
} println("%s (%s)" format (m.name, m.age))
I think you should try
val member = memberInfo.as[MongoDBList]("member").as[BasicDBObject](0)
println(member("name"))
This problem has not to do really with MongoDB, but rather with your data structure. Your JSON/BSON data structure includes
An object community, which includes
An array of members
Each member has properties name or age.
Your problem is completely equivalent to the following:
case class Community(members:List[Member])
case class Member(name:String, age:Int)
val a = List(member1,member2)
// a.name does not compile, name is a property defined on a member, not on the list
Yes, you can do this beautifully with comprehensions. You could do the following:
for { record <- mongoColl.find(search,fields).toList
community <- record.getAs[MongoDBObject]("community")
member <- record.getAs[MongoDBObject]("member")
name <- member.getAs[String]("name") } yield name
This would work just to get the name. To get multiple values, I think you would do:
for { record <- mongoColl.find(search,fields).toList
community <- record.getAs[MongoDBObject]("community")
member <- record.getAs[MongoDBObject]("member")
field <- List("name","age") } yield member.get(field).toString
I'm using Lift JSON's for-comprehensions to parse some JSON. The JSON is recursive, so e.g. the field id exists at each level. Here is an example:
val json = """
{
"id": 1
"children": [
{
"id": 2
},
{
"id": 3
}
]
}
"""
The following code
var ids = for {
JObject(parent) <- parse(json)
JField("id", JInt(id)) <- parent
} yield id
println(ids)
produces List(1, 2, 3). I was expecting it to product List(1).
In my program this results in quadratic computation, though I only need linear.
Is it possible to use for-comprehensions to match the top level id fields only?
I haven't delved deep enough to figure out why the default comprehension is recursive, however you can solve this by simply qualifying your search root:
scala> for ( JField( "id", JInt( id ) ) <- parent.children ) yield id
res4: List[BigInt] = List(1)
Note the use of parent.children.