I am attempting to perform an XPath based extraction using Lift JSON except that the xpath pattern of extraction is determined during runtime
To illustrate, I'd like to convert string "a.b.c.d" to Lift JSON extraction using (json \ "a" \ "b" \ "c").extract[List[Int]]
import net.liftweb.json._
import net.liftweb.json.JsonDSL._
import net.liftweb.json.JsonAST._
import net.liftweb.json.Extraction._
implicit val formats = net.liftweb.json.DefaultFormats
val x = """{ "a" : { "b" : [ {"c" : 10},{ "c" : 20 } ] } }"""
val json = parse(x)
val dataString = "a.b.c"
val dataList = dataString.split("\\.").toList
// List(a,b,c)
// I'd want to convert the above string to - (json \ "a" \ "b" \ "c").extract[List[Int]]
Is it possible to use foldLeft to achieve this pattern?
If I understand your question, this is what you want:
List("a", "b", "c").foldLeft(json){ _ \ _ }.extract[List[Int]]
And to help understand why:
foldLeft takes two arguments; an initial state, and a function that takes the "current" state and the next input, returning the "next" state. The "inputs" are the elements of the thing you're calling foldLeft on, i.e. the List("a", "b", "c").
So if you use json as your initial state, the first state change call will be json \ "a". The next will be the result of that calculation, backslash "b", and so on.
Related
Say I have some params coming from our API clients like this:
val params = (request \ "params")
I want to filter them and remove certain key/values. Like if I get:
{
"foo": "bar",
"hello": "world"
}
I want to filter it to
{
"foo": "bar"
}
Here's my WIP code but, as more advanced Scala people will probably tell right away, it doesn't work.
val params = (request \ "params").get.as[List[JsObject]]
val blacklistedParams = Seq("foo")
val approvedParams = params.filter((param: JsObject) => {
!blacklistedParams.contains(param)
})
That first line always fails. I've tried doing .get.as in all sorts of types but always get errors. I'm still new to Scala and types in general.
I think I figured out a way:
val params = (request \ "params").get.as[Map[String, JsValue]]
val blacklistedParams = Seq("foo")
val approvedParams = params.filter((param) => {
!blacklistedParams.contains(param._1)
})
My only annoyance with this method is that ._1. To me its not very clear for the next person that this is the key of the key/val pair of the params.
You can use -
val filtered = (request \ "params").as[JsObject] - "hello"
Full example:
def index = Action{
val json = Json.obj(
"params" -> Json.obj(
"foo" -> "bar",
"hello" -> "world"))
val filtered = (json \ "params").as[JsObject] - "hello"
Ok(filtered)
}
output:
{
foo: "bar"
}
I am trying to parse a Travis-ci api response which has the following structure :
{
repos: [
{"id": ..., "slug": ...},
{"id": ..., "slug": ...},
{"id": ..., "slug": ...}
]
}
So I have decided to create case classes reflecting the json structure :
case class TravisRepository(id: String, slug: String)
case class TravisUserRepositories(repos: Seq[TravisRepository])
And I have added the implicit Read methods :
implicit val travisRepositoryReads: Reads[TravisRepository] = (
(JsPath \ "id").read[String] and
(JsPath \ "slug").read[String]
)(TravisRepository.apply _)
implicit val travisUserRepositoriesReads: Reads[TravisUserRepositories] = (
(JsPath \ "repos").read[Seq[TravisReposity]]
)(TravisUserRepositories.apply _)
However the second read is not compiling with the following error :
Overloaded method value [read] cannot be applied to (Seq[utils.TravisRepository] => utils.TravisUserRepositories)
When adding another column to the second Read, this compiles. With a single column, this is not compiling anymore. Can someone explain why is this not compiling? Is it a limitation of the Play-Json parser?
That's simply because you have the case "only one single field in your case class"...
To be able to use the Functional combining, you need at least 2 fields.
// would be OK implicit val travisUserRepositoriesReads:
Reads[TravisUserRepositories] = (
(JsPath \ "repos").read[Seq[TravisReposity]] and
... )(TravisUserRepositories.apply _)
// should be OK implicit val travisUserRepositoriesReads:
Reads[TravisUserRepositories] = (
(JsPath \ "repos").read[Seq[TravisReposity]] map (TravisUserRepositories.apply _)
using net.liftweb.json what is the difference \ and \ operators when parsing json ?
import net.liftweb.json._
val parsed = JsonParser.parse(jsonString)
val name = parsed.\("firstName")
val userId = parsed.\\("userId")
"\\" will extract the value even if it's present within nested json whereas "\" will extract the value only if present as a top-level attribute.
Consider this json
val json = """{"nested1":{"nested2": {"myKey":"myValue"}}}"""
val jsonMsg = parse(json)
In this case
(jsonMsg \ "myKey").values
retruns None
where as
(jsonMsg \\ "myKey").values
returns myValue
I am experimenting with the json4s library (based on lift-json). One of the things I would like to do is to parse a JSON string into an AST, and then manipulate it.
For example, I would like to upsert a field (insert the field into the AST if it does not exist, or update its value if it does).
I have not been able to find how to do it in the documentation. Experimenting with the available methods, I have come up with the following, which works, but feels clumsy.
import org.json4s._
import org.json4s.JsonDSL._
import org.json4s.native.JsonMethods._
object TestJson {
implicit val formats = DefaultFormats
def main(args: Array[String]): Unit = {
val json = """{"foo":1, "bar":{"foo":2}}"""
val ast = parse(json).asInstanceOf[JObject]
println( upsertField(ast, ("foo" -> "3")) )
println( upsertField(ast, ("foobar" -> "3")) )
}
def upsertField(src:JObject, fld:JField): JValue = {
if(src \ fld._1 == JNothing){
src ~ fld
}
else{
src.replace(List(fld._1), fld._2)
}
}
}
I dislike it for many reasons:
Having to explicitly cast the results of parse(json) to JObject
The result of the upsertField function is a JValue, which I will have to recast if I want to manipulate the object further
The upsertField function just feels very unelegant
It does not work for fields that are not at the top level of the hierarchy
Is there a better way to transform the AST?
EDIT: as a workaround to the problem, I have managed to convert my JSON to Scala regular classes, and manipulate them with lenses (Using Lenses on Scala Regular Classes)
There is the merge function which creates or overrides a field. You can also update fields that are not at the root level of the tree.
import org.json4s._
import org.json4s.JsonDSL._
import org.json4s.jackson.JsonMethods._
object mergeJson extends App {
val json =
"""
|{
| "foo":1,
| "bar": {
| "foo": 2
| }
|}
|""".stripMargin
val ast = parse(json)
val updated = ast merge (("foo", 3) ~ ("bar", ("fnord", 5)))
println(pretty(updated))
// {
// "foo" : 3,
// "bar" : {
// "foo" : 2,
// "fnord" : 5
// }
// }
}
Let me also give you the SON of JSON version:
import nl.typeset.sonofjson._
val json = parse("""{ "foo" : 1, "bar" : { "foo" : 2 } }""")
// or, perhaps a little easier
val json = obj(foo = 1, bar = obj(foo = 2))
json.foo = "3"
json.foobar = "3"
When I was implementing some very specific json diff using lift json I used a lot of recursive functions to get to the jpath where I need to modify value, and modified json was constructed when recursion "collapsed". LiftJson is immutable after all. You mentioned lenses as another approach, which is very interesting by itself. But my current favourite is play-json library that is working like a charm in a situation when you need to do json-to-json transformation:
from Mandubian Blog:
val gizmo2gremlin = (
(__ \ 'name).json.put(JsString("gremlin")) and
(__ \ 'description).json.pickBranch(
(__ \ 'size).json.update( of[JsNumber].map{ case JsNumber(size) => JsNumber(size * 3) } ) and
(__ \ 'features).json.put( Json.arr("skinny", "ugly", "evil") ) and
(__ \ 'danger).json.put(JsString("always"))
reduce
) and
(__ \ 'hates).json.copyFrom( (__ \ 'loves).json.pick )
) reduce
Yummy Features: all transformers are combinators that can be mixed together, validation, shapeless support, auto marshaling of case classes with implicit overrides, stand-alone library.
I'm using https://github.com/json4s/json4s. How do I convert its values such as JDouble, JBool to corresponding Scala's data types -- Double and Boolean?
UPDATE:
scala> (json \ "status")
res8: org.json4s.JValue = JBool(false)
scala> (json \ "status").extract[Boolean]
<console>:16: error: No org.json4s.Formats found. Try to bring an instance of org.json4s.Formats in scope or use the org.json4s.DefaultFormats.
(json \ "status").extract[Boolean]
As mentioned in Read Me, heres how you do it.. :)
import org.json4s._
import org.json4s.native.JsonMethods._
implicit val formats = DefaultFormats
val json = parse("""
{
"mydouble" : 3.14,
"isPie" : true
}
""")
val dbl = (json \ "mydouble").extractOpt[Double]
//> dbl : Option[Double] = Some(3.14)
val bool = (json \ "isPie").extractOpt[Boolean]
//> bool : Option[Boolean] = Some(true)
Looking at the code (https://github.com/json4s/json4s/blob/7c70e9664232ffee4acf24c8969424cd37957736/ast/src/main/scala/org/json4s/JsonAST.scala) shows that you just need to call the JValue.values method.
UPDATE: From your comment it seems that it's not so much that you have a JDouble and want to extract its Double value (and similarly extract a Boolean from a JDouble. Instead, you have a JValue
and want to extract its value as a Double or Boolean (knowing in advance the expected type).
This can be done with extract, as explained in the README that you linked to:
(json \ "status").extract[Double]
or:
(json \ "status").extract[Boolean]
See also this test file for more examples:
https://github.com/json4s/json4s/blob/master/tests/src/test/scala/org/json4s/ExtractionExamplesSpec.scala