In the following function, I am parsing a json. I want to return JsError which describes all the errors (if any) in the json. But I am unable to figure out how to keep changing the JsError without using a var. I am looking for a more functional way of writing the code.
implicit object PQReads extends Reads[PQ] {
def reads(json:JsValue):JsResult[PQ] = {
val id = (json \ "id").asOpt[UUID]
val d = (json \ "d").asOpt[String]
val h = (json \"h").asOpt[List[String]]
val i = (json \ "i").asOpt[List[String]]
//HERE I WANT TO ADD LOGIC TO CHECK THAT ALL THE OPTIONS ARE DEFINED.
//THE ONES WHICH ARE NONE SHOULD BE INDICATED BACK IN JSERROR
// (EG D AND I FIELDS ARE MISSING). HOW DO I DO THIS WITHOUT
// USING VAR TYPE FOR JSERROR.
//IF ALL THE FIELDS ARE PRESENT THEN I'LL RETURN PQ INSTANCE IN JSSUCCESS
}
I have not used Play for quite some time, but I don't think you have to do the reading manually.
When you have case class PQ(id: UUID, d :String, h: List[String], i: List[String])
,
you can simply write implicit val pqReads = Json.reads[PQ].
See here for the documentation on the automated mapping.
But if you really want to do it yourself, take a look at the reading combinators.
implicit val pqReads: Reads[PQ] = (
(JsPath \ "id").read[UUID] and
(JsPath \ "d").read[String] and
(JsPath \ "h").read[List[String]] and
(JsPath \ "i").read[List[String]]
)(PQ.apply _)
For the conversions,
val validJson = Json.parse("""
{
"id": "586c154d-1e0f-428c-97bc-200dec9328bb",
"d": "d",
"h": ["h", "i"],
"i": []
}
""")
println(validJson.validate[PQ])
val invalidJson = Json.parse("""
{
"id": "586c154d-1e0f-428c-97bc-200dec9328bb",
"d": "d",
"h": 123
}
""")
println(invalidJson.validate[PQ])
both of the implicit vals provide the same result.
JsSuccess(PQ(586c154d-1e0f-428c-97bc-200dec9328bb,d,List(h, i),List()),)
JsError(List((/i,List(JsonValidationError(List(error.path.missing),WrappedArray()))), (/h,List(JsonValidationError(List(error.expected.jsarray),WrappedArray())))))
With play-json I use always case-classes!
I simplified your problem to the essence:
import play.api.libs.json._
Define a case class
case class PQ(id:UUID, d:String, h:List[String], i: List[String])
Add a formatter in the companion object:
object PQ {
implicit val jsonFormat: Format[PQ] = Json.format[PQ]
}
And use validate:
Json.parse(YOUR_JSON).validate[PQ] match {
case JsSuccess(pq, _) => println(pq)
case JSError(errors) => println(s"Handle exception $other")
}
This returns either the PQ or a List with the errors. No need to do anything here.
Compilation error:
No Json serializer found for type Seq[(models.Account, models.Company)]. Try to implement an implicit Writes or Format for this type.
How can I define an implicit writes for the result of a join query?
Controller:
def someEndpoint = Action.async { implicit request =>
val query = for {
a <- accounts if a.id === 10
c <- companies if a.companyID === c.id
} yield (a, c)
db.run(query.result).map(rows => Ok(Json.toJson(rows))) // Causes compilation error
}
Each of my models (account and company) have their own implicit writes (here's the company one):
case class Company(id: Int, name: String)
object Company {
implicit val writes = new Writes[Company] {
def writes(company: Company): JsValue = {
Json.obj(
"id" -> company.id,
"name" -> company.name
)
}
}
}
Is it possible to dynamically handle serializations for joins? I have a lot of things I will be joining together... Do I need to explicitly define a writes for each combination?
Writes.seq will help you
small writer
val w = (
(__ \ "account").write[Account] and
(__ \ "company").write[Company]
).tupled
helps you can transform Seq[(models.Account, models.Company)] to JsValue with
Writes.seq(w).writes(rows)
and last command will be
db.run(query.result).map(rows => Ok(Writes.seq(w).writes(rows))
or a more clear variant
db.run(query.result)
.map(
_.map{
case (a,c) => Json.obj("account" -> a, "company" -> c)
}
)
.map(rows =>
Ok(JsArray(rows))
)
it's the same thing, but you create object for every row yourself.
I think you expect the response of your query in JSON to be something like
[
{
"account" : { "number": "123", "companyID" : 1 },
"company" : { "id" : 1, "name" : "My company"}
} , ...
]
The problem is the response of the query is just a tuple, so "account" and "company" are not easy to calculate.
Instead of a tuple you could create a new case class with the joined data, but I understand you want to avoid that. In that case you can instead of a tuple use a Map that is something that will convert automatically to JSON.
Extra: Creating writers for case classes is very simple
import play.api.libs.json._
implicit val personWrites = Json.writes[Person]
Reference: https://www.playframework.com/documentation/2.4.x/ScalaJsonInception
I'm learning Json4s library.
I have a json fragment like this:
{
"records":[
{
"name":"John Derp",
"address":"Jem Street 21"
},
{
"name":"Scala Jo",
"address":"in my sweet dream"
}
]
}
And, I have Scala code, which converts a json string into a List of Maps, like this:
import org.json4s._
import org.json4s.JsonAST._
import org.json4s.native.JsonParser
val json = JsonParser.parse( """{"records":[{"name":"John Derp","address":"Jem Street 21"},{"name":"Scala Jo","address":"in my sweet dream"}]}""")
val records: List[Map[String, Any]] = for {
JObject(rec) <- json \ "records"
JField("name", JString(name)) <- rec
JField("address", JString(address)) <- rec
} yield Map("name" -> name, "address" -> address)
println(records)
The output of records to screen gives this:
List(Map(name -> John Derp, address -> Jem Street 21), Map(name ->
Scala Jo, address -> in my sweet dream))
I want to understand what the lines inside the for loop mean. For example, what is the meaning of this line:
JObject(rec) <- json \ "records"
I understand that the json \ "records" produces a JArray object, but why is it fetched as JObject(rec) at left of <-? What is the meaning of the JObject(rec) syntax? Where does the rec variable come from? Does JObject(rec) mean instantiating a new JObject class from rec input?
BTW, I have a Java programming background, so it would also be helpful if you can show me the Java equivalent code for the loop above.
You have the following types hierarchy:
sealed abstract class JValue {
def \(nameToFind: String): JValue = ???
def filter(p: (JValue) => Boolean): List[JValue] = ???
}
case class JObject(val obj: List[JField]) extends JValue
case class JField(val name: String, val value: JValue) extends JValue
case class JString(val s: String) extends JValue
case class JArray(val arr: List[JValue]) extends JValue {
override def filter(p: (JValue) => Boolean): List[JValue] =
arr.filter(p)
}
Your JSON parser returns following object:
object JsonParser {
def parse(s: String): JValue = {
new JValue {
override def \(nameToFind: String): JValue =
JArray(List(
JObject(List(
JField("name", JString("John Derp")),
JField("address", JString("Jem Street 21")))),
JObject(List(
JField("name", JString("Scala Jo")),
JField("address", JString("in my sweet dream"))))))
}
}
}
val json = JsonParser.parse("Your JSON")
Under the hood Scala compiler generates the following:
val res = (json \ "records")
.filter(_.isInstanceOf[JObject])
.flatMap { x =>
x match {
case JObject(obj) => //
obj //
.withFilter(f => f match {
case JField("name", _) => true
case _ => false
}) //
.flatMap(n => obj.withFilter(f => f match {
case JField("address", _) => true
case _ => false
}).map(a => Map(
"name" -> (n.value match { case JString(name) => name }),
"address" -> (a.value match { case JString(address) => address }))))
}
}
First line JObject(rec) <- json \ "records" is possible because JArray.filter returns List[JValue] (i.e. List[JObject]). Here each value of List[JValue] maps to JObject(rec) with pattern matching.
Rest calls are series of flatMap and map (this is how Scala for comprehensions work) with pattern matching.
I used Scala 2.11.4.
Of course, match expressions above are implemented using series of type checks and casts.
UPDATE:
When you use Json4s library there is an implicit conversion from JValue to org.json4s.MonadicJValue. See package object json4s:
implicit def jvalue2monadic(jv: JValue) = new MonadicJValue(jv)
This conversion is used here: JObject(rec) <- json \ "records". First, json is converted to MonadicJValue, then def \("records") is applied, then def filter is used on the result of def \ which is JValue, then it is again implicitly converted to MonadicJValue, then def filter of MonadicJValue is used. The result of MonadicJValue.filter is List[JValue]. After that steps described above are performed.
You are using a Scala for comprehension and I believe much of the confusion is about how for comprehensions work. This is Scala syntax for accessing the map, flatMap and filter methods of a monad in a concise way for iterating over collections. You will need some understanding of monads and for comprehensions in order to fully comprehend this. The Scala documentation can help, and so will a search for "scala for comprehension". You will also need to understand about extractors in Scala.
You asked about the meaning of this line:
JObject(rec) <- json \ "records"
This is part of the for comprehension.
Your statement:
I understand that the json \ "records" produces a JArray object,
is slightly incorrect. The \ function extracts a List[JSObject] from the parser result, json
but why is it fetched as JObject(rec) at left of <-?
The json \ "records" uses the json4s extractor \ to select the "records" member of the Json data and yield a List[JObject]. The <- can be read as "is taken from" and implies that you are iterating over the list. The elements of the list have type JObject and the construct JObject(rec) applies an extractor to create a value, rec, that holds the content of the JObject (its fields).
how come it's fetched as JObject(rec) at left of <-?
That is the Scala syntax for iterating over a collection. For example, we could also write:
for (x <- 1 to 10)
which would simply give us the values of 1 through 10 in x. In your example, we're using a similar kind of iteration but over the content of a list of JObjects.
What is the meaning of the JObject(rec)?
This is a Scala extractor. If you look in the json4s code you will find that JObject is defined like this:
case class JObject(obj: List[JField]) extends JValue
When we have a case class in Scala there are two methods defined automatically: apply and unapply. The meaning of JObject(rec) then is to invoke the unapply method and produce a value, rec, that corresponds to the value obj in the JObject constructor (apply method). So, rec will have the type List[JField].
Where does the rec variable come from?
It comes from simply using it and is declared as a placeholder for the obj parameter to JObject's apply method.
Does JObject(rec) mean instantiating new JObject class from rec input?
No, it doesn't. It comes about because the JArray resulting from json \ "records" contains only JObject values.
So, to interpret this:
JObject(rec) <- json \ "records"
we could write the following pseudo-code in english:
Find the "records" in the parsed json as a JArray and iterate over them. The elements of the JArray should be of type JObject. Pull the "obj" field of each JObject as a list of JField and assign it to a value named "rec".
Hopefully that makes all this a bit clearer?
it's also helpful if you can show me the Java equivalent code for the loop above.
That could be done, of course, but it is far more work than I'm willing to contribute here. One thing you could do is compile the code with Scala, find the associated .class files, and decompile them as Java. That might be quite instructive for you to learn how much Scala simplifies programming over Java. :)
why I can't do this? for ( rec <- json \ "records", so rec become JObject. What is the reason of JObject(rec) at the left of <- ?
You could! However, you'd then need to get the contents of the JObject. You could write the for comprehension this way:
val records: List[Map[String, Any]] = for {
obj: JObject <- json \ "records"
rec = obj.obj
JField("name", JString(name)) <- rec
JField("address", JString(address)) <- rec
} yield Map("name" -> name, "address" -> address)
It would have the same meaning, but it is longer.
I just want to understand what does the N(x) pattern mean, because I only ever see for (x <- y pattern before.
As explained above, this is an extractor which is simply the use of the unapply method which is automatically created for case classes. A similar thing is done in a case statement in Scala.
UPDATE:
The code you provided does not compile for me against 3.2.11 version of json4s-native. This import:
import org.json4s.JsonAST._
is redundant with this import:
import org.json4s._
such that JObject is defined twice. If I remove the JsonAST import then it compiles just fine.
To test this out a little further, I put your code in a scala file like this:
package example
import org.json4s._
// import org.json4s.JsonAST._
import org.json4s.native.JsonParser
class ForComprehension {
val json = JsonParser.parse(
"""{
|"records":[
|{"name":"John Derp","address":"Jem Street 21"},
|{"name":"Scala Jo","address":"in my sweet dream"}
|]}""".stripMargin
)
val records: List[Map[String, Any]] = for {
JObject(rec) <- json \ "records"
JField("name", JString(name)) <- rec
JField("address", JString(address)) <- rec
} yield Map("name" -> name, "address" -> address)
println(records)
}
and then started a Scala REPL session to investigate:
scala> import example.ForComprehension
import example.ForComprehension
scala> val x = new ForComprehension
List(Map(name -> John Derp, address -> Jem Street 21), Map(name -> Scala Jo, address -> in my sweet dream))
x: example.ForComprehension = example.ForComprehension#5f9cbb71
scala> val obj = x.json \ "records"
obj: org.json4s.JValue = JArray(List(JObject(List((name,JString(John Derp)), (address,JString(Jem Street 21)))), JObject(List((name,JString(Scala Jo)), (address,JString(in my sweet dream))))))
scala> for (a <- obj) yield { a }
res1: org.json4s.JValue = JArray(List(JObject(List((name,JString(John Derp)), (address,JString(Jem Street 21)))), JObject(List((name,JString(Scala Jo)), (address,JString(in my sweet dream))))))
scala> import org.json4s.JsonAST.JObject
for ( JObject(rec) <- obj ) yield { rec }
import org.json4s.JsonAST.JObject
scala> res2: List[List[org.json4s.JsonAST.JField]] = List(List((name,JString(John Derp)), (address,JString(Jem Street 21))), List((name,JString(Scala Jo)), (address,JString(in my sweet dream))))
So:
You are correct, the result of the \ operator is a JArray
The "iteration" over the JArray just treats the entire array as the only value in the list
There must be an implicit conversion from JArray to JObject that permits the extractor to yield the contents of JArray as a List[JField].
Once everything is a List, the for comprehension proceeds as normal.
Hope that helps with your understanding of this.
For more on pattern matching within assignments, try this blog
UPDATE #2:
I dug around a little more to discover the implicit conversion at play here. The culprit is the \ operator. To understand how json \ "records" turns into a monadic iterable thing, you have to look at this code:
org.json4s package object: This line declares an implicit conversion from JValue to MonadicJValue. So what's a MonadicJValue?
org.json4s.MonadicJValue: This defines all the things that make JValues iterable in a for comprehension: filter, map, flatMap and also provides the \ and \\ XPath-like operators
So, essentially, the use of the \ operator results in the following sequence of actions:
- implicitly convert the json (JValue) into MonadicJValue
- Apply the \ operator in MonadicJValue to yield a JArray (the "records")
- implicitly convert the JArray into MonadicJValue
- Use the MonadicJValue.filter and MonadicJValue.map methods to implement the for comprehension
Just simplified example, how for-comprehesion works here:
scala> trait A
defined trait A
scala> case class A2(value: Int) extends A
defined class A2
scala> case class A3(value: Int) extends A
defined class A3
scala> val a = List(1,2,3)
a: List[Int] = List(1, 2, 3)
scala> val a: List[A] = List(A2(1),A3(2),A2(3))
a: List[A] = List(A2(1), A3(2), A2(3))
So here is just:
scala> for(A2(rec) <- a) yield rec //will return and unapply only A2 instances
res34: List[Int] = List(1, 3)
Which is equivalent to:
scala> a.collect{case A2(rec) => rec}
res35: List[Int] = List(1, 3)
Collect is based on filter - so it's enough to have filter method as JValue has.
P.S. There is no foreach in JValue - so this won't work for(rec <- json \ "records") rec. But there is map, so that will: for(rec <- json \ "records") yield rec
If you need your for without pattern matching:
for {
rec <- (json \ "records").filter(_.isInstanceOf[JObject]).map(_.asInstanceOf[JObject])
rcobj = rec.obj
name <- rcobj if name._1 == "name"
address <- rcobj if address._1 == "address"
nm = name._2.asInstanceOf[JString].s
vl = address._2.asInstanceOf[JString].s
} yield Map("name" -> nm, "address" -> vl)
res27: List[scala.collection.immutable.Map[String,String]] = List(Map(name -> John Derp, address -> Jem Street 21), Map(name -> Scala Jo, address -> in my sweet dream))
Thanks to the answers to my previous question, I was able to create a function macro such that it returns a Map that maps each field name to its value of a class, e.g.
...
trait Model
case class User (name: String, age: Int, posts: List[String]) extends Model {
val numPosts: Int = posts.length
...
def foo = "bar"
...
}
So this command
val myUser = User("Foo", 25, List("Lorem", "Ipsum"))
myUser.asMap
returns
Map("name" -> "Foo", "age" -> 25, "posts" -> List("Lorem", "Ipsum"), "numPosts" -> 2)
This is where Tuples for the Map are generated (see Travis Brown's answer):
...
val pairs = weakTypeOf[T].declarations.collect {
case m: MethodSymbol if m.isAccessor =>
val name = c.literal(m.name.decoded)
val value = c.Expr(Select(model, m.name))
reify(name.splice -> value.splice).tree
}
...
Now I want to ignore fields that have #transient annotation. How would I check if a method has a #transient annotation?
I'm thinking of modifying the snippet above as
val pairs = weakTypeOf[T].declarations.collect {
case m: MethodSymbol if m.isAccessor && !m.annotations.exists(???) =>
val name = c.literal(m.name.decoded)
val value = c.Expr(Select(model, m.name))
reify(name.splice -> value.splice).tree
}
but I can't find what I need to write in exists part. How would I get #transient as an Annotation so I could pass it there?
Thanks in advance!
The annotation will be on the val itself, not on the accessor. The easiest way to access the val is through the accessed method on MethodSymbol:
def isTransient(m: MethodSymbol) = m.accessed.annotations.exists(
_.tpe =:= typeOf[scala.transient]
)
Now you can just write the following in your collect:
case m: MethodSymbol if m.isAccessor && !isTransient(m) =>
Note that the version of isTransient I've given here has to be defined in your macro, since it needs the imports from c.universe, but you could factor it out by adding a Universe argument if you're doing this kind of thing in several macros.
I am using built-in jerson with playframework 2, and all I want is to serialize map, containing values of different type:
object AppWriter extends Writes[Application] {
def writes(app: Application): JsValue = {
Json.toJson(Map(
"id" -> app.getId.toString,
"name" -> app.getName,
"users" -> Seq(1, 2, 3)
))
}
}
In this case I have:
No Json deserializer found for type scala.collection.immutable.Map[java.lang.String,java.lang.Object].
Try to implement an implicit Writes or Format for this type
Navigatin through framework code shows that there is serializer for Map[String,V] types implicit def mapWrites[V] .. , but I cannot understand why it doesn't applied.
Can anybody help me?
UPD: I found simple workaround:
object AppWriter extends Writes[Application] {
def writes(app: Application): JsValue = {
Json.toJson(Map[String, JsValue](
"id" -> JsNumber(BigDecimal(app.getId)),
"name" -> JsString(app.getName),
"users" -> JsArray(Seq(1, 2, 3).map(x => JsNumber(x)))
))
}
}
but this is not so elegant...
The standard way to do this is by creating a JsObject from the individual key-value pairs for the fields—not by putting the pairs into a map. For example, assuming your Application looks like this:
case class Application(getId: Int, getName: String)
You could write:
import play.api.libs.json._, Json.toJson
implicit object AppWriter extends Writes[Application] {
def writes(app: Application): JsValue = JsObject(
Seq(
"id" -> JsNumber(app.getId),
"name" -> JsString(app.getName),
"users" -> toJson(Seq(1, 2, 3))
)
)
}
And then:
scala> toJson(Application(1, "test"))
res1: play.api.libs.json.JsValue = {"id":1,"name":"test","users":[1,2,3]}
Note that you don't need to spell out how to serialize the Seq[Int]—the default Format instances will do the work for you.