I am using circe optics like this
import io.circe.parser._
import io.circe.optics._
import io.circe.optics.JsonPath._
val json = parse("""{"response": {"person": {"firstname": "foo", "lastname":"bar"}}}""").right.get
Now I want to extract the whole person object in string form ... from this json like
val p = root.response.person.string
and then decode it into a case class like
case class Person(firstname: String, lastname: String)
decode[Person](p.getOption(json).get)
But it doesn't work because root.response.person.string returns null. I think it only works on actual string and integer columns.
So can circe optics be used to extract entire sections of json (for example the person object inside of json)? and then that section is decoded into a case class?
This gets it done. No need to get the string in between, just work with the Json
object Some extends App {
import io.circe.optics.JsonPath._
import io.circe.parser._
import io.circe._
import io.circe.generic.semiauto._
val json = parse("""{"response": {"person": {"firstname": "foo", "lastname":"bar"}}}""").right.get
// this is just a lense to the person, not the person yet
val personJsonPath = root.response.person.json
case class Person(firstname: String, lastname: String)
implicit val personDecoder: Decoder[Person] = deriveDecoder[Person]
val maybePerson = personJsonPath
// here you get the person out
.getOption(json)
// transforming json directly to case class, error handling should not be done like this ;)
.map(_.as[Person].fold(throw _, identity))
println(maybePerson)
}
Related
I am not sure if this is achievable and I have a very basic understanding of how generics work in scala. But I was wondering if this is possible.
Say I have a method:
case class Person(id:String,name:String)
case class Student(id:String,name:String, class:String)
def convertToJson[A](fileName:String):{
//read file
parse[A]
}
Is it possible to write this generic method which would parse the json based on the type of class I send when I call the convertToJson method?
Something like:
convertToJson[Student](fileName)
convertToJson[Person](fileName)
BTW the above code gives me a :
No Manifest available for A. error.
Using json4s for parsing.
Any help is appreciated.
This will convert a JSON string to a case class
import org.json4s._
import org.json4s.jackson.JsonMethods._
def convertToJson[T](json: String)(implicit fmt: Formats = DefaultFormats, mf: Manifest[T]): T =
Extraction.extract(parse(json))
Once this is defined you can parse appropriate strings to the required type:
case class Person(id: String, name: String)
case class Student(id: String, name: String, `class`: String)
val person = convertToJson[Person]("""{"name":"Jane","id":45}""")
val student = convertToJson[Student]("""{"name":"John","id":63, "class": "101"}""")
Note that this will ignore JSON data that does not match fields in the case class. If a field is optional in the JSON, make it an Option in the case class and you will get None if the field is not there.
json4s allows user to convert a JsonAST object to a case class using extract.
import org.json4s._
import org.json4s.jackson.JsonMethods._
implicit val formats = DefaultFormats
case class Item(name: String, price: Double)
val json = parse("""{"name": "phone", "price": 1000.0}""") // JObject(List((name,JString(phone)), (price,JDouble(1000.0))))
val item = json.extract[Item] // Item(phone,1000.0)
However, to convert a case class into a JsonAST object, the only way I can think of is:
serialize a case class using write
deserialize a string using extract
Like below:
parse(write(item)) // JObject(List((name,JString(phone)), (price,JDouble(1000.0))))
Is there any better way for the conversion? Thank you!
Extraction.decompose converts a case class object into a JsonAST.
Extraction.decompose(item) // JObject(List((name,JString(phone)), (price,JDouble(1000.0))))
As the title states, I am trying to rename fields on generated json from case classes using Json4s.
If I try to rename fields on simple case classes like:
case class User(name: String, lastName: String)
Following examples that you can find in the documentation of json4s or here How can I rename a field during serialization with Json4s? will work.
But documentation does not mention how to do nested object renames like for example from deviceId to did in this example:
case class User(name: String, lastName: String, details: UserDetails)
case class UserDetails(deviceId: String)
I tried using things like:
FieldSerializer.renameFrom("deviceId", "did")
or
FieldSerializer.renameFrom("details.deviceId", "details.did")
or
parse(message) transformField {
case ("deviceId", value) => ("did", value)
}
or
parse(message) transformField {
case ("details.deviceId", value) => ("details.did", value)
}
And none of them worked, so my question is: Is this nested rename possible on scala4s? If yes, how can I do to for example rename deviceId to did?
For the nested object, you can create FieldSerializer to bind this nested type, like:
import org.json4s._
import org.json4s.FieldSerializer._
import org.json4s.jackson.Serialization.write
val rename = FieldSerializer[UserDetails](renameTo("deviceId", "did")) // bind UserDetails to FieldSerializer
implicit val format: Formats = DefaultFormats + rename
println(write(u))
> {"name":"name","lastName":"lastName","details":{"did":"deviceId"}}
I'm trying to define HttpService that receives json and parses it to case class with json4s library:
import org.http4s._
import org.http4s.dsl._
import org.json4s._
import org.json4s.native.JsonMethods._
case class Request(firstName: String, secondName: String)
HttpService {
case req # POST -> Root =>
val request = parse(<map req.body or req.bodyAsText to JsonInput>).extract[Request]
Ok()
}
How can I get org.json4s.JsonInput from req.body or req.bodyAsText?
I know that json4s also have StringInput and StreamInput that inherits from JsonInput for using with String and InputStream so I think that I need to convert req.body to InputStream or req.bodyAsText to String but I still do not understand how.
I'm new to Scala and I do not yet fully understand some concepts such as scalaz.stream.Process.
You can use the http4s-json4s-jackson (or http4s-json4s-native) packages and use an org.http4s.EntityDecoder to easily get a Foo (I renamed your Request case class to Foo below) from a request.
EntityDecoder is a type class which can decode an entity from the request body.
We want to get the Foo posted in JSON, so we need to create an EntityDecoder[Foo] which can decode JSON. If we want to create this decoder using json4s we need a Reader (or a JsonFormat).
If you have an EntityDecoder[Foo] instance, we can get the Foo from the request with req.as[Foo].
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.http4s._
import org.http4s.dsl._
import org.http4s.json4s.jackson._
case class Foo(firstName: String, secondName: String)
// create a json4s Reader[Foo]
implicit val formats = DefaultFormats
implicit val fooReader = new Reader[Foo] {
def read(value: JValue): Foo = value.extract[Foo]
}
// create a http4s EntityDecoder[Foo] (which uses the Reader)
implicit val fooDec = jsonOf[Foo]
val service = HttpService {
case req # POST -> Root =>
// req.as[Foo] gives us a Task[Foo]
// and since Ok(...) gives a Task[Response] we need to use flatMap
req.as[Foo] flatMap ( foo => Ok(foo.firstName + " " + foo.secondName) )
}
Note: The json libraries libraries used most often with http4s are probably argonaut and circe. So you might find more http4s examples using one of those libraries.
Peter's solution both corrects the question and answers it, but I stumbled here looking for the solution to OP's stated, but not intended, question: "how to get request body as [...] InputStream" in http4s. Thanks to the discussion in Issue 634 on GitHub, here's what I came up with:
import java.io.InputStream
import org.http4s._
implicit val inputStreamDecoder: EntityDecoder[InputStream] =
EntityDecoder.decodeBy(MediaRange.`*/*`) { msg =>
DecodeResult.success(scalaz.stream.io.toInputStream(msg.body))
}
And then in your HttpService, use that decoder like so:
request.as[InputStream].flatMap { inputStream => ...inputStream is an InputStream... }
Or skip the whole Decoder dance, if you want:
val inputStream = scalaz.stream.io.toInputStream(request.body)
You may use flatMap and as inside it before calling the Http4s service to decode responses from it:
#Test def `Get json gives valid contact`: Unit = {
val request = Request[IO](GET, uri"/contact")
val io = Main.getJsonWithContact.orNotFound.run(request)
// here is magic
val response = io.flatMap(_.as[Json]).unsafeRunSync()
val contact = contactEncoder(Contact(1, "Denis", "123")) // this is encoding to json for assertion
assertEquals(contact, response)
}
This is how types work here:
val io: IO[Response[IO]] = Main.getJsonWithContact.orNotFound.run(request)
val response: IO[Json] = io.flatMap(_.as[Json])
val res: Json = response.unsafeRunSync()
as[String] will return the string just like this.
I'm trying to use "pickling" serialization is Scala, and I see the same example demonstrating it:
import scala.pickling._
import json._
val pckl = List(1, 2, 3, 4).pickle
Unpickling is just as easy as pickling:
val lst = pckl.unpickle[List[Int]]
This example raises some question. First of all, it skips converting of object to string. Apparently you need to call pckl.value to get json string representation.
Unpickling is even more confusing. Deserialization is an act of turning string (or bytes) into an object. How come this "example" demonstrates deserialization if there is no string/binry representation of object?
So, how do I deserialize simple object with pickling library?
Use the type system and case classes to achieve your goals. You can unpickle to some superior type in your hierarchy (up to and including AnyRef). Here is an example:
trait Zero
case class One(a:Int) extends Zero
case class Two(s:String) extends Zero
object Test extends App {
import scala.pickling._
import json._
// String that can be sent down a wire
val wire: String = Two("abc").pickle.value
// On the other side, just use a case class
wire.unpickle[Zero] match {
case One(a) => println(a)
case Two(s) => println(s)
case unknown => println(unknown.getClass.getCanonicalName)
}
}
Ok, I think I understood it.
import scala.pickling._
import json._
var str = Array(1,2,3).pickle.value // this is JSON string
println(str)
val x = str.unpickle[Array[Int]] // unpickle from string
will produce JSON string:
{
"tpe": "scala.Array[scala.Int]",
"value": [
1,
2,
3
]
}
So, the same way we pickle any type, we can unpickle string. Type of serialization is regulated by implicit formatter declared in "json." and can be replaced by "binary."
It does look like you will be starting with a pickle to unpickle to a case class. But the JSON string can be fed to the JSONPickle class to get the starting pickle.
Here's an example based on their array-json test
package so
import scala.pickling._
import json._
case class C(arr: Array[Int]) { override def toString = s"""C(${arr.mkString("[", ",", "]")})""" }
object PickleTester extends App {
val json = """{"arr":[ 1, 2, 3 ]}"""
val cPickle = JSONPickle( json )
val unpickledC: C = cPickle.unpickle[C]
println( s"$unpickledC, arr.sum = ${unpickledC.arr.sum}" )
}
The output printed is:
C([1,2,3]), arr.sum = 6
I was able to drop the "tpe" in from the test as well as the .stripMargin.trim on the input JSON from the test. It works all in one line, but I thought it might be more apparent split up. It's unclear to me if that "tpe" from the test is supposed to provide a measure of type safety for the incoming JSON.
Looks like the only other class they support for pickling is a BinaryPickle unless you want to roll your own. The latest scala-pickling snapshot jar requires quasiquotes to compile the code in this answer.
I tried someting more complicated this morning and discovered that the "tpe" is required for non-primatives in the incoming JSON - which points out that the serialized string really must be compatible with the pickler( which I mixed into the above code ):
case class J(a: Option[Boolean], b: Option[String], c: Option[Int]) { override def toString = s"J($a, $b, $c)" }
...
val jJson = """{"a": {"tpe": "scala.None.type"},
| "b":{"tpe": "scala.Some[java.lang.String]","x":"donut"},
| "c":{"tpe": "scala.Some[scala.Int]","x":47}}"""
val jPickle = JSONPickle( jJson.stripMargin.trim )
val unpickledJ: J = jPickle.unpickle[J]
println( s"$unpickledJ" )
...
where naturually, I had to use .value on a J(None, Some("donut"), Some(47)) to figure out how to create the jJson input value to prevent the unpickling from throwing an exception.
The output for J is like:
J(None, Some(donut), Some(47))
Looking at this test, it appears that if the incoming JSON is all primatives or case classes (or combinations) that the JSONPickle magic works, but some other classes like Options require extra "tpe" type information to unpickle correctly.