Serializing/Deserializing case objects with Gson in Scala - scala

I am using Gson to serialize and deserialize objects, and saving the results in Redis. ie object is serialized into json string then put in Redis, when the object is retreived, it is string then I use Gson.fromjson(str, className) to deserialize into object.
I am beginner with Scala so I assume my usage is incorrect.
I have the following class:
case class Status(id: String, state: State)
where State is the following:
sealed trait State {}
case object COMPLETED_SUCCESSFULLY extends State {}
case object FINISHED_POLLING extends State {}
case object CURRENTLY_DOWNLOADING extends State {}
case object FINISHED_DOWNLOADING extends State {}
case object CURRENTLY_UPLOADING extends State {}
case object FINISHED_UPLOADING extends State {}
I want to serialize Status into a json string then deserialize it back into an object.
But, when I serialize Status using Gson, I get:
"{\"id\":\"foo\",\"state\":{}}"
Why is that?
Ex:
val Status = new Status("foo", COMPLETED_SUCCESSFULLY)
I expect the serialized output to be
"{\"id\":\"foo\",\"state\":\"COMPLETED_SUCCESSFULLY\"}"

By default, case objects are serialized by Gson to empty json objects: {}. You have to write custom serializer to get expected behaviour:
object StateSerializer extends JsonSerializer[State] {
override def serialize(t1: State, t2: Type, jsonSerializationContext: JsonSerializationContext): JsonElement = {
val res = new JsonObject()
res.add("name", new JsonPrimitive(t1.toString))
res
}
}
val gson = new GsonBuilder().registerTypeHierarchyAdapter(classOf[State], StateSerializer)
.registerTypeHierarchyAdapter(classOf[State], StateDeserializer).setPrettyPrinting().create()
println(gson.toJson(COMPLETED_SUCCESSFULLY))
Will print:
{
"name": "COMPLETED_SUCCESSFULLY"
}
Also if you want to transform json to case object you have to implement JsonDeserializer:
object StateDeserializer extends JsonDeserializer[State] {
override def deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext): State = {
val res = json match {
case o: JsonObject if o.has("name") && o.entrySet().size() == 1 =>
val name = o.get("name").getAsString
name match {
case "FINISHED_POLLING" => FINISHED_POLLING
case "FINISHED_DOWNLOADING" => FINISHED_DOWNLOADING
case "FINISHED_UPLOADING" => FINISHED_UPLOADING
case "CURRENTLY_DOWNLOADING" => CURRENTLY_DOWNLOADING
case "CURRENTLY_UPLOADING" => CURRENTLY_UPLOADING
case "COMPLETED_SUCCESSFULLY" => COMPLETED_SUCCESSFULLY
case _ => null
}
case _ => null
}
Option(res).getOrElse(throw new JsonParseException(s"$json can't be parsed to State"))
}
}
println(gson.fromJson("{\"name\": \"COMPLETED_SUCCESSFULLY\"}", classOf[State]))
Will print:
COMPLETED_SUCCESSFULLY

Related

Comparing the json data types at runtime using Jackson and Scala

I have an incoming JSON data that looks like below:
{"id":"1000","premium":29999,"eventTime":"2021-12-22 00:00:00"}
Now, I have created a class that will accept this record and will check whether the data type of the incoming record is according to the data types defined in the case class. However, when I am calling the method it is always calling the Failure part of the match case.
case class Premium(id: String, premium: Long, eventTime: String)
class Splitter extends ProcessFunction[String, Premium] {
val outputTag = new OutputTag[String]("failed")
def fromJson[T](json: String)(implicit m: Manifest[T]): Either[String, T] = {
Try {
println("inside")
lazy val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
mapper.readValue[T](json)
} match {
case Success(x) => {
Right(x)
}
case Failure(err) => {
Left(json)
}
}
}
override def processElement(i: String, context: ProcessFunction[String, Premium]#Context, collector: Collector[Premium]): Unit = {
fromJson(i) match {
case Right(data) => {
collector.collect(data)
println("Good Records: " + data)
}
case Left(json) => {
context.output(outputTag, json)
println("Bad Records: " + json)
}
}
}
}
Based on the sample record above, it should pass the Success value but no matter what I pass, it always enters the Failure part. What else is missing?
I am using Scala 2.11.12 and I tried examples from this link and this link but no luck.

Unable to JSON Marshal B defined as List[A] using AKKA-Http and Spray-Json

as stated in the title, I'm not able to marshal List[A] into the proper Json (Array of Objects).
I'm using AKKA-Http and Spray-Json.
I defined two case classes:
final case class Item(id: String, pid: String, title: String)
final case class Items(items: List[Item])
And on that call GET http://localhost:8080/item received:
class ItemEndpoint extends Directives with ItemJsonSupport {
val route: Route = {
path("item") {
get {
parameters("page".?, "size".?) {
(page, size) => (page, size) match {
case (_, _) =>
onSuccess(Server.requestHandler ? GetItemsRequest){
case response: Items =>
complete(response)
case _ =>
complete(StatusCodes.InternalServerError)
}
}
}
}
}
}
}
GetItemsRequest is called. The latter is defined as
case class GetItemsRequest
And the RequestHandler as
class RequestHandler extends Actor with ActorLogging {
var items : Items = _
def receive: Receive = {
case GetItemsRequest =>
items = itemFactory.getItems
sender() ! items
}
}
Where getItems performs a query on Cassandra via Spark
def getItems() : Items = {
val query = sc.sql("SELECT * FROM mydb")
Items(query.map(row => Item(row.getAs[String]("item"),
row.getAs[String]("pid"), row.getAs[String]("title")
)).collect().toList)
}
returning Items containing List[Item]. All the objects are printed correctly (some of them have null fields).
Using ItemJsonFormatter
trait ItemJsonSupport extends SprayJsonSupport with DefaultJsonProtocol {
implicit val itemFormat: RootJsonFormat[Item] = jsonFormat3(Item)
implicit val itemsFormat: RootJsonFormat[Items] = jsonFormat1(Items)
}
Leads to the following error:
[simple-rest-system-akka.actor.default-dispatcher-9] [akka.actor.ActorSystemImpl(simple-rest-system)] Error during processing of request: 'requirement failed'. Completing with 500 Internal Server Error response. To change default exception handling behavior, provide a custom ExceptionHandler.
java.lang.IllegalArgumentException: requirement failed
I tried catching the exception and work on it but I haven't obtained more intel on the problem.
I followed AKKA DOCS on marshalling.
When doing the same thing, but getting only 1 item, it works just fine, I obtain a json containing all Item's parameters well formatted.
{
"id": "78289232389",
"pid": "B007ILCQ8I",
"title": ""
}
Even looking at other related Q/A I was not able to find an answer, so
What's Causing this? How can I fix it?
All the objects are printed correctly (some of them have null fields).
The exception could be thrown because getItems is returning Item objects that have one or more null field values.
One way to handle this is to replace nulls with empty strings:
def rowToItem(row: Row): Item = {
Item(Option(row.getAs[String]("item")).getOrElse(""),
Option(row.getAs[String]("pid")).getOrElse(""),
Option(row.getAs[String]("title")).getOrElse(""))
}
def getItems(): Items = {
val query = sc.sql("SELECT * FROM mydb")
Items(query.map(row => rowToItem(row)).collect().toList)
}

JSON4S does not serialize internal case class members

I have a case class inheriting from a trait:
trait Thing {
val name: String
val created: DateTime = DateTime.now
}
case class Door(override val name: String) extends Thing
This is akka-http, and I'm trying to return JSON to a get request:
...
~
path ("get" / Segment) { id =>
get {
onComplete(doorsManager ? ThingsManager.Get(id)) {
case Success(d: Door) => {
complete(200, d)
}
case Success(_) => {
complete(404, s"door $id not found")
}
case Failure(reason) => complete(500, reason)
}
}
} ~
...
but I only get the JSON of name. I do have the implicit Joda serializers in scope.
if i override the 'created' timestamp in the constructor of the case class, it does get serialized, but it defines the purpose, as I don't need (or want) the user to provide the timestamp. I've tried moving the timestamp into Door (either as override or just by skipping the trait) and the result is the same (that is, no 'created').
how do I tell JSON4S to serialize internal members (and inherited ones) too?
You have to define a custom format.
import org.json4s.{FieldSerializer, DefaultFormats}
import org.json4s.native.Serialization.write
case class Door(override val name: String) extends Thing
trait Thing {
val name: String
val created: DateTime = DateTime.now
}
implicit val formats = DefaultFormats + FieldSerializer[Door with Thing()]
val obj = new Door("dooor")
write(obj)

Why is data always empty in this akka FSM actor?

Here is the code:
package com.packt.akka
import akka.actor.{ActorSystem, FSM, Props, Stash}
object UserStorageFSM {
// FSM State
sealed trait State
case object Connected extends State
case object Disconnected extends State
// FSM Data
sealed trait Data {
def data: List[User]
}
case object EmptyData extends Data {
val data = List.empty
}
trait DBOperation
object DBOperation {
case object Create extends DBOperation
case object Update extends DBOperation
case object Read extends DBOperation
case object Delete extends DBOperation
}
case object Connect
case object Disconnect
case class Operation(op: DBOperation, user: User)
case class User(username: String, email: String)
}
class UserStorageFSM extends FSM[UserStorageFSM.State, UserStorageFSM.Data] with Stash {
import UserStorageFSM._
// 1. define start with
startWith(Disconnected, EmptyData)
// 2. define states
when(Disconnected) {
case Event(Connect, _) =>
println("UserStorage Connected to DB")
unstashAll()
goto(Connected) using (EmptyData)
case Event(_, _) =>
stash()
stay using (EmptyData)
}
when(Connected) {
case Event(Disconnect, _) =>
println("UserStorage disconnected from DB")
goto(Disconnected) using EmptyData
case Event(Operation(op, user), oldData) =>
op match {
case DBOperation.Create =>
stay using new Data {
override def data = user :: oldData.data
}
case DBOperation.Delete => stay using new Data {
override def data = oldData.data.filter(_ != user)
}
case DBOperation.Read => {
println(oldData.data)
stay using oldData
}
case DBOperation.Update => {
stay using new Data {
override def data: List[User] = user :: oldData.data.filter(_.username != user.username)
}
}
}
stay using EmptyData
}
// 3. initialize
initialize()
}
object FiniteStateMachine extends App {
import UserStorageFSM._
val system = ActorSystem("Hotswap-FSM")
val userStorage = system.actorOf(Props[UserStorageFSM], "userStorage-fsm")
userStorage ! Connect
userStorage ! Operation(DBOperation.Create, User("Admin", "admin#packt.com"))
userStorage ! Operation(DBOperation.Create, User("Admin1", "admin#packt.com"))
userStorage ! Operation(DBOperation.Read, User("Admin", "admin#packt.com"))
userStorage ! Disconnect
Thread.sleep(1000)
system.terminate()
}
It simulates a storage system that allows CRUD operations. The problem here is that the system seems to always contain empty data. What went wrong here?
In your handler for Operation you call stay in the pattern match on op, but then at the bottom you call stay using EmptyData and that is the one that gets used. Remove stay using EmptyData from the bottom of case Event(Operation(op, user), oldData) => and you should start seeing the updated Data.

How to represent optional fields in spray-json?

I have an optional field on my requests:
case class SearchRequest(url: String, nextAt: Option[Date])
My protocol is:
object SearchRequestJsonProtocol extends DefaultJsonProtocol {
implicit val searchRequestFormat = jsonFormat(SearchRequest, "url", "nextAt")
}
How do I mark the nextAt field optional, such that the following JSON objects will be correctly read and accepted:
{"url":"..."}
{"url":"...", "nextAt":null}
{"url":"...", "nextAt":"2012-05-30T15:23Z"}
I actually don't really care about the null case, but if you have details, it would be nice. I'm using spray-json, and was under the impression that using an Option would skip the field if it was absent on the original JSON object.
Works for me (spray-json 1.1.1 scala 2.9.1 build)
import cc.spray.json._
import cc.spray.json.DefaultJsonProtocol._
// string instead of date for simplicity
case class SearchRequest(url: String, nextAt: Option[String])
// btw, you could use jsonFormat2 method here
implicit val searchRequestFormat = jsonFormat(SearchRequest, "url", "nextAt")
assert {
List(
"""{"url":"..."}""",
"""{"url":"...", "nextAt":null}""",
"""{"url":"...", "nextAt":"2012-05-30T15:23Z"}""")
.map(_.asJson.convertTo[SearchRequest]) == List(
SearchRequest("...", None),
SearchRequest("...", None),
SearchRequest("...", Some("2012-05-30T15:23Z")))
}
You might have to create an explicit format (warning: psuedocodish):
object SearchRequestJsonProtocol extends DefaultJsonProtocol {
implicit object SearchRequestJsonFormat extends JsonFormat[SearchRequest] {
def read(value: JsValue) = value match {
case JsObject(List(
JsField("url", JsString(url)),
JsField("nextAt", JsString(nextAt)))) =>
SearchRequest(url, Some(new Instant(nextAt)))
case JsObject(List(JsField("url", JsString(url)))) =>
SearchRequest(url, None)
case _ =>
throw new DeserializationException("SearchRequest expected")
}
def write(obj: SearchRequest) = obj.nextAt match {
case Some(nextAt) =>
JsObject(JsField("url", JsString(obj.url)),
JsField("nextAt", JsString(nextAt.toString)))
case None => JsObject(JsField("url", JsString(obj.url)))
}
}
}
Use NullOptions trait to disable skipping nulls:
https://github.com/spray/spray-json#nulloptions
Example:
https://github.com/spray/spray-json/blob/master/src/test/scala/spray/json/ProductFormatsSpec.scala
Don't know if this will help you but you can give that field a default value in the case class definition, so if the field is not in the json, it will assign the default value to it.
Easy.
import cc.spray.json._
trait MyJsonProtocol extends DefaultJsonProtocol {
implicit val searchFormat = new JsonWriter[SearchRequest] {
def write(r: SearchRequest): JsValue = {
JsObject(
"url" -> JsString(r.url),
"next_at" -> r.nextAt.toJson,
)
}
}
}
class JsonTest extends FunSuite with MyJsonProtocol {
test("JSON") {
val search = new SearchRequest("www.site.ru", None)
val marshalled = search.toJson
println(marshalled)
}
}
For anyone who is chancing upon this post and wants an update to François Beausoleil's answer for newer versions of Spray (circa 2015+?), JsField is deprecated as a public member of JsValue; you should simply supply a list of tuples instead of JsFields. Their answer is spot-on, though.