Basic Create-from-Json Method Walkthrough - scala

I'm new to play, scala, and reactivemongo and was wondering if someone could explain to me the following code in easy terms to understand.
def createFromJson = Action.async(parse.json) { request =>
import play.api.libs.json.Reads._
val transformer: Reads[JsObject] =
Reads.jsPickBranch[JsString](__ \ "name") and
Reads.jsPickBranch[JsNumber](__ \ "age") and
Reads.jsPut(__ \ "created", JsNumber(new java.util.Date().getTime())) reduce
request.body.transform(transformer).map { result =>
collection.insert(result).map { lastError =>
Logger.debug(s"Successfully inserted with LastError: $lastError")
Created
}
}.getOrElse(Future.successful(BadRequest("invalid json")))}
I know that it creates a user from a JSON user with name and age attributes. What I don't understand is the way that input JSON is read in this method. ALSO the concept of Action.async(par.json), request => getorElse, Future, etc.
ALSO any easier/simpler ways of writing this method would be greatly appreciated.
Thanks in advance!

I believe you found this code in a template I have made following excellent reactive mongo documentation.
http://reactivemongo.org/releases/0.11/documentation/index.html
http://reactivemongo.org/releases/0.11/documentation/tutorial/play2.html
I feel a bit obliged to explain it. Let's run through the code.
def createFromJson = Action.async(parse.json) { request =>
The function createFromJson will return an Action (play stuff) that is asynchronous (returns a future of a result) that handles a body in json format. To do that it will use the request.
Documentation: https://www.playframework.com/documentation/2.5.x/ScalaAsync
A json can be anything that follows the json formats, for example an array a String, an object, ...
Our transformer is going to take only the data that we are interested in from the json and will return a clean json object
val transformer: Reads[JsObject] =
Reads.jsPickBranch[JsString](__ \ "name") and
Reads.jsPickBranch[JsNumber](__ \ "age") and
Reads.jsPut(__ \ "created", JsNumber(new java.util.Date().getTime())) reduce
As you see, it will pick the branch name as a string and the branch age as a number. It will also add to the final json object a field created with the time of creation.
As you see we are not transforming it to a Person instance, it is just a JsObject instance as it is defined in
val transformer: Reads[JsObject] ....
Play offers you a few ways to handle json in a simpler way. This examples tries to show the power of manipulating directly the json values without converting to a model.
For example if you have a case class
case class Person(name: String, age: Int)
You could create automatically a reads from it,
val personReads: Person[Person] = Json.reads[Person]
But to just store it in Mongo
DB there is no reason to build this instance and then transform it to json again.
Of course if you need to do some logic with the models before inserting them, you might need to create the model.
Documentation:
https://www.playframework.com/documentation/2.5.x/ScalaJson
https://www.playframework.com/documentation/2.5.x/ScalaJsonCombinators
https://www.playframework.com/documentation/2.5.x/ScalaJsonAutomated
https://www.playframework.com/documentation/2.5.x/ScalaJsonTransformers
With this in mind the rest of the code should be clear
request.body.transform(transformer).map { result =>
collection.insert(result).map { lastError =>
Logger.debug(s"Successfully inserted with LastError: $lastError")
Created
}
}
From the request, we take the body (a JsValue) we transform it into a JsObject (result) and we insert it in the collection.
Insert returns a Future with the last error, when the Person is stored last error will be logged and a Created (201 code) will be returned to the client of the API.
The last bit should be also clear by now
}.getOrElse(Future.successful(BadRequest("invalid json")))
If there is any problem parsing and transforming the json body of the request into our JsObject an already completed future with the result BadRequest (400 code) will be returned to the client.
It is a future because Action.Async needs future of result as the return type.
Enjoy scala.

Related

How to iterate over result of Future List in Scala?

I am new to Scala and was trying my hands on with akka. I am trying to access data from MongoDB in Scala and want to convert it into JSON and XML format.
This code attached below is using path /getJson and calling getJson() function to get data in a form of future.
get {
concat(
path("getJson"){
val f = Patterns.ask(actor1,getJson(),10.seconds)
val res = Await.result(f,10.seconds)
val result = res.toString
complete(res.toString)
}
}
The getJson() method is as follows:
def getJson()= {
val future = collection.find().toFuture()
future
}
I have a Greeting Case class in file Greeting.scala:
case class Greeting(msg:String,name:String)
And MyJsonProtocol.scala file for Marshelling of scala object to JSON format as follows:
trait MyJsonProtocol extends SprayJsonSupport with DefaultJsonProtocol {
implicit val templateFormat = jsonFormat2(Greeting)
}
I am getting output of complete(res.toString) in Postman as :
Future(Success(List(
Iterable(
(_id,BsonObjectId{value=5fc73944986ced2b9c2527c4}),
(msg,BsonString{value='Hiiiiii'}),
(name,BsonString{value='Ruchirrrr'})
),
Iterable(
(_id,BsonObjectId{value=5fc73c35050ec6430ec4b211}),
(msg,BsonString{value='Holaaa Amigo'}),
(name,BsonString{value='Pablo'})),
Iterable(
(_id,BsonObjectId{value=5fc8c224e529b228916da59d}),
(msg,BsonString{value='Demo'}),
(name,BsonString{value='RuchirD'}))
)))
Can someone please tell me how to iterate over this output and to display it in JSON format?
When working with Scala, its very important to know your way around types. First step toweards this is at least knowing the types of your variables and values.
If you look at this method,
def getJson() = {
val future = collection.find().toFuture()
future
}
Is lacks the type type information at all levels, which is a really bad practice.
I am assuming that you are using mongo-scala-driver. And your collection is actually a MongoCollection[Document].
Which means that the output of collection.find() should be a FindOberservable[Document], hence collection.find().toFuture() should be a Future[Seq[Document]]. So, your getJson method should be written as,
def getJson(): Future[Seq[Document]] =
collection.find().toFuture()
Now, this means that you are passing a Future[Seq[Document]] to your actor1, which is again a bad practice. You should never send any kind of Future values among actors. It looks like your actor1 does nothing but sends the same message back. Why does this actor1 even required when it does nothing ?
Which means your f is a Future[Future[Seq[Document]]]. Then you are using Await.result to get the result of this future f. Which is again an anti-pattern, since Await blocks your thread.
Now, your res is a Future[Seq[Document]]. And you are converting it to a String and sending that string back with complete.
Your JsonProtocol is not working because you are not even passing it any Greeting's.
You have to do the following,
Read raw Bson objects from mongo.
convert raw Bson objects to your Gretting objects.
comlete your result with these Gretting objects. The JsonProtocol should take case of converting these Greeting objects to Json.
The easist way to do all this is by using the mongo driver's CodecRegistreis.
case class Greeting(msg:String, name:String)
Now, your MongoDAL object will look like following (it might be missing some imports, fill any missing imports as you did in your own code).
import org.mongodb.scala.bson.codecs.Macros
import org.mongodb.scala.bson.codecs.DEFAULT_CODEC_REGISTRY
import org.bson.codecs.configuration.CodecRegistries
import org.mongodb.scala.{MongoClient, MongoCollection, MongoDatabase}
object MongoDAL {
val greetingCodecProvider = Macros.createCodecProvider[Greeting]()
val codecRegistry = CodecRegistries.fromRegistries(
CodecRegistries.fromProviders(greetingCodecProvider),
DEFAULT_CODEC_REGISTRY
)
val mongoClient: MongoClient = ... // however you are connecting to mongo and creating a mongo client
val mongoDatabase: MongoDatabase =
mongoClient
.getDatabase("database_name")
.withCodecRegistry(codecRegistry)
val greetingCollection: MongoCollection[Greeting] =
mongoDatabase.getCollection[Greeting]("greeting_collection_name")
def fetchAllGreetings(): Future[Seq[Greeting]] =
greetingCollection.find().toFuture()
}
Now, your route can be defined as
get {
concat(
path("getJson") {
val greetingSeqFuture: Future[Seq[Greeting]] = MongoDAL.fetchAllGreetings()
// I don't see any need for that actor thing,
// but if you really need to do that, then you can
// do that by using flatMap to chain future computations.
val actorResponseFuture: Future[Seq[Greeting]] =
greetingSeqFuture
.flatMap(greetingSeq => Patterns.ask(actor1, greetingSeq, 10.seconds))
// complete can handle futures just fine
// it will wait for futre completion
// then convert the seq of Greetings to Json using your JsonProtocol
complete(actorResponseFuture)
}
}
First of all, don't call toString in complete(res.toString).
As it said in AkkaHTTP json support guide if you set everything right, your case class will be converted to json automatically.
But as I see in the output, your res is not an object of a Greeting type. Looks like it is somehow related to the Greeting and has the same structure. Seems to be a raw output of the MongoDB request. If it is a correct assumption, you should convert the raw output from MongoDB to your Greeting case class.
I guess it could be done in getJson() after collection.find().

How to use Argonaut to decode poorly structured JSON where key name is meaningful

Hi the Decode Person example in the documentation is great if the JSON has a key and value and you can use the key name to extract its value, but what about if the string that makes up the key is arbitrary but meaningful.
for Fxample one open cryptocurrency api can give historic prices of coins and the structure of the JSON returned is different depending on the base currency of the coin I'm asking for and the various quote currencies I want it priced in.. for example lets say I want the price at a particular date of 'DOGE' in 'AUD' and 'XRP' the returned JSON looks like
{"DOGE":{"AUD":0.008835,"XRP":0.004988}}
I can't navigate to base and get its value and then prices and get them as the JSON is not stuctured that way, I need to look for 'DOGE' as a Key then in the Object retrned know that there will be a 'AUD' key and 'XRP' key. And of course that will be different for every result depending on my query.
Of course I know these keys as I create the search based on them but how can I use Argonaut to parse this JSON? Can I somehow create a Decode that closes over my key names?
Any help or guidance is appreciated, thanks.
Since you don't know what the property names are going to be ahead of time, you can't create a codec and decode the raw JSON directly to a Scala class.
You want to parse the raw JSON as a generic argonaut.Json object, then you can pattern match or use fold to examine the contents. For example:
val rawJson: String = ...
val parsed: Either[String, argonaut.Json] = argonaut.Parse.parse(rawJson)
You can see the methods available on argonaut's Json object by inspecting the source code.
As per Fried Brice's answer I did go down the parse route then mapped the resulting Either to produce my data type see code snippet below, suggestions, improvements welcome.
def parseHistoricPriceJSON(rawJson: String, fromcurrency: Currency, toCurrencies: List[Currency]): Either[String, PricedAsset] = {
import argonaut._, Argonaut._
import monocle.macros.syntax.lens._
val parsed: Either[String, Json] = Parse.parse(rawJson)
val myTocurrs = Currency("XRP") :: toCurrencies
parsed.right.map(outer => {
val cursor = outer.cursor
val ps = for {
toC <- myTocurrs
prices <- cursor.downField(fromcurrency.sym)
price <- prices.downField(toC.sym)
thep <- price.focus.number
} yield (toC, thep.toDouble.get)
PricedAsset(fromcurrency, ps)
})
}
case class Currency(sym: String) extends AnyVal {
def show = sym
}
case class PricedAsset(base:Currency, quotePrices: List[(Currency,Double)])

Add element to JsValue?

I'm trying to add in a new element to a JsValue, but I'm not sure how to go about it.
val rJson = Json.parse(response)
val imgId = //stuff to get the id :Long
rJson.apply("imgId", imgId)
Json.stringify(rJson)
Should I be converting to a JSONObject or is there some method that can be applied directly to the JsValue to insert a new element to the JSON?
Edit:
response is coming from another server, but I do have control over it. So, if I need to add an empty "imgId" element to the JSON Object, that's fine.
You can do this as a JsObject, which extends JsValue and has a + method:
val rJson: JsValue = Json.parse(response)
val imgId = ...
val returnJson: JsObject = rJson.as[JsObject] + ("imgId" -> Json.toJson(imgId))
Json.stringify(returnJson)
I use the following helper in a project I'm working on:
/** Insert a new value at the given path */
def insert(path: JsPath, value: JsValue) =
__.json.update(path.json.put(value))
, which can be used as a JSON transformer as such:
val rJson = Json.parse(response)
val imgId = //stuff to get the id :Long
Json.stringify(rJson.transform(insert(__ \ 'imgId, imgId)))
You could definitely just use the body of that insert method, but I personally find the transformer API to be really counterintuitive.
The nice thing about this is that any number of transforms can be composed using andThen. We typically use this to convert API responses to the format of our models so that we can use Reads deserializers to instantiate model instances. We use this insert helper to mock parts of the API response that don't yet exist, so that we can build models we need ahead of the API.
Even though the API is convoluted, I highly, highly recommend investing some time in reading the Play framework docs on JSON handling, all 5 pages. They're not the world's greatest docs, but they actually are pretty thorough.

spray-can how to unwrap unmarshalled FormData

I have something like this:
case HttpRequest(POST, Uri.Path("/userAction"), headers, entity: HttpEntity.NonEmpty, protocol) =>
val x = entity.as[spray.http.FormData].merge.asInstanceOf[spray.http.FormData].fields
sender ! HttpResponse(entity="Got it: " + x)
It seems a little bit inconvinient to unwrap the HttpEntity, is there a much more elegant way?
Furthermore: is it possible to get the data from inputs with multiple delacred variables? Is there a better way to access FormData, e.g. as Json or HashMap?
Is there an actual reason why you are implementing this with spray-can? In routing it's more easier and natural:
lazy val yourRoute: Route = {
(post & path("userAction")) {
entity(as[SomeType]) { data =>
// do what you want
}
}
}
In this version it all comes to the proper unmarshaller for your type (SomeType). But if you want to stick with spray-can, you can use spray-httpx unmarshaller. I don't have IDE near to me, but this can like this (actual code from my project with some Scalaz tricks):
case request # HttpRequest(...) =>
fromTryCatch(request.entity |> unmarshal[SomeType]) // returns Throwable \/ SomeType, like scala's Either type
Here it also comes to the right unmarshaller, e.g if you want to get your entity in Json format (i.e spray JsObject), then you need to emit a request with json payload, write entity(as[JsObject]) and provide an implicit unmarshaller from spray.json.SprayJsonSupport. I think that using this approach for FormData is a bit overhead, cause for simple Post payload you have formFields directive, and you can write:
lazy val yourRoute: Route = {
(post & path("userAction")) {
formFields(...) { fields => // just enumerate field names from your FormData
// do what you want
}
}
}
For Map approach if the same, just add spray.json.DefaultJsonProtocol into the scope, to add unmarshaller for maps.
Still using spray-routing DSL is far better then dealing with low-level can code. If you still want to use, then take a look at spray.httpx.unmarshalling package, it contains different types of unmarshallers, for entity, request and response.

Play Scala No Json deserializer found for type (String, String). Try to implement an implicit Reads or Format for this type

These Json serializers in Play with Scala are driving me nuts.
I have read dozens of posts and the tutorials and the documentation. Tried four different ways of implementing Reads / Writes / Format overrides and all to no avail.
So I backed off the custom type and decided to go uber simple:
def suggest = Action(parse.json) {
request =>
request.body.validate[(String, String)].map {
case (suggestion, categories) => Ok("You suggested " + suggestion + " for categories " + categories)
}.recoverTotal {
e => BadRequest(JsError.toFlatJson(e))
}
}
And the error comes back as noted in the subject.
Do I really need to provide a custom Reads / Writes / Format implementation for such a basic body?
A sample input body could be:
{"suggestion":"add generics", "categories":"request;language;updates"}
What simple thing am I missing?
Play! gives you a LOT of ways to work with Json. From the looks of your code, you're going down the Tuple road. Which essentially lets you convert Json into a Tuple. You're also using 'reads' which has the downside that you don't get precise error reporting (ie: if invalid json was passed you would know its invalid... but you wouldn't necessarily know why it was invalid). If you wanted more error handling then you need to start using the 'validate' method (details here: http://www.playframework.com/documentation/2.1.1/ScalaJsonCombinators).
Another way you could go is to map Json to case classes doing something like:
import play.api.libs.json._
case class MyClass(
suggestion: String,
categories: String
)
object MyClass {
implicit val readsMyClass: Reads[MyClass] = new Reads[MyClass] {
def reads(json: JsValue): JsResult[MyClass] = {
for {
suggestion <- (json \ "suggestion").validate[String]
categories <- (json \ "categories").validate[String]
} yield MyClass(json,categories)
}
}
}
This seems like a lot of code so Play 2.1 introduced Json 'Inception' which I have yet to try (http://www.playframework.com/documentation/2.1.1/ScalaJsonInception).
Finally, if you want the Json validation but don't necessary need to marshall/unmarshall case classes, then you can try the 'coast-to-coast' method. This will keep all your json as JsObject types but still give you validation. Sample code here: https://github.com/mandubian/play2-json-demo/blob/master/json-coast-to-coast/app/controllers/Application.scala
Hope this helps.
So I added this:
implicit val rds = (
(__ \ 'suggestion).read[String] and
(__ \ 'categories).read[String]
) tupled
And that seems to work.
Curious, though, is this really the best way to do this? It seems like a LOT of code if you have many many types to serialize / deserialize.