spray-can how to unwrap unmarshalled FormData - scala

I have something like this:
case HttpRequest(POST, Uri.Path("/userAction"), headers, entity: HttpEntity.NonEmpty, protocol) =>
val x = entity.as[spray.http.FormData].merge.asInstanceOf[spray.http.FormData].fields
sender ! HttpResponse(entity="Got it: " + x)
It seems a little bit inconvinient to unwrap the HttpEntity, is there a much more elegant way?
Furthermore: is it possible to get the data from inputs with multiple delacred variables? Is there a better way to access FormData, e.g. as Json or HashMap?

Is there an actual reason why you are implementing this with spray-can? In routing it's more easier and natural:
lazy val yourRoute: Route = {
(post & path("userAction")) {
entity(as[SomeType]) { data =>
// do what you want
}
}
}
In this version it all comes to the proper unmarshaller for your type (SomeType). But if you want to stick with spray-can, you can use spray-httpx unmarshaller. I don't have IDE near to me, but this can like this (actual code from my project with some Scalaz tricks):
case request # HttpRequest(...) =>
fromTryCatch(request.entity |> unmarshal[SomeType]) // returns Throwable \/ SomeType, like scala's Either type
Here it also comes to the right unmarshaller, e.g if you want to get your entity in Json format (i.e spray JsObject), then you need to emit a request with json payload, write entity(as[JsObject]) and provide an implicit unmarshaller from spray.json.SprayJsonSupport. I think that using this approach for FormData is a bit overhead, cause for simple Post payload you have formFields directive, and you can write:
lazy val yourRoute: Route = {
(post & path("userAction")) {
formFields(...) { fields => // just enumerate field names from your FormData
// do what you want
}
}
}
For Map approach if the same, just add spray.json.DefaultJsonProtocol into the scope, to add unmarshaller for maps.
Still using spray-routing DSL is far better then dealing with low-level can code. If you still want to use, then take a look at spray.httpx.unmarshalling package, it contains different types of unmarshallers, for entity, request and response.

Related

Access variables in Gatling feeder with gRPC

I am trying to create my gRPC payload by reading data from a csv file(it has guestID and category as columns).
I followed the example here https://github.com/phiSgr/gatling-grpc/blob/244ab372da6773102d79c65a7e4086f409d3fe94/src/test/scala/com/github/phisgr/example/GrpcExample.scala but I see a type mismatch error when I try the same. (it expects Seq[ContextKey] but here i am able to form Seq[Expression[ContextKey]])
val scn2: ScenarioBuilder = scenario("gRPC call - 50 users repeated 100 times")
.feed(csv("testtext.csv"))
.exec(
grpc("gRPC request with test message")
.rpc(RecommenderGrpc.METHOD_GET_RECOMMENDATIONS)
.payload(RequestContext.of(Map("test" -> "test"),
Seq(ContextKey.defaultInstance.updateExpr(
_.id :~ $("guestID"),
_.`type` :~ Type.GUEST
), ContextKey.defaultInstance.updateExpr(
_.id :~ $("category"),
_.`type` :~ Type.CATEGORY
)),
Seq())
)
)
(payload is a RequestContext object which takes in metadata, keys and items. metadata is a map, keys is a Seq of ContextKey and items is empty Seq. ContextKey contains string guestID or category and type).
How to use the variables in the feeder here?
Skip to the bottom for the solution.
Expression[T] is an alias for Session => Validation[T]. In plain English, that is a function that constructs the payload from the session with a possibility of failure.
You can consider an Expression[T], abstractly, "contains" a T.
Like how a Promise in JavaScript "contains" a future value. You cannot give a Promise of T to a function that expects a T. If one wants to transform or combine Promises, that code has to be turned inside out, and supplied as an argument to .then.1
aPromise + 1 // wrong
aPromise.then(a => a + 1)
This is the same reason why your code sample does not compile.
Users of Gatling are not necessarily familiar with Scala, or functional programming in general. It will be counterproductive to make them understand this "wrapping" stuff.2 So there are code that help you combine the Expressions.
For HTTP and other untyped stuff, the EL string is parsed and transformed in to an Expression under the hood.
Protobuf messages are strongly typed, the payload cannot be easily constructed using string interpolation. So the :~ operators on lenses are used to handle the plumbing so that you do not have to manually handle the Expression wrapping.
But now you have a function, RequestContext.of, that constructs the payload. The lens magic cannot help if you need that function. You have to write the Expression lambda yourself.3
.payload { session =>
for {
guestId <- session("guestId").validate[String]
category <- session("category").validate[String]
} yield RequestContext.of(
Map("test" -> "test"),
Seq(
ContextKey(id = guestID, `type` = Type.GUEST),
ContextKey(id = category, `type` = Type.CATEGORY)
),
Seq()
)
}
Needless to say this is very cumbersome, and people now use async-await with Promises.
An Expression is just the Either monad and the Reader monad stacked together, what's the problem?
I may be able to write a version with lenses if I know what RequestContext.of does.

Basic Create-from-Json Method Walkthrough

I'm new to play, scala, and reactivemongo and was wondering if someone could explain to me the following code in easy terms to understand.
def createFromJson = Action.async(parse.json) { request =>
import play.api.libs.json.Reads._
val transformer: Reads[JsObject] =
Reads.jsPickBranch[JsString](__ \ "name") and
Reads.jsPickBranch[JsNumber](__ \ "age") and
Reads.jsPut(__ \ "created", JsNumber(new java.util.Date().getTime())) reduce
request.body.transform(transformer).map { result =>
collection.insert(result).map { lastError =>
Logger.debug(s"Successfully inserted with LastError: $lastError")
Created
}
}.getOrElse(Future.successful(BadRequest("invalid json")))}
I know that it creates a user from a JSON user with name and age attributes. What I don't understand is the way that input JSON is read in this method. ALSO the concept of Action.async(par.json), request => getorElse, Future, etc.
ALSO any easier/simpler ways of writing this method would be greatly appreciated.
Thanks in advance!
I believe you found this code in a template I have made following excellent reactive mongo documentation.
http://reactivemongo.org/releases/0.11/documentation/index.html
http://reactivemongo.org/releases/0.11/documentation/tutorial/play2.html
I feel a bit obliged to explain it. Let's run through the code.
def createFromJson = Action.async(parse.json) { request =>
The function createFromJson will return an Action (play stuff) that is asynchronous (returns a future of a result) that handles a body in json format. To do that it will use the request.
Documentation: https://www.playframework.com/documentation/2.5.x/ScalaAsync
A json can be anything that follows the json formats, for example an array a String, an object, ...
Our transformer is going to take only the data that we are interested in from the json and will return a clean json object
val transformer: Reads[JsObject] =
Reads.jsPickBranch[JsString](__ \ "name") and
Reads.jsPickBranch[JsNumber](__ \ "age") and
Reads.jsPut(__ \ "created", JsNumber(new java.util.Date().getTime())) reduce
As you see, it will pick the branch name as a string and the branch age as a number. It will also add to the final json object a field created with the time of creation.
As you see we are not transforming it to a Person instance, it is just a JsObject instance as it is defined in
val transformer: Reads[JsObject] ....
Play offers you a few ways to handle json in a simpler way. This examples tries to show the power of manipulating directly the json values without converting to a model.
For example if you have a case class
case class Person(name: String, age: Int)
You could create automatically a reads from it,
val personReads: Person[Person] = Json.reads[Person]
But to just store it in Mongo
DB there is no reason to build this instance and then transform it to json again.
Of course if you need to do some logic with the models before inserting them, you might need to create the model.
Documentation:
https://www.playframework.com/documentation/2.5.x/ScalaJson
https://www.playframework.com/documentation/2.5.x/ScalaJsonCombinators
https://www.playframework.com/documentation/2.5.x/ScalaJsonAutomated
https://www.playframework.com/documentation/2.5.x/ScalaJsonTransformers
With this in mind the rest of the code should be clear
request.body.transform(transformer).map { result =>
collection.insert(result).map { lastError =>
Logger.debug(s"Successfully inserted with LastError: $lastError")
Created
}
}
From the request, we take the body (a JsValue) we transform it into a JsObject (result) and we insert it in the collection.
Insert returns a Future with the last error, when the Person is stored last error will be logged and a Created (201 code) will be returned to the client of the API.
The last bit should be also clear by now
}.getOrElse(Future.successful(BadRequest("invalid json")))
If there is any problem parsing and transforming the json body of the request into our JsObject an already completed future with the result BadRequest (400 code) will be returned to the client.
It is a future because Action.Async needs future of result as the return type.
Enjoy scala.

Spray marshalling collection of ToResponseMarshallable

I have a simple Spray scenario that doesn't work as I've expected.
I have a spray server that dispatches work to different modules, and then return their composed responses.
As I don't want to limit Module responses, I chose that the Module's method return ToResponseMarshallable, as I have Modules that need to return a plain String.
Module signature could be:
def moduleProcessing(): ToResponseMarshallable = randomString()
And my "complete" block look similar to this:
complete {
val response1 = moduleProcessing()
val response2 = moduleProcessing()
Seq(response1,response2)
}
In the previous example, I would expect to get:
[{"someRandomString"},{"anotherRandomString"}]
But I am getting:
[{},{}]
Of course it will propagate as expected, if I return a single response or if I change the signature of the moduleProcessing return type to any Marshallable Type.
Thanks in advance for your help!
I think it's strange that your moduleProcessing method returns directly a ToResponseMarshallable[T]. The spray way would be to have this method return a T and have an in scope Marshaller[T] for when you complete requests by providing a T.
Finally, I don't think Marshaller.of[Seq[T]] is a built in marshaller in spray.
This means you'll need to provide your own Seq marshaller. This could delegate the actual marshalling of each item in the seq to an implicit Marshaller[T] and concatenate the results. Many builtin spray marshallers require implicit Marshaller[T] values like this. Alternatively, if you want a simple Seq[T] to string marshaller, try something like the following:
implicit val CustomMarshaller = Marshaller.delegate[Seq[T], String](`text/html`)(_.mkString("{", ",", "}"))

Can I return Future[Option[Foo]] in spray?

My attempt so far ran into java.lang.ClassCastException: scala.runtime.BoxedUnit cannot be cast to scala.Option. In fact I even can not think how the appropriate response may look like. Is it possible?
Here could be two options: either it's a bug in your program or you didn't provide spray any way to marshall your Foo type, cause Future and Option are handled by default. E.g this route can be handled by standard spray marshallers without any problems:
val route = {
(get & path("test")) {
complete {
Future(Option("Hello Spray!"))
}
}
}
Now if you make a GET request on /test you'll get a correct response.
If you have a specific type, then you need to provide you own marshaller of type ToResponseMarshallable to sray through implicit context. I think the most common and easiest way would be to make a Json response, for this you need a spray-json (or some other supported json lib) and just provide a converter to json, like:
import spray.json.DefaultJsonProtocol._
case class Boy(name: String, age: String)
object Boy {
implicit val boyJson = jsonFormat2(Boy.apply)
}
Now the only thing left to do is place a json marshaller into the scope:
import spray.httpx.SprayJsonSupport._ // import json marshaller
val route = {
(get & path("test") {
complete {
Future(Option(Boy("Name", 0)))
}
}
}
Now you'll get a json response. IF you need some other kind of response, make a custom marshaller for a Boy type.
In short - Yes, but you need to make compiler happy.
Spray lets you return Future or plain response as long as it can be marshalled back to a response. You need to make sure that you either have implicit conversions in the scope that do the marshalling for you or transform your Foo object explicitly.
Here is how response transformation is performed: http://spray.io/documentation/1.2.0/spray-httpx/response-transformation/.
To anyone still getting issues returning a future in the complete block, make sure to have an execution context such as scala.concurrent.ExecutionContext.Implicits.global in scope! IDE tends to not understand this is the problem and can send you down a long rabbit hole.

Play Scala No Json deserializer found for type (String, String). Try to implement an implicit Reads or Format for this type

These Json serializers in Play with Scala are driving me nuts.
I have read dozens of posts and the tutorials and the documentation. Tried four different ways of implementing Reads / Writes / Format overrides and all to no avail.
So I backed off the custom type and decided to go uber simple:
def suggest = Action(parse.json) {
request =>
request.body.validate[(String, String)].map {
case (suggestion, categories) => Ok("You suggested " + suggestion + " for categories " + categories)
}.recoverTotal {
e => BadRequest(JsError.toFlatJson(e))
}
}
And the error comes back as noted in the subject.
Do I really need to provide a custom Reads / Writes / Format implementation for such a basic body?
A sample input body could be:
{"suggestion":"add generics", "categories":"request;language;updates"}
What simple thing am I missing?
Play! gives you a LOT of ways to work with Json. From the looks of your code, you're going down the Tuple road. Which essentially lets you convert Json into a Tuple. You're also using 'reads' which has the downside that you don't get precise error reporting (ie: if invalid json was passed you would know its invalid... but you wouldn't necessarily know why it was invalid). If you wanted more error handling then you need to start using the 'validate' method (details here: http://www.playframework.com/documentation/2.1.1/ScalaJsonCombinators).
Another way you could go is to map Json to case classes doing something like:
import play.api.libs.json._
case class MyClass(
suggestion: String,
categories: String
)
object MyClass {
implicit val readsMyClass: Reads[MyClass] = new Reads[MyClass] {
def reads(json: JsValue): JsResult[MyClass] = {
for {
suggestion <- (json \ "suggestion").validate[String]
categories <- (json \ "categories").validate[String]
} yield MyClass(json,categories)
}
}
}
This seems like a lot of code so Play 2.1 introduced Json 'Inception' which I have yet to try (http://www.playframework.com/documentation/2.1.1/ScalaJsonInception).
Finally, if you want the Json validation but don't necessary need to marshall/unmarshall case classes, then you can try the 'coast-to-coast' method. This will keep all your json as JsObject types but still give you validation. Sample code here: https://github.com/mandubian/play2-json-demo/blob/master/json-coast-to-coast/app/controllers/Application.scala
Hope this helps.
So I added this:
implicit val rds = (
(__ \ 'suggestion).read[String] and
(__ \ 'categories).read[String]
) tupled
And that seems to work.
Curious, though, is this really the best way to do this? It seems like a LOT of code if you have many many types to serialize / deserialize.