Extract Json from Spray POST as string, not by marshaling to entity - apache-kafka

There is an existing question that has much of what I'm after:
Extracting Raw JSON as String inside a Spray POST route
But it stops short without explaining how to get the actual Json string representation out of the Directive[String]. I'm trying to send Json data to Kafka as a string (which the Kafka Producer serializes), so I'm trying to extract the Json in string form. I will do the marshalling to an entity at the other end in the Kafka consumer. The answer link above gets me close:
def rawJson = extract { _.request.entity.asString}
case "value2" => rawJson{ json =>// use the json }
But I end up with Directive[String]. How do I get the String out?

The example you referenced should work. You would use the rawJson directive they defined to wrap your inner route, and the json string would be made available within that inner route.
In the example below, personJson is a String, extracted by the body of the request via the rawJson directive, and made available to the inner route where the rest of the work is done.
def rawJson = extract { _.request.entity.asString}
val personRoute = {
(post & path("persons")){
rawJson{ personJson =>
onSuccess(personService.addPerson(person)){ personWithId =>
complete(StatusCodes.Created, personWithId)
}
}
}

I came up with the following syntax which accomplishes my need to extract the Json in String form. At first I thought it inefficient that I was unmarshaling and then remarshaling again, but then I realized that this provides a form of immediate Json validation. But there may be more efficient ways to do that.
The API is all Spray. handleWith uses an implicit conversion to the RawWeatherData case class.
path("weather"/"data"/"json") {
handleWith { rawRecord: RawWeatherData =>
val rawJsonStr = rawRecord.toJson.toString
kafkaJsonRecordIngest(rawJsonStr)
rawRecord
}
}

Related

How to complete Akka Http response with Stream and Custom Status Code

I've got an akka-http application that uses akka-streams for data processing. So, it makes some sense to complete the request with Source[Result, _] to get backpressure across HTTP boundary for free.
Versions:
akka-http 10.0.7
akka-streams 2.5.2
akka 2.5.2
This is the simplified version of the code, and it works just fine.
pathEnd { post { entity(asSourceOf[Request]) { _ =>
complete {
Source.single("ok")
}
}}}
Since this enpoint is supposed to create and entity, instead of returning 200 OK to the requester I'd like to return 204 CREATED status code. However, I wasn't able to find a way to do that:
complete { Created -> source.single("ok") } fails compilation with Type mismatch, expected: ToResponseMarshallable, actual: (StatusCodes.Success, Source[String, NotUsed])
complete { source.single((Created, "ok")) } fails with Type mismatch, expected: ToResponseMarshallable, actual: Source[(StatusCodes.Success, String), NotUsed]
complete(Created) { Source.single("ok") } fails with Type mismatch, expected: RequestContext, actual: Source[String,NotUsed]
complete(Created, Source.signle("ok") fails with too many arguments for method complete(m: => ToResponseMarshallable)
It looks like custom marshaller might be a way to achieve that, but it'll basically mean I'll need one unmarshaller per endpoint, which isn't quite convenient or clear.
So, the question is, are there a (more convenient than custom unmarshaller) way to complete the request with Source[_,_] while also providing status code.
From the documentation:
complete(Created -> "bar")
If you want to provide some Source of data then construct the HttpResponse and pass it to complete:
import akka.http.scaladsl.model.HttpEntity.Chunked
import akka.http.scaladsl.model.ContentTypes
import akka.http.scaladsl.model.HttpEntity.ChunkStreamPart
complete {
val entity =
Chunked(ContentTypes.`text/plain(UTF-8)`,
Source.single("ok").map(ChunkStreamPart.apply))
HttpResponse(status = Created, entity=entity)
}
I hit this problem and took the approach of using mapResponse to override the status code. This is the simplest approach I've found.
mapResponse(_.copy(status = StatusCodes.Accepted)) {
complete {
Source.single("ok")
}
}
The drawback of Ramon's answer is that you become responsible for both marshalling the stream (to a ByteString), and for the content negotiation.

Create two or more APIs with same URL in play framework

I have use case where I need to read value from query string.
Currently I have two different APIs(Some other person has created the code) which maps to same URL
GET /service/class/:className/details controllers.Student.getStudentDetails(studentId)
GET /service/class/:className/details controllers.Student.getAllStudentsDetails()
If query string is present in URL then API1 should execute, otherwise API2.
As URL is same for both APIs, I am able to hit only get-student-details API(Because it has higher priority in routes file).
I am looking for alternatives to fix this problem.
As per my knowledge we don't need to create different APIs just to handle query strings.
I am thinking to merge 2 different APIs in single APIs which takes action depending upon presence of query string in request.
What I want to know is if there is way to execute two different APIs which maps to same URL(Only difference is with query string).
NOTE: I am using play 2.4.6.
I see few ways using a single controller function (say we chose getStudentDetails)
1) Having an Option parameter:
def getStudentDetails(studentId: Option[String]) = Action { studentId match {
case Some(id) => // do something
case None => // do something else
}
// ..
}
2) Look for your query string parameters inside your http request:
def getStudentDetails = Action { request =>
request.queryString.get("studentId") match {
case Some(list) => // do something...beware this is a List
case None => // do something else
}
//...
}

Spray routing: How to respond with different content-types?

In spray I would like to respond with different content-types, depending on the given Accept header. I've seen a couple of suggestions in the question by rompetroll, but I would like to hear if there are any canonical way of doing it (i. e. simple or already implemented).
In essence what I imagine should happen is something like:
path("somepath") {
get {
// Find whatever we would like to return (lazily)
...
// Marshall resource and complete depending on the `Accept` header
...
}
}
Thanks in advance.
See the tests in this commit.
I copied it here for reference:
case class Data(name: String, age: Int)
object Data {
import spray.json.DefaultJsonProtocol._
import spray.httpx.SprayJsonSupport._
// don't make those `implicit` or you will "ambiguous implicit" errors when compiling
val jsonMarshaller: Marshaller[Data] = jsonFormat2(Data.apply)
val xmlMarshaller: Marshaller[Data] =
Marshaller.delegate[Data, xml.NodeSeq](MediaTypes.`text/xml`) { (data: Data) ⇒
<data><name>{ data.name }</name><age>{ data.age }</age></data>
}
implicit val dataMarshaller: ToResponseMarshaller[Data] =
ToResponseMarshaller.oneOf(MediaTypes.`application/json`, MediaTypes.`text/xml`) (jsonMarshaller, xmlMarshaller)
}
You then using complete should suffice in your route, content-type negotiation is automatically taken care of:
get {
complete(Data("Ida", 83))
}
Spray is actually looking into the Accept header value and validates against it. So if route is returning application/json or text/plain and client accepts image/jpeg than spray will return 406 Not Acceptable. If client will request application/json ortext/plain from this route than he will receive repsonse with matching Content-Type.
The main trick here is to use correct marshallers for return objects.
You can read more about marshalling here.
Also you can override MediaType with respondWithMediaType directive, but I think it is better to use correct marshallers.

Scala drivers for couchdb and partial schemas

One question I have about current Scala couchdb drivers is whether they can work with "partial" schemas". I'll try to explain what I mean: the libraries I've see seem to all want to do a complete conversion from JSON docs in the database to a Scala object, handle the Scala object, and convert it back to JSON. This is is fine if your application knows everything about that type of object --- especially if it is the sole piece of software interacting with that database. However, what if I want to write a little application that only knows about part of the JSON object: for example, what if I'm only interested in a 'mybook' component embedded like this:
{
_id: "0ea56a7ec317138700743cdb740f555a",
_rev: "2-3e15c3acfc3936abf10ea4f84a0aeced",
type: "user",
profiles: {
mybook: {
key: "AGW45HWH",
secret: "g4juh43ui9hg929gk4"
},
.. 6 or 7 other profiles
},
.. lots of other stuff
}
I really don't want to convert the whole JSON AST to a Scala object. On the other hand, in couchdb, you must save back the entire JSON doc, so this needs to be preserved somehow. I think what I really what is something like this:
class MyBook {
private val userJson: JObject = ... // full JSON retrieved from the database
lazy val _id: String = ... // parsed from the JSON
lazy val _rev: String = ... // parsed from the JSON
lazy val key: String = ... // parsed from the JSON
lazy val secret: String = ... // (ditto)
def withSecret(secret: String): MyBook = ... // new object with altered userJson
def save(db: CouchDB) = ... // save userJson back to couchdb
}
Advantages:
computationally cheaper to extract only needed fields
don't have to sync with database evolution except for 'mybook' part
more suitable for development with partial schemas
safer, because there is less change as inadvertently deleting fields if we didn't keep up with the database schema
Disadavantages:
domain objects in Scala are not pristinely independent of couch/JSON
more memory use per object
Is this possible with any of the current Scala drivers? With either of scouchdb or the new Sohva library, it seems not.
As long as you have a good JSON library and a good HTTP client library, implementing a schemaless CouchDB client library is really easy.
Here is an example in Java: code, tests.
My couchDB library uses spray-json for (de)serialization, which is very flexible and would enable you to ignore parts of a document but still save it. Let's look at a simplified example:
Say we have a document like this
{
dontcare: {
...
},
important: "foo"
}
Then you could declare a class to hold information from this document and define how the conversion is done:
case class Dummy(js:JsValue)
case class PartialDoc(dontcare: Dummy, important: String)
implicit object DummyFormat extends JsonFormat[Dummy] {
override def read(js:JsValue):Dummy = Dummy(js)
override def write(d:Dummy):JsValue = d.js
}
implicit val productFormat = jsonFormat2(PartialDoc)
This will ignore anything in dontcare but still safe it as a raw JSON AST. Of course this example is not as complex as the one in your question, but it should give you an idea how to solve your problem.

Request Content-Type in Play! Framework for REST webservices

I'm trying to implement a REST webservice with the Play! framework. I know how I can return a response in different formats (JSON, XML, HTML, ...) by specifying multiple templates. However, I didn't find any information on how you should process different Content-Types in a (e.g. POST) request (form encoded, JSON, XML, ...).
Is it possible to annotate a method to match only certain Content-Types (something like #Consumes)? Do I have to differentiate between the different request Content-Types with an if-clause in the controller method?
Have a look at the Play documentation for combining body parsers:
http://www.playframework.com/documentation/2.2.0/ScalaBodyParsers
If you want to constrain a post body to only xml or json you can write something along these lines:
val xmlOrJson = parse.using {
request =>
request.contentType.map(_.toLowerCase(Locale.ENGLISH)) match {
case Some("application/json") | Some("text/json") => play.api.mvc.BodyParsers.parse.json
case Some("application/xml") | Some("text/xml") => play.api.mvc.BodyParsers.parse.xml
case _ => play.api.mvc.BodyParsers.parse.error(Future.successful(UnsupportedMediaType("Invalid content type specified")))
}
}
def test = Action(xmlOrJson) {
request =>
request.body match {
case json: JsObject => Ok(json) //echo back posted json
case xml: NodeSeq => Ok(xml) //echo back posted XML
}
}
The xmlOrJson function looks at the content type request header to determine the body parser. If it is not xml or json then it returns the error body parser with an UnsupportedMediaType response (HTTP 415).
You then pass this function in as the body parser of any action you want to constrain to xml or json content. Within the action you can look at the body to determine if xml or json was parsed and process accordingly.
You don't do it through annotation, but rather through your routes file, or through an if statement in your action. Depends on your use case as to which one is best suited.
The following URL gives you some information on the routes file for content negotiation. http://www.playframework.org/documentation/1.2.4/routes#content-types
Example
GET /index.xml Application.index(format:'xml')
GET /index.json Application.indexJson(format:'json')
The above calls different actions, but you could call the same action with a different format value if you wish.
You may use default parser together with pattern matching in Play 2 with Scala. Something like this:
def myAction() = Action { req =>
req.body match {
case AnyContentAsFormUrlEncoded(m) =>
val f1 = m.get("field1").flatMap(_.headOption)
...
case AnyContentAsJson(j) =>
val f1 = (j \ "field1").asOpt[String]
...
case _ => //handle content types you don't support
...
}