Scala, Hammock - retrieve http response headers and convert JSON to custom object - scala

I have created a simple program which use Hammock(https://github.com/pepegar/hammock) and now I would like to get response from github API with reposne's headers. I created a code like this:
object GitHttpClient extends App {
implicit val decoder = jsonOf[IO, List[GitRepository]]
implicit val interpreter = ApacheInterpreter.instance[IO]
val response = Hammock
.request(Method.GET, uri"https://api.github.com/orgs/github/repos?per_page=3", Map())
.as[List[GitRepository]]
.exec[IO]
.unsafeRunSync()
println(response)
}
case class GitRepository(full_name: String, contributors_url: String)
And it works fine, I got Git data mapped to my object. But now I also want to get headers from response and I cannot do this by simple response.headers. Only when I remove .as[List[GitRepository]] line and have whole HttpResponse I could access headers. Is it possible to get headers without parsing whole HttpResponse?

I solved this problem by using Decoder after received reponse:
val response = Hammock
.request(Method.GET, uri"https://api.github.com/orgs/github/repos?per_page=3", Map())
.exec[IO]
.unsafeRunSync()
println(response.headers("Link") contains ("next"))
println(HammockDecoder[List[GitRepository]].decode(response.entity))

Related

json4s used in scalatra application throws "com.fasterxml.jackson.databind.JsonMappingException: No content to map due to end-of-input"

json4s used in scalatra application throws "com.fasterxml.jackson.databind.JsonMappingException: No content to map due to end-of-input" when a POST request through a browser.
I have a ScalatraServlet to serve FORM submit from browser. Here is the Servlet.
class PagesController(service: RecordService) extends ScalatraServlet with JacksonJsonSupport {
post("/addRecord") {
contentType = "text/html"
//implicit val formats = DefaultFormats
val jsonPayload = request.body
println(s"payload: $jsonPayload")
val x = parse(request.body)
println(s"parsed: $x")
val record = x.extract[MRecord]
println(s"object: $record")
service.add(Record(0, "Mocked data"))
println(s"added $recordModel")
redirect(URL.LANDING_PAGE_URL)
}
When I run the POST request through cli/rest-client with content-type as appplication/www-form-url-encode, there is no such error and I can confirm from the println statements. However, when a browser submits a form, "com.fasterxml.jackson.databind.JsonMappingException: No content to map due to end-of-input"
What is the cause of this exception to occur only when the form is submitted and not when submitted through REST client/cli?
Looks like your code expects that a request body is JSON but a browser form submits param_name1=param_value1&param_name2=param_value2 as a request body. If you have a field named json that contains JSON in your form, probably, you can get a JSON as follows:
post("/addRecord") {
val jsonPayload = params("json")
val x = parse(jsonPayload)
...
}
By the way, the json4s version that is used in Scalatra 2.7.0 is 3.6.7. It would be better to upgrade to this version: https://github.com/scalatra/scalatra/blob/v2.7.0/project/Dependencies.scala#L55

set header as accept and value as application/json in Requestbuilding in play framework and scala Akka-type

beans.scala - class contains connection to server
lazy val ConnectionFlow: Flow[HttpRequest, HttpResponse, Any] =
Http().outgoingConnection(config.getString("host"), config.getInt("port"))
lazy val AppService = new Service(config, ConnectionFlow)
Service.scala class
def Request(request: HttpRequest): Future[HttpResponse] =
Source.single(request).via(ConnectionFlow).runWith(Sink.head)
//building Json request
val reqJs = Json.obj("PARAMS" -> Json.obj("param1" -> value1))
Request(RequestBuilding.Post("/services/serviceName",reqJS).flatMap { response =>
// need response to be in JSobject format but the service returns application/xml format
If I understand correctly, you're asking how to modify the request s.t. it indicates to the server it expects a JSON response.
To do that, you can try
val builder = RequestBuilding.Post("/services/serviceName",reqJS)
builder.addHeader(Accept(MediaRange(`application/json`)))
// send out the request as you did
According to https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept, Accept Header only advertisers which content types the client is able to understand, while it is fully on the server's mercy to respect this request. That being said, the server could literally respond with anything they want, despite the header you send.

Akka-http and handling an empty body during POST

Suppose I am handling a post message.
The Content-Type may be application/xml or application/json. I can unmarshal XML and JSON documents just fine.
However, I would like to handle the case where the POST method is called with an empty body (the content-type may still be xml or json, but the document itself is an empty string). This is for various logging and analytics purposes. Right now, the method fails since the empty body cannot be processed by either one of my unmarshallers.
How do I write an unmarshaller for this case? Or can a unmarshaller handle a null body?
Somewhere in my code I have the following. Do I need more marshallers?
implicit def myUnmarshaller(implicit mat: Materializer): FromEntityUnmarshaller[MyClass] =
Unmarshaller.firstOf[HttpEntity, MyClass](
/* myNullBodyMarshaller, */ // do I need this?
myJsonUnmarshaller,
myXmlUnmarshaller
)
or can I modify an existing one? For example, this one?
def myJsonUnmarshaller(implicit mat: Materializer): FromEntityUnmarshaller[MyClass] =
Unmarshaller.byteStringUnmarshaller.forContentTypes(MediaTypes.`application/json`).mapWithCharset { (data, charset) ⇒
val input: String = if (charset == HttpCharsets.`UTF-8`) data.utf8String else data.decodeString(charset.nioCharset.name)
val tmp = input.parseJson.convertTo[MyClass]
MyClass(tmp.A, tmp.B)
}

Play (Scala) handling different Content-Type within the same action

I am developing a web service which accepts JSON data. Sometimes input data comes with attachments like image or some PDF file. In that case this data comes as multi-part data.
I need to create action which accepts both content type. And depending on content type it should be able to parse the json and retrieve the attachment related metadata from json and then download the attachment.
I have two actions which handles things separately
def multiPartAction: Action[MultipartFormData[Array[Byte]]] = Action(multipartFormDataAsBytes)={request =>
...
}
Second action
def handleJSon: Action[JsValue] = Action.async(parse.json) {
request =>
...
}
How do I handle these two actions together in one action?
You can either specify your own body parser that is a combination of two, similarly to how it is done here, or leave the body parser out, sticking to default AnyContent body type. Then:
def action = Action { request =>
val body: AnyContent = request.body
val jsonBody: Option[JsValue] = body.asJson
val multipartBody: Option[MultipartFormData[TemporaryFile] =
body.asMultipartFormData
(jsonBody map getResponseForJson) orElse
(multipartBody map getResponseForAttachment) getOrElse
BadRequest("Unsupported request body")
}

Play Framework Testing using MultipartFormData in a FakeRequest

I am currently in the process of writing some Specs2 tests for may Play Framework 2.2.x application which accepts MultipartFormData submissions as part of it's function.
I have successfully written other tests with text and JSON bodies using the following form:
"respond to POST JSON with description field present" in {
running(FakeApplication()) {
val response = route(FakeRequest(POST, "/submission.json").withJsonBody(toJson(Map("content" -> toJson("test-content"), "description" -> toJson("test-description"))))).get
status(response) must equalTo(OK)
contentType(response) must beSome.which(_ == "application/json")
contentAsString(response) must contain(""""description":"test-description"""")
contentAsString(response) must contain(""""content":"test-content"""")
}
}
However, when I use the .withMultipartFormData method I get the following errors:
Cannot write an instance of play.api.mvc.AnyContentAsMultipartFormData to HTTP response. Try to define a Writeable[play.api.mvc.AnyContentAsMultipartFormData]
val response = route(FakeRequest(PUT,"/submission.json/1/files").withMultipartFormDataBody(data)).get
^
The MultipartFormData test I have been attempting to debug is of the form:
"respond to file PUT form with details not specififed" in {
running(FakeApplication()) {
val basePath:String = Play.application.path.getCanonicalPath();
val data:MultipartFormData[TemporaryFile] = MultipartFormData(Map[String,Seq[String]](),
List(
FilePart("file_upload","",Some("Content-Type: multipart/form-data"),TemporaryFile(new java.io.File(basePath + "/test-data/testUpload.jpg")))
),
List(),
List())
val response = route(FakeRequest(PUT,"/submission.json/1/files").withMultipartFormDataBody(data)).get
status(response) must equalTo(CREATED)
}
}
Looking at the Play Framework documentation for the relevant version of the FakeRequest class I can't see too much to help me trace down the problem: play.api.test.FakeRequest
And in terms of other documentation on the matter it seems the Play Framework website and Google are rather lacking.
I have tried the following alternative means of attempting to test my MultipartFormData code:
Writing a test case for file uploads in Play 2.1 and Scala
Test MultipartFormData in Play 2.0 FakeRequest
How do I test multipart form data requests for file uploads in Play Framework 2.0 using Java? (Translating to Scala code first).
However, I have not had any success with any of these approaches either.
Rather than testing in a FakeApplication which is slow and (in my experience) can be error-prone when tests are running in parallel, I've been unit testing my Multipart form upload handlers like this:
Split out the Play wiring from your actual upload handling in your controller; e.g.:
def handleUpload = Action(parse.multipartFormData) { implicit request =>
doUpload(request)
}
def doUpload(request:Request[MultipartFormData[TemporaryFile]]) = {
...
}
(Where handleUpload is the method in your routes file that handles the POST)
Now you've got an endpoint that's easier to get at, you can mock out your service layer to respond appropriately to good/bad requests, and inject the mock service into your controller under test (I won't show that here, there are a million different ways to do that)
Now mock out a multipart request that will arrive at your doUpload method:
val request= mock[Request[MultipartFormData[TemporaryFile]]]
val tempFile = TemporaryFile("do_upload","spec")
val fileName = "testFile.txt"
val part = FilePart("key: String", fileName, None, tempFile)
val files = Seq[FilePart[TemporaryFile]](part)
val multipartBody = MultipartFormData(Map[String, Seq[String]](), files, Seq[BadPart](), Seq[MissingFilePart]())
request.body returns multipartBody
And finally, you can call your doUpload method and verify functionality:
val result = controller.doUpload(request)
status(result) must beEqualTo(201)
By testing like this, you can quickly and easily test all the error-handling paths in your Controller (which is probably what you're trying to do after all) without the overhead of needing to start the entire application.
In Play 2.5.x, it is easy to test file upload
val file = new java.io.File("the.file")
val part = FilePart[File](key = "thekey", filename = "the.file", contentType = None, ref = file)
val request = FakeRequest().withBody(
MultipartFormData[File](dataParts = Map.empty, files = Seq(part), badParts = Nil)
)
val response = controller.create().apply(request)
status(response) must beEqualTo(201)
(I've answered in the other thread: PlayFramework Testing: Uploading File in Fake Request Errors)
In short, you need a Writeable[AnyContentAsMultipartFormData], which turns MultipartFormData[TemporaryFile] into Array[Byte], and you can take it from here: http://tech.fongmun.com/post/125479939452/test-multipartformdata-in-play