http4s - get request body as String or InputStream - scala

I'm trying to define HttpService that receives json and parses it to case class with json4s library:
import org.http4s._
import org.http4s.dsl._
import org.json4s._
import org.json4s.native.JsonMethods._
case class Request(firstName: String, secondName: String)
HttpService {
case req # POST -> Root =>
val request = parse(<map req.body or req.bodyAsText to JsonInput>).extract[Request]
Ok()
}
How can I get org.json4s.JsonInput from req.body or req.bodyAsText?
I know that json4s also have StringInput and StreamInput that inherits from JsonInput for using with String and InputStream so I think that I need to convert req.body to InputStream or req.bodyAsText to String but I still do not understand how.
I'm new to Scala and I do not yet fully understand some concepts such as scalaz.stream.Process.

You can use the http4s-json4s-jackson (or http4s-json4s-native) packages and use an org.http4s.EntityDecoder to easily get a Foo (I renamed your Request case class to Foo below) from a request.
EntityDecoder is a type class which can decode an entity from the request body.
We want to get the Foo posted in JSON, so we need to create an EntityDecoder[Foo] which can decode JSON. If we want to create this decoder using json4s we need a Reader (or a JsonFormat).
If you have an EntityDecoder[Foo] instance, we can get the Foo from the request with req.as[Foo].
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.http4s._
import org.http4s.dsl._
import org.http4s.json4s.jackson._
case class Foo(firstName: String, secondName: String)
// create a json4s Reader[Foo]
implicit val formats = DefaultFormats
implicit val fooReader = new Reader[Foo] {
def read(value: JValue): Foo = value.extract[Foo]
}
// create a http4s EntityDecoder[Foo] (which uses the Reader)
implicit val fooDec = jsonOf[Foo]
val service = HttpService {
case req # POST -> Root =>
// req.as[Foo] gives us a Task[Foo]
// and since Ok(...) gives a Task[Response] we need to use flatMap
req.as[Foo] flatMap ( foo => Ok(foo.firstName + " " + foo.secondName) )
}
Note: The json libraries libraries used most often with http4s are probably argonaut and circe. So you might find more http4s examples using one of those libraries.

Peter's solution both corrects the question and answers it, but I stumbled here looking for the solution to OP's stated, but not intended, question: "how to get request body as [...] InputStream" in http4s. Thanks to the discussion in Issue 634 on GitHub, here's what I came up with:
import java.io.InputStream
import org.http4s._
implicit val inputStreamDecoder: EntityDecoder[InputStream] =
EntityDecoder.decodeBy(MediaRange.`*/*`) { msg =>
DecodeResult.success(scalaz.stream.io.toInputStream(msg.body))
}
And then in your HttpService, use that decoder like so:
request.as[InputStream].flatMap { inputStream => ...inputStream is an InputStream... }
Or skip the whole Decoder dance, if you want:
val inputStream = scalaz.stream.io.toInputStream(request.body)

You may use flatMap and as inside it before calling the Http4s service to decode responses from it:
#Test def `Get json gives valid contact`: Unit = {
val request = Request[IO](GET, uri"/contact")
val io = Main.getJsonWithContact.orNotFound.run(request)
// here is magic
val response = io.flatMap(_.as[Json]).unsafeRunSync()
val contact = contactEncoder(Contact(1, "Denis", "123")) // this is encoding to json for assertion
assertEquals(contact, response)
}
This is how types work here:
val io: IO[Response[IO]] = Main.getJsonWithContact.orNotFound.run(request)
val response: IO[Json] = io.flatMap(_.as[Json])
val res: Json = response.unsafeRunSync()
as[String] will return the string just like this.

Related

In Scala Play Framework, is there a simple way to verify signatures on form requests

I am trying to write a Scala Play Framework action that will verify a HmacSHA256 signature on an incoming POST request containing form-url-encoded data.
This does not seem straightforward in the Play framework because: i) actions builders only have access to headers, but do not have access to the request body, and ii) in order to calculate the signature we have to treat the request body as Array[ByteString], but when we come to process the form data we have to treat it as Map[String, Seq[String]], the problem being thatPlay forces us to choose a single type for our request, and we cannot easily "cast" the request body to a different type.
The only solution I have been able to come up with is to use an ActionRefiner that returns a WrappedRequest that embeds a callback to validate the signature. The callback in turn reparses the data using FormUrlEncodedParser.parse(new String(request.body.toArray)). This approach is illustrated in the code below.
This all seems overly convoluted. Is there a simpler way to verify Hmac signatures in Play, or am I simply running up against limitations of the API?
package actions
import akka.util.ByteString
import com.google.inject.Inject
import play.api.Logging
import play.api.mvc.Results.Unauthorized
import play.api.mvc._
import play.core.parsers.FormUrlEncodedParser
import services.SlackSignatureVerifierService
import scala.concurrent.{ExecutionContext, Future}
import scala.util.Try
class SlackRequest[A](
val validateSignature: String => Try[String],
request: Request[A]
) extends WrappedRequest[A](request)
object SlackSignatureVerifyAction {
implicit class SlackRequestByteStringValidator(
slackRequest: SlackRequest[ByteString]
) {
def validateSignatureAgainstBody(): Try[Map[String, Seq[String]]] = {
val raw = slackRequest.body.utf8String
slackRequest.validateSignature(raw) map { _ =>
FormUrlEncodedParser.parse(new String(slackRequest.body.toArray))
}
}
}
val HEADERS_TIMESTAMP: String = "X-Slack-Request-Timestamp"
val HEADERS_SIGNATURE: String = "X-Slack-Signature"
}
class SlackSignatureVerifyAction #Inject() (
val parser: BodyParsers.Default,
slackSignatureVerifierService: SlackSignatureVerifierService
)(implicit ec: ExecutionContext)
extends ActionBuilder[SlackRequest, AnyContent]
with ActionRefiner[Request, SlackRequest]
with Logging {
override protected def executionContext: ExecutionContext = ec
override protected def refine[A](
request: Request[A]
): Future[Either[Result, SlackRequest[A]]] = {
val timestamp =
request.headers.get(SlackSignatureVerifyAction.HEADERS_TIMESTAMP)
val signature =
request.headers.get(SlackSignatureVerifyAction.HEADERS_SIGNATURE)
(timestamp, signature) match {
case (Some(timestamp), Some(signature)) =>
Future.successful {
val validate = (body: String) =>
slackSignatureVerifierService.validate(timestamp, body, signature)
Right(new SlackRequest[A](validate, request))
}
case _ =>
Future { Left(Unauthorized("Invalid signature headers")) }
}
}
}
You are right, there isn't an easy way to verify Hmac signatures on Play! projects. In the end, your approach seems to have a very reasonable implementation and could be easier adapted to other providers, such as GitHub and Stripe, that uses Hmac signatures.
I really think it could be a good open-source project to provide an Action with a wrapped Request or even a Service with a method to do custom signature validation for other providers. Why don't you help the Play community with an open-source project over GitHub?
I have created a new Play module to validate Hmac signatures. Details can be found here:
https://github.com/phelps-sg/play-hmac-signatures

What is the right way to send JSON response in http4s?

Not so long time ago I switched from akka-http to http4s. One of the basic things which I wanted to do correctly — JSON handling, in particular sending a JSON response.
I decided to use http4s with ZIO instead of cats, so here is how an http route looks like:
import fs2.Stream
import org.http4s._
import org.http4s.dsl.io._
import org.http4s.implicits._
import scalaz.zio.Task
import scalaz.zio.interop.catz._
import io.circe.generic.auto._
import io.circe.syntax._
class TweetsRoutes {
case class Tweet(author: String, tweet: String)
val helloWorldService = HttpRoutes.of[Task] {
case GET -> Root / "hello" / name => Task {
Response[Task](Ok)
.withBodyStream(Stream.emits(
Tweet(name, "dummy tweet text").asJson.toString.getBytes
))
}
}.orNotFound
}
As you see, JSON serialization part is pretty verbose:
.withBodyStream(Stream.emits(
Tweet(name, "dummy tweet text").asJson.toString.getBytes
))
Is there any other way to send JSON in a response?
Yes, there is: define and Encoder and Decoder for Task:
implicit def circeJsonDecoder[A](
implicit decoder: Decoder[A]
): EntityDecoder[Task, A] = jsonOf[Task, A]
implicit def circeJsonEncoder[A](
implicit encoder: Encoder[A]
): EntityEncoder[Task, A] = jsonEncoderOf[Task, A]
this way there is no need to transform to bytes.
EDIT: there is a full example here: https://github.com/mschuwalow/zio-todo-backend/blob/develop/src/main/scala/com/schuwalow/zio/todo/http/TodoService.scala
HT: #mschuwalow
There is even simpler solution for this. If you want to handle case class JSON encoding for HTTP responses, you just can add these imports:
import io.circe.generic.auto._
import org.http4s.circe.CirceEntityCodec._
BTW, the same imports handle decoding of incoming JSON requests into case classes as well

Transforming Scala case class into JSON

I have two case classes. The main one, Request, contains two maps.
The first map has a string for both key and value.
The second map has a string key, and value which is an instance of the second case class, KVMapList.
case class Request (var parameters:MutableMap[String, String] = MutableMap[String, String](), var deps:MutableMap[String, KVMapList] = MutableMap[String, KVMapList]())
case class KVMapList(kvMap:MutableMap[String, String], list:ListBuffer[MutableMap[String, String]])
The requirement is to transform Request into a JSON representation.
The following code is trying to do this:
import com.fasterxml.jackson.annotation.PropertyAccessor
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper
import com.fasterxml.jackson.annotation.JsonAutoDetect.Visibility
import com.fasterxml.jackson.databind.ObjectMapper
def test(req:Request):String {
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.setVisibility(PropertyAccessor.ALL, Visibility.ANY)
var jsonInString: String = null
try {
jsonInString = mapper.writeValueAsString(request)
}
catch {
=case e: IOException => {
e.printStackTrace
}
jsonString
}
This however is not working. Even when the Request class is populated, the output is :
{"parameters":{"underlying":{"some-value":""},"empty":false,"traversableAgain":true},"deps":{"sizeMapDefined":false,"empty":false,"traversableAgain":true}}
Using the JSON object mapper with corresponding Java classes is straightforward, but have not yet got it working in Scala. Any assistance is very much appreciated.
Jackson is more of a bad old memory in Scala to some degree. You should use a native Scala library for JSON processing, particularly one really good at compile time derivation of JSON serializers, such as circe.
I'm aware this doesn't directly answer your question, but after using circe I would never go back to anything else.
import io.circe.generic.auto._
import io.circe.parser._
import io.circe.syntax._
val req = new Request(...)
val json = req.asJson.noSpaces
val reparsed = decode[Request](json)
On a different note, using mutable maps inside case classes is as non-idiomatic as it gets, and it should be quite trivial to implement immutable ops for your maps using the auto-generated copy method.
case class Request(parameters: Map[String, String] {
def +(key: String, value: String): Request = {
this.copy(parameters = parameters + (key -> value))
}
}
You should really avoid mutability wherever possible, and it looks like avoiding it here wouldn't be much work at all.
I am not sure what this ScalaObjectMapper does, doesn't look like it is useful.
If you add mapper.registerModule(DefaultScalaModule) in the beginning, it should work ... assuming that by MutableMap you mean mutable.Map, and not some sort of home-made class (because, if you do, you'd have to provide a serializer for it yourself).
(DefaultScalaModule is in jackson-module-scala library. Just add it to your build if you don't already have it).

https4s how to make a POST call to a REST web service

I am trying to use the http4s library. I am trying to make a POST request to a REST web service with some json payload.
when I read the documentation http://http4s.org/docs/0.15/ I can only see a GET method example.
does anyone know how to make a POST?
It looks like the get/getAs methods mentioned in the example are just convenience wrappers for the fetch method. See https://github.com/http4s/http4s/blob/a4b52b042338ab35d89d260e0bcb39ccec1f1947/client/src/main/scala/org/http4s/client/Client.scala#L116
Use the Request constructor and pass Method.POST as the method.
fetch(Request(Method.POST, uri))
https4s version: 0.14.11
The hard part is how to set the post body. When you dive into the code, you may find type EntityBody = Process[Task, ByteVector]. But wtf is it? However, if you have not been ready to dive into scalaz, just use withBody.
object Client extends App {
val client = PooledHttp1Client()
val httpize = Uri.uri("http://httpize.herokuapp.com")
def post() = {
val req = Request(method = Method.POST, uri = httpize / "post").withBody("hello")
val task = client.expect[String](req)
val x = task.unsafePerformSync
println(x)
}
post()
client.shutdownNow()
}
P.S. my helpful post about http4s client(Just skip the Chinese and read the scala code): http://sadhen.com/blog/2016/11/27/http4s-client-intro.html
import org.http4s.circe._
import org.http4s.dsl._
import io.circe.generic.auto._
case class Name(name: String)
implicit val nameDecoder: EntityDecoder[Name] = jsonOf[Name]
def routes: PartialFunction[Request, Task[Response]] = {
case req # POST -> Root / "hello" =>
req.decode[Name] { name =>
Ok(s"Hello, ${name.name}")
}
Hope this helps.

JSON Rendering Scala/Play 2.1

I am working on a small project try to get a Scala/Play backend working. I am trying to have it return and also process JSON on the web service side. I cannot seem to figure out how to get the JSON marshalling and unmarshalling to work. Could someone help me with this issue? I am using Play 2.1 and Scala 2.10. The error that I get is
"overriding method reads in trait Reads of type (json: play.api.libs.json.JsValue)play.api.libs.json.JsResult[models.Address]; method reads has incompatible type"
Edited. Someone else gave me the solution. For read you must use JsSuccess, not JsResult.
case class Address(id: Long, name: String)
object Address {
implicit object AddressFormat extends Format[Address] {
def reads(json: JsValue):Address = JsSuccess(Address(
(json \ "id").as[Long],
(json \ "name").as[String]
))
def writes(address: Address): JsValue = JsObject(Seq(
"id" -> JsNumber(address.id),
"name" -> JsString(address.name)
))
}
}
With Play 2.1 you could simplify your code:
import play.api.libs.json._
import play.api.libs.functional.syntax._
implicit val addressFormat = (
(__ \ "id").format[String] and
(__ \ "name").format[Long]
)(Address.apply, unlift(Address.unapply))
More detailed information can be found here: ScalaJsonCombinators
You can simplify your code even further by using macros, although they are marked as experimental:
import play.api.libs.json._
import play.api.libs.functional.syntax._
case class Address(id: Long, name: String)
implicit val addressFormat = Json.format[Address]
More details on this technique in the official Play documentation.
Hey my solution would be:
import play.api.libs.json.JsonNaming.SnakeCase
import play.api.libs.json._
object Test {
implicit val config = JsonConfiguration(SnakeCase)
implicit val userFormat: OFormat[Test] = Json.format[Test]
}
case class Test(
testName: String,
testWert: String,
testHaus: String
)
In conclusion, you get a compenion object. The config converts all keys of the case class into snakecase. The implicit values ensure that a valid Json can be parsed into a model. So you get your test model back.
The Json should look like this:
{
"test_name" : "Hello",
"test_wert": "Hello",
"test_haus": "Hello"
}
https://www.playframework.com/documentation/2.6.x/ScalaJsonAutomated