Not so long time ago I switched from akka-http to http4s. One of the basic things which I wanted to do correctly — JSON handling, in particular sending a JSON response.
I decided to use http4s with ZIO instead of cats, so here is how an http route looks like:
import fs2.Stream
import org.http4s._
import org.http4s.dsl.io._
import org.http4s.implicits._
import scalaz.zio.Task
import scalaz.zio.interop.catz._
import io.circe.generic.auto._
import io.circe.syntax._
class TweetsRoutes {
case class Tweet(author: String, tweet: String)
val helloWorldService = HttpRoutes.of[Task] {
case GET -> Root / "hello" / name => Task {
Response[Task](Ok)
.withBodyStream(Stream.emits(
Tweet(name, "dummy tweet text").asJson.toString.getBytes
))
}
}.orNotFound
}
As you see, JSON serialization part is pretty verbose:
.withBodyStream(Stream.emits(
Tweet(name, "dummy tweet text").asJson.toString.getBytes
))
Is there any other way to send JSON in a response?
Yes, there is: define and Encoder and Decoder for Task:
implicit def circeJsonDecoder[A](
implicit decoder: Decoder[A]
): EntityDecoder[Task, A] = jsonOf[Task, A]
implicit def circeJsonEncoder[A](
implicit encoder: Encoder[A]
): EntityEncoder[Task, A] = jsonEncoderOf[Task, A]
this way there is no need to transform to bytes.
EDIT: there is a full example here: https://github.com/mschuwalow/zio-todo-backend/blob/develop/src/main/scala/com/schuwalow/zio/todo/http/TodoService.scala
HT: #mschuwalow
There is even simpler solution for this. If you want to handle case class JSON encoding for HTTP responses, you just can add these imports:
import io.circe.generic.auto._
import org.http4s.circe.CirceEntityCodec._
BTW, the same imports handle decoding of incoming JSON requests into case classes as well
Related
I am trying to write a Scala Play Framework action that will verify a HmacSHA256 signature on an incoming POST request containing form-url-encoded data.
This does not seem straightforward in the Play framework because: i) actions builders only have access to headers, but do not have access to the request body, and ii) in order to calculate the signature we have to treat the request body as Array[ByteString], but when we come to process the form data we have to treat it as Map[String, Seq[String]], the problem being thatPlay forces us to choose a single type for our request, and we cannot easily "cast" the request body to a different type.
The only solution I have been able to come up with is to use an ActionRefiner that returns a WrappedRequest that embeds a callback to validate the signature. The callback in turn reparses the data using FormUrlEncodedParser.parse(new String(request.body.toArray)). This approach is illustrated in the code below.
This all seems overly convoluted. Is there a simpler way to verify Hmac signatures in Play, or am I simply running up against limitations of the API?
package actions
import akka.util.ByteString
import com.google.inject.Inject
import play.api.Logging
import play.api.mvc.Results.Unauthorized
import play.api.mvc._
import play.core.parsers.FormUrlEncodedParser
import services.SlackSignatureVerifierService
import scala.concurrent.{ExecutionContext, Future}
import scala.util.Try
class SlackRequest[A](
val validateSignature: String => Try[String],
request: Request[A]
) extends WrappedRequest[A](request)
object SlackSignatureVerifyAction {
implicit class SlackRequestByteStringValidator(
slackRequest: SlackRequest[ByteString]
) {
def validateSignatureAgainstBody(): Try[Map[String, Seq[String]]] = {
val raw = slackRequest.body.utf8String
slackRequest.validateSignature(raw) map { _ =>
FormUrlEncodedParser.parse(new String(slackRequest.body.toArray))
}
}
}
val HEADERS_TIMESTAMP: String = "X-Slack-Request-Timestamp"
val HEADERS_SIGNATURE: String = "X-Slack-Signature"
}
class SlackSignatureVerifyAction #Inject() (
val parser: BodyParsers.Default,
slackSignatureVerifierService: SlackSignatureVerifierService
)(implicit ec: ExecutionContext)
extends ActionBuilder[SlackRequest, AnyContent]
with ActionRefiner[Request, SlackRequest]
with Logging {
override protected def executionContext: ExecutionContext = ec
override protected def refine[A](
request: Request[A]
): Future[Either[Result, SlackRequest[A]]] = {
val timestamp =
request.headers.get(SlackSignatureVerifyAction.HEADERS_TIMESTAMP)
val signature =
request.headers.get(SlackSignatureVerifyAction.HEADERS_SIGNATURE)
(timestamp, signature) match {
case (Some(timestamp), Some(signature)) =>
Future.successful {
val validate = (body: String) =>
slackSignatureVerifierService.validate(timestamp, body, signature)
Right(new SlackRequest[A](validate, request))
}
case _ =>
Future { Left(Unauthorized("Invalid signature headers")) }
}
}
}
You are right, there isn't an easy way to verify Hmac signatures on Play! projects. In the end, your approach seems to have a very reasonable implementation and could be easier adapted to other providers, such as GitHub and Stripe, that uses Hmac signatures.
I really think it could be a good open-source project to provide an Action with a wrapped Request or even a Service with a method to do custom signature validation for other providers. Why don't you help the Play community with an open-source project over GitHub?
I have created a new Play module to validate Hmac signatures. Details can be found here:
https://github.com/phelps-sg/play-hmac-signatures
When attempting to decode a json message with circe, I have a requirement to return the original value that caused the decoding failure, if possible.
As an example, I receive some invalidJson that contains an invalidUuid. Having access to both the invalidJson and the history: List[CursorOp], how would I grab the invalidValue?
Some sample code, illustrating the problem:
import java.util.UUID
import io.circe.{CursorOp, DecodingFailure}
import org.scalatest.{FlatSpec, Matchers}
import cats.implicits._
import io.circe.generic.auto._
class JsonSpec extends FlatSpec with Matchers {
case class TestDecode(test: Test)
case class Test(uuidKey: Option[UUID])
"a token decoder" should "return the invalid value for a decoding failure" in {
val invalidUuid = "invalid"
val invalidJson = s"""{"test": {"uuidKey": "$invalidUuid"}}"""
io.circe.parser.decode[TestDecode](invalidJson) match {
case Left(DecodingFailure(_, history)) =>
getInvalidValue(invalidJson, history) shouldEqual invalidUuid.some
case Left(_) => fail("should have returned a DecodingFailure")
case Right(_) => fail("should have returned a DecodingFailure")
}
def getInvalidValue(invalidJson: String, history: List[CursorOp]): Option[String] = ???
}
}```
Yes, this is exactly the kind of use case that replay is designed to support:
import io.circe.{CursorOp, Decoder, DecodingFailure}
import io.circe.generic.auto._
import io.circe.jawn.parse
import java.util.UUID
case class TestDecode(test: Test)
case class Test(uuidKey: Option[UUID])
val Right(doc) = parse("""{"test": {"uuidKey": "invalid"}}""")
val Left(DecodingFailure(_, ops)) = Decoder[TestDecode].decodeJson(doc)
doc.hcursor.replay(ops).focus
// res0: Option[io.circe.Json] = Some("invalid")
It's a method on ACursor that takes a list of operations and applies them to the cursor (in reverse order, since the operations are generally gathered as a stack but in this case we want FIFO).
Note that the focus is an Option because the library doesn't have any static guarantee that you're applying this list of operations to the same JSON value you initially tried to decode, so it could fail.
I have a function of the form:
(inp:CaseClass) => SomeOtherCaseClass
In other words, it takes a function with a single argument whose type is a Case Class. It returns a single value whose argument is a different kind of case class.
Just about every public function in my application will be of this form.
I'd like to expose this as a web-service so that a client can use this function by HTTP-Posting some JSON. The client will receive the response as a JSON encoded document.
An ideal solution will be:
Simple (e.g. few lines of code, using mostly non-obscure language features)
Should automatically marshal and unmarshal JSON - I don't want to have to write manual converters.
As easy to use as Flask (a popular Python micro-framework for web-stuff).
Things I don't yet care about:
High performance
Authentication / Encryption
I do have a working implementation based on Scalatra. It's OK-ish, but not particularly pretty because it includes quite a bit of boilerplate code just to make it start a server. I'm wondering if I can go for something even more minimal.
This solution was based on some 3-year old code samples I found at work. I'm sure that there must be something more appropriate which has been developed in the last few years?
Have a look at akka-http. It's relatively new (rewrite of the now obsolete spray library) and has a large support community. It's easy to get started for your simple use case but if you ever need more advanced features in the future, they're probably supported. JSON de-/serialization is done using spray-json adapter. You may also use other libraries, like circe with minimal amount of boilerplate. Here's a simple web server implementation accepting POST requests (copied from here):
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.stream.ActorMaterializer
import akka.Done
import akka.http.scaladsl.server.Route
import akka.http.scaladsl.server.Directives._
import akka.http.scaladsl.model.StatusCodes
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._
import spray.json.DefaultJsonProtocol._
import scala.io.StdIn
import scala.concurrent.Future
object WebServer {
// domain model
final case class Item(name: String, id: Long)
final case class Order(items: List[Item])
// formats for unmarshalling and marshalling
implicit val itemFormat = jsonFormat2(Item)
implicit val orderFormat = jsonFormat1(Order)
// (fake) async database query api
def fetchItem(itemId: Long): Future[Option[Item]] = ???
def saveOrder(order: Order): Future[Done] = ???
def main(args: Array[String]) {
// needed to run the route
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
// needed for the future map/flatmap in the end
implicit val executionContext = system.dispatcher
val route: Route =
get {
pathPrefix("item" / LongNumber) { id =>
// there might be no item for a given id
val maybeItem: Future[Option[Item]] = fetchItem(id)
onSuccess(maybeItem) {
case Some(item) => complete(item)
case None => complete(StatusCodes.NotFound)
}
}
} ~
post {
path("create-order") {
entity(as[Order]) { order =>
val saved: Future[Done] = saveOrder(order)
onComplete(saved) { done =>
complete("order created")
}
}
}
}
val bindingFuture = Http().bindAndHandle(route, "localhost", 8080)
println(s"Server online at http://localhost:8080/\nPress RETURN to stop...")
StdIn.readLine() // let it run until user presses return
bindingFuture
.flatMap(_.unbind()) // trigger unbinding from the port
.onComplete(_ ⇒ system.terminate()) // and shutdown when done
}
}
Does this suit your needs?
I'm trying to define HttpService that receives json and parses it to case class with json4s library:
import org.http4s._
import org.http4s.dsl._
import org.json4s._
import org.json4s.native.JsonMethods._
case class Request(firstName: String, secondName: String)
HttpService {
case req # POST -> Root =>
val request = parse(<map req.body or req.bodyAsText to JsonInput>).extract[Request]
Ok()
}
How can I get org.json4s.JsonInput from req.body or req.bodyAsText?
I know that json4s also have StringInput and StreamInput that inherits from JsonInput for using with String and InputStream so I think that I need to convert req.body to InputStream or req.bodyAsText to String but I still do not understand how.
I'm new to Scala and I do not yet fully understand some concepts such as scalaz.stream.Process.
You can use the http4s-json4s-jackson (or http4s-json4s-native) packages and use an org.http4s.EntityDecoder to easily get a Foo (I renamed your Request case class to Foo below) from a request.
EntityDecoder is a type class which can decode an entity from the request body.
We want to get the Foo posted in JSON, so we need to create an EntityDecoder[Foo] which can decode JSON. If we want to create this decoder using json4s we need a Reader (or a JsonFormat).
If you have an EntityDecoder[Foo] instance, we can get the Foo from the request with req.as[Foo].
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.http4s._
import org.http4s.dsl._
import org.http4s.json4s.jackson._
case class Foo(firstName: String, secondName: String)
// create a json4s Reader[Foo]
implicit val formats = DefaultFormats
implicit val fooReader = new Reader[Foo] {
def read(value: JValue): Foo = value.extract[Foo]
}
// create a http4s EntityDecoder[Foo] (which uses the Reader)
implicit val fooDec = jsonOf[Foo]
val service = HttpService {
case req # POST -> Root =>
// req.as[Foo] gives us a Task[Foo]
// and since Ok(...) gives a Task[Response] we need to use flatMap
req.as[Foo] flatMap ( foo => Ok(foo.firstName + " " + foo.secondName) )
}
Note: The json libraries libraries used most often with http4s are probably argonaut and circe. So you might find more http4s examples using one of those libraries.
Peter's solution both corrects the question and answers it, but I stumbled here looking for the solution to OP's stated, but not intended, question: "how to get request body as [...] InputStream" in http4s. Thanks to the discussion in Issue 634 on GitHub, here's what I came up with:
import java.io.InputStream
import org.http4s._
implicit val inputStreamDecoder: EntityDecoder[InputStream] =
EntityDecoder.decodeBy(MediaRange.`*/*`) { msg =>
DecodeResult.success(scalaz.stream.io.toInputStream(msg.body))
}
And then in your HttpService, use that decoder like so:
request.as[InputStream].flatMap { inputStream => ...inputStream is an InputStream... }
Or skip the whole Decoder dance, if you want:
val inputStream = scalaz.stream.io.toInputStream(request.body)
You may use flatMap and as inside it before calling the Http4s service to decode responses from it:
#Test def `Get json gives valid contact`: Unit = {
val request = Request[IO](GET, uri"/contact")
val io = Main.getJsonWithContact.orNotFound.run(request)
// here is magic
val response = io.flatMap(_.as[Json]).unsafeRunSync()
val contact = contactEncoder(Contact(1, "Denis", "123")) // this is encoding to json for assertion
assertEquals(contact, response)
}
This is how types work here:
val io: IO[Response[IO]] = Main.getJsonWithContact.orNotFound.run(request)
val response: IO[Json] = io.flatMap(_.as[Json])
val res: Json = response.unsafeRunSync()
as[String] will return the string just like this.
I am working on a small project try to get a Scala/Play backend working. I am trying to have it return and also process JSON on the web service side. I cannot seem to figure out how to get the JSON marshalling and unmarshalling to work. Could someone help me with this issue? I am using Play 2.1 and Scala 2.10. The error that I get is
"overriding method reads in trait Reads of type (json: play.api.libs.json.JsValue)play.api.libs.json.JsResult[models.Address]; method reads has incompatible type"
Edited. Someone else gave me the solution. For read you must use JsSuccess, not JsResult.
case class Address(id: Long, name: String)
object Address {
implicit object AddressFormat extends Format[Address] {
def reads(json: JsValue):Address = JsSuccess(Address(
(json \ "id").as[Long],
(json \ "name").as[String]
))
def writes(address: Address): JsValue = JsObject(Seq(
"id" -> JsNumber(address.id),
"name" -> JsString(address.name)
))
}
}
With Play 2.1 you could simplify your code:
import play.api.libs.json._
import play.api.libs.functional.syntax._
implicit val addressFormat = (
(__ \ "id").format[String] and
(__ \ "name").format[Long]
)(Address.apply, unlift(Address.unapply))
More detailed information can be found here: ScalaJsonCombinators
You can simplify your code even further by using macros, although they are marked as experimental:
import play.api.libs.json._
import play.api.libs.functional.syntax._
case class Address(id: Long, name: String)
implicit val addressFormat = Json.format[Address]
More details on this technique in the official Play documentation.
Hey my solution would be:
import play.api.libs.json.JsonNaming.SnakeCase
import play.api.libs.json._
object Test {
implicit val config = JsonConfiguration(SnakeCase)
implicit val userFormat: OFormat[Test] = Json.format[Test]
}
case class Test(
testName: String,
testWert: String,
testHaus: String
)
In conclusion, you get a compenion object. The config converts all keys of the case class into snakecase. The implicit values ensure that a valid Json can be parsed into a model. So you get your test model back.
The Json should look like this:
{
"test_name" : "Hello",
"test_wert": "Hello",
"test_haus": "Hello"
}
https://www.playframework.com/documentation/2.6.x/ScalaJsonAutomated