The most trivial self-hosted REST-like web service in Scala? - scala

I have a function of the form:
(inp:CaseClass) => SomeOtherCaseClass
In other words, it takes a function with a single argument whose type is a Case Class. It returns a single value whose argument is a different kind of case class.
Just about every public function in my application will be of this form.
I'd like to expose this as a web-service so that a client can use this function by HTTP-Posting some JSON. The client will receive the response as a JSON encoded document.
An ideal solution will be:
Simple (e.g. few lines of code, using mostly non-obscure language features)
Should automatically marshal and unmarshal JSON - I don't want to have to write manual converters.
As easy to use as Flask (a popular Python micro-framework for web-stuff).
Things I don't yet care about:
High performance
Authentication / Encryption
I do have a working implementation based on Scalatra. It's OK-ish, but not particularly pretty because it includes quite a bit of boilerplate code just to make it start a server. I'm wondering if I can go for something even more minimal.
This solution was based on some 3-year old code samples I found at work. I'm sure that there must be something more appropriate which has been developed in the last few years?

Have a look at akka-http. It's relatively new (rewrite of the now obsolete spray library) and has a large support community. It's easy to get started for your simple use case but if you ever need more advanced features in the future, they're probably supported. JSON de-/serialization is done using spray-json adapter. You may also use other libraries, like circe with minimal amount of boilerplate. Here's a simple web server implementation accepting POST requests (copied from here):
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.stream.ActorMaterializer
import akka.Done
import akka.http.scaladsl.server.Route
import akka.http.scaladsl.server.Directives._
import akka.http.scaladsl.model.StatusCodes
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._
import spray.json.DefaultJsonProtocol._
import scala.io.StdIn
import scala.concurrent.Future
object WebServer {
// domain model
final case class Item(name: String, id: Long)
final case class Order(items: List[Item])
// formats for unmarshalling and marshalling
implicit val itemFormat = jsonFormat2(Item)
implicit val orderFormat = jsonFormat1(Order)
// (fake) async database query api
def fetchItem(itemId: Long): Future[Option[Item]] = ???
def saveOrder(order: Order): Future[Done] = ???
def main(args: Array[String]) {
// needed to run the route
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
// needed for the future map/flatmap in the end
implicit val executionContext = system.dispatcher
val route: Route =
get {
pathPrefix("item" / LongNumber) { id =>
// there might be no item for a given id
val maybeItem: Future[Option[Item]] = fetchItem(id)
onSuccess(maybeItem) {
case Some(item) => complete(item)
case None => complete(StatusCodes.NotFound)
}
}
} ~
post {
path("create-order") {
entity(as[Order]) { order =>
val saved: Future[Done] = saveOrder(order)
onComplete(saved) { done =>
complete("order created")
}
}
}
}
val bindingFuture = Http().bindAndHandle(route, "localhost", 8080)
println(s"Server online at http://localhost:8080/\nPress RETURN to stop...")
StdIn.readLine() // let it run until user presses return
bindingFuture
.flatMap(_.unbind()) // trigger unbinding from the port
.onComplete(_ ⇒ system.terminate()) // and shutdown when done
}
}
Does this suit your needs?

Related

In Scala Play Framework, is there a simple way to verify signatures on form requests

I am trying to write a Scala Play Framework action that will verify a HmacSHA256 signature on an incoming POST request containing form-url-encoded data.
This does not seem straightforward in the Play framework because: i) actions builders only have access to headers, but do not have access to the request body, and ii) in order to calculate the signature we have to treat the request body as Array[ByteString], but when we come to process the form data we have to treat it as Map[String, Seq[String]], the problem being thatPlay forces us to choose a single type for our request, and we cannot easily "cast" the request body to a different type.
The only solution I have been able to come up with is to use an ActionRefiner that returns a WrappedRequest that embeds a callback to validate the signature. The callback in turn reparses the data using FormUrlEncodedParser.parse(new String(request.body.toArray)). This approach is illustrated in the code below.
This all seems overly convoluted. Is there a simpler way to verify Hmac signatures in Play, or am I simply running up against limitations of the API?
package actions
import akka.util.ByteString
import com.google.inject.Inject
import play.api.Logging
import play.api.mvc.Results.Unauthorized
import play.api.mvc._
import play.core.parsers.FormUrlEncodedParser
import services.SlackSignatureVerifierService
import scala.concurrent.{ExecutionContext, Future}
import scala.util.Try
class SlackRequest[A](
val validateSignature: String => Try[String],
request: Request[A]
) extends WrappedRequest[A](request)
object SlackSignatureVerifyAction {
implicit class SlackRequestByteStringValidator(
slackRequest: SlackRequest[ByteString]
) {
def validateSignatureAgainstBody(): Try[Map[String, Seq[String]]] = {
val raw = slackRequest.body.utf8String
slackRequest.validateSignature(raw) map { _ =>
FormUrlEncodedParser.parse(new String(slackRequest.body.toArray))
}
}
}
val HEADERS_TIMESTAMP: String = "X-Slack-Request-Timestamp"
val HEADERS_SIGNATURE: String = "X-Slack-Signature"
}
class SlackSignatureVerifyAction #Inject() (
val parser: BodyParsers.Default,
slackSignatureVerifierService: SlackSignatureVerifierService
)(implicit ec: ExecutionContext)
extends ActionBuilder[SlackRequest, AnyContent]
with ActionRefiner[Request, SlackRequest]
with Logging {
override protected def executionContext: ExecutionContext = ec
override protected def refine[A](
request: Request[A]
): Future[Either[Result, SlackRequest[A]]] = {
val timestamp =
request.headers.get(SlackSignatureVerifyAction.HEADERS_TIMESTAMP)
val signature =
request.headers.get(SlackSignatureVerifyAction.HEADERS_SIGNATURE)
(timestamp, signature) match {
case (Some(timestamp), Some(signature)) =>
Future.successful {
val validate = (body: String) =>
slackSignatureVerifierService.validate(timestamp, body, signature)
Right(new SlackRequest[A](validate, request))
}
case _ =>
Future { Left(Unauthorized("Invalid signature headers")) }
}
}
}
You are right, there isn't an easy way to verify Hmac signatures on Play! projects. In the end, your approach seems to have a very reasonable implementation and could be easier adapted to other providers, such as GitHub and Stripe, that uses Hmac signatures.
I really think it could be a good open-source project to provide an Action with a wrapped Request or even a Service with a method to do custom signature validation for other providers. Why don't you help the Play community with an open-source project over GitHub?
I have created a new Play module to validate Hmac signatures. Details can be found here:
https://github.com/phelps-sg/play-hmac-signatures

Completing request outside of main controller in Akka-http

I am pretty new to the Akka world and I have to migrate a project from Spray to akka-http.
In spray, a route was of type Route = RequestContext ⇒ Unit.
But in akka-http, it is of type Route = RequestContext ⇒ Future[RouteResult].
So in spray, we would often handle and complete our requests through a chain of Actors (outside of the main controller) using only fire-and-forget so we didn't have to "ask" and the performance was great. Now, we have to use "ask" every time we pass the request to another Actor (correct me if I'm wrong)
I've been searching a lot and I found a few options which I'm not sure if they fully satisfy me (the need to complete a request in another Actor without the need to return it back to the controller). So that's where you could help me :)
Option 1: Using onSuccess/onComplete
Does using this block my main controller from receiving more requests?
Option 2: Using Futures and using RouteResult.Complete
I've found the following example at How to complete a request in another actor when using akka-http:
import akka.actor.{ActorSystem, Actor, Props}
import akka.pattern.ask
import akka.stream.ActorMaterializer
import akka.http.scaladsl.Http
import akka.http.scaladsl.server.Directives._
import akka.http.scaladsl.server.{RequestContext, RouteResult}
import scala.concurrent.Future
import akka.http.scaladsl.model.HttpResponse
import akka.util.Timeout
class RequestActor extends Actor {
//business logic - returns empty HttpResponse
def handleRequestMessage(requestContext : RequestContext) =
RouteResult.Complete(new HttpResponse())
override def receive = {
case reqContext : RequestContext =>
sender ! handleRequestMessage(reqContext)
}
}//end class RequestActor
object RouteActorTest extends App {
implicit val as = ActorSystem("RouteActorTest")
implicit val timeout = new Timeout(1000)
val actorRef = as actorOf Props[RequestActor]
def handleRequest(reqContext : RequestContext) : Future[RouteResult] =
ask(actorRef,reqContext).mapTo[RouteResult]
val route = path("") { get(handleRequest) }
//rest of application...
}//end object RouteActorTest
But this actually passes the response back every time to the previous Actor (the sender) until it reaches the main controller. Another thing about this option is that in the code, it says:
/**
* The result of handling a request.
*
* As a user you typically don't create RouteResult instances directly.
* Instead, use the methods on the [[RequestContext]] to achieve the desired effect.
*/
So I'm not sure if it's a recommended way of doing it.
I've tried using requestContext.complete() in another actor but it doesn't work (no error thrown, simply doesn't send response)
Does anybody know what the best way is to implement the previous architecture we had?
Thanks a lot!
the need to complete a request in another Actor without the need to
return it back to the controller
You don't need to complete the request in another actor. All you need to do is for the actor that handles the request to send back a result, and you can use complete to signal completion of the future:
case class MyRequest(data: Int)
case class MyResult(data: Int)
class RequestActor extends Actor {
override def receive: PartialFunction[Any, Unit] = {
case MyRequest(data) => sender ! MyResult(data + 1)
}
}
And route:
get {
path("yuval") {
import scala.concurrent.ExecutionContext.Implicits.global
implicit val timeout = Timeout(5 seconds)
complete {
val result = (completionActor ? MyRequest(1)).mapTo[MyResult]
result.map(r => HttpResponse(StatusCodes.OK, entity = s"Result was ${r.data}"))
}
}
}
If you want to handle to handcraft the HttpResponse yourself, you can always use context.complete:
get {
path("yuval") {
import scala.concurrent.ExecutionContext.Implicits.global
implicit val timeout = Timeout(5 seconds)
context => {
val result = (completionActor ? MyRequest(1)).mapTo[MyResult]
result.flatMap(r => context.complete(HttpResponse(StatusCodes.OK, entity = s"Result was ${r.data}")))
}
}
}

http4s - get request body as String or InputStream

I'm trying to define HttpService that receives json and parses it to case class with json4s library:
import org.http4s._
import org.http4s.dsl._
import org.json4s._
import org.json4s.native.JsonMethods._
case class Request(firstName: String, secondName: String)
HttpService {
case req # POST -> Root =>
val request = parse(<map req.body or req.bodyAsText to JsonInput>).extract[Request]
Ok()
}
How can I get org.json4s.JsonInput from req.body or req.bodyAsText?
I know that json4s also have StringInput and StreamInput that inherits from JsonInput for using with String and InputStream so I think that I need to convert req.body to InputStream or req.bodyAsText to String but I still do not understand how.
I'm new to Scala and I do not yet fully understand some concepts such as scalaz.stream.Process.
You can use the http4s-json4s-jackson (or http4s-json4s-native) packages and use an org.http4s.EntityDecoder to easily get a Foo (I renamed your Request case class to Foo below) from a request.
EntityDecoder is a type class which can decode an entity from the request body.
We want to get the Foo posted in JSON, so we need to create an EntityDecoder[Foo] which can decode JSON. If we want to create this decoder using json4s we need a Reader (or a JsonFormat).
If you have an EntityDecoder[Foo] instance, we can get the Foo from the request with req.as[Foo].
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.http4s._
import org.http4s.dsl._
import org.http4s.json4s.jackson._
case class Foo(firstName: String, secondName: String)
// create a json4s Reader[Foo]
implicit val formats = DefaultFormats
implicit val fooReader = new Reader[Foo] {
def read(value: JValue): Foo = value.extract[Foo]
}
// create a http4s EntityDecoder[Foo] (which uses the Reader)
implicit val fooDec = jsonOf[Foo]
val service = HttpService {
case req # POST -> Root =>
// req.as[Foo] gives us a Task[Foo]
// and since Ok(...) gives a Task[Response] we need to use flatMap
req.as[Foo] flatMap ( foo => Ok(foo.firstName + " " + foo.secondName) )
}
Note: The json libraries libraries used most often with http4s are probably argonaut and circe. So you might find more http4s examples using one of those libraries.
Peter's solution both corrects the question and answers it, but I stumbled here looking for the solution to OP's stated, but not intended, question: "how to get request body as [...] InputStream" in http4s. Thanks to the discussion in Issue 634 on GitHub, here's what I came up with:
import java.io.InputStream
import org.http4s._
implicit val inputStreamDecoder: EntityDecoder[InputStream] =
EntityDecoder.decodeBy(MediaRange.`*/*`) { msg =>
DecodeResult.success(scalaz.stream.io.toInputStream(msg.body))
}
And then in your HttpService, use that decoder like so:
request.as[InputStream].flatMap { inputStream => ...inputStream is an InputStream... }
Or skip the whole Decoder dance, if you want:
val inputStream = scalaz.stream.io.toInputStream(request.body)
You may use flatMap and as inside it before calling the Http4s service to decode responses from it:
#Test def `Get json gives valid contact`: Unit = {
val request = Request[IO](GET, uri"/contact")
val io = Main.getJsonWithContact.orNotFound.run(request)
// here is magic
val response = io.flatMap(_.as[Json]).unsafeRunSync()
val contact = contactEncoder(Contact(1, "Denis", "123")) // this is encoding to json for assertion
assertEquals(contact, response)
}
This is how types work here:
val io: IO[Response[IO]] = Main.getJsonWithContact.orNotFound.run(request)
val response: IO[Json] = io.flatMap(_.as[Json])
val res: Json = response.unsafeRunSync()
as[String] will return the string just like this.

ReactiveMongo w/ Play Scala

New to play,scala, and reactivemongo and the documentation is not very noob friendly.
I see the Bulk Insert section at See Bulk Insert
but I don't know why they aren't showing it contained in a method?
I am expecting a request with JSON data containing multiple objects in it. How do I set up a bulk insert that handles multiple inserts with errors that can be returned.
For example by single insert method is as follows:
def createFromJson = Action(parse.json) {
request =>
try {
val person = request.body.validate[Person].get
val mongoResult = Await.result(collection.insert(person),Duration.apply(20,"seconds"))
if(mongoResult.hasErrors) throw new Exception(mongoResult.errmsg.getOrElse("something unknown"))
Created(Json.toJson(person))
}
catch {
case e: Exception => BadRequest(e.getMessage)
}
}
Here is a full example how you can do it:
class ExampleController #Inject()(database: DefaultDB) extends Controller {
case class Person(firstName: String, lastName: String)
val personCollection: BSONCollection = database.collection("persons")
implicit val PersonJsonReader: Reads[Person] = Json.reads[Person]
implicit val PersonSeqJsonReader: Reads[Seq[Person]] = Reads.seq(PersonJsonReader)
implicit val PersonJsonWriter: Writes[Person] = Json.writes[Person]
implicit val PersonSeqJsonWriter: Writes[Seq[Person]] = Writes.seq(PersonJsonWriter)
implicit val PersonBsonWriter = Macros.writer[Person]
def insertMultiple = Action.async(parse.json) { implicit request =>
val validationResult: JsResult[Seq[Person]] = request.body.validate[Seq[Person]]
validationResult.fold(
invalidValidationResult => Future.successful(BadRequest),
// [1]
validValidationResult => {
val bulkDocs = validValidationResult.
map(implicitly[personCollection.ImplicitlyDocumentProducer](_))
personCollection.bulkInsert(ordered = true)(bulkDocs: _*).map {
case insertResult if insertResult.ok =>
Created(Json.toJson(validationResult.get))
case insertResult =>
InternalServerError
}
}
)
}
}
The meat of it all sits in the lines after [1]. validValidationResult is a variable of type Seq[Person] and contains valid data at this point. Thats what we want to insert into the database.
To do that we need to prepare the documents by mapping each document through the ImplicitlyDocumentProducer of your target collection (here personCollection). Thats leaves you with bulkDocs of type Seq[personCollection.ImplicitlyDocumentProducer]. You can just use bulkInsert() with that:
personCollection.bulkInsert(ordered = true)(bulkDocs: _*)
We use _* here to splat the Seq since bulkInsert() expects varargs and not a Seq. See this thread for more info about it. And thats basically it already.
The remaing code is handling play results and validating the received request body to make sure it contains valid data.
Here are a few general tips to work with play/reactivemongo/scala/futures:
Avoid Await.result. You basically never need it in production code. The idea behind futures is to perform non-blocking operations. Making them blocking again with Await.result defeats the purpose. It can be useful for debugging or test code, but even then there are usually better ways to go about things. Scala futures (unlike java ones) are very powerful and you can do a lot with them, see e.g. flatMap/map/filter/foreach/.. in the Future scaladoc. The above code for instance makes use of exactly that. It uses Action.async instead of Action at the controller method. This means it has to return a Future[Result] instead of a Result. Which is great because ReactiveMongo returns a bunch of Futures for all operations. So all you have to do is execute bulkInsert, which returns a Future and use map() to map the returned Future[MultiBulkWriteResult] to a Future[Result]. This results in no blocking and play can work with the returned future just fine.
Of course the above example can be improved a bit, I tried to keep it simple.
For instance you should return proper error messages when returning BadRequest (request body validation failed) or InternalServerError (database write failed). You can get more info about the errors from invalidValidationResult and insertResult. And you could use Formats instead of that many Reads/Writes (and also use them for ReactiveMongo). Check the play json documentation as well as the reactive mongo doc for more info on that.
Although the previous answer is correct.
We can reduce the boilerplate using JSONCollection
package controllers
import javax.inject._
import play.api.libs.json._
import play.api.mvc._
import play.modules.reactivemongo._
import reactivemongo.play.json.collection.{JSONCollection, _}
import utils.Errors
import scala.concurrent.{ExecutionContext, Future}
case class Person(name: String, age: Int)
object Person {
implicit val formatter = Json.format[Person]
}
#Singleton
class PersonBulkController #Inject()(val reactiveMongoApi: ReactiveMongoApi)(implicit exec: ExecutionContext) extends Controller with MongoController with ReactiveMongoComponents {
val persons: JSONCollection = db.collection[JSONCollection]("person")
def createBulkFromJson = Action.async(parse.json) { request =>
Json.fromJson[Seq[Person]](request.body) match {
case JsSuccess(newPersons, _) =>
val documents = newPersons.map(implicitly[persons.ImplicitlyDocumentProducer](_))
persons.bulkInsert(ordered = true)(documents: _*).map{ multiResult =>
Created(s"Created ${multiResult.n} persons")
}
case JsError(errors) =>
Future.successful(BadRequest("Could not build an array of persons from the json provided. " + errors))
}
}
}
In build.sbt
libraryDependencies ++= Seq(
"org.reactivemongo" %% "play2-reactivemongo" % "0.11.12"
)
Tested with play 2.5.1 although it should compile in previous versions of play.
FYI, as previous answers said, there are two ways to manipulate JSON data: use ReactiveMongo module + Play JSON library, or use ReactiveMongo's BSON library.
The documentation of ReactiveMongo module for Play Framework is available online. You can find code examples there.

How to test Zentasks sample app from Play 2.0

I play with Play 2.0, Scala version. Currently, I analyze Zentasks sample app.
One of the part of this app is authentication mechanism mostly covered in Secured trait. I'm wondering how I can test secured actions, ex. index from Projects controller.
For not-secured action, I'd probably do something like
val result = controllers.Projects.index(FakeRequest())
to run an action and get its result.
What should I do in case of the secured action?
Disclaimer: I'm totally new to both Scala and Play, so all hints are very valuable. Thanks!
There is a fix for the integrated approach to this in Playframewrk v2.1 I have a backport of the fix on the 2.0.x branch
Until it gets merged and released, here is what I did (it works on Play 2.0.3+):
I defined my own Helpers object in a libs package like so.
package libs
import play.api.mvc._
import play.api.libs.iteratee._
import play.api.libs.concurrent._
import play.api.test._
object Helpers {
def routeAndCall[T](request: FakeRequest[T]): Option[Result] = {
routeAndCall(this.getClass.getClassLoader.loadClass("Routes").asInstanceOf[Class[play.core.Router.Routes]], request)
}
/**
* Use the Router to determine the Action to call for this request and executes it.
*/
def routeAndCall[T, ROUTER <: play.core.Router.Routes](router: Class[ROUTER], request: FakeRequest[T]): Option[play.api.mvc.Result] = {
val routes = router.getClassLoader.loadClass(router.getName + "$").getDeclaredField("MODULE$").get(null).asInstanceOf[play.core.Router.Routes]
routes.routes.lift(request).map {
case a: Action[_] =>
val action = a.asInstanceOf[Action[T]]
val parsedBody: Option[Either[play.api.mvc.Result, T]] = action.parser(request).fold(
(a, in) => Promise.pure(Some(a)),
k => Promise.pure(None),
(msg, in) => Promise.pure(None)
).await.get
parsedBody.map{resultOrT =>
resultOrT.right.toOption.map{innerBody =>
action(FakeRequest(request.method, request.uri, request.headers, innerBody))
}.getOrElse(resultOrT.left.get)
}.getOrElse(action(request))
}
}
}
Then in my test I import my Helpers and the whole play Helpers context, except for routeAndCall :
import libs.Helpers._
import play.api.test.Helpers.{routeAndCall => _,_}
I then use an Around to setup my app (I need the provide an application.secret as I store the authenticated user name in the session which is based on a signed cookie)
def appWithSecret():Map[String,String]={
Map(("application.secret","the answer is 42 !"))
}
object emptyApp extends Around {
def around[T <% Result](t: => T) = {
running(FakeApplication(additionalConfiguration = inMemoryMongoDatabase("emptyApp")++appWithSecret())) {
User(new ObjectId, "Jane Doe", "foobar#example.com", "id1").save()
t // execute t inside a http session
}
}
}
This allows me to write the following tests:
"respond to the index Action" in emptyApp {
val request: FakeRequest[AnyContent] = FakeRequest(GET, "/expenses").withSession(("email", "foobar#example.com"))
val Some(result) = routeAndCall(request)
status(result) must equalTo(OK)
contentType(result) must beSome("application/json")
charset(result) must beSome("utf-8")
contentAsString(result) must contain("Hello Bob")
}
It allows you to exercise the secured code even though it is not a unit test.
ok, I am no great expert either, but here is an idea.
Create a trait InSecure trait extends Secured which overrides the Secured actions and always permits access.
Then you can make an object InSecureProjects extends Projects with InSecture in your test, this should override just the security checks and let you test the actions without any security.
Now, instead of running the tests on Projects, you run them on InSecureProjects. You can do exactly the same for the other secured controllers.
I haven't tested it, so let me know if it works ;)