I am trying to use the http4s library. I am trying to make a POST request to a REST web service with some json payload.
when I read the documentation http://http4s.org/docs/0.15/ I can only see a GET method example.
does anyone know how to make a POST?
It looks like the get/getAs methods mentioned in the example are just convenience wrappers for the fetch method. See https://github.com/http4s/http4s/blob/a4b52b042338ab35d89d260e0bcb39ccec1f1947/client/src/main/scala/org/http4s/client/Client.scala#L116
Use the Request constructor and pass Method.POST as the method.
fetch(Request(Method.POST, uri))
https4s version: 0.14.11
The hard part is how to set the post body. When you dive into the code, you may find type EntityBody = Process[Task, ByteVector]. But wtf is it? However, if you have not been ready to dive into scalaz, just use withBody.
object Client extends App {
val client = PooledHttp1Client()
val httpize = Uri.uri("http://httpize.herokuapp.com")
def post() = {
val req = Request(method = Method.POST, uri = httpize / "post").withBody("hello")
val task = client.expect[String](req)
val x = task.unsafePerformSync
println(x)
}
post()
client.shutdownNow()
}
P.S. my helpful post about http4s client(Just skip the Chinese and read the scala code): http://sadhen.com/blog/2016/11/27/http4s-client-intro.html
import org.http4s.circe._
import org.http4s.dsl._
import io.circe.generic.auto._
case class Name(name: String)
implicit val nameDecoder: EntityDecoder[Name] = jsonOf[Name]
def routes: PartialFunction[Request, Task[Response]] = {
case req # POST -> Root / "hello" =>
req.decode[Name] { name =>
Ok(s"Hello, ${name.name}")
}
Hope this helps.
Related
I am trying to write a Scala Play Framework action that will verify a HmacSHA256 signature on an incoming POST request containing form-url-encoded data.
This does not seem straightforward in the Play framework because: i) actions builders only have access to headers, but do not have access to the request body, and ii) in order to calculate the signature we have to treat the request body as Array[ByteString], but when we come to process the form data we have to treat it as Map[String, Seq[String]], the problem being thatPlay forces us to choose a single type for our request, and we cannot easily "cast" the request body to a different type.
The only solution I have been able to come up with is to use an ActionRefiner that returns a WrappedRequest that embeds a callback to validate the signature. The callback in turn reparses the data using FormUrlEncodedParser.parse(new String(request.body.toArray)). This approach is illustrated in the code below.
This all seems overly convoluted. Is there a simpler way to verify Hmac signatures in Play, or am I simply running up against limitations of the API?
package actions
import akka.util.ByteString
import com.google.inject.Inject
import play.api.Logging
import play.api.mvc.Results.Unauthorized
import play.api.mvc._
import play.core.parsers.FormUrlEncodedParser
import services.SlackSignatureVerifierService
import scala.concurrent.{ExecutionContext, Future}
import scala.util.Try
class SlackRequest[A](
val validateSignature: String => Try[String],
request: Request[A]
) extends WrappedRequest[A](request)
object SlackSignatureVerifyAction {
implicit class SlackRequestByteStringValidator(
slackRequest: SlackRequest[ByteString]
) {
def validateSignatureAgainstBody(): Try[Map[String, Seq[String]]] = {
val raw = slackRequest.body.utf8String
slackRequest.validateSignature(raw) map { _ =>
FormUrlEncodedParser.parse(new String(slackRequest.body.toArray))
}
}
}
val HEADERS_TIMESTAMP: String = "X-Slack-Request-Timestamp"
val HEADERS_SIGNATURE: String = "X-Slack-Signature"
}
class SlackSignatureVerifyAction #Inject() (
val parser: BodyParsers.Default,
slackSignatureVerifierService: SlackSignatureVerifierService
)(implicit ec: ExecutionContext)
extends ActionBuilder[SlackRequest, AnyContent]
with ActionRefiner[Request, SlackRequest]
with Logging {
override protected def executionContext: ExecutionContext = ec
override protected def refine[A](
request: Request[A]
): Future[Either[Result, SlackRequest[A]]] = {
val timestamp =
request.headers.get(SlackSignatureVerifyAction.HEADERS_TIMESTAMP)
val signature =
request.headers.get(SlackSignatureVerifyAction.HEADERS_SIGNATURE)
(timestamp, signature) match {
case (Some(timestamp), Some(signature)) =>
Future.successful {
val validate = (body: String) =>
slackSignatureVerifierService.validate(timestamp, body, signature)
Right(new SlackRequest[A](validate, request))
}
case _ =>
Future { Left(Unauthorized("Invalid signature headers")) }
}
}
}
You are right, there isn't an easy way to verify Hmac signatures on Play! projects. In the end, your approach seems to have a very reasonable implementation and could be easier adapted to other providers, such as GitHub and Stripe, that uses Hmac signatures.
I really think it could be a good open-source project to provide an Action with a wrapped Request or even a Service with a method to do custom signature validation for other providers. Why don't you help the Play community with an open-source project over GitHub?
I have created a new Play module to validate Hmac signatures. Details can be found here:
https://github.com/phelps-sg/play-hmac-signatures
I'm trying to define HttpService that receives json and parses it to case class with json4s library:
import org.http4s._
import org.http4s.dsl._
import org.json4s._
import org.json4s.native.JsonMethods._
case class Request(firstName: String, secondName: String)
HttpService {
case req # POST -> Root =>
val request = parse(<map req.body or req.bodyAsText to JsonInput>).extract[Request]
Ok()
}
How can I get org.json4s.JsonInput from req.body or req.bodyAsText?
I know that json4s also have StringInput and StreamInput that inherits from JsonInput for using with String and InputStream so I think that I need to convert req.body to InputStream or req.bodyAsText to String but I still do not understand how.
I'm new to Scala and I do not yet fully understand some concepts such as scalaz.stream.Process.
You can use the http4s-json4s-jackson (or http4s-json4s-native) packages and use an org.http4s.EntityDecoder to easily get a Foo (I renamed your Request case class to Foo below) from a request.
EntityDecoder is a type class which can decode an entity from the request body.
We want to get the Foo posted in JSON, so we need to create an EntityDecoder[Foo] which can decode JSON. If we want to create this decoder using json4s we need a Reader (or a JsonFormat).
If you have an EntityDecoder[Foo] instance, we can get the Foo from the request with req.as[Foo].
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.http4s._
import org.http4s.dsl._
import org.http4s.json4s.jackson._
case class Foo(firstName: String, secondName: String)
// create a json4s Reader[Foo]
implicit val formats = DefaultFormats
implicit val fooReader = new Reader[Foo] {
def read(value: JValue): Foo = value.extract[Foo]
}
// create a http4s EntityDecoder[Foo] (which uses the Reader)
implicit val fooDec = jsonOf[Foo]
val service = HttpService {
case req # POST -> Root =>
// req.as[Foo] gives us a Task[Foo]
// and since Ok(...) gives a Task[Response] we need to use flatMap
req.as[Foo] flatMap ( foo => Ok(foo.firstName + " " + foo.secondName) )
}
Note: The json libraries libraries used most often with http4s are probably argonaut and circe. So you might find more http4s examples using one of those libraries.
Peter's solution both corrects the question and answers it, but I stumbled here looking for the solution to OP's stated, but not intended, question: "how to get request body as [...] InputStream" in http4s. Thanks to the discussion in Issue 634 on GitHub, here's what I came up with:
import java.io.InputStream
import org.http4s._
implicit val inputStreamDecoder: EntityDecoder[InputStream] =
EntityDecoder.decodeBy(MediaRange.`*/*`) { msg =>
DecodeResult.success(scalaz.stream.io.toInputStream(msg.body))
}
And then in your HttpService, use that decoder like so:
request.as[InputStream].flatMap { inputStream => ...inputStream is an InputStream... }
Or skip the whole Decoder dance, if you want:
val inputStream = scalaz.stream.io.toInputStream(request.body)
You may use flatMap and as inside it before calling the Http4s service to decode responses from it:
#Test def `Get json gives valid contact`: Unit = {
val request = Request[IO](GET, uri"/contact")
val io = Main.getJsonWithContact.orNotFound.run(request)
// here is magic
val response = io.flatMap(_.as[Json]).unsafeRunSync()
val contact = contactEncoder(Contact(1, "Denis", "123")) // this is encoding to json for assertion
assertEquals(contact, response)
}
This is how types work here:
val io: IO[Response[IO]] = Main.getJsonWithContact.orNotFound.run(request)
val response: IO[Json] = io.flatMap(_.as[Json])
val res: Json = response.unsafeRunSync()
as[String] will return the string just like this.
I'm using Play! 2.4 with Deadbolt2 for authorization. However, since I introduced the authorization rules, I'm unable to write successful tests for my controllers. As an example:
class VisitController #Inject() (authorization: DeadboltActions) extends Controller {
def fetchDailyVisits(date: Date) = authorization.Restrict(List(Array(ADMIN_ROLE), Array(MANAGER_ROLE))) {
Action.async {
visitService.findDailyVisits(date).map(result =>
Ok(Json.toJson(result))
)
}
}
}
I'm using specs2 in the tests. My test looks like this atm:
class VisitControllerSpec extends PlaySpecification with Mockito with ScalaFutures {
val deadboltActions = mock[DeadboltActions]
"VisitControllerSpec#fetchDailyVisits" should {
val testDate = Date.from(LocalDate.of(2016, 2, 25)
.atStartOfDay(ZoneId.systemDefault()).toInstant)
"Return Status Ok with returned list" in {
val expected = List(completeVisitWithId, anotherCompleteVisitWithId)
visitService.findDailyVisits(testDate) returns Future { expected }
val request = FakeRequest(GET, "/visits?date=2016-02-25")
val result = new VisitController(deadboltActions)
.fetchDailyVisits(testDate)(request)
result.futureValue.header.status must beEqualTo(OK)
contentAsJson(result) must_== Json.toJson(expected)
}
}
}
How do I mock deadboltActions in a way I can specify the user will be allowed access?
Is there another way? Maybe by providing a different DeadboltHandler? It seems kind of obvious this would be the way to go, I just don't seem to be able to figure it out and there aren't a lot of Deadbolt2 examples out there (at least for scala).
Or, being more extreme, any other authorization framework out there that works well with scala play and allows to handle security as a cross-cutting concern without poluting the controllers? Deadbolt2 is too limited for this reason, but I honestly can't find a better authorization framework (unless I write my own).
There are a number of different ways you can do this.
If your DeadboltHandler has a DAO injected for accessing the subject, you can override the binding of the DAO to provide one containing test subjects.
abstract class AbstractControllerSpec extends PlaySpecification {
sequential
isolated
def testApp: Application = new GuiceApplicationBuilder().in(Mode.Test).bindings(bind[SubjectDao].to[TestSubjectDao]).build()
}
See the test app for an example of using this approach.
Alternatively, you can extend your DeadboltHandler implementation to override getSubject and provide a test subject from here. The binding is handled in the same way as above.
Finally, you can keep all your code as-is and populate the test database with subjects; the requests you send will be shaped by your authentication requirements (headers, something in a cookie, etc).
For unit testing, something similar applies. Given a SubjectDao that has some hard-coded subjects for test purposes, you can use WithApplication and an injector look-up to get what you need.
class TestSubjectDao extends SubjectDao {
val subjects: Map[String, Subject] = Map("greet" -> new SecuritySubject("greet",
List(SecurityRole("foo"),
SecurityRole("bar")),
List(SecurityPermission("killer.undead.zombie"))),
"lotte" -> new SecuritySubject("lotte",
List(SecurityRole("hurdy")),
List(SecurityPermission("killer.undead.vampire"))),
"steve" -> new SecuritySubject("steve",
List(SecurityRole("bar")),
List(SecurityPermission("curator.museum.insects"))),
"mani" -> new SecuritySubject("mani",
List(SecurityRole("bar"),
SecurityRole("hurdy")),
List(SecurityPermission("zombie.movie.enthusiast"))),
"trippel" -> new SecuritySubject("trippel",
List(SecurityRole("foo"),
SecurityRole("hurdy")),
List[SecurityPermission]()))
override def user(userName: String): Option[Subject] = subjects.get(userName)
}
With a controller that looks something like this:
class Subject #Inject()(deadbolt: DeadboltActions) extends Controller {
def subjectMustBePresent = deadbolt.SubjectPresent()() { authRequest =>
Future {
Ok("Content accessible")
}
}
}
We can then unit test it like this:
import be.objectify.deadbolt.scala.DeadboltActions
import be.objectify.deadbolt.scala.test.controllers.composed.Subject
import be.objectify.deadbolt.scala.test.dao.{SubjectDao, TestSubjectDao}
import play.api.Mode
import play.api.inject._
import play.api.inject.guice.GuiceApplicationBuilder
import play.api.mvc.{Result, Results}
import play.api.test.{FakeRequest, PlaySpecification, WithApplication}
import scala.concurrent.Future
object SubjectPresentUnitSpec extends PlaySpecification with Results {
"Subject present " should {
"should result in a 401 when no subject is present" in new WithApplication(new GuiceApplicationBuilder().in(Mode.Test).bindings(bind[SubjectDao].to[TestSubjectDao]).build()) {
val deadbolt: DeadboltActions = implicitApp.injector.instanceOf[DeadboltActions]
val controller = new Subject(deadbolt)
val result: Future[Result] = call(controller.subjectMustBePresent(), FakeRequest())
val statusCode: Int = status(result)
statusCode must be equalTo 401
}
"should result in a 200 when a subject is present" in new WithApplication(new GuiceApplicationBuilder().in(Mode.Test).bindings(bind[SubjectDao].to[TestSubjectDao]).build()) {
val deadbolt: DeadboltActions = implicitApp.injector.instanceOf[DeadboltActions]
val controller = new Subject(deadbolt)
val result: Future[Result] = call(controller.subjectMustBePresent(), FakeRequest().withHeaders(("x-deadbolt-test-user", "greet")))
val statusCode: Int = status(result)
statusCode must be equalTo 200
}
}
}
It doesn't answer exactly to my original question, which was mostly related with Deadbolt2, but I kept getting frustrated with the fact I had to specify my authorization rules in my controllers, which is not truly cross cutting.
The answer provided by Steve Chaloner helps, but still forced me to go through a few hoops.
Enter Panoptes. This authorization framework is based on Filters instead of Action chaining, so it allows to easily specify authorization rules in a central location and outside of the controllers.
Setting your security rules in Panoptes is somewhat similar to Spring Security and it looks like this:
class BasicAuthHandler extends AuthorizationHandler {
override def config: Set[(Pattern, _ <: AuthorizationRule)] = {
Set(
Pattern(Some(POST), "/products") -> atLeastOne(withRole("Admin"), withRole("Manager"))
Pattern(Some(GET), "/cart[/A-Za-z0-9]*") -> withRole("Admin"),
Pattern(None, "/orders[/A-Za-z0-9]*") -> withRole("Admin")
)
}
}
Other than that, you need a couple of lines to declare the filter and plug in your AuthorizationHandler.
class Filters #Inject()(securityFilter: SecurityFilter) extends HttpFilters {
override def filters = Seq(securityFilter)
}
class ControllerProviderModule extends AbstractModule {
override def configure(): Unit = { bind(classOf[AuthorizationHandler]).to(classOf[MyAuthorizationHandler])
}
}
The README file in the git repository has more details and code samples.
It's also customizable to the point it allows to create your own AuthorizationRules. In my project I have a requirement where I need to check the mobile device that makes the call is registered in the system. I can write an AuthorizationRule to handle this for me for every request whose path matches my pattern.
Unit testing controllers is extra simple, because any mocking of the security layer can be ommited. They can be tested like any other class.
If you're having similar issues or also believe authorization rules don't belong in the controllers, have a go at Panoptes, it might suit your needs. Hope this helps someone else.
New to play,scala, and reactivemongo and the documentation is not very noob friendly.
I see the Bulk Insert section at See Bulk Insert
but I don't know why they aren't showing it contained in a method?
I am expecting a request with JSON data containing multiple objects in it. How do I set up a bulk insert that handles multiple inserts with errors that can be returned.
For example by single insert method is as follows:
def createFromJson = Action(parse.json) {
request =>
try {
val person = request.body.validate[Person].get
val mongoResult = Await.result(collection.insert(person),Duration.apply(20,"seconds"))
if(mongoResult.hasErrors) throw new Exception(mongoResult.errmsg.getOrElse("something unknown"))
Created(Json.toJson(person))
}
catch {
case e: Exception => BadRequest(e.getMessage)
}
}
Here is a full example how you can do it:
class ExampleController #Inject()(database: DefaultDB) extends Controller {
case class Person(firstName: String, lastName: String)
val personCollection: BSONCollection = database.collection("persons")
implicit val PersonJsonReader: Reads[Person] = Json.reads[Person]
implicit val PersonSeqJsonReader: Reads[Seq[Person]] = Reads.seq(PersonJsonReader)
implicit val PersonJsonWriter: Writes[Person] = Json.writes[Person]
implicit val PersonSeqJsonWriter: Writes[Seq[Person]] = Writes.seq(PersonJsonWriter)
implicit val PersonBsonWriter = Macros.writer[Person]
def insertMultiple = Action.async(parse.json) { implicit request =>
val validationResult: JsResult[Seq[Person]] = request.body.validate[Seq[Person]]
validationResult.fold(
invalidValidationResult => Future.successful(BadRequest),
// [1]
validValidationResult => {
val bulkDocs = validValidationResult.
map(implicitly[personCollection.ImplicitlyDocumentProducer](_))
personCollection.bulkInsert(ordered = true)(bulkDocs: _*).map {
case insertResult if insertResult.ok =>
Created(Json.toJson(validationResult.get))
case insertResult =>
InternalServerError
}
}
)
}
}
The meat of it all sits in the lines after [1]. validValidationResult is a variable of type Seq[Person] and contains valid data at this point. Thats what we want to insert into the database.
To do that we need to prepare the documents by mapping each document through the ImplicitlyDocumentProducer of your target collection (here personCollection). Thats leaves you with bulkDocs of type Seq[personCollection.ImplicitlyDocumentProducer]. You can just use bulkInsert() with that:
personCollection.bulkInsert(ordered = true)(bulkDocs: _*)
We use _* here to splat the Seq since bulkInsert() expects varargs and not a Seq. See this thread for more info about it. And thats basically it already.
The remaing code is handling play results and validating the received request body to make sure it contains valid data.
Here are a few general tips to work with play/reactivemongo/scala/futures:
Avoid Await.result. You basically never need it in production code. The idea behind futures is to perform non-blocking operations. Making them blocking again with Await.result defeats the purpose. It can be useful for debugging or test code, but even then there are usually better ways to go about things. Scala futures (unlike java ones) are very powerful and you can do a lot with them, see e.g. flatMap/map/filter/foreach/.. in the Future scaladoc. The above code for instance makes use of exactly that. It uses Action.async instead of Action at the controller method. This means it has to return a Future[Result] instead of a Result. Which is great because ReactiveMongo returns a bunch of Futures for all operations. So all you have to do is execute bulkInsert, which returns a Future and use map() to map the returned Future[MultiBulkWriteResult] to a Future[Result]. This results in no blocking and play can work with the returned future just fine.
Of course the above example can be improved a bit, I tried to keep it simple.
For instance you should return proper error messages when returning BadRequest (request body validation failed) or InternalServerError (database write failed). You can get more info about the errors from invalidValidationResult and insertResult. And you could use Formats instead of that many Reads/Writes (and also use them for ReactiveMongo). Check the play json documentation as well as the reactive mongo doc for more info on that.
Although the previous answer is correct.
We can reduce the boilerplate using JSONCollection
package controllers
import javax.inject._
import play.api.libs.json._
import play.api.mvc._
import play.modules.reactivemongo._
import reactivemongo.play.json.collection.{JSONCollection, _}
import utils.Errors
import scala.concurrent.{ExecutionContext, Future}
case class Person(name: String, age: Int)
object Person {
implicit val formatter = Json.format[Person]
}
#Singleton
class PersonBulkController #Inject()(val reactiveMongoApi: ReactiveMongoApi)(implicit exec: ExecutionContext) extends Controller with MongoController with ReactiveMongoComponents {
val persons: JSONCollection = db.collection[JSONCollection]("person")
def createBulkFromJson = Action.async(parse.json) { request =>
Json.fromJson[Seq[Person]](request.body) match {
case JsSuccess(newPersons, _) =>
val documents = newPersons.map(implicitly[persons.ImplicitlyDocumentProducer](_))
persons.bulkInsert(ordered = true)(documents: _*).map{ multiResult =>
Created(s"Created ${multiResult.n} persons")
}
case JsError(errors) =>
Future.successful(BadRequest("Could not build an array of persons from the json provided. " + errors))
}
}
}
In build.sbt
libraryDependencies ++= Seq(
"org.reactivemongo" %% "play2-reactivemongo" % "0.11.12"
)
Tested with play 2.5.1 although it should compile in previous versions of play.
FYI, as previous answers said, there are two ways to manipulate JSON data: use ReactiveMongo module + Play JSON library, or use ReactiveMongo's BSON library.
The documentation of ReactiveMongo module for Play Framework is available online. You can find code examples there.
I play with Play 2.0, Scala version. Currently, I analyze Zentasks sample app.
One of the part of this app is authentication mechanism mostly covered in Secured trait. I'm wondering how I can test secured actions, ex. index from Projects controller.
For not-secured action, I'd probably do something like
val result = controllers.Projects.index(FakeRequest())
to run an action and get its result.
What should I do in case of the secured action?
Disclaimer: I'm totally new to both Scala and Play, so all hints are very valuable. Thanks!
There is a fix for the integrated approach to this in Playframewrk v2.1 I have a backport of the fix on the 2.0.x branch
Until it gets merged and released, here is what I did (it works on Play 2.0.3+):
I defined my own Helpers object in a libs package like so.
package libs
import play.api.mvc._
import play.api.libs.iteratee._
import play.api.libs.concurrent._
import play.api.test._
object Helpers {
def routeAndCall[T](request: FakeRequest[T]): Option[Result] = {
routeAndCall(this.getClass.getClassLoader.loadClass("Routes").asInstanceOf[Class[play.core.Router.Routes]], request)
}
/**
* Use the Router to determine the Action to call for this request and executes it.
*/
def routeAndCall[T, ROUTER <: play.core.Router.Routes](router: Class[ROUTER], request: FakeRequest[T]): Option[play.api.mvc.Result] = {
val routes = router.getClassLoader.loadClass(router.getName + "$").getDeclaredField("MODULE$").get(null).asInstanceOf[play.core.Router.Routes]
routes.routes.lift(request).map {
case a: Action[_] =>
val action = a.asInstanceOf[Action[T]]
val parsedBody: Option[Either[play.api.mvc.Result, T]] = action.parser(request).fold(
(a, in) => Promise.pure(Some(a)),
k => Promise.pure(None),
(msg, in) => Promise.pure(None)
).await.get
parsedBody.map{resultOrT =>
resultOrT.right.toOption.map{innerBody =>
action(FakeRequest(request.method, request.uri, request.headers, innerBody))
}.getOrElse(resultOrT.left.get)
}.getOrElse(action(request))
}
}
}
Then in my test I import my Helpers and the whole play Helpers context, except for routeAndCall :
import libs.Helpers._
import play.api.test.Helpers.{routeAndCall => _,_}
I then use an Around to setup my app (I need the provide an application.secret as I store the authenticated user name in the session which is based on a signed cookie)
def appWithSecret():Map[String,String]={
Map(("application.secret","the answer is 42 !"))
}
object emptyApp extends Around {
def around[T <% Result](t: => T) = {
running(FakeApplication(additionalConfiguration = inMemoryMongoDatabase("emptyApp")++appWithSecret())) {
User(new ObjectId, "Jane Doe", "foobar#example.com", "id1").save()
t // execute t inside a http session
}
}
}
This allows me to write the following tests:
"respond to the index Action" in emptyApp {
val request: FakeRequest[AnyContent] = FakeRequest(GET, "/expenses").withSession(("email", "foobar#example.com"))
val Some(result) = routeAndCall(request)
status(result) must equalTo(OK)
contentType(result) must beSome("application/json")
charset(result) must beSome("utf-8")
contentAsString(result) must contain("Hello Bob")
}
It allows you to exercise the secured code even though it is not a unit test.
ok, I am no great expert either, but here is an idea.
Create a trait InSecure trait extends Secured which overrides the Secured actions and always permits access.
Then you can make an object InSecureProjects extends Projects with InSecture in your test, this should override just the security checks and let you test the actions without any security.
Now, instead of running the tests on Projects, you run them on InSecureProjects. You can do exactly the same for the other secured controllers.
I haven't tested it, so let me know if it works ;)