Composing with Play's EssentialAction - scala

I'm using Play 2.2.1 and I'm trying to write my own Action to deal with CORS requests. I found this but unfortunately it doesn't compile.
Just for reference here's the (slightly modified) code:
import play.api.mvc._
import scala.concurrent.ExecutionContext
case class CorsAction(action: EssentialAction) extends EssentialAction {
def apply(request: RequestHeader) = {
implicit val executionContext: ExecutionContext = play.api.libs.concurrent.Execution.defaultContext
val origin = request.headers.get("Origin").getOrElse("*")
if (request.method == "OPTIONS") {
val cors = Action { request =>
Ok("").withHeaders(
"Access-Control-Allow-Origin" -> origin,
"Access-Control-Allow-Methods" -> "GET, POST, PUT, DELETE, OPTIONS",
"Access-Control-Allow-Headers" -> "Accept, Origin, Content-type, Authorization, X-Auth-Token, " +
"X-HTTP-Method-Override, X-Json, X-Prototype-Version, X-Requested-With",
"Access-Control-Allow-Credentials" -> "true",
"Access-Control-Max-Age" -> (60 * 60 * 24 * 30).toString)
}
cors(request)
} else {
action(request).map(res =>
res.withHeaders(
"Access-Control-Allow-Origin" -> origin,
"Access-Control-Allow-Credentials" -> "true"
))
}
}
}
The error is:
Cors.scala:13: not found: value Ok
I'm very new to Scala and even more so to Play! and can't figure out what's going on. As far as I know I have to use EssentialAction and not just Action b/c I want to get to the header of the request. All the examples I found so far involve only Action.

As Will said you are missing the the Results trait.
A probably cleaner way to implement the CorsAction would be to use ActionBuildersas described in
http://www.playframework.com/documentation/2.2.x/ScalaActionsComposition
The implementation would look like this:
import play.api.mvc._
import scala.concurrent.{ExecutionContext, Future}
object CorsAction extends ActionBuilder[Request] with Results{
val MaxAge = 60 * 60 * 24 * 30
val AllowHeaders = List(
"Accept", "Origin", "Content-type", "Authorization", "X-Auth-Token",
"X-HTTP-Method-Override", "X-Json", "X-Prototype-Version", "X-Requested-With")
val AllowMethods = List("GET", "POST", "PUT", "DELETE", "OPTIONS")
val AllowCredentials = true
def cors[A](origin: String) =
Ok.withHeaders(
"Access-Control-Allow-Origin" -> origin,
"Access-Control-Allow-Methods" -> AllowMethods.mkString(", "),
"Access-Control-Allow-Headers" -> AllowHeaders.mkString(", "),
"Access-Control-Allow-Credentials" -> AllowCredentials.toString,
"Access-Control-Max-Age" -> MaxAge.toString)
def invokeBlock[A](request: Request[A], block: (Request[A]) => Future[SimpleResult]) = {
implicit val executionContext = play.api.libs.concurrent.Execution.defaultContext
val origin = request.headers.get("Origin").getOrElse("*")
if (request.method == "OPTIONS") {
Future.successful(cors(origin))
} else {
block(request).map(res =>
res.withHeaders(
"Access-Control-Allow-Origin" -> origin,
"Access-Control-Allow-Credentials" -> AllowCredentials.toString
))
}
}
}

You need to import the Results trait -- the Ok val is defined there.
http://www.playframework.com/documentation/2.2.x/api/scala/index.html#play.api.mvc.Results

Related

How to test a Scala Play Framework websocket?

If I have a websocket like the following:
def websocket: WebSocket = WebSocket.accept[String, String] { _ =>
ActorFlow.actorRef(out => LightWebSocketActor.props(out))
}
For reference, this is the LightWebSocketActor:
class LightWebSocketActor(out: ActorRef) extends Actor {
val topic: String = service.topic
override def receive: Receive = {
case message: String =>
play.Logger.debug(s"Message: $message")
PublishService.publish("true")
out ! message
}
}
object LightWebSocketActor {
var list: ListBuffer[ActorRef] = ListBuffer.empty[ActorRef]
def props(out: ActorRef): Props = {
list += out
Props(new LightSocketActor(out))
}
def sendMessage(message: String): Unit = {
list.foreach(_ ! message)
}
}
This is using the akka websocket approach.
How should a test for this kind of controller be created?
How should I send information an expect a response?
What kind of information should be sent in the fake request?
For example I have this test for a regular html-returning controller:
"Application" should {
"render the index page" in new WithApplication {
val home = route(app, FakeRequest(GET, "/")).get
status(home) must equalTo(OK)
contentType(home) must beSome.which(_ == "text/html")
contentAsString(home) must contain ("shouts out")
}
}
Play 2.6
I followed this Example: play-scala-websocket-example
Main steps:
Create or provide a WebSocketClient that you can use in your
tests.
Create the client:
val asyncHttpClient: AsyncHttpClient = wsClient.underlying[AsyncHttpClient]
val webSocketClient = new WebSocketClient(asyncHttpClient)
Connect to the serverURL:
val listener = new WebSocketClient.LoggingListener(message => queue.put(message))
val completionStage = webSocketClient.call(serverURL, origin, listener)
val f = FutureConverters.toScala(completionStage)
Test the Messages sent by the Server:
whenReady(f, timeout = Timeout(1.second)) { webSocket =>
await().until(() => webSocket.isOpen && queue.peek() != null)
checkMsg1(queue.take())
checkMsg2(queue.take())
assert(queue.isEmpty)
}
For example, like:
private def checkMsg1(msg: String) {
val json: JsValue = Json.parse(msg)
json.validate[AdapterMsg] match {
case JsSuccess(AdapterNotRunning(None), _) => // ok
case other => fail(s"Unexpected result: $other")
}
}
The whole example can be found here: scala-adapters (JobCockpitControllerSpec)
Adapted to Playframework 2.7
import java.util.concurrent.ExecutionException
import java.util.function.Consumer
import com.typesafe.scalalogging.StrictLogging
import play.shaded.ahc.org.asynchttpclient.AsyncHttpClient
import play.shaded.ahc.org.asynchttpclient.netty.ws.NettyWebSocket
import play.shaded.ahc.org.asynchttpclient.ws.{WebSocket, WebSocketListener, WebSocketUpgradeHandler}
import scala.compat.java8.FutureConverters
import scala.concurrent.Future
class LoggingListener(onMessageCallback: Consumer[String]) extends WebSocketListener with StrictLogging {
override def onOpen(websocket: WebSocket): Unit = {
logger.info("onClose: ")
websocket.sendTextFrame("hello")
}
override def onClose(webSocket: WebSocket, i: Int, s: String): Unit =
logger.info("onClose: ")
override def onError(t: Throwable): Unit =
logger.error("onError: ", t);
override def onTextFrame(payload: String, finalFragment: Boolean, rsv: Int): Unit = {
logger.debug(s"$payload $finalFragment $rsv")
onMessageCallback.accept(payload)
}
}
class WebSocketClient(client: AsyncHttpClient) {
#throws[ExecutionException]
#throws[InterruptedException]
def call(url: String, origin: String, listener: WebSocketListener): Future[NettyWebSocket] = {
val requestBuilder = client.prepareGet(url).addHeader("Origin", origin)
val handler = new WebSocketUpgradeHandler.Builder().addWebSocketListener(listener).build
val listenableFuture = requestBuilder.execute(handler)
FutureConverters.toScala(listenableFuture.toCompletableFuture)
}
}
And in test:
val myPublicAddress = s"localhost:$port"
val serverURL = s"ws://$myPublicAddress/api/alarm/ws"
val asyncHttpClient = client.underlying[AsyncHttpClient]
val webSocketClient = new WebSocketClient(asyncHttpClient)
val origin = "ws://example.com/ws"
val consumer: Consumer[String] = (message: String) => logger.debug(message)
val listener = new LoggingListener(consumer)
val f = webSocketClient.call(serverURL, origin, listener)
Await.result(f, atMost = 1000.millis)
This is a complete example which uses the Akka Websocket Client to test a Websocket controller. There is some custom code, but it shows multiple test scenarios. This works for Play 2.7.
package controllers
import java.util.concurrent.{ LinkedBlockingDeque, TimeUnit }
import actors.WSBridge
import akka.Done
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.headers.{ Origin, RawHeader }
import akka.http.scaladsl.model.ws.{ BinaryMessage, Message, TextMessage, WebSocketRequest }
import akka.http.scaladsl.model.{ HttpResponse, StatusCodes, Uri }
import akka.stream.scaladsl.{ Flow, Keep, Sink, Source, SourceQueueWithComplete }
import akka.stream.{ ActorMaterializer, OverflowStrategy }
import models.WSTopic
import org.specs2.matcher.JsonMatchers
import play.api.Logging
import play.api.inject.guice.GuiceApplicationBuilder
import play.api.test._
import scala.collection.immutable.Seq
import scala.concurrent.Future
/**
* Test case for the [[WSController]] actor.
*/
class WSControllerSpec extends ForServer with WSControllerSpecContext with JsonMatchers {
"The `socket` method" should {
"return a 403 status code if the origin doesn't match" >> { implicit rs: RunningServer =>
val maybeSocket = await(websocketClient.connect(WebSocketRequest(endpoint)))
maybeSocket must beLeft[HttpResponse].like { case response =>
response.status must be equalTo StatusCodes.Forbidden
}
}
"return a 400 status code if the topic cannot be found" >> { implicit rs: RunningServer =>
val headers = Seq(Origin("http://localhost:9443"))
val maybeSocket = await(websocketClient.connect(WebSocketRequest(endpoint, headers)))
maybeSocket must beLeft[HttpResponse].like { case response =>
response.status must be equalTo StatusCodes.BadRequest
}
}
"return a 400 status code if the topic syntax isn't valid in query param" >> { implicit rs: RunningServer =>
val headers = Seq(Origin("http://localhost:9443"))
val request = WebSocketRequest(endpoint.withRawQueryString("?topic=."), headers)
val maybeSocket = await(websocketClient.connect(request))
maybeSocket must beLeft[HttpResponse].like { case response =>
response.status must be equalTo StatusCodes.BadRequest
}
}
"return a 400 status code if the topic syntax isn't valid in header param" >> { implicit rs: RunningServer =>
val headers = Seq(Origin("http://localhost:9443"), RawHeader("X-TOPIC", "."))
val maybeSocket = await(websocketClient.connect(WebSocketRequest(endpoint, headers)))
maybeSocket must beLeft[HttpResponse].like { case response =>
response.status must be equalTo StatusCodes.BadRequest
}
}
"receive an acknowledge message when connecting to a topic via query param" >> { implicit rs: RunningServer =>
val headers = Seq(Origin("http://localhost:9443"))
val request = WebSocketRequest(endpoint.withRawQueryString("topic=%2Fflowers%2Frose"), headers)
val maybeSocket = await(websocketClient.connect(request))
maybeSocket must beRight[(SourceQueue, MessageQueue)].like { case (_, messages) =>
messages.poll(1000, TimeUnit.MILLISECONDS) must be equalTo
WSBridge.Ack(WSTopic("/flowers/rose")).message.toJson.toString()
}
}
"receive an acknowledge message when connecting to a topic via query param" >> { implicit rs: RunningServer =>
val headers = Seq(Origin("http://localhost:9443"), RawHeader("X-TOPIC", "/flowers/tulip"))
val maybeSocket = await(websocketClient.connect(WebSocketRequest(endpoint, headers)))
maybeSocket must beRight[(SourceQueue, MessageQueue)].like { case (_, messages) =>
messages.poll(1000, TimeUnit.MILLISECONDS) must be equalTo
WSBridge.Ack(WSTopic("/flowers/tulip")).message.toJson.toString()
}
}
"receive a pong message when sending a ping" >> { implicit rs: RunningServer =>
val headers = Seq(Origin("http://localhost:9443"), RawHeader("X-TOPIC", "/flowers/tulip"))
val maybeSocket = await(websocketClient.connect(WebSocketRequest(endpoint, headers)))
maybeSocket must beRight[(SourceQueue, MessageQueue)].like { case (queue, messages) =>
queue.offer(WSBridge.Ping.toJson.toString())
messages.poll(1000, TimeUnit.MILLISECONDS) must be equalTo
WSBridge.Ack(WSTopic("/flowers/tulip")).message.toJson.toString()
messages.poll(1000, TimeUnit.MILLISECONDS) must be equalTo
WSBridge.Pong.toJson.toString()
}
}
}
}
/**
* The context for the [[WSControllerSpec]].
*/
trait WSControllerSpecContext extends ForServer with PlaySpecification with ApplicationFactories {
type SourceQueue = SourceQueueWithComplete[String]
type MessageQueue = LinkedBlockingDeque[String]
/**
* Provides the application factory.
*/
protected def applicationFactory: ApplicationFactory = withGuiceApp(GuiceApplicationBuilder())
/**
* Gets the WebSocket endpoint.
*
* #param rs The running server.
* #return The WebSocket endpoint.
*/
protected def endpoint(implicit rs: RunningServer): Uri =
Uri(rs.endpoints.httpEndpoint.get.pathUrl("/ws").replace("http://", "ws://"))
/**
* Provides an instance of the WebSocket client.
*
* This should be a method to return a fresh client for every test.
*/
protected def websocketClient = new AkkaWebSocketClient
/**
* An Akka WebSocket client that is optimized for testing.
*/
class AkkaWebSocketClient extends Logging {
/**
* The queue of received messages.
*/
private val messageQueue = new LinkedBlockingDeque[String]()
/**
* Connect to the WebSocket.
*
* #param wsRequest The WebSocket request instance.
* #return Either an [[HttpResponse]] if the upgrade process wasn't successful or a source and a message queue
* to which new messages may be offered.
*/
def connect(wsRequest: WebSocketRequest): Future[Either[HttpResponse, (SourceQueue, MessageQueue)]] = {
implicit val system: ActorSystem = ActorSystem()
implicit val materializer: ActorMaterializer = ActorMaterializer()
import system.dispatcher
// Store each incoming message in the messages queue
val incoming: Sink[Message, Future[Done]] = Sink.foreach {
case TextMessage.Strict(s) => messageQueue.offer(s)
case TextMessage.Streamed(s) => s.runFold("")(_ + _).foreach(messageQueue.offer)
case BinaryMessage.Strict(s) => messageQueue.offer(s.utf8String)
case BinaryMessage.Streamed(s) => s.runFold("")(_ + _.utf8String).foreach(messageQueue.offer)
}
// Out source is a queue to which we can offer messages that will be sent to the WebSocket server.
// All offered messages will be transformed into WebSocket messages.
val sourceQueue = Source.queue[String](Int.MaxValue, OverflowStrategy.backpressure)
.map { msg => TextMessage.Strict(msg) }
val (sourceMat, source) = sourceQueue.preMaterialize()
// The outgoing flow sends all messages which are offered to the queue (our stream source) to the WebSocket
// server.
val flow: Flow[Message, Message, Future[Done]] = Flow.fromSinkAndSourceMat(incoming, source)(Keep.left)
// UpgradeResponse is a Future[WebSocketUpgradeResponse] that completes or fails when the connection succeeds
// or fails and closed is a Future[Done] representing the stream completion from above
val (upgradeResponse, closed) = Http().singleWebSocketRequest(wsRequest, flow)
closed.foreach(_ => logger.info("Channel closed"))
upgradeResponse.map { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Right((sourceMat, messageQueue))
} else {
Left(upgrade.response)
}
}
}
}
}

Akka flow for multiple http requests

In a project of mine I have an akka actor for sending post requests to my google fcm server. The actor takes a list of ids and should make as many requests as there are in the list. I print out the response from the server in runForeach(println(_)) but I only get one printout for a whole list of ids. Why does this happen?
class FCMActor(val key: String) extends Actor{
import fcm.FCMActor._
import akka.pattern.pipe
import context.dispatcher
private implicit def system: ActorSystem = ActorSystem()
final implicit val materializer: ActorMaterializer = ActorMaterializer(ActorMaterializerSettings(context.system))
def buildBody(id: Option[String]): String = {
Json.obj(
"to" -> id,
"priority" -> "high",
"data" -> Json.obj("message" -> "Firebase Clud Message"),
"time_to_live" -> 60
).toString()
}
def buildHttpRequest(body: String): HttpRequest = {
HttpRequest(method = HttpMethods.POST,
uri = s"/fcm/send",
entity = HttpEntity(MediaTypes.`application/json`, body),
headers = List(RawHeader("Authorization", s"key=$key")))
}
val connectionFlow: Flow[HttpRequest, HttpResponse, Future[Http.OutgoingConnection]] = {
Http().outgoingConnection("fcm.googleapis.com")
}
def send(ids: List[Option[String]]) = {
val httpRequests: List[HttpRequest] = ids.map(buildBody).map(buildHttpRequest)
println(httpRequests)
Source(httpRequests).via(connectionFlow).runForeach(println(_)) // << here I only get one println
}
override def receive: Receive = {
case SendToIds(ids: List[Option[String]]) =>
send(ids)
}
}
You are not consuming the response entity that the server sends you. To understand why this is important, check out the related docs page.
A quick code change to try and fix this is:
... .runForeach{ response =>
response.discardEntityBytes()
println(response)
}
Or, if you're actually interested in the entity, something along the lines of
... .runForeach{ _.entity.dataBytes
.runFold(ByteString.empty) { case (acc, b) => acc ++ b }
.map(println(_))
}

How do I set a non-standard User-Agent using spray-client?

I'm building an application for a Telco, using Scala and Akka, and need to communicate with Account Information and Refill servers using the UCIP protocol.
UCIP is a simple protocol, built on XMLRPC; the only issue I'm having is that it requires clients to set the User-Agent header in the specific format User-Agent: <client name>/<protocol version>/<client version>, which spray parses as invalid.
I tried creating a custom User-Agent header, inheriting from spray.http.HttpHeader but it still doesn't work. Here's what I've got so far:
import akka.actor.ActorSystem
import akka.event.{Logging, LoggingAdapter}
import spray.client.pipelining._
import spray.http._
import spray.httpx._
case class `User-Agent`(value: String) extends HttpHeader {
def lowercaseName: String = "user-agent"
def name: String = "User-Agent"
def render[R <: Rendering](r: R): r.type = r ~~ s"User-Agent: $value"
}
class UcipClient(val url: String, val protocol: String, username: String, password: String) (implicit system: ActorSystem) {
val log = Logging.getLogger(system, this)
val logRequest: HttpRequest => HttpRequest = { r => log.debug(r.toString); r }
val logResponse: HttpResponse => HttpResponse = { r => log.debug(r.toString); r }
val pipeline = (
addHeader(`User-Agent`("USSD-UCIP/%s/1.0".format(protocol)))
~> addCredentials(BasicHttpCredentials(username, password))
~> logRequest
~> sendReceive
~> logResponse
)
def send(req: UcipRequest) = pipeline(Post(url, req.getRequest))
}
My requests keep returning "Sorry, Error occured: 403, Invalid protocol version Not defined", however, they return the correct response when I send the same details using curl.
What am I missing, and is this even possible with spray-client? I've spent a fair bit of time checking the internets (which led me towards the custom header route), but still haven't figured this out...would really appreciate any help :-)
Turns out I wasn't far from the answer. While examining the headers being sent over the wire, I noticed the User-Agent was being set twice: once by my code, and again by Spray (because it considered my header invalid).
Setting the spray.can.client.user-agent-header to the empty string "" removed the second header, and requests were successful. Here's the final version of the custom header:
import spray.http._
object CustomHttpHeaders {
case class `User-Agent`(val value: String) extends HttpHeader with Product with Serializable {
def lowercaseName: String = "user-agent"
def name: String = "User-Agent"
def render[R <: Rendering](r: R): r.type = r ~~ s"User-Agent: $value"
}
}
And the final UCIP client:
import akka.actor.ActorRefFactory
import com.typesafe.config.Config
import scala.concurrent.ExecutionContext.Implicits.global
import scala.xml.NodeSeq
import spray.client.pipelining._
import spray.http._
import spray.httpx._
class UcipFault(val code: Int, msg: String) extends RuntimeException(s"$code: $msg")
class AirException(val code: Int) extends RuntimeException(s"$code")
class UcipClient(config: Config, val url: String)(implicit context: ActorRefFactory) {
import CustomHttpHeaders._
val throwOnFailure: NodeSeq => NodeSeq = {
case f if (f \\ "fault").size != 0 =>
val faultData = (f \\ "fault" \\ "member" \ "value")
throw new UcipFault((faultData \\ "i4").text.toInt,
(faultData \\ "string").text)
case el =>
val responseCode = ((el \\ "member")
.filter { n => (n \\ "name").text == "responseCode" }
.map { n => (n \\ "i4").text.toInt }).head
if (responseCode == 0) el else throw new AirException(responseCode)
}
val pipeline = (
addHeader(`User-Agent`("USSD-UCIP/%s/1.0".format(config.getString("ucip.server-protocol"))))
~> addCredentials(BasicHttpCredentials(config.getString("ucip.server-username"), config.getString("ucip.server-password")))
~> sendReceive
~> unmarshal[NodeSeq]
~> throwOnFailure
)
def send(req: UcipRequest) = pipeline(Post(url, req.getRequest))
}

In the scala spray framework, how can I create multiple http clients that have different configurations (e.g., timeouts, retries)

I have two spray http clients, such as the following:
val pipelineFoo: HttpRequest => Future[Foo] = (
sendReceive
~> unmarshal[Message.Foo])
val pipelineBar: HttpRequest => Future[Bar] = (
sendReceive
~> unmarshal[Message.Bar])
def execFoo(h: String, p: Int): Future[Foo] = {
val uri = Uri.from(scheme = "http", host = h, port = p, path = "/foo")
pipelineFoo(Get(uri))
}
def execBar(h: String, p: Int): Future[Bar] = {
val uri = Uri.from(scheme = "http", host = h, port = p, path = "/bar")
pipelineBar(Get(uri))
}
I would like have the foo request retry several times with a long timeout, and have the bar request not retry and have a short timeout (say 1 second). How can I achieve this in spray (sorry if this is somewhere in the documentation, but I've been unable to find it -- I've only found some documentation on setting such config parameters globally).
This shouldn't be too difficult. sendReceive can actually take more parameters. For example, here is the signature for one of the alternatives:
def sendReceive(transport: ActorRef)(implicit ec: ExecutionContext, futureTimeout: Timeout): SendReceive
I use this myself for similar scenarios where I have to have a bigger number of retries and longer timeouts when I hit an external service vs hitting one of our internal ones.
Here's an example of my pipeline that I use:
lazy val pipeline: HttpRequest => Future[HttpResponse] = (
addCredentials(BasicHttpCredentials(clientConnection.credentials._1, clientConnection.credentials._2))
~> addHeader(`User-Agent`(ProductVersion("<YOUR NAME HERE>", "<YOUR VERSION HERE>", "http://github.com/<WHEREVER YOUR PROJECT IS>"), ProductVersion("spray-client", "1.3.1", "http://spray.io")))
~> logRequest(log)
~> sendReceive(clientConnection.connection)(clientConnection.context, clientConnection.timeout)
~> decode(Deflate)
~> decode(Gzip)
)
The clientConnection is nothing special. It's just a case class that I made that can be filled in manually via code or maybe some config in your application.conf
2 years latter, but maybe worth for other people.
We had the same need and we based our solution on a copy/paste of Spray connector files.
import akka.actor.{ActorRef, ActorSystem}
import akka.io.IO
import akka.pattern.ask
import com.typesafe.config.Config
import spray.can.Http
import spray.can.Http.HostConnectorSetup
import spray.can.client.HostConnectorSettings
import spray.client.pipelining.sendReceive
import spray.http.Uri.Host
import spray.http.{HttpRequest, HttpResponse, Uri}
import scala.concurrent.duration._
import scala.concurrent.{ExecutionContextExecutor, Future}
case class HttpCustomSettings(
requestTimeout: Duration,
maxRetries: Int,
maxConnections: Int
)
/**
* Implement a new HTTP client on top of akka IO and spray HTTP
* to provide a way for caller to set client parameters on request basis instead
* of globally in application.conf
*
* This client defaults all its configuration with the one set in spray.conf
* see spray.can.client and spray.can.host-connector
* But you can override some of them on demand
* - maxRetries
* - requestTimeout
* - maxConnections
*/
class HttpClient(actorSystem: ActorSystem, config: Config) {
private implicit val context: ActorSystem = actorSystem
private implicit val dispatcher: ExecutionContextExecutor = actorSystem.dispatcher
private val HTTP = "http"
private val HTTPS = "https"
private val defaultSettings: HostConnectorSettings =
HostConnectorSettings.fromSubConfig(config.getConfig("spray.can"))
//not configurable since this timeout has little to no use practically
//this timeout DOES NOT kill the open connection
//http://kamon.io/teamblog/2014/11/02/understanding-spray-client-timeout-settings/
private implicit val clientFutureTimeout: akka.util.Timeout = 5.seconds
def send(
request: HttpRequest,
customSettings: Option[HttpCustomSettings] = None
): Future[HttpResponse] = {
val pipeline: Future[HttpRequest ⇒ Future[HttpResponse]] =
pipelineForUri(request.uri, customSettings)
pipeline.flatMap(send ⇒ send(request))
}
/**
* To understand more this method
* #see http://kamon.io/assets/img/diagrams/spray-client-actors.png
* #see [[spray.can.HttpManager]]
* #see [[spray.can.client.HttpHostConnector]]
* #see [[spray.can.Http]]
*/
private def pipelineForUri(
uri: Uri,
customSettings: Option[HttpCustomSettings]
): Future[HttpRequest ⇒ Future[HttpResponse]] = {
for {
Http.HostConnectorInfo(connector, _) ← IO(Http) ? connectorSetup(uri, customSettings)
} yield sendReceive(connector)
}
private def connectorSetup(
uri: Uri,
customSettings: Option[HttpCustomSettings]
): HostConnectorSetup = {
require(
uri.scheme == HTTP || uri.scheme == HTTPS,
s"Not a valid $HTTP URI scheme: '${uri.scheme}' in '$uri'. (Did you forget $HTTP:// ?)"
)
val connector: HostConnectorSetup = HostConnectorSetup(
uri.authority.host.toString,
uri.effectivePort,
sslEncryption = uri.scheme == HTTPS
)
customSettings match {
case Some(custom) ⇒ connector.copy(settings = Option(mapCustomSettings(defaultSettings, custom)))
case None ⇒ connector.copy(settings = Option(defaultSettings))
}
}
private def mapCustomSettings(
settings: HostConnectorSettings,
customSettings: HttpCustomSettings
): HostConnectorSettings = {
settings.copy(
maxRetries = customSettings.maxRetries,
maxConnections = customSettings.maxConnections,
connectionSettings = settings.connectionSettings.copy(requestTimeout = customSettings.requestTimeout)
)
}
}

Writing a test case for file uploads in Play 2.1 and Scala

I found the following question/answer:
Test MultipartFormData in Play 2.0 FakeRequest
But it seems things have changed in Play 2.1. I've tried adapting the example like so:
"Application" should {
"Upload Photo" in {
running(FakeApplication()) {
val data = new MultipartFormData(Map(), List(
FilePart("qqfile", "message", Some("Content-Type: multipart/form-data"),
TemporaryFile(getClass().getResource("/test/photos/DSC03024.JPG").getFile()))
), List())
val Some(result) = routeAndCall(FakeRequest(POST, "/admin/photo/upload", FakeHeaders(), data))
status(result) must equalTo(CREATED)
headers(result) must contain(LOCATION)
contentType(result) must beSome("application/json")
However whenever I attempt to run the request, I get a null-pointer exception:
[error] ! Upload Photo
[error] NullPointerException: null (PhotoManagementSpec.scala:25)
[error] test.PhotoManagementSpec$$anonfun$1$$anonfun$apply$3$$anonfun$apply$4.apply(PhotoManagementSpec.scala:28)
[error] test.PhotoManagementSpec$$anonfun$1$$anonfun$apply$3$$anonfun$apply$4.apply(PhotoManagementSpec.scala:25)
[error] play.api.test.Helpers$.running(Helpers.scala:40)
[error] test.PhotoManagementSpec$$anonfun$1$$anonfun$apply$3.apply(PhotoManagementSpec.scala:25)
[error] test.PhotoManagementSpec$$anonfun$1$$anonfun$apply$3.apply(PhotoManagementSpec.scala:25)
If I try to replace the deprecated routeAndCall with just route (and remove the Option around result), I get a compile error stating that it can't write an instance of MultipartFormData[TemporaryFile] to the HTTP response.
What's the right way to design this test in Play 2.1 with Scala?
Edit: Tried to modify the code to test just the controller:
"Application" should {
"Upload Photo" in {
val data = new MultipartFormData(Map(), List(
FilePart("qqfile", "message", Some("Content-Type: multipart/form-data"),
TemporaryFile(getClass().getResource("/test/photos/DSC03024.JPG").getFile()))
), List())
val result = controllers.Photo.upload()(FakeRequest(POST, "/admin/photo/upload",FakeHeaders(),data))
status(result) must equalTo(OK)
contentType(result) must beSome("text/html")
charset(result) must beSome("utf-8")
contentAsString(result) must contain("Hello Bob")
}
But I now get a type error on all the test conditions around the results like so:
[error] found : play.api.libs.iteratee.Iteratee[Array[Byte],play.api.mvc.Result]
[error] required: play.api.mvc.Result
I don't understand why I'm getting an Interator for byte arrays mapped to Results. Could this have something to do with how I'm using a custom body parser? My controller's definition looks like this:
def upload = Action(CustomParsers.multipartFormDataAsBytes) { request =>
request.body.file("qqfile").map { upload =>
Using the form parser from this post: Pulling files from MultipartFormData in memory in Play2 / Scala
Play 2.3 includes a newer version of httpmime.jar, requiring some minor corrections. Building on Marcus's solution using Play's Writeable mechanism, while retaining some of the syntactic sugar from my Play 2.1 solution, this is what I've come up with:
import scala.language.implicitConversions
import java.io.{ByteArrayOutputStream, File}
import org.apache.http.entity.ContentType
import org.apache.http.entity.mime.MultipartEntityBuilder
import org.apache.http.entity.mime.content._
import org.specs2.mutable.Specification
import play.api.http._
import play.api.libs.Files.TemporaryFile
import play.api.mvc.MultipartFormData.FilePart
import play.api.mvc.{Codec, MultipartFormData}
import play.api.test.Helpers._
import play.api.test.{FakeApplication, FakeRequest}
trait FakeMultipartUpload {
implicit def writeableOf_multiPartFormData(implicit codec: Codec): Writeable[MultipartFormData[TemporaryFile]] = {
val builder = MultipartEntityBuilder.create().setBoundary("12345678")
def transform(multipart: MultipartFormData[TemporaryFile]): Array[Byte] = {
multipart.dataParts.foreach { part =>
part._2.foreach { p2 =>
builder.addPart(part._1, new StringBody(p2, ContentType.create("text/plain", "UTF-8")))
}
}
multipart.files.foreach { file =>
val part = new FileBody(file.ref.file, ContentType.create(file.contentType.getOrElse("application/octet-stream")), file.filename)
builder.addPart(file.key, part)
}
val outputStream = new ByteArrayOutputStream
builder.build.writeTo(outputStream)
outputStream.toByteArray
}
new Writeable[MultipartFormData[TemporaryFile]](transform, Some(builder.build.getContentType.getValue))
}
/** shortcut for generating a MultipartFormData with one file part which more fields can be added to */
def fileUpload(key: String, file: File, contentType: String): MultipartFormData[TemporaryFile] = {
MultipartFormData(
dataParts = Map(),
files = Seq(FilePart[TemporaryFile](key, file.getName, Some(contentType), TemporaryFile(file))),
badParts = Seq(),
missingFileParts = Seq())
}
/** shortcut for a request body containing a single file attachment */
case class WrappedFakeRequest[A](fr: FakeRequest[A]) {
def withFileUpload(key: String, file: File, contentType: String) = {
fr.withBody(fileUpload(key, file, contentType))
}
}
implicit def toWrappedFakeRequest[A](fr: FakeRequest[A]) = WrappedFakeRequest(fr)
}
class MyTest extends Specification with FakeMultipartUpload {
"uploading" should {
"be easier than this" in {
running(FakeApplication()) {
val uploadFile = new File("/tmp/file.txt")
val req = FakeRequest(POST, "/upload/path").
withFileUpload("image", uploadFile, "image/gif")
val response = route(req).get
status(response) must equalTo(OK)
}
}
}
}
I managed to get this working with Play 2.1 based on various mailing list suggestions. Here's how I do it:
import scala.language.implicitConversions
import java.io.{ ByteArrayOutputStream, File }
import org.apache.http.entity.mime.MultipartEntity
import org.apache.http.entity.mime.content.{ ContentBody, FileBody }
import org.specs2.mutable.Specification
import play.api.http.Writeable
import play.api.test.{ FakeApplication, FakeRequest }
import play.api.test.Helpers._
trait FakeMultipartUpload {
case class WrappedFakeRequest[A](fr: FakeRequest[A]) {
def withMultipart(parts: (String, ContentBody)*) = {
// create a multipart form
val entity = new MultipartEntity()
parts.foreach { part =>
entity.addPart(part._1, part._2)
}
// serialize the form
val outputStream = new ByteArrayOutputStream
entity.writeTo(outputStream)
val bytes = outputStream.toByteArray
// inject the form into our request
val headerContentType = entity.getContentType.getValue
fr.withBody(bytes).withHeaders(CONTENT_TYPE -> headerContentType)
}
def withFileUpload(fileParam: String, file: File, contentType: String) = {
withMultipart(fileParam -> new FileBody(file, contentType))
}
}
implicit def toWrappedFakeRequest[A](fr: FakeRequest[A]) = WrappedFakeRequest(fr)
// override Play's equivalent Writeable so that the content-type header from the FakeRequest is used instead of application/octet-stream
implicit val wBytes: Writeable[Array[Byte]] = Writeable(identity, None)
}
class MyTest extends Specification with FakeMultipartUpload {
"uploading" should {
"be easier than this" in {
running(FakeApplication()) {
val uploadFile = new File("/tmp/file.txt")
val req = FakeRequest(POST, "/upload/path").
withFileUpload("image", uploadFile, "image/gif")
val response = route(req).get
status(response) must equalTo(OK)
}
}
}
}
I've modified Alex's code to act as a Writable which better integrates into Play 2.2.2
package test
import play.api.http._
import play.api.mvc.MultipartFormData.FilePart
import play.api.libs.iteratee._
import play.api.libs.Files.TemporaryFile
import play.api.mvc.{Codec, MultipartFormData }
import java.io.{FileInputStream, ByteArrayOutputStream}
import org.apache.commons.io.IOUtils
import org.apache.http.entity.mime.MultipartEntity
import org.apache.http.entity.mime.content._
object MultipartWriteable {
/**
* `Writeable` for multipart/form-data.
*
*/
implicit def writeableOf_multiPartFormData(implicit codec: Codec): Writeable[MultipartFormData[TemporaryFile]] = {
val entity = new MultipartEntity()
def transform(multipart: MultipartFormData[TemporaryFile]):Array[Byte] = {
multipart.dataParts.foreach { part =>
part._2.foreach { p2 =>
entity.addPart(part._1, new StringBody(p2))
}
}
multipart.files.foreach { file =>
val part = new FileBody(file.ref.file, file.filename, file.contentType.getOrElse("application/octet-stream"), null)
entity.addPart(file.key, part)
}
val outputStream = new ByteArrayOutputStream
entity.writeTo(outputStream)
val bytes = outputStream.toByteArray
outputStream.close
bytes
}
new Writeable[MultipartFormData[TemporaryFile]](transform, Some(entity.getContentType.getValue))
}
}
This way it is possible to write something like this:
val filePart:MultipartFormData.FilePart[TemporaryFile] = MultipartFormData.FilePart(...)
val fileParts:Seq[MultipartFormData.FilePart[TemporaryFile]] = Seq(filePart)
val dataParts:Map[String, Seq[String]] = ...
val multipart = new MultipartFormData[TemporaryFile](dataParts, fileParts, List(), List())
val request = FakeRequest(POST, "/url", FakeHeaders(), multipart)
var result = route(request).get
Following EEColor's suggestion, I got the following to work:
"Upload Photo" in {
val file = scala.io.Source.fromFile(getClass().getResource("/photos/DSC03024.JPG").getFile())(scala.io.Codec.ISO8859).map(_.toByte).toArray
val data = new MultipartFormData(Map(), List(
FilePart("qqfile", "DSC03024.JPG", Some("image/jpeg"),
file)
), List())
val result = controllers.Photo.upload()(FakeRequest(POST, "/admin/photos/upload",FakeHeaders(),data))
status(result) must equalTo(CREATED)
headers(result) must haveKeys(LOCATION)
contentType(result) must beSome("application/json")
}
Here's my version of Writeable[AnyContentAsMultipartFormData]:
import java.io.File
import play.api.http.{HeaderNames, Writeable}
import play.api.libs.Files.TemporaryFile
import play.api.mvc.MultipartFormData.FilePart
import play.api.mvc.{AnyContentAsMultipartFormData, Codec, MultipartFormData}
object MultipartFormDataWritable {
val boundary = "--------ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890"
def formatDataParts(data: Map[String, Seq[String]]) = {
val dataParts = data.flatMap { case (key, values) =>
values.map { value =>
val name = s""""$key""""
s"--$boundary\r\n${HeaderNames.CONTENT_DISPOSITION}: form-data; name=$name\r\n\r\n$value\r\n"
}
}.mkString("")
Codec.utf_8.encode(dataParts)
}
def filePartHeader(file: FilePart[TemporaryFile]) = {
val name = s""""${file.key}""""
val filename = s""""${file.filename}""""
val contentType = file.contentType.map { ct =>
s"${HeaderNames.CONTENT_TYPE}: $ct\r\n"
}.getOrElse("")
Codec.utf_8.encode(s"--$boundary\r\n${HeaderNames.CONTENT_DISPOSITION}: form-data; name=$name; filename=$filename\r\n$contentType\r\n")
}
val singleton = Writeable[MultipartFormData[TemporaryFile]](
transform = { form: MultipartFormData[TemporaryFile] =>
formatDataParts(form.dataParts) ++
form.files.flatMap { file =>
val fileBytes = Files.readAllBytes(Paths.get(file.ref.file.getAbsolutePath))
filePartHeader(file) ++ fileBytes ++ Codec.utf_8.encode("\r\n")
} ++
Codec.utf_8.encode(s"--$boundary--")
},
contentType = Some(s"multipart/form-data; boundary=$boundary")
)
}
implicit val anyContentAsMultipartFormWritable: Writeable[AnyContentAsMultipartFormData] = {
MultipartFormDataWritable.singleton.map(_.mdf)
}
It's adapted from (and some bugs fixed): https://github.com/jroper/playframework/blob/multpart-form-data-writeable/framework/src/play/src/main/scala/play/api/http/Writeable.scala#L108
See the whole post here, if you are interested: http://tech.fongmun.com/post/125479939452/test-multipartformdata-in-play
For me, the best solution for this problem is the Alex Varju one
Here is a version updated for Play 2.5:
object FakeMultipartUpload {
implicit def writeableOf_multiPartFormData(implicit codec: Codec): Writeable[AnyContentAsMultipartFormData] = {
val builder = MultipartEntityBuilder.create().setBoundary("12345678")
def transform(multipart: AnyContentAsMultipartFormData): ByteString = {
multipart.mdf.dataParts.foreach { part =>
part._2.foreach { p2 =>
builder.addPart(part._1, new StringBody(p2, ContentType.create("text/plain", "UTF-8")))
}
}
multipart.mdf.files.foreach { file =>
val part = new FileBody(file.ref.file, ContentType.create(file.contentType.getOrElse("application/octet-stream")), file.filename)
builder.addPart(file.key, part)
}
val outputStream = new ByteArrayOutputStream
builder.build.writeTo(outputStream)
ByteString(outputStream.toByteArray)
}
new Writeable(transform, Some(builder.build.getContentType.getValue))
}
}
In Play 2.6.x you can write test cases in the following way to test file upload API:
class HDFSControllerTest extends Specification {
"HDFSController" should {
"return 200 Status for file Upload" in new WithApplication {
val tempFile = SingletonTemporaryFileCreator.create("txt","csv")
tempFile.deleteOnExit()
val data = new MultipartFormData[TemporaryFile](Map(),
List(FilePart("metadata", "text1.csv", Some("text/plain"), tempFile)), List())
val res: Option[Future[Result]] = route(app, FakeRequest(POST, "/api/hdfs").withMultipartFormDataBody(data))
print(contentAsString(res.get))
res must beSome.which(status(_) == OK)
}
}
}
Made Alex's version compatible with Play 2.8
import akka.util.ByteString
import java.io.ByteArrayOutputStream
import org.apache.http.entity.mime.content.StringBody
import org.apache.http.entity.ContentType
import org.apache.http.entity.mime.content.FileBody
import org.apache.http.entity.mime.MultipartEntityBuilder
import play.api.http.Writeable
import play.api.libs.Files.TemporaryFile
import play.api.mvc.Codec
import play.api.mvc.MultipartFormData
import play.api.mvc.MultipartFormData.FilePart
import play.api.test.FakeRequest
trait FakeMultipartUpload {
implicit def writeableOf_multiPartFormData(
implicit codec: Codec
): Writeable[MultipartFormData[TemporaryFile]] = {
val builder = MultipartEntityBuilder.create().setBoundary("12345678")
def transform(multipart: MultipartFormData[TemporaryFile]): ByteString = {
multipart.dataParts.foreach { part =>
part._2.foreach { p2 =>
builder.addPart(part._1, new StringBody(p2, ContentType.create("text/plain", "UTF-8")))
}
}
multipart.files.foreach { file =>
val part = new FileBody(
file.ref.file,
ContentType.create(file.contentType.getOrElse("application/octet-stream")),
file.filename
)
builder.addPart(file.key, part)
}
val outputStream = new ByteArrayOutputStream
builder.build.writeTo(outputStream)
ByteString(outputStream.toByteArray)
}
new Writeable(transform, Some(builder.build.getContentType.getValue))
}
/** shortcut for generating a MultipartFormData with one file part which more fields can be added to */
def fileUpload(
key: String,
file: TemporaryFile,
contentType: String
): MultipartFormData[TemporaryFile] = {
MultipartFormData(
dataParts = Map(),
files = Seq(FilePart[TemporaryFile](key, file.file.getName, Some(contentType), file)),
badParts = Seq()
)
}
/** shortcut for a request body containing a single file attachment */
case class WrappedFakeRequest[A](fr: FakeRequest[A]) {
def withFileUpload(key: String, file: TemporaryFile, contentType: String) = {
fr.withBody(fileUpload(key, file, contentType))
}
}
implicit def toWrappedFakeRequest[A](fr: FakeRequest[A]) = WrappedFakeRequest(fr)
}