Scala and Akka HTTP: Processing form-data requests - scala

Suppose we have the following request:
curl --location --request POST 'localhost:8080/api' \
--header 'Content-Type: multipart/form-data' \
--form 'field1=value1' \
--form 'field2=value2'
The request handler below gets the whole entity, but I am struggling to see how I could instead get value1 and value2.
val requestHandler: Flow[HttpRequest, HttpResponse, _] = Flow[HttpRequest].mapAsync(1) {
case HttpRequest(HttpMethods.POST, Uri.Path("/api"), _, entity, _) =>
val entityTextFuture: Future[String] = entity.toStrict(3 seconds).map(_.data.utf8String)
entityTextFuture.flatMap { text =>
Future(HttpResponse(
StatusCodes.OK,
entity = text
))
}
}
Important: I have to use the Akka HTTP low-level server API, so I cannot use routes.
Many thanks for your time and help in advance!

If all you want are the string values in the form data, you just need to unmarshal to StrictForm, and then unmarshal each of the field values as strings.
Here's a proof of concept Ammonite script that responds to your curl request with value1 & value2:
import $ivy.`com.typesafe.akka::akka-actor:2.6.3`
import $ivy.`com.typesafe.akka::akka-stream:2.6.3`
import $ivy.`com.typesafe.akka::akka-http:10.1.11`
import scala.concurrent.Future
import akka.actor.ActorSystem
import akka.stream.scaladsl.Flow
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.http.scaladsl.unmarshalling.Unmarshal
import akka.http.scaladsl.common.StrictForm
implicit val system = ActorSystem()
implicit val ec = system.dispatcher
val requestHandler: Flow[HttpRequest, HttpResponse, _] = Flow[HttpRequest].mapAsync(1) {
case HttpRequest(HttpMethods.POST, Uri.Path("/api"), _, entity, _) =>
for {
strictForm <- Unmarshal(entity).to[StrictForm]
fieldsSeq <- Future.traverse(strictForm.fields) {
case (n, v) => Unmarshal(v).to[String].map(n -> _)
}
fields = fieldsSeq.toMap
response = fields("field1") + " & " + fields("field2")
} yield HttpResponse(StatusCodes.OK, entity = response)
}
Http().bindAndHandle(requestHandler, "localhost", 8080)

Related

Identify Akka HttpRequest and HttpResponse?

While using Akka HttpRequest and pipe the request to an actor, i couldn't identify the response.
The actor will handle each message that will receive but it doesn't know which request used to get this response. Is there any way to identify each request to match the response with ?
Note: i don't have the server to resend any part of request body again.
Thanks in advance
MySelf.scala
import akka.actor.{ Actor, ActorLogging }
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.stream.{ ActorMaterializer, ActorMaterializerSettings }
import akka.util.ByteString
class Myself extends Actor with ActorLogging {
import akka.pattern.pipe
import context.dispatcher
final implicit val materializer: ActorMaterializer =
ActorMaterializer(ActorMaterializerSettings(context.system))
def receive = {
case HttpResponse(StatusCodes.OK, headers, entity, _) =>
entity.dataBytes.runFold(ByteString(""))(_ ++ _).foreach { body =>
log.info("Got response, body: " + body.utf8String)
}
case resp # HttpResponse(code, _, _, _) =>
log.info("Request failed, response code: " + code)
resp.discardEntityBytes()
}
}
Main.scala
import akka.actor.{ActorSystem, Props}
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.stream.ActorMaterializer
object HttpServerMain extends App {
import akka.pattern.pipe
// import system.dispatcher
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
// needed for the future flatMap/onComplete in the end
implicit val executionContext = system.dispatcher
val http = Http(system)
val myActor = system.actorOf(Props[MySelf])
http.singleRequest(HttpRequest(uri = "http://akka.io"))
.pipeTo(myActor)
http.singleRequest(HttpRequest(uri = "http://akka.io/another-request"))
.pipeTo(myActor)
Thread.sleep(2000)
system.terminate()
You can simply use map to transform the Future and add some kind of ID (usually called correlation ID for such purposes) to it before you pipe it to myActor:
http.singleRequest(HttpRequest(uri = "http://akka.io"))
.map(x => (1, x)).pipeTo(myActor)
You'll need to change you pattern match blocks to take a tupple:
case (id, HttpResponse(StatusCodes.OK, headers, entity, _)) =>
If you can't/don't want to change your pattern match block for some reason you can use same approach, but instead add a unique HTTP header into your completed request (using copy) with something like this (not checked if compiles):
// make a unique header name that you are sure will not be
// received from http response:
val correlationHeader: HttpHeader = ... // mycustomheader
// Basically hack the response to add your header:
http.singleRequest(HttpRequest(uri = "http://akka.io"))
.map(x => x.copy(headers = correlationHeader +: headers)).pipeTo(myActor)
// Now you can check your header to see which response that was:
case HttpResponse(StatusCodes.OK, headers, entity, _) =>
headers.find(_.is("mycustomheader")).map(_.value).getOrElse("NA")
This is more of a hack though compared to previous option because you are modifying a response.
I think you cannot do that directly using pipeTo because it essentially just adds andThen call to your Future. One option is tomap and then send a (request, response) tuple to actor:
val request = HttpRequest(uri = "http://akka.io")
http.singleRequest(request).map {
response => myActor ! (request, response)
}
class Myself extends Actor with ActorLogging {
...
def receive = {
case (request, HttpResponse(StatusCodes.OK, headers, entity, _)) =>
...
case (request, resp # HttpResponse(code, _, _, _)) =>
log.info(request.toString)
...
}
}

Akka Http Client Set Cookie on a HttpRequest

I am trying to make a GET request to a REST web service using Akka Http Client.
I am not able to figure out how do I set a cookie on the request before I make the GET.
I searched the web and I found ways to read the cookie on the server side. but I could not find anything which showed me how to set the cookie on the client side request.
Based on my own research I tried the following approach to set a cookie on http request
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.http.scaladsl.unmarshalling.Unmarshal
import akka.stream.scaladsl.{Sink, Source}
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport
import akka.http.scaladsl.model.headers.HttpCookie
import akka.stream.ActorMaterializer
import spray.json._
import scala.util.{Failure, Success}
case class Post(postId: Int, id: Int, name: String, email: String, body: String)
trait JsonSupport extends SprayJsonSupport with DefaultJsonProtocol {
implicit val postFormat = jsonFormat5(Post.apply)
}
object AkkaHttpClient extends JsonSupport{
def main(args: Array[String]) : Unit = {
val cookie = headers.`Set-Cookie`(HttpCookie(name="foo", value="bar"))
implicit val system = ActorSystem("my-Actor")
implicit val actorMaterializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
val mycookie = HttpCookie(name="foo", value="bar")
val httpClient = Http().outgoingConnection(host = "jsonplaceholder.typicode.com")
val request = HttpRequest(uri = Uri("/comments"), headers = List(cookie))
val flow = Source.single(request)
.via(httpClient)
.mapAsync(1)(r => Unmarshal(r.entity).to[List[Post]])
.runWith(Sink.head)
flow.andThen {
case Success(list) => println(s"request succeded ${list.size}")
case Failure(_) => println("request failed")
}.andThen {
case _ => system.terminate()
}
}
}
But this gives an error
[WARN] [08/05/2016 10:50:11.134] [my-Actor-akka.actor.default-dispatcher-3] [akka.actor.ActorSystemImpl(my-Actor)]
HTTP header 'Set-Cookie: foo=bar' is not allowed in requests
The idiomatic way to construct any header for an akka-http client is by
using akka.http.scaladsl.model.headers.
In your case it would be
val cookieHeader = akka.http.scaladsl.model.headers.Cookie("name","value")
HttpRequest(uri = Uri("/comments"), headers = List(cookieHeader, ...))
The outgoing header must be 'Cookie' not 'Set-Cookie':
val cookie = HttpCookiePair("foo", "bar")
val headers: immutable.Seq[HttpHeader] = if (cookies.isEmpty) immutable.Seq.empty else immutable.Seq(Cookie(cookies))
val request = HttpRequest(uri = uri).withHeadersAndEntity(headers, HttpEntity(msg))

How to read json array in scala using the Play framework

I have the following Json as var dataObject ={"files": ["code.R", "data.cv", "input.txt"]}.
I am posting this json as a body from the client side and I want to parse the Json and read these files names in the server side in play scala.
Please help
Because you have only one field, you can't use json combinators,
But you can do as follow:
case class Selection(files:List[String])
object Selection{
implicit val selectionReads = (__ \ 'files).read[List[String]].map{ l => Selection(l) }
implicit val selectionWrites = (__ \ 'files).write[List[String]].contramap { (selection: Selection) => selection.files}
//You can replace the above 2 lines with this line - depends on you.
implicit val selectionFormat: Format[Selection] = (__ \ 'files).format[List[String]].inmap(files => Selection(files), (selection: Selection) => selection.files)
}
Make sure you import:
import play.api.libs.functional.syntax._
This is the documentation: https://www.playframework.com/documentation/2.5.x/ScalaJson
And the solution is simple:
import play.api.libs.json._
val json: JsValue = Json.parse("{ "files": ["code.R","data.csv","input.txt"] }")
val files = (json \ "files").get

akka HttpResponse read body as String scala

So I have a function with this signature (akka.http.model.HttpResponse):
def apply(query: Seq[(String, String)], accept: String): HttpResponse
I simply get a value in a test like:
val resp = TagAPI(Seq.empty[(String, String)], api.acceptHeader)
I want to check its body in a test something like:
resp.entity.asString == "tags"
My question is how I can get the response body as string?
import akka.http.scaladsl.unmarshalling.Unmarshal
implicit val system = ActorSystem("System")
implicit val materializer = ActorFlowMaterializer()
val responseAsString: Future[String] = Unmarshal(entity).to[String]
Since Akka Http is streams based, the entity is streaming as well. If you really need the entire string at once, you can convert the incoming request into a Strict one:
This is done by using the toStrict(timeout: FiniteDuration)(mat: Materializer) API to collect the request into a strict entity within a given time limit (this is important since you don't want to "try to collect the entity forever" in case the incoming request does actually never end):
import akka.stream.ActorFlowMaterializer
import akka.actor.ActorSystem
implicit val system = ActorSystem("Sys") // your actor system, only 1 per app
implicit val materializer = ActorFlowMaterializer() // you must provide a materializer
import system.dispatcher
import scala.concurrent.duration._
val timeout = 300.millis
val bs: Future[ByteString] = entity.toStrict(timeout).map { _.data }
val s: Future[String] = bs.map(_.utf8String) // if you indeed need a `String`
You can also try this one also.
responseObject.entity.dataBytes.runFold(ByteString(""))(_ ++ _).map(_.utf8String) map println
Unmarshaller.stringUnmarshaller(someHttpEntity)
works like a charm, implicit materializer needed as well
Here is simple directive that extracts string from request's body
def withString(): Directive1[String] = {
extractStrictEntity(3.seconds).flatMap { entity =>
provide(entity.data.utf8String)
}
}
Unfortunately in my case, Unmarshal to String didn't work properly complaining on: Unsupported Content-Type, supported: application/json. That would be more elegant solution, but I had to use another way. In my test I used Future extracted from entity of the response and Await (from scala.concurrent) to get the result from the Future:
Put("/post/item", requestEntity) ~> route ~> check {
val responseContent: Future[Option[String]] =
response.entity.dataBytes.map(_.utf8String).runWith(Sink.lastOption)
val content: Option[String] = Await.result(responseContent, 10.seconds)
content.get should be(errorMessage)
response.status should be(StatusCodes.InternalServerError)
}
If you need to go through all lines in a response, you can use runForeach of Source:
response.entity.dataBytes.map(_.utf8String).runForeach(data => println(data))
Here is my working example,
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.stream.ActorMaterializer
import akka.util.ByteString
import scala.concurrent.Future
import scala.util.{ Failure, Success }
def getDataAkkaHTTP:Unit = {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
// needed for the future flatMap/onComplete in the end
implicit val executionContext = system.dispatcher
val url = "http://localhost:8080/"
val responseFuture: Future[HttpResponse] = Http().singleRequest(HttpRequest(uri = url))
responseFuture.onComplete {
case Success(res) => {
val HttpResponse(statusCodes, headers, entity, _) = res
println(entity)
entity.dataBytes.runFold(ByteString(""))(_ ++ _).foreach (body => println(body.utf8String))
system.terminate()
}
case Failure(_) => sys.error("something wrong")
}
}

How do I set a non-standard User-Agent using spray-client?

I'm building an application for a Telco, using Scala and Akka, and need to communicate with Account Information and Refill servers using the UCIP protocol.
UCIP is a simple protocol, built on XMLRPC; the only issue I'm having is that it requires clients to set the User-Agent header in the specific format User-Agent: <client name>/<protocol version>/<client version>, which spray parses as invalid.
I tried creating a custom User-Agent header, inheriting from spray.http.HttpHeader but it still doesn't work. Here's what I've got so far:
import akka.actor.ActorSystem
import akka.event.{Logging, LoggingAdapter}
import spray.client.pipelining._
import spray.http._
import spray.httpx._
case class `User-Agent`(value: String) extends HttpHeader {
def lowercaseName: String = "user-agent"
def name: String = "User-Agent"
def render[R <: Rendering](r: R): r.type = r ~~ s"User-Agent: $value"
}
class UcipClient(val url: String, val protocol: String, username: String, password: String) (implicit system: ActorSystem) {
val log = Logging.getLogger(system, this)
val logRequest: HttpRequest => HttpRequest = { r => log.debug(r.toString); r }
val logResponse: HttpResponse => HttpResponse = { r => log.debug(r.toString); r }
val pipeline = (
addHeader(`User-Agent`("USSD-UCIP/%s/1.0".format(protocol)))
~> addCredentials(BasicHttpCredentials(username, password))
~> logRequest
~> sendReceive
~> logResponse
)
def send(req: UcipRequest) = pipeline(Post(url, req.getRequest))
}
My requests keep returning "Sorry, Error occured: 403, Invalid protocol version Not defined", however, they return the correct response when I send the same details using curl.
What am I missing, and is this even possible with spray-client? I've spent a fair bit of time checking the internets (which led me towards the custom header route), but still haven't figured this out...would really appreciate any help :-)
Turns out I wasn't far from the answer. While examining the headers being sent over the wire, I noticed the User-Agent was being set twice: once by my code, and again by Spray (because it considered my header invalid).
Setting the spray.can.client.user-agent-header to the empty string "" removed the second header, and requests were successful. Here's the final version of the custom header:
import spray.http._
object CustomHttpHeaders {
case class `User-Agent`(val value: String) extends HttpHeader with Product with Serializable {
def lowercaseName: String = "user-agent"
def name: String = "User-Agent"
def render[R <: Rendering](r: R): r.type = r ~~ s"User-Agent: $value"
}
}
And the final UCIP client:
import akka.actor.ActorRefFactory
import com.typesafe.config.Config
import scala.concurrent.ExecutionContext.Implicits.global
import scala.xml.NodeSeq
import spray.client.pipelining._
import spray.http._
import spray.httpx._
class UcipFault(val code: Int, msg: String) extends RuntimeException(s"$code: $msg")
class AirException(val code: Int) extends RuntimeException(s"$code")
class UcipClient(config: Config, val url: String)(implicit context: ActorRefFactory) {
import CustomHttpHeaders._
val throwOnFailure: NodeSeq => NodeSeq = {
case f if (f \\ "fault").size != 0 =>
val faultData = (f \\ "fault" \\ "member" \ "value")
throw new UcipFault((faultData \\ "i4").text.toInt,
(faultData \\ "string").text)
case el =>
val responseCode = ((el \\ "member")
.filter { n => (n \\ "name").text == "responseCode" }
.map { n => (n \\ "i4").text.toInt }).head
if (responseCode == 0) el else throw new AirException(responseCode)
}
val pipeline = (
addHeader(`User-Agent`("USSD-UCIP/%s/1.0".format(config.getString("ucip.server-protocol"))))
~> addCredentials(BasicHttpCredentials(config.getString("ucip.server-username"), config.getString("ucip.server-password")))
~> sendReceive
~> unmarshal[NodeSeq]
~> throwOnFailure
)
def send(req: UcipRequest) = pipeline(Post(url, req.getRequest))
}