I am trying to make a GET request to a REST web service using Akka Http Client.
I am not able to figure out how do I set a cookie on the request before I make the GET.
I searched the web and I found ways to read the cookie on the server side. but I could not find anything which showed me how to set the cookie on the client side request.
Based on my own research I tried the following approach to set a cookie on http request
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.http.scaladsl.unmarshalling.Unmarshal
import akka.stream.scaladsl.{Sink, Source}
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport
import akka.http.scaladsl.model.headers.HttpCookie
import akka.stream.ActorMaterializer
import spray.json._
import scala.util.{Failure, Success}
case class Post(postId: Int, id: Int, name: String, email: String, body: String)
trait JsonSupport extends SprayJsonSupport with DefaultJsonProtocol {
implicit val postFormat = jsonFormat5(Post.apply)
}
object AkkaHttpClient extends JsonSupport{
def main(args: Array[String]) : Unit = {
val cookie = headers.`Set-Cookie`(HttpCookie(name="foo", value="bar"))
implicit val system = ActorSystem("my-Actor")
implicit val actorMaterializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
val mycookie = HttpCookie(name="foo", value="bar")
val httpClient = Http().outgoingConnection(host = "jsonplaceholder.typicode.com")
val request = HttpRequest(uri = Uri("/comments"), headers = List(cookie))
val flow = Source.single(request)
.via(httpClient)
.mapAsync(1)(r => Unmarshal(r.entity).to[List[Post]])
.runWith(Sink.head)
flow.andThen {
case Success(list) => println(s"request succeded ${list.size}")
case Failure(_) => println("request failed")
}.andThen {
case _ => system.terminate()
}
}
}
But this gives an error
[WARN] [08/05/2016 10:50:11.134] [my-Actor-akka.actor.default-dispatcher-3] [akka.actor.ActorSystemImpl(my-Actor)]
HTTP header 'Set-Cookie: foo=bar' is not allowed in requests
The idiomatic way to construct any header for an akka-http client is by
using akka.http.scaladsl.model.headers.
In your case it would be
val cookieHeader = akka.http.scaladsl.model.headers.Cookie("name","value")
HttpRequest(uri = Uri("/comments"), headers = List(cookieHeader, ...))
The outgoing header must be 'Cookie' not 'Set-Cookie':
val cookie = HttpCookiePair("foo", "bar")
val headers: immutable.Seq[HttpHeader] = if (cookies.isEmpty) immutable.Seq.empty else immutable.Seq(Cookie(cookies))
val request = HttpRequest(uri = uri).withHeadersAndEntity(headers, HttpEntity(msg))
Related
I am rewriting some application layer code in scala from using scalaj to akka-http
in order to reduce the number of third party dependencies in the project (we already use akka for other things in the same project.) The code simply wraps common types of request to an underlying general request provided by the library
Mostly it has been fine, but I am stuck on the problem of optionally adding a proxy to a request.
Requests should either be direct to the destination or via a proxy, determined by a parameter at runtime.
In my scalaj implementation, I have the following helper class and methods
object HttpUtils {
private def request(
host: Host,
method: HttpMethod,
params: Map[String, String],
postData: Option[String],
timeout: Duration,
headers: Seq[(String, String)],
proxy: Option[ProxyConfig]
): HttpResponse[String] = {
// most general request builder. Other methods in the object fill in parameters and wrap this in a Future
val baseRequest = Http(host.url)
val proxiedRequest = addProxy(proxy, baseRequest)
val fullRequest = addPostData(postData)(proxiedRequest)
.method(method.toString)
.params(params)
.headers(headers)
.option(HttpOptions.connTimeout(timeout.toMillis.toInt))
.option(HttpOptions.readTimeout(timeout.toMillis.toInt))
fullRequest.asString // scalaj for send off request and block until response
}
// Other methods ...
private def addProxy(proxy: Option[ProxyConfig], request: HttpRequest): HttpRequest =
proxy.fold(request)((p: ProxyConfig) => request.proxy(p.host, p.port))
}
case class ProxyConfig(host: String, port: Int)
Is there a way to build a similar construct with akka-http?
Akka HTTP does have proxy support that, as of version 10.0.9, is still unstable. Keeping in mind that the API could change, you could do something like the following to handle optional proxy settings:
import java.net.InetSocketAddress
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.http.scaladsl.{ClientTransport, Http}
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
case class ProxyConfig(host: String, port: Int)
val proxyConfig = Option(ProxyConfig("localhost", 8888))
val clientTransport =
proxyConfig.map(p => ClientTransport.httpsProxy(InetSocketAddress.createUnresolved(p.host, p.port)))
.getOrElse(ClientTransport.TCP)
val settings = ConnectionPoolSettings(system).withTransport(clientTransport)
Http().singleRequest(HttpRequest(uri = "https://google.com"), settings = settings)
In Akka Http 10.2.0, use bindflow for a Flow[HttpRequest, HttpResponse, NotUsed] defined by a RunnableGraph with Flowshape. Insided the RunnableGraph, an Http() outgoingConnection is used to connect to the remote proxy. Some example code:
import akka.actor.typed.ActorSystem
import akka.actor.typed.scaladsl.Behaviors
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.{HttpRequest, HttpResponse}
import akka.stream._
import akka.stream.scaladsl.{Broadcast, Flow, GraphDSL, Merge}
import scala.concurrent.ExecutionContextExecutor
import scala.concurrent.duration.DurationInt
import scala.io.StdIn
import scala.util.{Failure, Success}
object Main {
def main(args: Array[String]) {
implicit val system: ActorSystem[Nothing] = ActorSystem(Behaviors.empty, "testproxy")
implicit val executionContext: ExecutionContextExecutor = system.executionContext
system.log.info("TestAkkaHttpProxy Main started...")
val remoteHost = "xxx.xxx.xxx.x"
val remotePort = 8000
val proxyHost = "0.0.0.0"
val proxyPort = 8080
val gateway = Flow.fromGraph(GraphDSL.create() { implicit b =>
import GraphDSL.Implicits._
// Broadcast for flow input
val broadcaster = b.add(Broadcast[HttpRequest](1))
// Merge for flow output
val responseMerge = b.add(Merge[HttpResponse](1))
// outgoing client for remote proxy
val remote = Http().outgoingConnection(remoteHost, remotePort)
// filter out header that creates Akka Http warning
val requestConvert = Flow[HttpRequest]
.map(req => { req.mapHeaders(headers => headers.filter(h => h.isNot("timeout-access")))
})
// connect graph
broadcaster.out(0) ~> requestConvert ~> remote ~> responseMerge
// expose ports
FlowShape(broadcaster.in, responseMerge.out)
})
// Akka Http server that binds to Flow (for remote proxy)
Http().newServerAt(proxyHost, proxyPort).bindFlow(gateway)
.onComplete({
case Success(binding) ⇒
println(s"Server is listening on 0.0.0.0:8080")
binding.addToCoordinatedShutdown(hardTerminationDeadline = 10.seconds)
case Failure(e) ⇒
println(s"Binding failed with ${e.getMessage}")
system.terminate()
})
system.log.info("Press RETURN to stop...")
StdIn.readLine()
system.terminate()
}
}
I'm trying to build a REST server using this tutorial:
https://spindance.com/reactive-rest-services-akka-http/
However, having reached the "Responding with JSON" section, I've noticed that my code doesn't compile, and I can't make POST requests. This is the error that I'm getting:
Error:(59, 18) could not find implicit value for parameter um: akka.http.scaladsl.unmarshalling.FromRequestUnmarshaller[Health]
entity(as[Health]) { statusReport =>
^
Having looked at other tutorials, I've found out that you need to include an object containing an implicit variable for the class that I'm trying to unmarshall. I did that, and I even imported the httpx library, but I'm still getting this error. My code is given below.
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.StatusCodes
import akka.http.scaladsl.server.Directives._
import akka.http.scaladsl.server.Route
import akka.stream.ActorMaterializer
import akka.pattern.ask
import akka.util.Timeout
import spray.json._
import DefaultJsonProtocol._
import spray.httpx.SprayJsonSupport.sprayJsonUnmarshaller
import scala.concurrent.duration._
import scala.io.StdIn
object JsonImplicits extends DefaultJsonProtocol {
implicit val healthFormat = jsonFormat2(Health)
}
object MyApplication {
val host = "localhost"
val port = 8080
def main(args: Array[String]): Unit = {
implicit val system = ActorSystem("simple-rest-system")
// Something to do with flows
implicit val materializer = ActorMaterializer()
// A reference to a specific thread pool
// You can configure thread pool options through it
// It is the engine that executes the actors
implicit val executionContext = system.dispatcher
val requestHandler = system.actorOf(RequestHandler.props(), "requestHandler")
//Define the route
val route : Route = {
implicit val timeout = Timeout(20 seconds)
import JsonImplicits._
import spray.httpx.SprayJsonSupport._
path("health") {
get {
onSuccess(requestHandler ? GetHealthRequest) {
case response: HealthResponse =>
complete(StatusCodes.OK, s"Everything is ${response.health.status}!")
case _ =>
complete(StatusCodes.InternalServerError)
}
}
} ~ post {
// Entity extracts the body of the POST request and then converts it into a
// Health object
entity(as[Health]) { statusReport =>
onSuccess(requestHandler ? SetStatusRequest(statusReport)) {
case response: HealthResponse =>
complete(StatusCodes.OK,s"Posted health as ${response.health.status}!")
case _ =>
complete(StatusCodes.InternalServerError)
}
}
}
}
//Start up and listen for requests
val bindingFuture = Http().bindAndHandle(route, host, port)
println(s"Waiting for requests at http://$host:$port/...\nHit RETURN to terminate")
StdIn.readLine()
//Shutdown
bindingFuture.flatMap(_.unbind())
system.terminate()
}
}
I'm trying to use akka-http in order to make http requests to a single host (e.g. "akka.io"). The problem is that the created flow (Http().cachedHostConnectionPool) starts emitting responses only after N http requests are made, where N is equal to max-connections.
import scala.util.Failure
import scala.util.Success
import com.typesafe.config.ConfigFactory
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.HttpRequest
import akka.http.scaladsl.model.Uri.apply
import akka.http.scaladsl.settings.ConnectionPoolSettings
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.Sink
import akka.stream.scaladsl.Source
object ConnectionPoolExample extends App {
implicit val system = ActorSystem()
implicit val executor = system.dispatcher
implicit val materializer = ActorMaterializer()
val config = ConfigFactory.load()
val connectionPoolSettings = ConnectionPoolSettings(config).withMaxConnections(10)
lazy val poolClientFlow = Http().cachedHostConnectionPool[Unit]("akka.io", 80, connectionPoolSettings)
val fakeSource = Source.fromIterator[Unit] { () => Iterator.continually { Thread.sleep(1000); () } }
val requests = fakeSource.map { _ => println("Creating request"); HttpRequest(uri = "/") -> (()) }
val responses = requests.via(poolClientFlow)
responses.runForeach {
case (tryResponse, jsonData) =>
tryResponse match {
case Success(httpResponse) =>
httpResponse.entity.dataBytes.runWith(Sink.ignore)
println(s"status: ${httpResponse.status}")
case Failure(e) => {
println(e)
}
}
}
}
The output looks like this:
Creating request
Creating request
Creating request
Creating request
Creating request
Creating request
Creating request
Creating request
Creating request
Creating request
status: 200 OK
Creating request
status: 200 OK
Creating request
status: 200 OK
...
I am failing to find any configuration parameters which would allow emitting responses as soon as they are ready and not when the pool is out of free connections.
Thanks!
The reason is that you block the client from doing other work by calling Thread.sleep—that method is simply forbidden inside reactive programs. The proper and simpler approach is to use Source.tick.
So I have a function with this signature (akka.http.model.HttpResponse):
def apply(query: Seq[(String, String)], accept: String): HttpResponse
I simply get a value in a test like:
val resp = TagAPI(Seq.empty[(String, String)], api.acceptHeader)
I want to check its body in a test something like:
resp.entity.asString == "tags"
My question is how I can get the response body as string?
import akka.http.scaladsl.unmarshalling.Unmarshal
implicit val system = ActorSystem("System")
implicit val materializer = ActorFlowMaterializer()
val responseAsString: Future[String] = Unmarshal(entity).to[String]
Since Akka Http is streams based, the entity is streaming as well. If you really need the entire string at once, you can convert the incoming request into a Strict one:
This is done by using the toStrict(timeout: FiniteDuration)(mat: Materializer) API to collect the request into a strict entity within a given time limit (this is important since you don't want to "try to collect the entity forever" in case the incoming request does actually never end):
import akka.stream.ActorFlowMaterializer
import akka.actor.ActorSystem
implicit val system = ActorSystem("Sys") // your actor system, only 1 per app
implicit val materializer = ActorFlowMaterializer() // you must provide a materializer
import system.dispatcher
import scala.concurrent.duration._
val timeout = 300.millis
val bs: Future[ByteString] = entity.toStrict(timeout).map { _.data }
val s: Future[String] = bs.map(_.utf8String) // if you indeed need a `String`
You can also try this one also.
responseObject.entity.dataBytes.runFold(ByteString(""))(_ ++ _).map(_.utf8String) map println
Unmarshaller.stringUnmarshaller(someHttpEntity)
works like a charm, implicit materializer needed as well
Here is simple directive that extracts string from request's body
def withString(): Directive1[String] = {
extractStrictEntity(3.seconds).flatMap { entity =>
provide(entity.data.utf8String)
}
}
Unfortunately in my case, Unmarshal to String didn't work properly complaining on: Unsupported Content-Type, supported: application/json. That would be more elegant solution, but I had to use another way. In my test I used Future extracted from entity of the response and Await (from scala.concurrent) to get the result from the Future:
Put("/post/item", requestEntity) ~> route ~> check {
val responseContent: Future[Option[String]] =
response.entity.dataBytes.map(_.utf8String).runWith(Sink.lastOption)
val content: Option[String] = Await.result(responseContent, 10.seconds)
content.get should be(errorMessage)
response.status should be(StatusCodes.InternalServerError)
}
If you need to go through all lines in a response, you can use runForeach of Source:
response.entity.dataBytes.map(_.utf8String).runForeach(data => println(data))
Here is my working example,
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.stream.ActorMaterializer
import akka.util.ByteString
import scala.concurrent.Future
import scala.util.{ Failure, Success }
def getDataAkkaHTTP:Unit = {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
// needed for the future flatMap/onComplete in the end
implicit val executionContext = system.dispatcher
val url = "http://localhost:8080/"
val responseFuture: Future[HttpResponse] = Http().singleRequest(HttpRequest(uri = url))
responseFuture.onComplete {
case Success(res) => {
val HttpResponse(statusCodes, headers, entity, _) = res
println(entity)
entity.dataBytes.runFold(ByteString(""))(_ ++ _).foreach (body => println(body.utf8String))
system.terminate()
}
case Failure(_) => sys.error("something wrong")
}
}
I’m getting started with a Finagle server (twitter/finagle):
import com.twitter.finagle.{Http, Service}
import com.twitter.util.{Await, Future}
import java.net.InetSocketAddress
import org.jboss.netty.handler.codec.http._
object Server extends App {
val service = new Service[HttpRequest, HttpResponse] {
def apply(req: HttpRequest): Future[HttpResponse] =
Future.value(new DefaultHttpResponse(
req.getProtocolVersion, HttpResponseStatus.OK))
}
val server = Http.serve(":8080", service)
Await.ready(server)
}
Client (twitter/finagle):
import com.twitter.finagle.{Http, Service}
import com.twitter.util.{Await, Future}
import java.net.InetSocketAddress
import org.jboss.netty.handler.codec.http._
object Client extends App {
val client: Service[HttpRequest, HttpResponse] =
Http.newService("localhost:8080")
val request = new DefaultHttpRequest(
HttpVersion.HTTP_1_1, HttpMethod.GET, "/")
val response: Future[HttpResponse] = client(request)
response onSuccess { resp: HttpResponse =>
println("GET success: " + resp)
}
Await.ready(response)
}
How do I send data like Map("data_id" -> 5) from the client to the server? And where in the server do I receive it? Do I have to add a callback to the server?
I haven’t found it by searching. If you can give me a link with an example, that will be enough.
Finagle is a very thin library. That means that you'll have to handle most of the "magic" by yourself.
To make the request with parameters from the Client, I use these helper methods:
def buildUri(base: String, path: String, params: Map[String, String] = Map.empty): String = {
val p = if (params.isEmpty) ""
else params map { case (k,v) => urlEncode(k) + "=" + urlEncode(v) } mkString ("?", "&", "")
base + path + p
}
def urlEncode(url: String): String = URLEncoder.encode(url, "UTF-8")
And then I call it like this:
val url = buildUri(baseAddress, path, defaultParams ++ params)
val req = RequestBuilder().url(url).setHeader("Accept", "*/*").buildGet
client(req)
As for the server you have to do basically the same thing and parse the parameters by hand. Either using java.net.URI or even org.jboss.netty.handler.codec.http.QueryStringDecoder.
Of course you can also use URI and QueryStringEncoder to encode as well, instead of using my helper methods.
That said, if you want to do that on higher level, you can use one of these libraries above Finagle:
https://github.com/fwbrasil/zoot
http://finatra.info/ (this is for the server part only)