I can't figure out how to create cachedHostConnectionPool in akka-http using scala for sending https requests. queueRequest(HttpRequest(uri = "https://example.com") sends a request to http, cachedHostConnectionPool[Promise[HttpResponse]]("https://example.com") throws an error that : isn't expected character.
import scala.util.{ Failure, Success }
import scala.concurrent.{ Future, Promise }
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.stream.ActorMaterializer
import akka.stream.scaladsl._
import akka.stream.{ OverflowStrategy, QueueOfferResult }
implicit val system = ActorSystem()
import system.dispatcher // to get an implicit ExecutionContext into scope
implicit val materializer = ActorMaterializer()
val QueueSize = 10
// This idea came initially from this blog post:
// http://kazuhiro.github.io/scala/akka/akka-http/akka-streams/2016/01/31/connection-pooling-with-akka-http-and-source-queue.html
val poolClientFlow = Http().cachedHostConnectionPool[Promise[HttpResponse]]("example.com")
val queue =
Source.queue[(HttpRequest, Promise[HttpResponse])](QueueSize, OverflowStrategy.dropNew)
.via(poolClientFlow)
.toMat(Sink.foreach({
case ((Success(resp), p)) => p.success(resp)
case ((Failure(e), p)) => p.failure(e)
}))(Keep.left)
.run()
def queueRequest(request: HttpRequest): Future[HttpResponse] = {
val responsePromise = Promise[HttpResponse]()
queue.offer(request -> responsePromise).flatMap {
case QueueOfferResult.Enqueued => responsePromise.future
case QueueOfferResult.Dropped => Future.failed(new RuntimeException("Queue overflowed. Try again later."))
case QueueOfferResult.Failure(ex) => Future.failed(ex)
case QueueOfferResult.QueueClosed => Future.failed(new RuntimeException("Queue was closed (pool shut down) while running the request. Try again later."))
}
}
val responseFuture: Future[HttpResponse] = queueRequest(HttpRequest(uri = "/"))
It seems like scala version supports only plain host names while in java you can provide a protocol too (from their tests):
http.cachedHostConnectionPool("akka.io", materializer());
http.cachedHostConnectionPool("https://akka.io", materializer());
http.cachedHostConnectionPool("https://akka.io:8080", materializer());
Any known workarounds?
You have to use cachedHostConnectionPoolHttps instead:
val poolClientFlow = Http().cachedHostConnectionPoolHttps[Promise[HttpResponse]]("example.com")
Related
In akka, I want to put the elements in stream and return an object. I know the elements could be a source to run a graph. But how can I put the element and return an object on runtime?
import akka.actor.ActorSystem
import akka.stream.QueueOfferResult.{Dropped, Enqueued, Failure, QueueClosed}
import akka.stream.{ActorMaterializer, OverflowStrategy}
import akka.stream.scaladsl.{Keep, Sink, Source}
import scala.Array.range
import scala.util.Success
object StreamElement {
implicit val system = ActorSystem("StreamElement")
implicit val materializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
def main(args: Array[String]): Unit = {
val (queue, value) = Source
.queue[Int](10, OverflowStrategy.backpressure)
.map(x => {
x * x
})
.toMat(Sink.asPublisher(false))(Keep.both)
.run()
range(0, 10)
.map(x => {
queue.offer(x).onComplete {
case Success(Enqueued) => {
}
case Success(Dropped) => {}
case _ => {
println("others")
}
}
})
}
}
How can I get the value returned?
Actually, you want to return the int value for each element.
So you could create the flow, then connect to source and Sink for each time.
package tech.parasol.scala.akka
import akka.actor.ActorSystem
import akka.stream.QueueOfferResult.{Dropped, Enqueued, Failure, QueueClosed}
import akka.stream.{ActorMaterializer, OverflowStrategy}
import akka.stream.scaladsl.{Flow, Keep, Sink, Source}
import scala.Array.range
import scala.util.Success
object StreamElement {
implicit val system = ActorSystem("StreamElement")
implicit val materializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
val flow = Flow[Int]
.buffer(16, OverflowStrategy.backpressure)
.map(x => x * x)
def main(args: Array[String]): Unit = {
range(0, 10)
.map(x => {
Source.single(x).via(flow).runWith(Sink.head)
}.map( v => println("v ===> " + v)
))
}
}
It's unclear to me why the Scala collection isn't fed to the Stream as a Source in your sample code. Given that you've already composed a Stream with materialized values to be captured in a Source Queue and a publisher Sink, you could create a subscriber Source using Source.fromPublisher to collect the wanted values, as shown below:
import akka.actor.ActorSystem
import akka.stream.scaladsl._
import akka.stream._
implicit val system = ActorSystem("system")
implicit val materializer = ActorMaterializer() // Not needed for Akka 2.6+
val (queue, pub) = Source
.queue[Int](10, OverflowStrategy.backpressure)
.map(x => x * x)
.toMat(Sink.asPublisher(false))(Keep.both)
.run()
val fromQueue = Source(0 until 10).runForeach(queue.offer(_))
val source = Source.fromPublisher(pub)
source.runForeach(x => print(x + " "))
// Output:
// 0 1 4 9 16 25 36 49 64 81
I am trying to add support for serializing and deserializing LocalDateTime in akka-http. I'm a little confused by the following error:
Error:(42, 20) could not find implicit value for parameter um:
akka.http.scaladsl.unmarshalling.FromRequestUnmarshaller[SomeData]
entity(as[SomeData]) { evt => complete(StatusCodes.Created) }
Error:(42, 20) not enough arguments for method as: (implicit um:
akka.http.scaladsl.unmarshalling.FromRequestUnmarshaller[SomeData])akka.http.scaladsl.unmarshalling.FromRequestUnmarshaller[SomeData].
Unspecified value parameter um.
entity(as[SomeData]) { evt => complete(StatusCodes.Created) }
What an I missing in the following code? I pasted my code into the minimalist sample from the Akka-http documentation. I am running Akka-http 10.0.6.
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.http.scaladsl.server.Directives._
import akka.stream.ActorMaterializer
import scala.io.StdIn
import java.time.LocalDateTime
import java.time.format.DateTimeFormatter
import spray.json.{DefaultJsonProtocol, JsString, JsValue, JsonFormat}
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport
import akka.http.scaladsl.server.Route
final case class SomeData (dateTime: LocalDateTime)
trait JsonSupport extends SprayJsonSupport with DefaultJsonProtocol {
implicit val eventDataFormat: JsonFormat[SomeData] = jsonFormat1(SomeData)
implicit val localDateTimeFormat = new JsonFormat[LocalDateTime] {
private val iso_date_time = DateTimeFormatter.ISO_DATE_TIME
def write(x: LocalDateTime) = JsString(iso_date_time.format(x))
def read(value: JsValue) = value match {
case JsString(x) => LocalDateTime.parse(x, iso_date_time)
case x => throw new RuntimeException(s"Unexpected type ${x.getClass.getName} when trying to parse LocalDateTime")
}
}
// implicit val eventDataFormat: JsonFormat[SomeData] = jsonFormat1(SomeData) ** moved here **
}
object Main extends JsonSupport {
def main(args: Array[String]): Unit = {
implicit val system = ActorSystem("my-system")
implicit val materializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
val route:Route =
path("hello") {
post {
entity(as[SomeData]) { evt => complete(StatusCodes.Created) }
}
}
val bindingFuture = Http().bindAndHandle(route, "localhost", 8080)
println(s"Server online at http://localhost:8080/\nPress RETURN to stop...")
StdIn.readLine() // let it run until user presses return
bindingFuture
.flatMap(_.unbind()) // trigger unbinding from the port
.onComplete(_ => system.terminate()) // and shutdown when done
}
}
implicit val eventDataFormat: JsonFormat[SomeData] = jsonFormat1(SomeData)
↓
implicit val eventDataFormat = jsonFormat1(SomeData)
From the Scala Spray documentation it is not clear how to check if it is not able to bind to a particular port
implicit val system = ActorSystem("mediaiton")
implicit val timeout = Timeout(5, TimeUnit.SECONDS)
val service = system.actorOf(Props[IotRestNB], "mediaiton")
println(s"Going to start the REST NB at $host $port")
IO(Http) ! Http.Bind(service, interface = host, port = port)
Spend a day trying to figure out from different post
import java.util.concurrent.TimeUnit
import akka.actor.{ActorSystem, Props}
import akka.io.IO
import akka.util.Timeout
import nb.rest.IotRestNB
import spray.can.Http
implicit val system = ActorSystem("lwm2m-mediaiton")
implicit val timeout = Timeout(5, TimeUnit.SECONDS)
val service = system.actorOf(Props[IotClassNB], "lwm2m-mediaiton")
println(s"Going to start the REST NB at $host $port")
IO(Http).tell(Http.Bind(service, interface = host, port = port), sender = service)
Now the actor Class - IotClassNB
import java.util.concurrent.Executors
import akka.actor.Actor
import lwm2m.server.BootUp
import org.eclipse.leshan.core.request.ContentFormat
import spray.can.Http._
import spray.http.MediaTypes
import spray.routing.HttpService
import scala.concurrent.{ExecutionContext, Future}
class IotClassNBextends Actor with MediationRoute {
//mixin class
def actorRefFactory = context
def receive = handleConnection orElse runRoute(route)
def handleConnection: Receive = {
case b: Bound =>
println("***REST Server Started***")
Future.successful(b)
case failed: CommandFailed =>
println("***REST Server Could not be Started***")//this is what we want
Future.failed(new
RuntimeException("Binding failed"))
}
}
trait MediationRoute extends HttpService {
// Execution Context for blocking ops
val blockingExecutionContext = {
ExecutionContext.fromExecutor(Executors.newFixedThreadPool(10))
}
val route = {
pathPrefix("v1") {
pathPrefix("mediation") {
respondWithMediaType(MediaTypes.`application/json`) {
pathPrefix("get_connected_clients") {
pathEnd {
get {
complete(
// Future.apply {
get_registered_clients())
// }(blockingExecutionContext))
}
}.....
And here is how to test your Spary server via Spray Client
#Test def test_RestNB(): Unit = {
implicit val system = ActorSystem("test")
import system.dispatcher
val pipeline: HttpRequest => Future[HttpResponse] = sendReceive
implicit val timeout = Timeout(25, TimeUnit.SECONDS)
val server_url = s"http://${host}:${port}/xxx/"
val response: Future[HttpResponse] = pipeline(Get(server_url))
val result = Await.result(response, timeout.duration) //wait for timeout
// println(s"Await Result is ${result.entity.asString}")
response.onComplete {
case Success(result: HttpResponse) =>
logger.info("Result: " + result.entity.asString)
assert(result.entity.asString === xxxxx")
case Failure(error) =>
logger.error(error + "Couldn't get list of items")
case _ =>
assert(false)
}
I have this crude test example with akka-http client and server.
Server.scala:
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.Sink
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.HttpMethods._
import akka.http.scaladsl.model._
import scala.concurrent.Future
class Server extends Runnable {
def run() = {
implicit val system = ActorSystem("server")
implicit val materializer = ActorMaterializer()
val serverSource = Http().bind(interface = "localhost", port = 8200)
val requestHandler: HttpRequest => HttpResponse = {
case HttpRequest(GET, Uri.Path("/stream"), _, _, _) =>
HttpResponse(entity = HttpEntity(MediaTypes.`text/plain`, "test"))
}
val bindingFuture: Future[Http.ServerBinding] = serverSource.to(Sink.foreach { connection =>
connection handleWithSyncHandler requestHandler
}).run()
}
}
Client.scala:
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.{Uri, HttpRequest}
import akka.stream.ActorMaterializer
object Client extends App {
implicit val system = ActorSystem("client")
import system.dispatcher
new Thread(new Server).start()
implicit val materializer = ActorMaterializer()
val source = Uri("http://localhost:8200/stream")
val finished = Http().singleRequest(HttpRequest(uri = source)).flatMap { response =>
response.entity.dataBytes.runForeach { chunk =>
println(chunk.utf8String)
}
}
}
At the moment Server just replies with a single "test".
How do I change the HttpResponse in Server to send "test" as chunked (stream) in an endless loop every 1 second?
Found the answer.
Server.scala:
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.{Source, Sink}
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.HttpMethods._
import akka.http.scaladsl.model._
import scala.concurrent.Future
import scala.concurrent.duration._
class Server extends Runnable {
def run() = {
implicit val system = ActorSystem("server")
implicit val materializer = ActorMaterializer()
val serverSource = Http().bind(interface = "localhost", port = 8200)
val requestHandler: HttpRequest => HttpResponse = {
case HttpRequest(GET, Uri.Path("/stream"), _, _, _) =>
HttpResponse(entity = HttpEntity.Chunked(ContentTypes.`text/plain`, Source(0 seconds, 1 seconds, "test")))
}
val bindingFuture: Future[Http.ServerBinding] = serverSource.to(Sink.foreach { connection =>
connection handleWithSyncHandler requestHandler
}).run()
}
}
I'm trying to understand how to use the new akka.http library. I would like to send an http request to a server and read the whole response body as a single String in order to produce a Source[String,?].
Here is the best solution I was able to produce so far:
def get(
modelID: String,
pool: Flow[(HttpRequest,Int),(Try[HttpResponse],Int),Http.HostConnectionPool]
): Source[String,Unit] = {
val uri = reactionsURL(modelID)
val req = HttpRequest(uri = uri)
Source.single( (req,0) )
.via( pool )
.map {
case (Success(resp),_) =>
resp.entity.dataBytes.map( _.decodeString("utf-8") )
}.flatten(FlattenStrategy.concat)
.grouped( 1024 )
.map( _.mkString )
It seems to work well (except the missing error path), but it is a bit clunky for such simple tasks. Is there a smarter solution ? Can I avoid the grouped/mkString ?
You can use toStrict method of HttpResponse with timeout. It gathers whole answer as Future.
def toStrict(timeout: FiniteDuration)(implicit ec: ExecutionContext, fm: Materializer): Future[Strict] Returns a sharable and serializable
copy of this message with a strict entity.
Example:
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.{HttpResponse, HttpRequest}
import akka.stream.{Materializer, ActorMaterializer}
import akka.stream.scaladsl.{Sink, Flow, Source}
import scala.concurrent.{ExecutionContext, Future}
import scala.concurrent.duration._
import scala.util.{Try, Success}
object Main extends App {
implicit val system = ActorSystem()
import system.dispatcher
implicit val materializer = ActorMaterializer()
val host = "127.0.0.1"
lazy val pool = Http().newHostConnectionPool[Int](host, 9000)
FlowBuilder.get("/path", pool).to(Sink.foreach(_.foreach(println))).run()
}
object FlowBuilder {
def get(modelID: String, pool: Flow[(HttpRequest, Int), (Try[HttpResponse], Int), Http.HostConnectionPool])
(implicit ec: ExecutionContext, mat: Materializer): Source[Future[String], Unit] = {
val uri = modelID
val req = HttpRequest(uri = modelID)
Source.single((req, 0)).via(pool)
.map {
case (Success(resp), _) => resp.entity.toStrict(5 seconds).map(_.data.decodeString("UTF-8"))
}
}
}
You can use Unmarshall which will also work on other types e.g. json from spray-json. This also as strict returns Future[_].
Example:
authedReq.via(authServerReqResFlow).mapAsync(1) { case (tryRes, _) =>
tryRes match {
case Failure(exception) => Future.failed[Principal](exception)
case Success(response # HttpResponse(StatusCodes.OK,_,_,_)) =>
val userContext = Unmarshal(response).to[UserContextData]
userContext.map {
case UserContextData(UserInfo(_, userName, fullName, email, title), _, _) =>
Principal(userName, fullName, email, title)
}
case Success(response # HttpResponse(responseCode,_,entity,_)) =>
Unmarshal(entity).to[String].flatMap(msg => Future.failed(new AuthenticationFailure(s"$responseCode\n$msg")))
}
}