I got one API which calls two another Downstream APIs. One downstream api (https://test/foo) is really important and it is very fast. Another slow downstream api (https://test/bar) has its limitation, the throughput of it only can handle 50 requests per sec.
I would like to make sure the downstream api https://test/foo has more priority than https://test/bar. For example, if the API thread pool is 75, I only allow 50 parallel incoming connection to go through https://test/bar. Rest of the connections should be used for https://test/bar. It would make https://test/bar never fails.
I guess I should apply throttle or maybe buffer with OverflowStrategy.dropNew for https://test/bar.
Here is the code snippet.
implicit val actorSystem = ActorSystem("api")
implicit val flowMaterializer = ActorMaterializer()
val httpService = Http()
val serverSource: Source[Http.IncomingConnection, Future[Http.ServerBinding]] =
httpService.bind(interface = "0.0.0.0", 3000)
val binding: Future[Http.ServerBinding] =
serverSource
.to(Sink.foreach { connection =>
connection.handleWith(
Flow[HttpRequest]
.map {
case HttpRequest(GET, Uri.Path("/priority-1"), _, _, _) =>
HttpResponse(entity = scala.io.Source.fromURL("https://test/foo").mkString)
case HttpRequest(GET, Uri.Path("/priority-2"), _, _, _) =>
HttpResponse(entity = scala.io.Source.fromURL("https://test/bar").mkString)
}
)
}).run()
Question 1: where should I put throttle(50, 1 seconds, 5000, ThrottleMode.Shaping) to conform to only https://test/bar threshold.
Question 2: do I need to apply buffer and OverflowStrategy.dropNew if I want to prioritise https://test/foo requests. In another words, all unnecessary connections for https://test/bar should be removed.
Question 3: Is there a better way to implement this requirement. I am using connection.handleWith[Flow[HttpRequest, HttpResponse]] in Sink and I am not sure this is right place.
If there are some code snippet provided, that would be much appreciated and super awesome :)
Thanks in advance
Related
I want to implement a high-throughput server that accepts multiple clients. Every request should query a database, so I need some kind of async behavior.
I followed the ROUTER-to-REQ pattern from documentation + Futures, so I ended with this "architecture":
trait ZmqProtocol extends Protocol {
private val pool = Executors.newCachedThreadPool()
private implicit val ec: ExecutionContextExecutor = ExecutionContext.fromExecutor(pool)
val context: ZMQ.Context = ZMQ.context(1)
val socket: ZMQ.Socket = context.socket(ZMQ.ROUTER)
socket.bind("tcp://*:5555")
override def receiveMessages(): String = {
while (true) {
val address = socket.recv(0)
val empty = socket.recv(0)
val request = socket.recv(0)
Future {
val message = new String(request)
getResponseFromDb(message)
} onComplete {
case Success(response) =>
// Send reply back to client
socket.send(address, ZMQ.SNDMORE)
socket.send("".getBytes, ZMQ.SNDMORE)
socket.send(response.getBytes(), 0)
case Failure(ex) => println(ex)
}
}
"DONE"
}
}
I understand this won't work because I'm sharing socket in Future so I need a better model. I know the ZeroMQ sockets are fast and creating several worker threads would be enough on input side, but if the bottleneck is on the database side and if I need to do some other work while waiting for DB, I presume all my threads would soon be exhausted.
Would it be too much of an overhead if I create new socket and bind on ROUTER in every Future or is there some better solution?
Also, for Scala developers: is there a way to force onComplete being executed on main thread (I suppose it would solve the issue)? Thanks!
Using akka stream and akka HTTP, I have created a stream which polls an api every 3 seconds, Unmarshalls the result to a JsValue object and sends this result to an actor. As can be seen in the following code:
// Source wich performs an http request every 3 seconds.
val source = Source.tick(0.seconds,
3.seconds,
HttpRequest(uri = Uri(path = Path("/posts/1"))))
// Processes the result of the http request
val flow = Http().outgoingConnectionHttps("jsonplaceholder.typicode.com").mapAsync(1) {
// Able to reach the API.
case HttpResponse(StatusCodes.OK, _, entity, _) =>
// Unmarshal the json response.
Unmarshal(entity).to[JsValue]
// Failed to reach the API.
case HttpResponse(code, _, entity, _) =>
entity.discardBytes()
Future.successful(code.toString())
}
// Run stream
source.via(flow).runWith(Sink.actorRef[Any](processJsonActor,akka.actor.Status.Success(("Completed stream"))))
This works, however the stream closes after 100 HttpRequests (ticks).
What is the cause of this behaviour?
Definitely something to do with outgoingConnectionHttps. This is a low level DSL and there could be some misconfigured setting somewhere which is causing this (although I couldn't figure out which one).
Usage of this DSL is actually discouraged by the docs.
Try using a higher level DSL like cached connection pool
val flow = Http().cachedHostConnectionPoolHttps[NotUsed]("akka.io").mapAsync(1) {
// Able to reach the API.
case (Success(HttpResponse(StatusCodes.OK, _, entity, _)), _) =>
// Unmarshal the json response.
Unmarshal(entity).to[String]
// Failed to reach the API.
case (Success(HttpResponse(code, _, entity, _)), _) =>
entity.discardBytes()
Future.successful(code.toString())
case (Failure(e), _) ⇒
throw e
}
// Run stream
source.map(_ → NotUsed).via(flow).runWith(...)
A potential issue is that there is no backpressure signal with Sink.actorRef, so the actor's mailbox could be getting full. If the actor, whenever it receives a JsValue object, is doing something that could take a long time, use Sink.actorRefWithAck instead. For example:
val initMessage = "start"
val completeMessage = "done"
val ackMessage = "ack"
source
.via(flow)
.runWith(Sink.actorRefWithAck[Any](
processJsonActor, initMessage, ackMessage, completeMessage))
You would need to change the actor to handle an initMessage and reply to the stream for every stream element with an ackMessage (with sender ! ackMessage). More information on Sink.actorRefWithAck is found here.
At the moment I use a websocket with
1 client
2 sources on server side
The server pushes out keepAlive messages and also answers requests from the client.
Now I would like to spice things up a bit and add the possibility to handle n clients.
So I had a look at:
https://github.com/playframework/play-scala-chatroom-example
This is basically a n-Inlet ~> n-Outlet so if any of the n clients writes something through their respective websocket, all of them get notified (including itself).
What I need is a bit more sophisticated as the server should
still send keepAlive messages to all the connected clients AND
if one of the clients asks for something/triggers a server-side "event", again all the clients should be notified of this.
So it's basically just one step in between in my abstract way of thinking.
Naive as I am I thought it could maybe be done by:
type AllowedWSMessage = String
val myActor = system.actorOf(Props{new myActor}, "myActor")
val myActorSink = Sink.actorRefWithAck(myActor, "init", "acknowledged", "completed")
import scala.concurrent.duration._
val tickingSource: Source[AllowedWSMessage, Cancellable] =
Source.tick(initialDelay = 1 second, interval = 10 seconds, tick = NotUsed)
.map(_ => "Staying Alive")
val serverMessageSource = Source
.queue[AllowedWSMessage](10, OverflowStrategy.backpressure)
.mapMaterializedValue { queue => myActor ! InitTunnel(queue)}
val serverSource: Source[AllowedWSMessage, Cancellable] = tickingSource.merge(serverMessageSource)
private val (clientSink, clientSource) =
{
// Don't log MergeHub$ProducerFailed as error if the client disconnects.
// recoverWithRetries -1 is essentially "recoverWith"
val source = MergeHub.source[AllowedWSMessage]
.log("source")
.recoverWithRetries(-1, { case _: Exception ⇒ Source.empty})
val sink: Sink[AllowedWSMessage, Source[AllowedWSMessage, NotUsed]] = BroadcastHub.sink[AllowedWSMessage]
source.via(serverSource).toMat(sink)(Keep.both).run()
}
(Note the source.via(serverSource)...)
but of course it's not that easy.
In the end what I want is basically:
(Client -> WebSocket ->) MergeHub ~> myActor ~> BroadcastHub (-> WebSocket -> Client)
Now I wonder, what is an elegant way of doing this?
Or are MergeHub and BroadcastHub the wrong tools for that challenge?
You have your server source and sink, you said they already work, so I'm not digging into them.
val fanIn = MergeHub.source[AllowedWSMessage].to(myActorSink).run()
val fanOut = serverSource.toMat(BroadcastHub.sink[AllowedWSMessage])(Keep.right).run()
// Now, somewhere in a (route-)method where you handle the websocket connection
Flow.fromSinkAndSource(fanIn, fanOut)
Easy as that, hope the knot in your head unravels now :)
Continuing series of questions about akka-streams I have another problem.
Variables:
Single http client flow with throttling
Multiple other flows that want to use first flow simultaneously
Goal:
Single http flow is flow that makes requests to particular API that limits number of calls to it. Otherwise it bans me. Thus it's very important to maintain rate of request regardless of how many clients in my code use it.
There are number of other flows that want to make requests to mentioned API but I'd like to have backpressure from http flow. Normally you connect whole thing to one graph and it works. But it my case I have multiple graphs.
How would you solve it ?
My attempt to solve it:
I use Source.queue for http flow so that I can queue http requests and have throttling. Problem is that Future from SourceQueue.offer fails if I exceed number of requests. Thus somehow I need to "reoffer" when previously offered event completes. Thus modified Future from SourceQueue would backpressure other graphs (inside their mapAsync) that make http requests.
Here is how I implemented it
object Main {
implicit val system = ActorSystem("root")
implicit val executor = system.dispatcher
implicit val materializer = ActorMaterializer()
private val queueHttp = Source.queue[(String, Promise[String])](2, OverflowStrategy.backpressure)
.throttle(1, FiniteDuration(1000, MILLISECONDS), 1, ThrottleMode.Shaping)
.mapAsync(4) {
case (text, promise) =>
// Simulate delay of http request
val delay = (Random.nextDouble() * 1000 / 2).toLong
Thread.sleep(delay)
Future.successful(text -> promise)
}
.toMat(Sink.foreach({
case (text, p) =>
p.success(text)
}))(Keep.left)
.run
val futureDeque = new ConcurrentLinkedDeque[Future[String]]()
def sendRequest(value: String): Future[String] = {
val p = Promise[String]()
val offerFuture = queueHttp.offer(value -> p)
def addToQueue(future: Future[String]): Future[String] = {
futureDeque.addLast(future)
future.onComplete {
case _ => futureDeque.remove(future)
}
future
}
offerFuture.flatMap {
case QueueOfferResult.Enqueued =>
addToQueue(p.future)
}.recoverWith {
case ex =>
val first = futureDeque.pollFirst()
if (first != null)
addToQueue(first.flatMap(_ => sendRequest(value)))
else
sendRequest(value)
}
}
def main(args: Array[String]) {
val allFutures = for (v <- 0 until 15)
yield {
val res = sendRequest(s"Text $v")
res.onSuccess {
case text =>
println("> " + text)
}
res
}
Future.sequence(allFutures).onComplete {
case Success(text) =>
println(s">>> TOTAL: ${text.length} [in queue: ${futureDeque.size()}]")
system.terminate()
case Failure(ex) =>
ex.printStackTrace()
system.terminate()
}
Await.result(system.whenTerminated, Duration.Inf)
}
}
Disadvantage of this solution is that I have locking on ConcurrentLinkedDeque which is probably not that bad for rate of 1 request per second but still.
How would you solve this task?
We have an open ticket (https://github.com/akka/akka/issues/19478) and some ideas for a "Hub" stage which would allow for dynamically combining streams, but I'm afraid I cannot give you any estimate for when it will be done.
So that is how we, in the Akka team, would solve the task. ;)
I'm trying to create an endpoint on my Akka Http Server which tells the users it's IP address using an external service (I know this can be performed way easier but I'm doing this as a challenge).
The code that doesn't make use of streams on the upper most layer is this:
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
val requestHandler: HttpRequest => Future[HttpResponse] = {
case HttpRequest(GET, Uri.Path("/"), _, _, _) =>
Http().singleRequest(HttpRequest(GET, Uri("http://checkip.amazonaws.com/"))).flatMap { response =>
response.entity.dataBytes.runFold(ByteString(""))(_ ++ _) map { string =>
HttpResponse(entity = HttpEntity(MediaTypes.`text/html`,
"<html><body><h1>" + string.utf8String + "</h1></body></html>"))
}
}
case _: HttpRequest =>
Future(HttpResponse(404, entity = "Unknown resource!"))
}
Http().bindAndHandleAsync(requestHandler, "localhost", 8080)
and it is working fine. However, as a challenge, I wanted to limit myself to only using streams (no Future's).
This is the layout I thought I'd use for this kind of an approach:
Source[Request] -> Flow[Request, Request] -> Flow[Request, Response] ->Flow[Response, Response] and to accommodate the 404 route, also Source[Request] -> Flow[Request, Response]. Now, if my Akka Stream knowledge serves me well, I need to use a Flow.fromGraph for such a thing, however, this is where I'm stuck.
In a Future I can do an easy map and flatMap for the various endpoints but in streams that would mean dividing up the Flow into multiple Flow's and I'm not quite sure how I'd do that. I thought about using UnzipWith and Options or a generic Broadcast.
Any help on this subject would be much appreciated.
I don't if this would be necessary? -- http://doc.akka.io/docs/akka-stream-and-http-experimental/2.0-M2/scala/stream-customize.html
You do not need to use Flow.fromGraph. Instead, a singular Flow that uses flatMapConcat will work:
//an outgoing connection flow
val checkIPFlow = Http().outgoingConnection("checkip.amazonaws.com")
//converts the final html String to an HttpResponse
def byteStrToResponse(byteStr : ByteString) =
HttpResponse(entity = new Default(MediaTypes.`text/html`,
byteStr.length,
Source.single(byteStr)))
val reqResponseFlow = Flow[HttpRequest].flatMapConcat[HttpResponse]( _ match {
case HttpRequest(GET, Uri.Path("/"), _, _, _) =>
Source.single(HttpRequest(GET, Uri("http://checkip.amazonaws.com/")))
.via(checkIPFlow)
.mapAsync(1)(_.entity.dataBytes.runFold(ByteString(""))(_ ++ _))
.map("<html><body><h1>" + _.utf8String + "</h1></body></html>")
.map(ByteString.apply)
.map(byteStrToResponse)
case _ =>
Source.single(HttpResponse(404, entity = "Unknown resource!"))
})
This Flow can then be used to bind to incoming requests:
Http().bindAndHandle(reqResponseFlow, "localhost", 8080)
And all without Futures...