Looking at the Scala code for this 5 REQ <--> 1 ROUTER setup:
Worker
class WorkerTask extends Runnable {
override def run: Unit = {
val rand = new Random(System.currentTimeMillis())
val context = ZMQ.context(1)
val worker = context.socket(ZMQ.REQ)
worker.connect("tcp://localhost:5555")
var total = 0
var workload = ""
do {
worker.send("Ready".getBytes, 0)
workload = new String(worker.recv(0))
Thread.sleep (rand.nextInt(1) * 1000)
total += 1
} while (workload.equalsIgnoreCase("END") == false)
printf("Completed: %d tasks\n", total)
}
}
main (Router)
def main(args: Array[String]): Unit = {
val NBR_WORKERS = 5
val context = ZMQ.context(1)
val client = context.socket(ZMQ.ROUTER)
assert(client.getType > -1)
client.bind("tcp://*:5555")
val workers = List.fill(NBR_WORKERS)(new Thread(new WorkerTask))
workers.foreach (_.start)
for (i <- 1 to (NBR_WORKERS * 10)) {
// LRU worker is next waiting in queue
val address = client.recv(0)
val empty = client.recv(0)
val ready = client.recv(0)
client.send(address, ZMQ.SNDMORE)
client.send("".getBytes, ZMQ.SNDMORE)
client.send("This is the workload".getBytes, 0)
}
for (i <- 1 to NBR_WORKERS) {
val address = client.recv(0)
val empty = client.recv(0)
val ready = client.recv(0)
client.send(address, ZMQ.SNDMORE)
client.send("".getBytes, ZMQ.SNDMORE)
client.send("END".getBytes, 0)
}
}
Running on my machine:
[info] Running net.server.RouterToReq
Completed: 21 tasks
Completed: 1 tasks
Completed: 27 tasks
Completed: 5 tasks
Completed: 1 tasks
As I partially understand the above code, 5 REQ workers accept requests from the ROUTER on port 5555.
Lastly, in the following code:
for (i <- 1 to NBR_WORKERS) {
val address = client.recv(0)
val empty = client.recv(0)
val ready = client.recv(0)
What message is the client, i.e. ROUTER, receiving?
The worker sends the message "Ready" repeatedly as a single frame message.
The REQ socket will add a blank delimiter frame on the front.
The ROUTER socket will prepend an identifier frame to the front of any received message to identify where it came from.
Thus the single ready message becomes a 3 frame message when you receive it on the router, this is why 3 recv calls are required.
When you send on a router socket the first frame will be removed and used to identify which client to send the message to. The REQ socket will remove all frames until it finds an empty frame and thus you only need a single recv call on the worker side.
Related
I am new to scala. I am trying to timeout api request. I am using spray to make API request. I have spray client to get response from some other server. In my application.conf i've specified timeout of request in spray.can something like:
spray.can {
server {
idle-timeout = ${idle-timeout}
request-timeout = ${request-timeout}
request-chunk-aggregation-limit = 20m
parsing {
max-content-length = 21m
}
}
client {
idle-timeout = ${idle-timeout}
request-timeout = ${request-timeout}
response-chunk-aggregation-limit = 20m
}
}
Now, I want to override this request-timeout in one my api. I have written api something like:
def completeService(jwttoken: String, completeRequest: String): Future[HttpResponse] = {
val pipeline: HttpRequest => Future[HttpResponse] = sendReceive ~> unmarshal[HttpResponse]
val response: Future[HttpResponse] = pipeline(Post(some-remote-url, completeRequest)
~> addHeader("FROM", jwttoken))
response
}
So, how can I put request-timeout here in this method? By overriding application.conf
I tried
implicit val timeoutVal: Timeout = Timeout(scala.concurrent.duration.Duration(100, MILLISECONDS).asInstanceOf[FiniteDuration])
and I got this:
test 2020-03-06 08:48:12.147 GMT [WARN] a.k.i.AskPatternInstrumentation - Timeout triggered for ask pattern to actor [IO-HTTP] at [pipelining.scala:38]
test 2020-03-06 08:48:12.573 GMT [INFO] akka.actor.DeadLetterActorRef - Message [spray.http.HttpResponse] from Actor[akka://test-app/user/IO-HTTP/host-connector-1/1#-1105043312] to Actor[akka://test-app/deadLetters] was not deliver
I think you can add an implicit parameter requestTimeout. The data type of that parameter is akka.util.Timeout. You can checkout this link for more details or this link
val _pipeline: Future[SendReceive] =
for (
Http.HostConnectorInfo(connector, _) <-
IO(Http) ? Http.HostConnectorSetup("www.spray.io", port = 80, settings = Some(new HostConnectorSettings(maxConnections = 3, maxRetries = 3, maxRedirects = 0, pipelining = false, idleTimeout = 5 seconds, connectionSettings = ClientConnectionSettings(...))))
) yield sendReceive(connector)
Let me know if it helps!!
in Scala, I have an akka http client class with some local binding:
class AkkaConPoolingHttpClient(
override val timeout: Option[FiniteDuration] = None,
val localBinding: Option[InetSocketAddress] = None,
val userAgentHeader: Option[String] = None)(
implicit val config: HttpClient.Config,
val system: ActorSystem,
val materializer: Materializer)
extends AkkaHttpClient {
protected val http = Http()
override def dispatch(request: HttpRequest): Future[HttpResponse] = {
val effectivePort = request.uri.effectivePort
val connection =
http.outgoingConnection(
request.uri.authority.host.address(),
port = effectivePort,
localAddress = localBinding)
val preparedRequest = userAgentHeader match {
case Some(userAgent) => fixUri(request.withHeaders(request.headers ++ Seq(headers.`User-Agent`(userAgent))))
case None => fixUri(request)
}
Source.single(preparedRequest) via connection runWith Sink.head
}
object AkkaConPoolingHttpClient {
private def fixUri(request: HttpRequest): HttpRequest =
request.withUri(request.uri.toRelative)
}
and I'm trying to see if it reuses the connections and it seems it doesn't:
val connectionCount = new AtomicInteger()
val testServerFuture = Http().bind("127.0.0.1", 0).to {
Sink.foreach { incomingConnection =>
connectionCount.incrementAndGet()
incomingConnection.flow.join(Flow[HttpRequest].map(_ => HttpResponse())).run()
}
}.run()
val testServerPort = Await.result(testServerFuture, defaultExpectTimeout)
.localAddress.getPort
val address = "127.0.0.1"
val addr = Some(new InetSocketAddress(address, 0))
val client = new AkkaConPoolingHttpClient(localBinding = addr)
// Send some requests concurrently
val requests = List(
Get(s"http://127.0.0.1:$testServerPort/1"),
Get(s"http://127.0.0.1:$testServerPort/2"),
Get(s"http://127.0.0.1:$testServerPort/3"))
val responses = Await.result(
Future.sequence(requests.map(client.sendRequest)),
defaultExpectTimeout)
// Send some more requests -- the connections from before should be reused
Thread.sleep(500)
val responses2 = Await.result(
Future.sequence(requests.map(client.sendRequest)),
defaultExpectTimeout)
// Usually this is "3", occasionally "4".
connectionCount.get() must beLessThanOrEqualTo(4)
Unfortunately, the test fails, connectionCount.get() has 6 connections. Why isn't it reuse the connections? what's wrong with this code?
I also tried with:
val effectivePort = request.uri.effectivePort
val clientSettings = ClientConnectionSettings(system).withSocketOptions(SO.ReuseAddress(true) :: Nil)
val connectionFlow: Flow[HttpRequest, HttpResponse, Future[Http.OutgoingConnection]] =
Http().outgoingConnection(
request.uri.authority.host.address(),
port = effectivePort,
localAddress = localBinding,
settings = clientSettings
)
..................
Source.single(preparedRequest)
.via(connectionFlow)
.runWith(Sink.head)
But I still have 6 connections in my test...
Problem
The problem is rooted in the fact that you are creation a new connection for each request. The client code is actually quite clear:
Source.single(preparedRequest) via connection runWith Sink.head
Each request is being sent through a newly instantiated connection. This is due to a general design flaw where you are getting the address from the request:
val connection =
http.outgoingConnection(
request.uri.authority.host.address(), //address comes from request
port = effectivePort,
localAddress = localBinding)
It would be more efficient to establish the address once (ensuring a single Connection), and then each Request would just need the path.
Solution
To use a single connection you'll have to create a single Flow and send all of your requests through that, as described here.
Given the following attempt to connect 1 DEALER to 1 ROUTER:
package net.async
import org.zeromq.ZMQ
import org.zeromq.ZMQ.Socket
import scala.annotation.tailrec
object Client {
val Empty = "".getBytes
def message(x: Int) = s"HELLO_#$x".getBytes
val Count = 5
}
class Client(name: String) extends Runnable {
import Client._
import AsyncClientServer.Port
override def run(): Unit = {
val context = ZMQ.context(1)
val dealer = context.socket(ZMQ.DEALER)
dealer.setIdentity(name.getBytes)
dealer.connect(s"tcp://localhost:$Port")
runHelper(dealer, Count)
}
#tailrec
private def runHelper(dealer: Socket, count: Int): Unit = {
dealer.send(dealer.getIdentity, ZMQ.SNDMORE)
dealer.send(Empty, ZMQ.SNDMORE)
dealer.send(message(count), 0)
println(s"Dealer: ${dealer.getIdentity} received message: " + dealer.recv(0))
runHelper(dealer, count - 1)
}
}
object AsyncClientServer {
val Port = 5555
val context = ZMQ.context(1)
val router = context.socket(ZMQ.ROUTER)
def main(args: Array[String]): Unit = {
router.bind(s"tcp://*:$Port")
mainHelper()
new Thread(new Client("Joe")).start()
}
private def mainHelper(): Unit = {
println("Waiting to receive messages from Dealer.")
val identity = router.recv(0)
val empty = router.recv(0)
val message = router.recv(0)
println(s"Router received message, ${new String(message)} from sender: ${new String(identity)}.")
mainHelper()
}
}
I see the following output, hanging on the second message.
[info] Running net.async.AsyncClientServer
[info] Waiting to receive messages from Dealer.
Why is that?
Not sure if its the cause of your problem but you don't need to send the identity frame from your dealer, zeromq will do this for you. By adding it your actually sending a 4 part message.
IDENTITY
IDENTITY
EMPTY
CONTENT
Given the following slightly modified Load Balancer ZeroMQ code:
package net.broker
import org.zeromq.ZMQ
object LruQueue2 {
class ClientTask(name: String) extends Runnable {
override def run(): Unit = {
val context = ZMQ.context(1)
val client = context.socket(ZMQ.REQ)
client.setIdentity(name.getBytes)
client.connect("tcp://localhost:5555")
// send request, get reply
client.send("HELLO".getBytes, 0)
val reply = client.recv(0)
println(s"${new String(client.getIdentity)} received: ${new String(reply)}")
}
}
class WorkerTask(name: String) extends Runnable {
override def run(): Unit = {
val context = ZMQ.context(1)
val worker = context.socket(ZMQ.REQ)
worker.connect("tcp://localhost:5556")
worker.setIdentity(name.getBytes)
worker.send("READY".getBytes, 0)
while(true) {
val clientAddr = worker.recv(0)
val empty = worker.recv(0)
val clientMsg = worker.recv(0)
worker.send(clientAddr, ZMQ.SNDMORE)
worker.send("".getBytes, ZMQ.SNDMORE)
worker.send("WORLD".getBytes, 0)
println(s"${new String(worker.getIdentity)}: 3-frames to client: ${new String(clientAddr)}")
}
}
}
def main(args: Array[String]): Unit = {
val NOFLAGS = 0
// worker using REQ socket to do LRU routing
val NBR_CLIENTS = 1
val NBR_WORKERS = 1
val context = ZMQ.context(1)
val frontend = context.socket(ZMQ.ROUTER)
val backend = context.socket(ZMQ.ROUTER)
frontend.bind("tcp://*:5555")
backend.bind("tcp://*:5556")
val clients = (1 to NBR_CLIENTS).toList.map{ i => new Thread(new ClientTask(s"CLIENT$i"))}
val workers = (1 to NBR_CLIENTS).toList.map{ i => new Thread(new WorkerTask(s"WORKER$i"))}
clients.foreach(_.start)
workers.foreach(_.start)
val workerQueue = scala.collection.mutable.Queue[Array[Byte]]()
val poller = context.poller(2)
poller.register(backend, ZMQ.Poller.POLLIN)
poller.register(frontend, ZMQ.Poller.POLLIN)
var clientNbr = NBR_CLIENTS
while(true) {
println("begin to poll")
poller.poll()
println("done polling")
println("clientNbr:" + clientNbr)
println("workerQueue.length: " + workerQueue.length)
if(clientNbr == 0) {
sys.exit(0)
}
else if(poller.pollin(0) && clientNbr > 0) {
val workerAddr = backend.recv(NOFLAGS)
val empty = backend.recv(NOFLAGS)
val clientAddrOrReadyMsg = backend.recv(NOFLAGS)
workerQueue.enqueue(workerAddr)
if(new String(clientAddrOrReadyMsg) == "READY") {
// nothing to do - worker is letting us know that he's ready to work
}
else {
// retrieve remaining 2 frames of client message
// [Empty][Client Message of "HELLO"]
val empty = backend.recv(0)
val workerResponse = backend.recv(0)
frontend.send(clientAddrOrReadyMsg, ZMQ.SNDMORE)
frontend.send("".getBytes, ZMQ.SNDMORE)
frontend.send(workerResponse, NOFLAGS)
clientNbr -= 1
}
}
else if (poller.pollin(1) && workerQueue.nonEmpty) {
val clientAddr = frontend.recv(0)
val empty = frontend.recv(0)
val clientMsg = frontend.recv(0)
backend.send(workerQueue.dequeue(), ZMQ.SNDMORE)
backend.send("".getBytes, ZMQ.SNDMORE)
backend.send(clientAddr, ZMQ.SNDMORE)
backend.send("".getBytes, ZMQ.SNDMORE)
backend.send(clientMsg, NOFLAGS)
}
else {}
}
}
}
When I run the above code, I see the following output:
[info] Running net.broker.LruQueue2
[info] begin to poll
[info] done polling
[info] clientNbr:1
[info] workerQueue.length: 0
[info] begin to poll
[info] done polling
[info] clientNbr:1
[info] workerQueue.length: 1
[info] begin to poll
[info] WORKER1: 3-frames to client: CLIENT1
[info] done polling
[info] clientNbr:1
[info] workerQueue.length: 0
[info] begin to poll
[info] CLIENT1 received: WORLD
It appears that it's stuck on poller.poll(), per the output print statements.
How can I debug this behavior?
I have a master Actor responsible for initializing some worker actors (there are two types of worker actors, namely, ParamServer actor and DataShard actor). For example, if I initiated 20 datashard actors via ClusterSharding(system).start(_,_,_,_,_) and after that I want to send some message to all datashard actors (say case object ReadyToProcess). I read that I can send messages to entities in Akka Cluster Shard via local ShardRegion(system).shardRegion(_). Is local shardRegion(_) will send to all datashards or just one. How can I send msgs to all datashard actors?
The master class given be:
class Master(ports: Seq[String],
dataSet: Seq[Example],
dataPerReplica: Int,
layerDimensions: Seq[Int],
activation: ActivationFunction,
activationFunctionDer: ActivationFunction,
learningRate: Double) extends Actor with ActorLogging {
val dataShards = dataSet.grouped(dataPerReplica).toSeq
val numLayers = layerDimensions.size
var numShardsFinished = 0
ports foreach { port =>
val config = ConfigFactory.parseString("akka.remote.netty.tcp.port=" + port).withFallback(ConfigFactory.load())
val clusterSystem = ActorSystem("ClusterSystem", config)
val paramServerRegions: Array[ActorRef] = new Array[ActorRef](numLayers - 1)
for (i <- 0 to numLayers - 2) {
paramServerRegions(i) = ClusterSharding(clusterSystem).start(
typeName = ParamServer.shardName,
entityProps = ParamServer.props(i, dataShards.size, learningRate, NeuralNetworkOps.randomMatrix(layerDimensions(i + 1), layerDimensions(i) + 1)),
settings = ClusterShardingSettings(clusterSystem),
extractEntityId = ParamServer.extractEntityId,
extractShardId = ParamServer.extractShardId
)
}
//create actors for each data shard/replica. Each replica needs to know about all parameter shards because they will
//be reading from them and updating them
val dataShardRegions: Array[ActorRef] = new Array[ActorRef](dataShards.size)
for (i <- 0 to dataShards.size) {
dataShardRegions(i) = ClusterSharding(clusterSystem).start(
typeName = DataShard.shardName,
entityProps = DataShard.props(i, clusterSystem, dataShards(i), activation, activationFunctionDer, paramServerRegions),
settings = ClusterShardingSettings(clusterSystem),
extractEntityId = ParamServer.extractEntityId,
extractShardId = ParamServer.extractShardId
)
}
}
def receive: Receive = {
case Start => {
val shardRegionSender = ClusterSharding(context.system).shardRegion(DataShard.shardName)
println("Tomosha boshlandi")
shardRegionSender ! ReadyToProcess
}
case ShardDone(id) => {
numShardsFinished+=1
log.info("")
if (numShardsFinished == dataShards.size) {
context.parent ! JobDone
context.stop(self)
}
}
}
}