Akka Actors - Creating Pool of Actors - scala

I created the following Akka Actor code in Scala. Code works fine when a single workerActor is created. But code silently fails when I try to create a pool of worker actors using round robin logic. Any idea how to fix this? How do I get more debug info to be printed?
import scala.collection.immutable.Map
import scala.collection.mutable.ArrayBuffer
import akka.actor.actorRef2Scala
import akka.actor.ActorSystem
import akka.actor.Props
import scala.concurrent.Await
import scala.concurrent.duration._
import akka.pattern.ask
import akka.util.Timeout
import akka.actor._
import org.junit._
import org.junit.Assert._
import messaging.actors._
import akka.routing.RoundRobinRouter
import akka.routing._
class MainEngineActorTest {
#Test
def testMainActor () = {
val _system = ActorSystem("MainEngineActor")
val master = _system.actorOf(Props[MainEngineActor], name = "EngineActor")
println ("Created Main Engine Actor")
implicit val timeout = Timeout(5 seconds)
val userID = new UserID ("test1")
println ("Sending messages")
for (i <- ( 1 to 10)) {
master ! "Hello"
master ! "World"
}
}
}
class MainEngineActor extends Actor with ActorLogging{
// works if we create only a single workerActor
//val workerActors = context.actorOf(Props[WorkerActor], name = "WorkerActors")
// Doesn't work when we create a pool of worker actors - how do we fix this?
// why doesn't this work and why aren't any error messages printed?
val workerActors = context.actorOf(RoundRobinPool(5).props(Props[WorkerActor]), name = "WorkerActors")
def receive: Receive = {
case request => {
workerActors forward request
}
}
}
class WorkerActor extends Actor {
def receive: Receive = {
case request => {
println ("RequestReceived =" + request)
}
}
}

Try creating your pool like this instead:
val workerActors = context.actorOf(Props[WorkerActor].withRouter(RoundRobinPool(5)), name = "WorkerActors")
In addition, when running this as a Junit test, the program is terminating before the child actors have a chance to receive the message. I verified this by adding a Thread.sleep(5000) after the loop that is sending the Hello and World messages to master. I then tweaked your code a little bit to use Akka's TestActorRef from akka-testkit which will force everything to use the CallingThreadDispatcher to get synchronous execution throughout the test and everything works as expected. The two lines I changed are:
implicit val _system = ActorSystem("MainEngineActor")
val master = TestActorRef(new MainEngineActor())

Related

How should I test akka-streams RestartingSource usage

I'm working on an application that has a couple of long-running streams going, where it subscribes to data about a certain entity and processes that data. These streams should be up 24/7, so we needed to handle failures (network issues etc).
For that purpose, we've wrapped our sources in RestartingSource.
I'm now trying to verify this behaviour, and while it looks like it functions, I'm struggling to create a test where I push in some data, verify that it processes correctly, then send an error, and verify that it reconnects after that and continues processing.
I've boiled that down to this minimal case:
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.{RestartSource, Sink, Source}
import akka.stream.testkit.TestPublisher
import org.scalatest.concurrent.Eventually
import org.scalatest.{FlatSpec, Matchers}
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext
class MinimalSpec extends FlatSpec with Matchers with Eventually {
"restarting a failed source" should "be testable" in {
implicit val sys: ActorSystem = ActorSystem("akka-grpc-measurements-for-test")
implicit val mat: ActorMaterializer = ActorMaterializer()
implicit val ec: ExecutionContext = sys.dispatcher
val probe = TestPublisher.probe[Int]()
val restartingSource = RestartSource
.onFailuresWithBackoff(1 second, 1 minute, 0d) { () => Source.fromPublisher(probe) }
var last: Int = 0
val sink = Sink.foreach { l: Int => last = l }
restartingSource.runWith(sink)
probe.sendNext(1)
eventually {
last shouldBe 1
}
probe.sendNext(2)
eventually {
last shouldBe 2
}
probe.sendError(new RuntimeException("boom"))
probe.expectSubscription()
probe.sendNext(3)
eventually {
last shouldBe 3
}
}
}
This test consistently fails on the last eventually block with Last failure message: 2 was not equal to 3. What am I missing here?
Edit: akka version is 2.5.31
I figured it out after having had a look at the TestPublisher code. Its subscription is a lazy val. So when RestartSource detects the error, and executes the factory method () => Source.fromPublisher(probe) again, it gets a new Source, but the subscription of the probe is still pointing to the old Source. Changing the code to initialize both a new Source and TestPublisher works.

items fail to be processed in Akka streams app that uses Source.queues and Sink.queues in a flow

I am trying to create an (Akka HTTP) stream procsesing flow using the classes akka.stream.scaladsl.Source and Sink queues.
I am using a queue because I have a processing step in my flow that issues http requests and I want this step to take as many
items off the queue as there are max-open-requests, and stop taking off the queue once max-open-requests are in flight.
The result is that backpressure is applied when my connection pool is overloaded.
Below, I have a very simplified test that reflects the main logic of my app. In the test 'Stress Spec' (below)
I am simulating a number of simultaneous connections via which I will send a 'Source' of 'Requesto' objects
to the getResponses method of the class ServiceImpl.
In the processing step 'pullOffSinkQueue' you will note that I am incrementing a counter to see how many items
I have pulled off the queue.
The test will send Serviceimpl a set of requests whose cardinality is set to equal
streamedRequestsPerConnection * numSimultaneousConnections.
When I send 20 requests my test passes fine. In particular the count of requests pulled off the
Sink.queue will be equal to the number of requests I send out. However, if
I increase the number of requests I send to above 50 or so, I see consistent failures in the test.
I get a message such as the one below
180 was not equal to 200
ScalaTestFailureLocation: com.foo.StressSpec at (StressSpec.scala:116)
Expected :200
Actual :180
<Click to see difference>
This indicates that the number of items pulled off the queue does not equal the number of items put on the queue.
I have a feeling this might be due to the fact that my test is not properly waiting for all items put into the stream
to be processed. If anyone has any suggestions, I'd be all ears ! Code is below.
package com.foo
import java.util.concurrent.atomic.AtomicInteger
import akka.stream.ActorAttributes.supervisionStrategy
import akka.stream.{Attributes, Materializer, QueueOfferResult}
import akka.stream.Supervision.resumingDecider
import akka.stream.scaladsl.{Flow, Keep, Sink, Source}
import scala.concurrent.{ExecutionContext, Future}
import akka.NotUsed
import akka.actor.ActorSystem
import akka.event.{Logging, LoggingAdapter}
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.{Sink, Source}
import org.scalatest.mockito.MockitoSugar
import org.scalatest.{FunSuite, Matchers}
import scala.collection.immutable
import scala.concurrent.duration._
import scala.concurrent.{Await, Future, _}
final case class Responso()
final case class Requesto()
object Handler {
val dbRequestCounter = new AtomicInteger(0)
}
class Handler(implicit ec: ExecutionContext, mat: Materializer) {
import Handler._
private val source =
Source.queue[(Requesto, String)](8, akka.stream.OverflowStrategy.backpressure)
private val sink =
Sink.queue[(Requesto, String)]().withAttributes(Attributes.inputBuffer(8, 8))
private val (sourceQueue, sinkQueue) = source.toMat(sink)(Keep.both).run()
def placeOnSourceQueue(ar: Requesto): Future[QueueOfferResult] = {
sourceQueue.offer((ar, "foo"))
}
def pullOffSinkQueue(qofr: QueueOfferResult): Future[Responso] = {
dbRequestCounter.incrementAndGet()
qofr match {
case QueueOfferResult.Enqueued =>
sinkQueue.pull().flatMap { maybeRequestPair: Option[(Requesto, String)] =>
Future.successful(Responso())
}
case error =>
println("enqueuing error: " + error)
Future.failed(new RuntimeException("enqueuing error: " + error))
}
}
}
class ServiceImpl(readHandler: Handler, writeHandler: Handler)
(implicit log: LoggingAdapter, mat: Materializer) {
private val readAttributeFlow: Flow[Requesto, Responso, NotUsed] = {
Flow[Requesto]
.mapAsyncUnordered(1)(readHandler.placeOnSourceQueue)
.mapAsyncUnordered(1)(readHandler.pullOffSinkQueue)
}
def getResponses(request: Source[Requesto, NotUsed]): Source[Responso, NotUsed] =
request
.via(readAttributeFlow)
.withAttributes(supervisionStrategy(resumingDecider))
}
class StressSpec
extends FunSuite
with MockitoSugar
with Matchers {
val streamedRequestsPerConnection = 10
val numSimultaneousConnections = 20
implicit val actorSystem: ActorSystem = ActorSystem()
implicit val materializer: ActorMaterializer = ActorMaterializer()
implicit val log: LoggingAdapter = Logging(actorSystem.eventStream, "test")
implicit val ec: ExecutionContext = actorSystem.dispatcher
import Handler._
lazy val requestHandler = new Handler()
lazy val svc: ServiceImpl =
new ServiceImpl(requestHandler, requestHandler)
test("can handle lots of simultaneous read requests") {
val totalExpected = streamedRequestsPerConnection * numSimultaneousConnections
def sendRequestAndAwaitResponse(): Unit = {
def getResponses(i: Integer) = {
val requestStream: Source[Requesto, NotUsed] =
Source(1 to streamedRequestsPerConnection)
.map { i =>
Requesto()
}
svc.getResponses(requestStream).runWith(Sink.seq)
}
val responses: immutable.Seq[Future[immutable.Seq[Responso]]] =
(1 to numSimultaneousConnections).map { getResponses(_) }
val flattenedResponses: Future[immutable.Seq[Responso]] =
Future.sequence(responses).map(_.flatten)
Await.ready(flattenedResponses, 1000.seconds).value.get
}
sendRequestAndAwaitResponse()
dbRequestCounter.get shouldBe(totalExpected)
}
}

Limit number of messages sent within time interval

Using below code I'm attempting to limit the amount of messages send to an actor within a specified time frame. But the messages are not being throttled and are being sent as quickly as possible. The downstream actor just makes a http request to the Google home page.
The throttler code where I attempt to limit 3 messages to be sent within 3 seconds :
val throttler: ActorRef =
Source.actorRef(bufferSize = 1000, OverflowStrategy.dropNew)
.throttle(3, 1.second)
.to(Sink.actorRef(printer, NotUsed))
.run()
How can I limit the number of messages sent within loop :
for( a <- 1 to 10000){
// Create the 'greeter' actors
val howdyGreeter: ActorRef =
system.actorOf(Greeter.props(String.valueOf(a), printer), String.valueOf(a))
howdyGreeter ! RequestActor("RequestActor")
howdyGreeter ! Greet
}
to 3 per second ?
entire code :
//https://developer.lightbend.com/guides/akka-quickstart-scala/full-example.html
import akka.NotUsed
import akka.stream.{OverflowStrategy, ThrottleMode}
import akka.stream.scaladsl.{Sink, Source}
import org.apache.http.client.methods.HttpGet
import org.apache.http.entity.StringEntity
import org.apache.http.impl.client.DefaultHttpClient
import net.liftweb.json._
import net.liftweb.json.Serialization.write
import org.apache.http.util.EntityUtils
//import akka.contrib.throttle.TimerBasedThrottler
import akka.actor.{Actor, ActorLogging, ActorRef, ActorSystem, Props}
import scala.concurrent.duration._
import akka.NotUsed
import akka.actor.ActorRef
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.stream.OverflowStrategy
import akka.stream.ThrottleMode
import akka.stream.scaladsl.Sink
import akka.stream.scaladsl.Source
object Greeter {
def props(message: String, printerActor: ActorRef): Props = Props(new Greeter(message, printerActor))
final case class RequestActor(who: String)
case object Greet
}
class Greeter(message: String, printerActor: ActorRef) extends Actor {
import Greeter._
import Printer._
var greeting = ""
def receive = {
case RequestActor(who) =>
val get = new HttpGet("http://www.google.com")
val response = (new DefaultHttpClient).execute(get)
// val responseString = EntityUtils.toString(response.getEntity, "UTF-8")
// System.out.println(responseString)
greeting = String.valueOf(response.getStatusLine.getStatusCode)
println("message is "+message)
// greeting = message + ", " + who
case Greet =>
printerActor ! Greeting(greeting)
}
}
object Printer {
def props: Props = Props[Printer]
final case class Greeting(greeting: String)
}
class Printer extends Actor with ActorLogging {
import Printer._
def receive = {
case Greeting(greeting) =>
log.info("Greeting received (from " + sender() + "): " + greeting)
}
}
object AkkaQuickstart extends App {
import Greeter._
// Create the 'helloAkka' actor system
val system: ActorSystem = ActorSystem("helloAkka")
// Create the printer actor,this is also the target actor
val printer: ActorRef = system.actorOf(Printer.props, "printerActor")
implicit val materializer = ActorMaterializer.create(system)
val throttler: ActorRef =
Source.actorRef(bufferSize = 1000, OverflowStrategy.dropNew)
.throttle(3, 1.second)
.to(Sink.actorRef(printer, NotUsed))
.run()
//Create a new actor for each request thread
for( a <- 1 to 10000){
// Create the 'greeter' actors
val howdyGreeter: ActorRef =
system.actorOf(Greeter.props(String.valueOf(a), printer), String.valueOf(a))
howdyGreeter ! RequestActor("RequestActor")
howdyGreeter ! Greet
}
}
An actor cannot influence what other actors do, in particular it has no control over who puts messages in its mailbox and when — this is how the actor model works. An actor only gets to decide what to do with the messages it finds in its mailbox, and over this it has full control. It can for example drop them, send back error replies, buffer them, etc.
If you want throttling and back-pressure, I recommend not using Actors at all for this part, but only use Akka Streams. The code that generates your request messages should be a Source, not a for-loop. Which source is most appropriate depends entirely on your real use-case, e.g. creating a stream from a strict collection with Source.from() or asynchronously pulling new elements out of a data structure with Source.unfoldAsync plus many more. Doing it this way ensures that the requests are only emitted when the time is right according to the downstream capacity or rate throttling.
It does not appear to me that you're actually using the throttler:
val throttler: ActorRef =
Source.actorRef(bufferSize = 1000, OverflowStrategy.dropNew)
.throttle(3, 1.second)
.to(Sink.actorRef(printer, NotUsed))
.run()
But I don't see any messages being sent to throttler in your code: throttler will only throttle messages sent to throttler.

How to ensure router actor with "group mode" already associate remote routees?

For router actor with "group mode", we new a series of actors in advance, and then we will use actorOf to associate one router with remote routees.
In fact, it internal will use actorSelection, my question is: how to assure the association already finish?
For actorSelection, we can use resolveOne to make sure the selection is successful, then send message. But how about router actor? See following code, how to assure line2 associate with remote routees before send message to it, just like line line1 does?
package local
import akka.actor._
import akka.routing.RoundRobinGroup
import akka.util.Timeout
import scala.concurrent.duration._
import com.typesafe.config.ConfigFactory
import scala.util.{Failure, Success}
object Local extends App {
val config = ConfigFactory.parseString(s"akka.remote.netty.tcp.port=2251")
val system = ActorSystem("LocalSystem", config.withFallback(ConfigFactory.load()))
system.actorOf(Props[LocalActor], "LocalActor")
val config2 = ConfigFactory.parseString(s"akka.remote.netty.tcp.port=2252")
val system2 = ActorSystem("LocalSystem2", config2.withFallback(ConfigFactory.load()))
system2.actorOf(Props[LocalActor2], "LocalActor2")
}
class LocalActor extends Actor {
def receive = {
case _ => println("hi")
}
}
class LocalActor2 extends Actor {
import scala.concurrent.ExecutionContext.Implicits.global
implicit val timeout = Timeout(5 seconds)
val a = context.actorSelection("akka.tcp://LocalSystem#127.0.0.1:2251/user/LocalActor")
a.resolveOne().onComplete { // line 1
case Success(actor) => println("ready")
a ! 1
case Failure(ex) => println("not ready")
}
val paths = List("akka.tcp://LocalSystem#127.0.0.1:2251/user/LocalActor")
val b = context.actorOf(RoundRobinGroup(paths).props(), "LocalActorRouter") // line 2
b ! 1
def receive = {
case _ =>
}
}

Akka Flow hangs when making http requests via connection pool

I'm using Akka 2.4.4 and trying to move from Apache HttpAsyncClient (unsuccessfully).
Below is simplified version of code that I use in my project.
The problem is that it hangs if I send more than 1-3 requests to the flow. So far after 6 hours of debugging I couldn't even locate the problem. I don't see exceptions, error logs, events in Decider. NOTHING :)
I tried reducing connection-timeout setting to 1s thinking that maybe it's waiting for response from the server but it didn't help.
What am I doing wrong ?
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.headers.Referer
import akka.http.scaladsl.model.{HttpRequest, HttpResponse}
import akka.http.scaladsl.settings.ConnectionPoolSettings
import akka.stream.Supervision.Decider
import akka.stream.scaladsl.{Sink, Source}
import akka.stream.{ActorAttributes, Supervision}
import com.typesafe.config.ConfigFactory
import scala.collection.immutable.{Seq => imSeq}
import scala.concurrent.{Await, Future}
import scala.concurrent.duration.Duration
import scala.util.Try
object Main {
implicit val system = ActorSystem("root")
implicit val executor = system.dispatcher
val config = ConfigFactory.load()
private val baseDomain = "www.google.com"
private val poolClientFlow = Http()(system).cachedHostConnectionPool[Any](baseDomain, 80, ConnectionPoolSettings(config))
private val decider: Decider = {
case ex =>
ex.printStackTrace()
Supervision.Stop
}
private def sendMultipleRequests[T](items: Seq[(HttpRequest, T)]): Future[Seq[(Try[HttpResponse], T)]] =
Source.fromIterator(() => items.toIterator)
.via(poolClientFlow)
.log("Logger")(log = myAdapter)
.recoverWith {
case ex =>
println(ex)
null
}
.withAttributes(ActorAttributes.supervisionStrategy(decider))
.runWith(Sink.seq)
.map { v =>
println(s"Got ${v.length} responses in Flow")
v.asInstanceOf[Seq[(Try[HttpResponse], T)]]
}
def main(args: Array[String]) {
val headers = imSeq(Referer("https://www.google.com/"))
val reqPair = HttpRequest(uri = "/intl/en/policies/privacy").withHeaders(headers) -> "some req ID"
val requests = List.fill(10)(reqPair)
val qwe = sendMultipleRequests(requests).map { case responses =>
println(s"Got ${responses.length} responses")
system.terminate()
}
Await.ready(system.whenTerminated, Duration.Inf)
}
}
Also what's up with proxy support ? Doesn't seem to work for me either.
You need to consume the body of the response fully so that the connection is made available for subsequent requests. If you don't care about the response entity at all, then you can just drain it to a Sink.ignore, something like this:
resp.entity.dataBytes.runWith(Sink.ignore)
By the default config, when using a host connection pool, the max connections is set to 4. Each pool has it's own queue where requests wait until one of the open connections becomes available. If that queue ever goes over 32 (default config, can be changed, must be a power of 2) then yo will start seeing failures. In your case, you only do 10 requests, so you don't hit that limit. But by not consuming the response entity you don't free up the connection and everything else just queues in behind, waiting for the connections to free up.