I have an Akka messaging engine that delivers millions of messages during the day, both SMS and Email. I need to introduce a new type of messaging (PushNotification) which consists in having each request consume a REST API (it will also process millions). I believe that consuming a Webservice is a blocking operation, so from what I have read I need to add a separate dispatcher for this new Actor, my questions is, does it necessarily need to be a thread-pool-executor with a fixed-pool-size as mentioned here? (See https://doc.akka.io/docs/akka-http/current/handling-blocking-operations-in-akka-http-routes.html) or is it possible to use a fork-join-executor instead? Also what is the best approach in order not to affect the current 2 types of Messaging ? (SMS and EMAIL) I mean how do I avoid to starve their thread-pool ? Currently EMAIL is using a separate Dispatcher and SMS is using the Default Dispatcher. Instead of creating a new Dispatcher for the Actor with blocking operation (calling WebService) is there any other way ? Like creating a reactive web service ?
Using a RESTful API from a web service does not have to be blocking.
An easy way to consume a RESTful API from an actor is to use Akka HTTP Client. This allows you to send an HTTP request and have the result sent back as a message to an actor using the pipeTo method.
This is a very cut-down example (lightly modified from the sample in the documentation).
import akka.http.scaladsl.Http
object RestWorker {
def props(replyTo: ActorRef): Props =
Props(new RestWorker(replyTo))
}
class RestWorker(replyTo: ActorRef) extends Actor
{
implicit val ec: ExecutionContext = context.system.dispatcher
override def preStart() = {
Http(context.system).singleRequest(HttpRequest(uri = "https://1.2.3.4/resource"))
.pipeTo(self)
}
def receive = {
case resp: HttpResponse =>
val response = ??? // Process response
replyTo ! response
self ! PoisonPill
}
}
Related
I'm trying to write an Akka HTTP webservice using domain-driven design but I'm struggling to pass technical data received by the webservice to the code doing the work inside a Future, namely a correlationId sent by the client to my webservice.
My understanding of DDD is that as a choice of implementation, the correlationId shouldn't appear in the domain:
package domain
trait MyRepository {
def startWork(): Future[Unit]
}
To make it available to my implementation code without it being a parameter of the method, I'm thinking of using thread-local storage like org.slf4j.MDC or a ThreadLocal. However, I don't know if that would work or if several calls to the webservice would be handled by the same thread and overwrite the value.
Here is the implementation:
package infra
class MyRepository(implicit executor: ExecutionContextExecutor) extends domain.MyRepository {
override def startWork(): Future[Unit] = {
Future {
val correlationId = ??? // MDC.get("correlationId") ?
log(s"The correlationId is $correlationId")
}
}
}
And the route in my webservice:
val repo = new infra.MyRepository()
val route = path("my" / "path") {
post {
parameter('correlationId) { correlationId =>
??? // MDC.put("correlationId", correlationId) ?
onComplete(repo.startWork()) {
complete(HttpResponse(StatusCodes.OK))
}
}
}
}
My question is twofold:
Is my general design sound and good DDD?
Would using org.slf4j.MDC or a ThreadLocal work or is there a better / more Akka-friendly way to implement it?
Thread-locals (including MDC, though a Lightbend subscription includes tooling to propagate the MDC alongside messages and futures) in Akka are generally a poor idea because it's generally not guaranteed that a given task (a Future in this case, or the actor handling a sent message) will execute on the same thread as the thread that requested the task (and in the specific case where that task is performing [likely-blocking] interactions with an external service/DB (implied by the use of Future {}), you pretty much don't want that to happen). Further, even if the task ends up executing on the same thread that requested the task, it's somewhat unlikely that no other task which could have mutated the MDC/thread-local would've executed in the meantime.
I myself don't see a problem with passing the correlation ID as an argument to startWork: you've already effectively exposed it by passing it through the HTTP endpoint.
First of all, I am not asking how to unit test an Akka Actor. I know the techniques to do that. My problem is slightly different.
I am developing a Scala program, let's call it the client, that does not use Akka Actors. However, the program uses a library, let's call it the server, whose interface is implemented using an Akka Actor.
Then, through the ask pattern, the client interacts with the server.
// This is the client program
class Client(uri: String) {
implicit val context: ActorSystem = ActorSystem("client-actor-system")
private val mainActor = context.actorSelection(uri)
def connect: Future[SomeClass] = {
implicit val timeout: Timeout = Timeout(5 seconds)
(mainActor ? Connect()).mapTo[CreationResponse]
}
}
Now, I want to write down some unit tests for Client class. Unit testing means to test a class in isolation. Every external dependency should be mocked.
How can I mock the reference to mainActor inside Client class? How can I inject in the actor system a mock actor for mainActor?
Thanks in advance.
I would inject the mainActor actor selection into the client, and create that using Akka TestKits.
Of course, you need an ActorSelection, not an ActorRef. Some solutions for that can be found here:
Giving an ActorPath for a testProbe
Or alternatively, you could make mainActor an ActorRef by sending an Identify message to the ActorSelection.
I am currently writing a client application in Scala which makes HTTP requests to an API. In this client application I have implemented a service connector which encapsulates all API related logic. Before making API calls, I want to authenticate the user, but I want to abstract this process. Which means that my actor would only call the service connector to initiate the API call, something like that:
class MessageProcessor(credentials: Credentials) extends Actor with ActorLogging {
implicit val ec = context.dispatcher
override def receive: Receive = {
case sendMsg: SendMessage =>
log.info(s"Sending message ${sendMsg.body}.")
sendMessage(sendMsg)
}
def sendMessage(msg: SendMessage) = {
ServiceConnector.sendMessage(credentials, msg).map { result =>
// My result
}
}
}
object MessageProcessor {
def props(credentials: Credentials) = Props(classOf[MessageProcessor], credentials)
}
In my service connector, I want to somehow save "the Scala way" the JWT token and if I am not yet authenticated, send an authentication request before making the actual API call.
How can I code such a service in an immutable manner with Futures in mind?
I thought about creating additional actors and just sending messages around with the token, but is this really necessary?
You need multiple Akka states to do it the proper "Scala way". I'm not completely sure how your API works, but the following example shows a basic approach. In its first state, it authenticates before sending the message. Once the authentication is confirmed, it sends the message. All following messages are immediately sent. If the authentication is lost somehow, you can also add a logout or timeout case that switches back to the first receive state.
class MessageProcessor(credentials: Credentials) extends Actor with ActorLogging {
implicit val ec = context.dispatcher
override def receive: Receive = {
case sendMsg: SendMessage =>
log.info(s"Authenticating...")
sender ! NotAuthenticated(credentials) // or authenticate somehow
context become waitingAuthentication(sendMsg)
}
def waitingAuthentication(sendMsg: SendMessage): Receive = {
case _: AuthenticationConfirmation =>
log.info(s"Sending message ${sendMsg.body}.")
sendMessage(sendMsg)
context become authenticated
}
def authenticated: Receive = {
case sendMsg: SendMessage =>
log.info(s"Sending message ${sendMsg.body}.")
sendMessage(sendMsg)
}
}
It's just an example and doesn't consider all cases (e.g., a SendMessage during waitingAuthentication, therefore you would need a queue of SendMessages). If multiple actors need to know the authentication state or the credentials, you need to broadcast to them if you don't want a bottleneck actor that handles and verifies all messages. In that case, all of them would also need multiple authentication states as described above.
Actors Are Designed For State
This seems like a false exercise. The entire point of Actors is that they can warehouse state. From the documentation specifically on state:
The good news is that Akka actors conceptually each have their own
light-weight thread, which is completely shielded from the rest of the
system. This means that instead of having to synchronize access using
locks you can just write your actor code without worrying about
concurrency at all.
Therefore, "the scala way" is just to keep it in a variable within Actors.
One of the reasons to use Actors instead of Futures is so you can maintain state. Why choose Actors and then dismiss one of their primary advantages???
Your question is the equivalent of "how do I use HashMap without doing any hashing (in a scala way)?"
I'm trying to subscribe to the log stream of a remote akka ActorSystem, basically to write a console that shows the running logs of the remote Actors.
The only way I can think to do this is to: create an Actor within the logging ActorSystem, have that Actor subscribe to the ActorSystem.eventStream and then subscribe to that Actor using actorSelection from within my console's ActorSystem.
But this seems very "indirect" since the log pipeline would look like:
logging Actor --> eventStream --> Actor subscribed to eventStream --> local Actor
Is there an easier way to subscribe to the event stream?
From a simplicity viewpoint, nothing forbids you to subscribe a remote actor to your event stream without an additional actor. The Akka documentation mentions:
The event stream is a local facility, meaning that it will not
distribute events to other nodes in a clustered environment (unless
you subscribe a Remote Actor to the stream explicitly). If you need to
broadcast events in an Akka cluster, without knowing your recipients
explicitly (i.e. obtaining their ActorRefs), you may want to look
into: Distributed Publish Subscribe in Cluster.
For illustration purposes, consider the following code fragment which corresponds to the remote system, the one you want to subscribe to:
class PublisherActor extends Actor with ActorLogging { // example publisher actor just to generate some logs
context.system.scheduler.schedule(1.second, 1.second, self, "echo")
def receive = {
case "echo" ⇒
val x = Random.nextInt(100)
log.info(s"I got a random number: $x")
}
}
def runPublisher() = {
println("=== running publisher node ===")
val system = ActorSystem("PublisherSystem")
val selection = system.actorSelection("akka.tcp://SubscriberSystem#127.0.0.1:2553/user/subscriber")
selection.resolveOne(10.seconds) onSuccess { // when the listener actor is available,
case listener ⇒ system.eventStream.subscribe(listener, classOf[LogEvent]) // subscribe it to the event stream
}
val publisher = system.actorOf(Props[PublisherActor], "publisher") // some example publisher
}
And then the corresponding subscriber in the "local" node, from where you want to show the logs:
class SubscriberActor extends Actor with ActorLogging {
log.info("subscriber listening...")
def receive = {
case msg ⇒ log.info(s"Got: $msg")
}
}
def runSubscriber() = {
println("=== running subscriber node ===")
val system = ActorSystem("SubscriberSystem")
val listener = system.actorOf(Props[SubscriberActor], "subscriber")
}
However, there are several caveats to this solution, as the fact that the publisher must be running before the subscriber (or the subscriber implement some retry policy until the publisher is up), the location is hardcoded and so on. If you want to have a more robust and resilient system and it's permissible, follow the advice in the documentation and use a distributed publisher-subscriber in a clustered environment which poses several advantages with a similar amount of boilerplate.
Hope it helped!
I have a shared external resource (say a file store) which a pool of actors is using. Each time a new request is made to the file store a new actor is created to fill the request with a reference to the external system passed in.
The current approach where I create a circuit breaker per actor defeats the purpose as a new actor is created for each 'request' which performs a sequence of operations on this external resource.
Not ideal - too many CB instances;
class MySharedResourceActor(externalResourceRef: ExtSystem) extends Actor with ActorLogging {
val breaker = new CircuitBreaker(context.system.scheduler,
maxFailures = 5,
callTimeout = 10.seconds,
resetTimeout = 1.minute).onOpen(notifyMeOnOpen())
def receive = {
case SomeExternalOp =>
breaker.withSyncCircuitBreaker(dangerousCallToExternalSystem()) pipeTo sender()
}
}
Better Approach - pass in a CB ref;
class MySharedResourceActor(externalResourceRef: ExtSystem, val breaker: CircuitBreaker) extends Actor with ActorLogging {
def receive = {
case SomeExternalOp =>
breaker.withSyncCircuitBreaker(dangerousCallToExternalSystem()) pipeTo sender()
}
}
Is it safe to pass in a Circuit-breaker reference from the parent actor which also maintains a reference to the external system and share this circuit breaker between multiple actors in a router pool, dynamically created or otherwise?
Yes it's safe to follow this approach. We share circuit breakers across related actors (pooled or otherwise) that are making http calls to the same host. If you didn't do this, and let each instance have it's own breaker, even if they were long lived instances, each one would need to hit the fail threshold separately before the breaker opened and I doubt this is the behavior you want. By sharing, it allows multiple actors to contribute stats (fails, successes) into the breaker so that the breaker is representative of all calls that have gone into the resource.
In looking at Akka's code, they are using atomics inside of the circuit breaker to represent state and handle state transitions, so they should be safe to use in multiple actors.