I want to implement logback MDC logging on my AKKA application to organize and have a more infomative log; however, I also read that MDC might not work well with AKKA because AKKA has asynchronous logging system (MDC might be stored on a different thread). I used the Custom Dispatcher for MDC Logging defined here hoping to solve my problem but I can't make it work on my application. My application is not a play framework app though.
I have a RequestHandler Actor that receives different types of request and delegates it to a RequestSpecificHandler Actor which will process it.
class RequestHandler() extends Actor with akka.actor.ActorLogging {
def receive: Receive = {
//Requests
case req: RequestA =>
org.slf4j.MDC.put("messageId", req.msgId)
org.slf4j.MDC.put("requestType", req.requestType)
log.debug("FIRST LOG Received a RequestA")
val actorA = context.ActorOf(ActorA.props)
actorA ! req.msg
case req: RequestB => //...
//other requests...
//Response
case res: ResponseA =>
log.debug("Received responseA")
org.slf4j.MDC.remove("messageId")
org.slf4j.MDC.remove("requestType")
//other response
}
}
In my RequestSpecificHandler Actors, I also create new or refer to other existing HelperActors
class ActorA () extends Actor with akka.actor.ActorLogging {
val helperA = context.actorSelection("/user/helperA")
val helperB = context.actorOf("HelperB.props")
def receive: Receive = {
case msg: MessageTypeA =>
//do some stuff
log.debug("received MessageTypeA")
helperA ! taskForA
case doneByA =>
//do some stuff
log.debug("received doneByA")
helperB ! taskForB
case doneByB =>
log.debug("send reponseA")
sender ! ResponseA
}
}
Logging differs everytime I send a request, sometimes it logs with correct MDC messageId and requestType, sometimes it does not have any value. Even the "FIRST LOG Received a RequestA" log behaves this way, I assume it should always have the correct logstamp as it is in the same class where I call MDC.put
Here is my application.conf:
akka {
log-dead-letters = 10
loggers = ["akka.event.slf4j.Slf4jLogger"]
loglevel = DEBUG
logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
actor{
default-dispatcher {
type = "some.package.monitoring.MDCPropagatingDispatcherConfigurator"
}
...
How can I do MDC logging where all the code logs (including dependency lib logs) executed during a certain request will have the same messageId, requestType logstamp? Are there other ways to do this aside from Custom Dispatcher for AKKA? Also, what is a more organize way to declare MDC.put and MDC.remove codes? Right now I'm having it on each case on receive.
Thanks
akka.actor.DiagnosticActorLogging might should solve your problem.
Related
I am using a third party library to provide parsing services (user agent parsing in my case) which is not a thread safe library and has to operate on a single threaded basis. I would like to write a thread safe API that can be called by multiple threads to interact with it via Futures API as the library might introduce some potential blocking (IO). I would also like to provide back pressure when necessary and return a failed future when the parser doesn't catch up with the producers.
It could actually be a generic requirement/question, how to interact with any client/library which is not thread safe (user agents/geo locations parsers, db clients like redis, loggers collectors like fluentd), with back pressure in a concurrent environments.
I came up with the following formula:
encapsulate the parser within a dedicated Actor.
create an akka stream source queue that receives ParseReuqest that contains the user agent and a Promise to complete, and using the ask pattern via mapAsync to interact with the parser actor.
create another actor to encapsulate the source queue.
Is this the way to go? Is there any other way to achieve this, maybe simpler ? maybe using graph stage? can it be done without the ask pattern and less code involved?
the actor mentioned in number 3, is because I'm not sure if the source queue is thread safe or not ?? I wish it was simply stated in the docs, but it doesn't. there are multiple versions over the web, some stating it's not and some stating it is.
Is the source queue, once materialized, is thread safe to push elements from different threads?
(the code may not compile and is prone to potential failures, and is only intended for this question in place)
class UserAgentRepo(dbFilePath: String)(implicit actorRefFactory: ActorRefFactory) {
import akka.pattern.ask
import akka.util.Timeout
import scala.concurrent.duration._
implicit val askTimeout = Timeout(5 seconds)
// API to parser - delegates the request to the back pressure actor
def parse(userAgent: String): Future[Option[UserAgentData]] = {
val p = Promise[Option[UserAgentData]]
parserBackPressureProvider ! UserAgentParseRequest(userAgent, p)
p.future
}
// Actor to provide back pressure that delegates requests to parser actor
private class ParserBackPressureProvider extends Actor {
private val parser = context.actorOf(Props[UserAgentParserActor])
val queue = Source.queue[UserAgentParseRequest](100, OverflowStrategy.dropNew)
.mapAsync(1)(request => (parser ? request.userAgent).mapTo[Option[UserAgentData]].map(_ -> request.p))
.to(Sink.foreach({
case (result, promise) => promise.success(result)
}))
.run()
override def receive: Receive = {
case request: UserAgentParseRequest => queue.offer(request).map {
case QueueOfferResult.Enqueued =>
case _ => request.p.failure(new RuntimeException("parser busy"))
}
}
}
// Actor parser
private class UserAgentParserActor extends Actor {
private val up = new UserAgentParser(dbFilePath, true, 50000)
override def receive: Receive = {
case userAgent: String =>
sender ! Try {
up.parseUa(userAgent)
}.toOption.map(UserAgentData(userAgent, _))
}
}
private case class UserAgentParseRequest(userAgent: String, p: Promise[Option[UserAgentData]])
private val parserBackPressureProvider = actorRefFactory.actorOf(Props[ParserBackPressureProvider])
}
Do you have to use actors for this?
It does not seem like you need all this complexity, scala/java hasd all the tools you need "out of the box":
class ParserFacade(parser: UserAgentParser, val capacity: Int = 100) {
private implicit val ec = ExecutionContext
.fromExecutor(
new ThreadPoolExecutor(
1, 1, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue(capacity)
)
)
def parse(ua: String): Future[Option[UserAgentData]] = try {
Future(Some(UserAgentData(ua, parser.parseUa(ua)))
.recover { _ => None }
} catch {
case _: RejectedExecutionException =>
Future.failed(new RuntimeException("parser is busy"))
}
}
New to both Scala + Akka here. I have a small network of actors that all send different messages to each other. Many of them need access to my DeadLetterChannel actor in case they receive a message that they are not sure how to handle:
class DeadLetterChannel(val queue : mutable.Queue[Any]) extends Actor with Logging {
override def receive: Receive = {
case any : Any =>
log.info(s"${this.getClass} just received a message of type ${} from ${sender().getClass}.")
queue.enqueue(any)
case _ =>
log.warn(s"${this.getClass} just received a non-Any message.")
}
}
Then from inside many other actors:
class DeviceManager(val dlc : DeadLetterChannel) extends ActorRef {
override def receive: Receive = {
case Heartbeat =>
// Handle Heartbeat here...
case Connect:
// Handle Connect here...
case Disconnect:
// Handle Disconnect here...
case _ =>
dlc ! ???
}
}
I have two problems here:
I'm getting a compiler error on the sending of the message (specifically the ! overload: "Cannot resolve symbol !"); and
I have no idea what to send to dlc in the _ case, any ideas? Obviously I'm just trying to send the message that DeviceManager received that is neither a Heartbeat, a Connect or a Disconnect type
You may have over-simplified the code for your example, but you can't just send a message to an Actor class instance. You need to create a named actor using actorOf on an ActorSystem, and then use actorSelection on that system to get a handle on the actor.
For the second part of your question, just put a match value in the case and send the result:
case msg =>
dlc ! msg
Also, you might want to use a different name for your class because a dead letter is message that can't be delivered, not a message that can't be handled by the recipient.
Actor classes should extend Actor instead of ActorRef. Also, use ActorRef to reference an actor. DeviceManager, therefore, should be defined as the following:
class DeviceManager(dlc: ActorRef) extends Actor {
// ...
}
The idea here is that you would use the ActorRef of the DeadLetterChannel actor when creating the device actors. For example:
val dlc: ActorRef = context.actorOf(/* .. */)
val device1: ActorRef = context.actorOf(Props(classOf[DeviceManager], dlc)
val device2: ActorRef = context.actorOf(Props(classOf[DeviceManager], dlc)
Then you can do as Tim suggested in his answer and capture the default case in the receive block and send this message to dlc:
class DeviceManager(dlc: ActorRef) extends Actor {
def receive = {
/* other case clauses */
case msg =>
dlc ! msg
}
}
Also, instead of passing a mutable queue to DeadLetterChannel as a constructor argument, encapsulate this state inside the actor as an immutable queue and expose it only via message passing:
class DeadLetterChannel extends Actor with akka.actor.ActorLogging {
var queue = collection.immutable.Queue[Any](0)
def receive = {
case GetMessages => // GetMessages is a custom case object
sender() ! queue
case msg =>
val s = sender()
log.info(s"DLC just received the following message from [$s]: $msg")
queue = queue.enqueue(msg)
}
}
Another approach is to not define a default case (i.e., leave out the case msg => clause) and use Akka's mechanisms for dealing with unhandled messages. Below are a couple of options:
Enable the built-in logging for unhandled messages, which logs these messages at the DEBUG level:
akka {
actor {
debug {
# enable DEBUG logging of unhandled messages
unhandled = on
}
}
}
This approach is perhaps the simplest if you just want to log unhandled messages.
As the documentation states:
If the current actor behavior does not match a received message...by default publishes an akka.actor.UnhandledMessage(message, sender, recipient) on the actor system’s event stream.
In light of this, you could create an actor that subscribes to the event stream and handles akka.actor.UnhandledMessage messages. Such an actor could look like the following:
import akka.actor.{ Actor, UnhandledMessage, Props }
class UnhandledMessageListener extends Actor {
def receive = {
case UnhandledMessage(msg, sender, recipient) =>
// do something with the message, such as log it and/or something else
}
}
val listener = system.actorOf(Props[UnhandledMessageListener])
system.eventStream.subscribe(listener, classOf[UnhandledMessage])
More information about the event stream and creating a subscriber to that stream can be found here.
I'm currently working on an application with a signup process. This signup process will, at some point, communicate with external systems in an asynchronous manner. To keep this question concise, I'm showing you two important actors I've written:
SignupActor.scala
class SignupActor extends PersistentFSM[SignupActor.State, Data, DomainEvt] {
private val apiActor = context.actorOf(ExternalAPIActor.props(new HttpClient))
// At a certain point, a CreateUser(data) message is sent to the apiActor
}
ExternalAPIActor.scala
class ExternalAPIActor(apiClient: HttpClient) extends Actor {
override def preRestart(reason: Throwable, message: Option[Any]) = {
message.foreach(context.system.scheduler.scheduleOnce(3 seconds, self, _))
super.preRestart(reason, message)
}
def receive: Receive = {
case CreateUser(data) =>
Await.result(
apiClient.post(data)
.map(_ => UserCreatedInAPI())
.pipeTo(context.parent),
Timeout(5 seconds).duration
)
}
}
This setup seems to work as expected. When there is an issue with the external API (such as a timeout or network problems), the Future returned by HttpClient::post fails and will result in an exception thanks to Await.result. This, in turn thanks to the SupervisorStrategy of the SignupActor parent actor, will restart the ExternalAPIActor where we can re-send the last message to itself with a small delay to avoid deadlock.
I see a couple of issues with this setup:
Within the receive method of ExternalAPIActor, blocking occurs. As far as I understand, blocking within Actors is considered an anti-pattern.
The delay used to re-send the message is static. If the API is unavailable for longer periods of time, we will keep on sending HTTP requests every 3 seconds. I'd like some kind of exponential backoff mechanism here instead.
To continue on with the latter, I've tried the following in the SignupActor:
SignupActor.scala
val supervisor = BackoffSupervisor.props(
Backoff.onFailure(
ExternalAPIActor.props(new HttpClient),
childName = "external-api",
minBackoff = 3 seconds,
maxBackoff = 30 seconds,
randomFactor = 0.2
)
)
private val apiActor = context.actorOf(supervisor)
Unfortunately, this doesn't seem to do anything at all -- the preRestart method of ExternalAPIActor isn't called at all. When replacing Backoff.onFailure with Backoff.onStop, the preRestart method is called, but without any kind of exponential backoff at all.
Given the above, my questions are as follows:
Is using Await.result the recommended (the only?) way to make sure exceptions thrown in a Future returned from services called within actors are caught and handled accordingly? An especially important part of my particular use case is the fact that messages shouldn't be dropped but retried when something went wrong. Or is there some other (idiomatic) way that exceptions thrown in asynchronous contexts should be handled within Actors?
How would one use the BackoffSupervisor as intended in this case? Again: it is very important that the message responsible for the exception is not dropped, but retried until a N-number of times (to be determined by the maxRetries argument of SupervisorStrategy.
Is using Await.result the recommended (the only?) way to make sure
exceptions thrown in a Future returned from services called within
actors are caught and handled accordingly?
No. Generally that's not how you want to handle failures in Akka. A better alternative is to pipe the failure to your own actor, avoiding the need to use Await.result at all:
def receive: Receive = {
case CreateUser(data) =>
apiClient.post(data)
.map(_ => UserCreatedInAPI())
.pipeTo(self)
case Success(res) => context.parent ! res
case Failure(e) => // Invoke retry here
}
This would mean no restart is required to handle failure, they are all part of the normal flow of your actor.
An additional way to handle this can be to create a "supervised future". Taken from this blog post:
object SupervisedPipe {
case class SupervisedFailure(ex: Throwable)
class SupervisedPipeableFuture[T](future: Future[T])(implicit executionContext: ExecutionContext) {
// implicit failure recipient goes to self when used inside an actor
def supervisedPipeTo(successRecipient: ActorRef)(implicit failureRecipient: ActorRef): Unit =
future.andThen {
case Success(result) => successRecipient ! result
case Failure(ex) => failureRecipient ! SupervisedFailure(ex)
}
}
implicit def supervisedPipeTo[T](future: Future[T])(implicit executionContext: ExecutionContext): SupervisedPipeableFuture[T] =
new SupervisedPipeableFuture[T](future)
/* `orElse` with the actor receive logic */
val handleSupervisedFailure: Receive = {
// just throw the exception and make the actor logic handle it
case SupervisedFailure(ex) => throw ex
}
def supervised(receive: Receive): Receive =
handleSupervisedFailure orElse receive
}
This way, you only pipe to self once you get a Failure, and otherwise send it to the actor the message was meant to be sent to, avoiding the need for the case Success I added to the receive method. All you need to do is replace supervisedPipeTo with the original framework provided pipeTo.
Alright, I've done some more thinking and tinkering and I've come up with the following.
ExternalAPIActor.scala
class ExternalAPIActor(apiClient: HttpClient) extends Actor with Stash {
import ExternalAPIActor._
def receive: Receive = {
case msg # CreateUser(data) =>
context.become(waitingForExternalServiceReceive(msg))
apiClient.post(data)
.map(_ => UserCreatedInAPI())
.pipeTo(self)
}
def waitingForExternalServiceReceive(event: InputEvent): Receive = LoggingReceive {
case Failure(_) =>
unstashAll()
context.unbecome()
context.system.scheduler.scheduleOnce(3 seconds, self, event)
case msg:OutputEvent =>
unstashAll()
context.unbecome()
context.parent ! msg
case _ => stash()
}
}
object ExternalAPIActor {
sealed trait InputEvent
sealed trait OutputEvent
final case class CreateUser(data: Map[String,Any]) extends InputEvent
final case class UserCreatedInAPI() extends OutputEvent
}
I've used this technique to prevent the original message from being lost in case there is something wrong with the external service we're calling. During the process of a request to an external service, I switch context, waiting for either a response of a failure and switch back afterwards. Thanks to the Stash trait, I can make sure other requests to external services aren't lost as well.
Since I have multiple actors in my application calling external services, I abstracted the waitingForExternalServiceReceive to its own trait:
WaitingForExternalService.scala
trait WaitingForExternalServiceReceive[-tInput, +tOutput] extends Stash {
def waitingForExternalServiceReceive(event: tInput)(implicit ec: ExecutionContext): Receive = LoggingReceive {
case akka.actor.Status.Failure(_) =>
unstashAll()
context.unbecome()
context.system.scheduler.scheduleOnce(3 seconds, self, event)
case msg:tOutput =>
unstashAll()
context.unbecome()
context.parent ! msg
case _ => stash()
}
}
Now, the ExternalAPIActor can extend this trait:
ExternalAPIActor.scala
class ExternalAPIActor(apiClient: HttpClient) extends Actor with WaitingForExternalServiceReceive[InputEvent,OutputEvent] {
import ExternalAPIActor._
def receive: Receive = {
case msg # CreateUser(data) =>
context.become(waitingForExternalServiceReceive(msg))
apiClient.post(data)
.map(_ => UserCreatedInAPI())
.pipeTo(self)
}
}
object ExternalAPIActor {
sealed trait InputEvent
sealed trait OutputEvent
final case class CreateUser(data: Map[String,Any]) extends InputEvent
final case class UserCreatedInAPI() extends OutputEvent
}
Now, the actor won't get restarted in case of failures/errors and the message isn't lost. What's more, the entire flow of the actor now is non-blocking.
This setup is (most probably) far from perfect, but it seems to work exactly as I need it to.
I'm trying to write an Actor which connects to an Amazon Kinesis stream and then relays any messages received via Comet to a Web UI. I'm using Source.actorPublisher for this and using the json method with Comet in Play described here. I got the events working just fine using Source.tick(), but when I tried using an ActorPublisher, the Actor never seems to be sent any Request messages as expected. How are requests for data usually sent down an Akka flow? I'm using v2.5 of the Play Framework.
My controller code:
def subDeviceSeen(id: Id): Action[AnyContent] = Action {
val deviceSeenSource: Source[DeviceSeenMessage, ActorRef] = Source.actorPublisher(DeviceSeenEventActor.props)
Ok.chunked(deviceSeenSource
.filter(m => m.id == id)
.map(Json.toJson(_))
via Comet.json("parent.deviceSeen")).as(ContentTypes.JSON)
}
Am I doing anything obviously wrong in the above? Here is my Actor code:
object DeviceSeenEventActor {
def props: Props = Props[DeviceSeenEventActor]
}
class DeviceSeenEventActor extends ActorPublisher[DeviceSeenMessage] {
implicit val mat = ActorMaterializer()(context)
val log = Logging(context.system, this)
def receive: Receive = {
case Request => log.debug("Received request message")
initKinesis()
context.become(run)
case Cancel => context.stop(self)
}
def run: Receive = {
case vsm:DeviceSeenMessage => onNext(vsm)
log.debug("Received request message")
onCompleteThenStop() //we are currently only interested in one message
case _:Any => log.warning("Unknown message received by event Actor")
}
private def initKinesis() = {
//init kinesis, a worker is created and given a reference to this Actor.
//The reference is used to send messages to the actor.
}
}
The 'Received request message' log line is never displayed. Am I missing some implicit? There are no warnings or anything else obvious displayed in the play console.
The issue was that I was pattern matching on case Request => ... instead of case Request() => .... Since I didn't have a default case in my receive() method, the message was simply dropped by the Actor.
In my Spray app, I delegate requests to actors. I want to be able to kill a actor that takes too long. I'm not sure whether I should be using Spray timeouts, Akka ask pattern or something else.
I have implemented:
def processRouteRequest(system: ActorSystem) = {
respondWithMediaType(`text/json`) {
params { p => ctx =>
val builder = newBuilderActor
builder ! Request(p) // the builder calls `ctx.complete`
builder ! PoisonPill
system.scheduler.scheduleOnce(routeRequestMaxLife, builder, Kill)
}
}
}
The idea being that the actor lives only for the duration of a single request and if it doesn't complete within routeRequestMaxLife it gets forcibly killed. This approach seems over-the-top (and spews a lot of info about undelivered messages). I'm not even certain it works correctly.
It seems like what I'm trying to achieve should be a common use-case. How should I approach it?
I would tend to using the ask pattern and handling the requests as follows:
class RequestHandler extends Actor {
def receive = {
case "quick" =>
sender() ! "Quick Reply"
self ! PoisonPill
case "slow" =>
val replyTo = sender()
context.system.scheduler.scheduleOnce(5 seconds, self, replyTo)
case a:ActorRef =>
a ! "Slow Reply"
self ! PoisonPill
}
}
class ExampleService extends HttpService with Actor {
implicit def actorRefFactory = context
import context.dispatcher
def handleRequest(mode: String):Future[String] = {
implicit val timeout = Timeout(1 second)
val requestHandler = context.actorOf(Props[RequestHandler])
(requestHandler ? mode).mapTo[String]
}
val route: Route =
path("endpoint" / Segment) { str =>
get {
onComplete(handleRequest(str)) {
case Success(str) => complete(str)
case Failure(ex) => complete(ex)
}
}
}
def receive = runRoute(route)
}
This way the actor takes care of stopping itself, and the semantics of Ask give you the information about whether or not the request timed out.