How to implement reliable Periodic Actor? - scala

Question

I am using PeriodicActor taken from akka-pattern with some minor changes
trait PeriodicActor[T] extends DecoratingActor with ActorLogging {
import context.dispatcher
var messages = new ListBuffer[T]
abstract override def preStart() = {
schedule()
super.preStart()
}
protected def schedule() {
context.system.scheduler.scheduleOnce(1 seconds, self, Tick)
}
receiver {
case Tick => {
flush()
schedule()
}
}
def flush() = {
handleMessages(this.messages)
.recover {
case NonFatal(e) => log.error(e, "error in actor")
}
this.messages.clear()
}
/**
* implement to handle buffered messages.
*/
def handleMessages(messages: ListBuffer[T]): Future[Any]
def buffer(msg: T) = {
messages.append(msg)
}
}
Every 1 second a Tick is sent to self, to flush all messages received during this second.
My problem is that scheduleOnce is not reliable enough. The Tick message might not get received, which means the Ticking mechanism will stop working.
So I thought of ways to make sure this will not stop:
Maybe add if statement on the buffer method, that will make sure for example the if the list get to a certain size, it will send a Tick to flush it. The problem here, is that if the there is one lost Tick message, the list might not get flushed for a while, which is not good for me.
thought maybe to add another scheduling of Tick message every 5 minutes to make sure the mechanism will keep working.
context.system.scheduler.schedule(5 minutes, longInterval, self, Tick)
What do you think? Is there a better way?

Answer 1

Your problem caused by the at-most-once delivery semantics which Akka guarantees. Since you're prepares consider sending additional Ticks every 5 minutes, let's assume that you don't need exactly-once delivery of Ticks, so the occasional extra Tick is acceptable. This means that you need at-least-once delivery of Ticks.
at-least-once is achieved using some sort of ACK-RETRY, in this case you could achieve this using the ask pattern.
protected def schedule() {
context.system.scheduler.scheduleOnce(1 seconds){
sendTick()
}
}
def sendTick() {
implicit val timeout = 1 second
(self ? Tick).onComplete{
case Success(_) => schedule() // <- queue up next tick
case Failure(_) => sendTick() // <- send another tick immediately
}
}
In your receiver, you need to provide the acknowledgement:
receiver {
case Tick => {
sender ! TickAck
flush()
schedule()
}
}
More information on akka delivery guarantees: http://www.mjlivesey.co.uk/2016/02/19/akka-delivery-guarantees.html
As well as lost messages another cause of failure is the actor dying. This requires a supervision strategy. The default strategy is to restart the actor, and the postRestart() method by default calls preStart() so unless you use a different supervision strategy or override postRestart() you should get acceptable behaviour; if the actor dies it will restart and schedule another Tick.