Make Scala Remote Actors more stable - scala

I was writing a little test program to try out some things with Remote Actors that I was going to need in a Scala project.
The basic goal was to write a test application of one server that could handle a bunch of clients and more important clients that can send multiple messages at the same time (like pings, requests for updates and user induced requests for data)
What I came up with was this:
brief overview: the client starts 3 different actors which again start actors in while loops with different offsets in order to simulate rather random messages.
import scala.actors.remote.RemoteActor
import scala.actors.remote.Node
import scala.actors.Actor
trait Request
trait Response
case object WhoAmI extends Request
case class YouAre(s:String) extends Response
case object Ping extends Request
case object Pong extends Response
case class PrintThis(s:String) extends Request
case object PrintingDone extends Response
object Server {
def main(args: Array[String]) {
val server = new Server
server.start
}
}
class Server extends Actor {
RemoteActor.alive(12345)
RemoteActor.register('server, this)
var count:Int = 0
def act() {
while(true) {
receive {
case WhoAmI => {
count += 1
sender ! YouAre(count.toString)
}
case Ping => sender ! Pong
case PrintThis(s) => {
println(s)
sender ! PrintingDone
}
case x => println("Got a bad request: " + x)
}
}
}
}
object Act3 extends scala.actors.Actor {
def act = {
var i = 0
Thread.sleep(900)
while (i <= 12) {
i += 1
val a = new Printer
a.start
Thread.sleep(900)
}
}
}
class Printer extends scala.actors.Actor {
def act = {
val server = RemoteActor.select(Node("localhost",12345), 'server)
server ! PrintThis("gagagagagagagagagagagagaga")
receive {
case PrintingDone => println("yeah I printed")
case _ => println("got something bad from printing")
}
}
}
object Act2 extends scala.actors.Actor {
def act = {
var i = 0
while (i < 10) {
i+=1
val a = new Pinger
a.start
Thread.sleep(700)
}
}
}
class Pinger extends scala.actors.Actor {
def act = {
val server = RemoteActor.select(Node("localhost",12345), 'server)
server ! Ping
receive {
case Pong => println("so I pinged and it fits")
case x => println("something wrong with ping. Got " + x)
}
}
}
object Act extends scala.actors.Actor {
def act = {
var i = 0
while(i < 10) {
i+=1
val a = new SayHi
a.start()
Thread.sleep(200)
}
}
}
class SayHi extends scala.actors.Actor {
def act = {
val server = RemoteActor.select(Node("localhost",12345), 'server)
server ! "Hey!"
}
}
object Client {
def main(args: Array[String]) {
Act.start()
//Act2.start()
Act3.start()
}
}
The problem is that things don't run as smoothly as I'd expect them to:
when I start only one of the client actors (by commenting the others out as I did with Act2in Client) things usually but not always go well. If I start two or more actors, quite often the printouts appear in bulk (meaning: there's nothing happening at once and then the printouts appear rather fast). Also the client sometimes terminates and sometimes doesn't.
This may not be the biggest problems but they're enough to make me feel quite uncomfortable. I did a lot of reading on Actors and Remote Actors but I find the available info rather lacking.
Tried to add exit statements where ever it seemed fit. But that didn't help.
Has anybody got an idea what I'm doing wrong? Any general tricks here? Some dos and donts?

My guess is that your issues stem from blocking your actor's threads by using receive and Thread.sleep. Blocking operations consume threads in the actors' thread pool, which can prevent other actors from executing until new threads are added to the pool. This question may provide some additional insight.
You can use loop, loopWhile, react, and reactWithin to rewrite many of your actors to use non-blocking operations. For example
import scala.actors.TIMEOUT
object Act extends scala.actors.Actor {
def act = {
var i = 0
loopWhile(i < 10) {
reactWithin(200) { case TIMEOUT =>
i+=1
val a = new SayHi
a.start()
}
}
}
}
Of course, you can eliminate some boilerplate by writing your own control construct:
def doWithin(msec: Long)(f: => Unit) = reactWithin(msec) { case TIMEOUT => f }
def repeat(times: Int)(f: => Unit) = {
var i = 0
loopWhile(i < times) {
f
i+=1
}
}
This would allow you to write
repeat(10) {
doWithin(200) {
(new SayHi).start
}
}

You may try Akka actors framework instead http://akkasource.org/

Related

Implement Actor model without Akka in Scala

I am doing my small research that implement Actor without Akka
I found one implementation of Actor in Scala. (How to implement actor model without Akka?)
It's very simple. Because I have not enough reputation to add the comment, so I create this question.
I wonder if I use Actor like below.
1/ How can I shutdown that actor from main thread?
2/ How can I add feature similar to Akka, like parent actor, kill request, and become method?
import scala.concurrent._
trait Actor[T] {
implicit val context = ExecutionContext.fromExecutor(java.util.concurrent.Executors.newFixedThreadPool(1))
def receive: T => Unit
def !(m: T) = Future { receive(m) }
}
This is my own example when trying to adapt the above code snippet
import scala.concurrent._
/**
* Created by hminle on 10/21/2016.
*/
trait Message
case class HelloMessage(hello: String) extends Message
case class GoodByeMessage(goodBye: String) extends Message
object State extends Enumeration {
type State = Value
val Waiting, Running, Terminating = Value
}
trait Actor[T] {
implicit val context = ExecutionContext.fromExecutor(java.util.concurrent.Executors.newFixedThreadPool(1))
private var state: State.State = State.Waiting
def handleMessage: T => Unit ={
if(state == State.Waiting) handleMessageWhenWaiting
else if(state == State.Running) handleMessageWhenRunning
else handleMessageWhenTerminating
}
def !(m: T) = Future {handleMessage(m)}
def handleMessageWhenWaiting: T => Unit
def handleMessageWhenRunning: T => Unit
def handleMessageWhenTerminating: T => Unit
def transitionTo(destinationState: State.State): Unit = {
this.state = destinationState
}
}
class Component1 extends Actor[Message]{
def handleMessageWhenRunning = {
case HelloMessage(hello) => {
println(Thread.currentThread().getName + hello)
}
case GoodByeMessage(goodBye) => {
println(Thread.currentThread().getName + goodBye)
transitionTo(State.Terminating)
}
}
def handleMessageWhenWaiting = {
case m => {
println(Thread.currentThread().getName + " I am waiting, I am not ready to run")
transitionTo(State.Running)
}
}
def handleMessageWhenTerminating = {
case m => {
println(Thread.currentThread().getName + " I am terminating, I cannot handle any message")
//need to shutdown here
}
}
}
class Component2(component1: Actor[Message]) extends Actor[Message]{
def handleMessageWhenRunning = {
case HelloMessage(hello) => {
println(Thread.currentThread().getName + hello)
component1 ! HelloMessage("hello 1")
}
case GoodByeMessage(goodBye) => {
println(Thread.currentThread().getName + goodBye)
component1 ! GoodByeMessage("goodbye 1")
transitionTo(State.Terminating)
}
}
def handleMessageWhenWaiting = {
case m => {
println(Thread.currentThread().getName + " I am waiting, I am not ready to run")
transitionTo(State.Running)
}
}
def handleMessageWhenTerminating = {
case m => {
println(Thread.currentThread().getName + " I am terminating, I cannot handle any message")
//need to shutdown here
}
}
}
object ActorExample extends App {
val a = new Component1
val b = new Component2(a)
b ! HelloMessage("hello World 2")
b ! HelloMessage("hello World 2, 2nd")
b ! GoodByeMessage("Good bye 2")
println(Thread.currentThread().getName)
}
You can look at Actor model implementation in scalazand take ideas from it, source code in scalaz actor is easier for insight than akka. You have freedom of choice about architecture: you can use mailboxes based on ConcurrentLinkedQueue like in Akka, use CAS for AtomicReffernce like in scalaz, in your case you use Future mechanism. IMO, you must write a context of your actor system, so solve first and second items in your question it's the variant of ActorContext:
val contextStack = new ThreadLocal[List[ActorContext]]
and shutdown can look like this:
1.
case Kill ⇒ throw new ActorKilledException("Kill")
case PoisonPill ⇒ self.stop()
2. For storing parent actor and similar task, you must store reference on them:
def parent: ActorRef
it's hard to say about advantages of every technique (CAS, mailboxes), it's possible variants to your research.

Testing Akka Typed behavior

How would I test that a given behavior sends the messages I expect?
Say, three messages of some type, one after the other...
With regular actors (untyped) there was the TestProbe from regular Akka with methods like expectedMsg:
http://doc.akka.io/api/akka/current/index.html#akka.testkit.TestProbe
With akka-typed I'm scratching my head still. There is something called EffectfulActorContext, but I've no idea how to use that.
Example
Say I am writing a simple PingPong service, that given a number n replies with Pong(n) n-times. So:
-> Ping(2)
Pong(2)
Pong(2)
-> Ping(0)
# nothing
-> Ping(1)
Pong(1)
Here is how this behavior might look:
case class Ping(i: Int, replyTo: ActorRef[Pong])
case class Pong(i: Int)
val pingPong: Behavior[Ping] = {
Static {
case Ping(i, replyTo) => (0 until i.max(0)).map(_=> replyTo ! Pong(i))
}
}
My Hack
Now since I can't figure out how to make this work, the "hack" that I am doing right now is making the actor always reply with a list of responses. So the behavior is:
case class Ping(i: Int, replyTo: ActorRef[List[Pong]])
case class Pong(i: Int)
val pingPong: Behavior[Ping] = {
Static {
case Ping(i, replyTo) => replyTo ! (0 until i.max(0)).map(_=>Pong(i)).toList
}
}
Given this hacky change, the tester is easy to write:
package com.test
import akka.typed.AskPattern._
import akka.typed.ScalaDSL._
import akka.typed.{ActorRef, ActorSystem, Behavior, Props}
import akka.util.Timeout
import com.test.PingPong.{Ping, Pong}
import org.scalatest.{FlatSpec, Matchers}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
import scala.concurrent.{Await, Future}
object PingPongTester {
/* Expect that the given messages arrived in order */
def expectMsgs(i: Int, msgs: List[Pong]) = {
implicit val timeout: Timeout = 5 seconds
val pingPongBe: ActorSystem[Ping] = ActorSystem("pingPongTester", Props(PingPong.pingPong))
val futures: Future[List[Pong]] = pingPongBe ? (Ping(i, _))
for {
pongs <- futures
done <- {
for ((actual, expected) <- pongs.zip(msgs)) {
assert(actual == expected, s"Expected $expected, but received $actual")
}
assert(pongs.size == msgs.size, s"Expected ${msgs.size} messages, but received ${pongs.size}")
pingPongBe.terminate
}
} Await.ready(pingPongBe.whenTerminated, 5 seconds)
}
}
object PingPong {
case class Ping(i: Int, replyTo: ActorRef[List[Pong]])
case class Pong(i: Int)
val pingPong: Behavior[Ping] = {
Static {
case Ping(i, replyTo) => replyTo ! (0 until i.max(0)).map(_=>Pong(i)).toList
}
}
}
class MainSpec extends FlatSpec with Matchers {
"PingPong" should "reply with empty when Pinged with zero" in {
PingPongTester.expectMsgs(0, List.empty)
}
it should "reply once when Pinged with one" in {
PingPongTester.expectMsgs(1, List(Pong(1)))
}
it should "reply with empty when Pinged with negative" in {
PingPongTester.expectMsgs(-1, List.empty)
}
it should "reply with as many pongs as Ping requested" in {
PingPongTester.expectMsgs(5, List(Pong(5), Pong(5), Pong(5), Pong(5), Pong(5)))
}
}
I'm using EffectfulActorContext for testing my Akka typed actors and here is an untested example based on your question.
Note: I'm also using the guardianactor provided in the Akka-typed test cases.
class Test extends TypedSpec{
val system = ActorSystem("actor-system", Props(guardian()))
val ctx: EffectfulActorContext[Ping] = new EffectfulActorContext[Ping]("ping", Ping.props(), system)
//This will send the command to Ping Actor
ctx.run(Ping)
//This should get you the inbox of the Pong created inside the Ping actor.
val pongInbox = ctx.getInbox("pong")
assert(pongInbox.hasMessages)
val pongMessages = pongInbox.receiveAll()
pongMessages.size should be(1) //1 or whatever number of messages you expect
}
Edit (Some more info): Cases where I need to add a replyTo ActorRef in my messages I do the following:
case class Pong(replyTo: ActorRef[Response])
val responseInbox: SyncInbox[Response] = Inbox.sync[Response]("responseInbox")
Pong(responseInbox.ref)
My initial approach to testing was to extend Behavior class
class TestQueueBehavior[Protocol] extends Behavior[Protocol] {
val messages: BlockingQueue[Protocol] = new LinkedBlockingQueue[Protocol]()
val behavior: Protocol => Unit = {
(p: Protocol) => messages.put(p)
}
def pollMessage(timeout: FiniteDuration = 3.seconds): Protocol = {
messages.poll(timeout.toMillis, TimeUnit.MILLISECONDS)
}
override def management(ctx: ActorContext[Protocol], msg: Signal): Behavior[Protocol] = msg match {
case _ ⇒ ScalaDSL.Unhandled
}
override def message(ctx: ActorContext[Protocol], msg: Protocol): Behavior[Protocol] = msg match {
case p =>
behavior(p)
Same
}
}
then I could call behavior.pollMessage(2.seconds) shouldBe somethingToCompareTo which was very similar to using TestProbe.
Although I think EffectfulActorContext is the right way to go, unfortunately couldn't figure out how to properly use it.

Accessing states of an Akka Actor through messages

I'm trying to access the state of a particular actor through messaging. What I don't know is how I can retrieve the state of the actor. Here, I need to access the state variable of Node, state1. I want to avoid using promises/futures in the code. How should I go about modifying this code if I want to do that?
class Node extends Actor {
val state1:Int = 4
def receive = {
case getState => {
sender ! ??? //How should I send the 'state1' to the sender?
}
}
}
class Master extends Actor {
def recieve = {
case someCase(node_actor:ActorRef) => {
// My aim here is to get the "state1" from node actor into var 's'
var s:Int = node_actor ! getState
}
}
}
Actors are designed perfectly to avoid manual handling of scala.concurrent. things.
Just separate request and response handling into different receive cases:
class Node extends Actor {
import Node._
val state1: Int = 4
def receive: Receive = {
case getState =>
sender ! State(state1)
}
}
class Master extends Actor {
import Master._
def receive: Receive = {
case Action(node) =>
// My aim here is to get the "state1" from node actor into var 's'
node ! Node.GetState
case Node.State(state) =>
// do something with state
}
}
object Master {
case class Action(node: ActorRef)
}
object Node {
case object GetState
case class State(state: Int)
}
Sometimes you could have also some intermediate values calculated and don't want to do something until you'll get the answer but being unreachable also is not acceptable. So you could just wait for node response in separate Receive while stashing incoming messages, like this:
class Master extends Actor with Stash {
import Master._
def receive: Receive = {
case Action(node) =>
val intermediate = scala.util.Random.nextLong()
node ! Node.GetState
context.become(waitingForState(calc = intermediate), discardOld = false)
}
def waitingForState(calc: Long): Receive = {
case Node.State(state) =>
// do something with state and calc
context.unbecome()
unstashAll()
case _ => stash()
}
}

Akka Actors Still Available After Stopped by PoisonPill

I'm using akka to dynamically create actors and destroy them when they're finished with a particular job. I've got a handle on actor creation, however stopping the actors keeps them in memory regardless of how I've terminated them. Eventually this causes an out of memory exception, despite the fact that I should only have a handful of active actors at any given time.
I've used:
self.tell(PoisonPill, self)
and:
context.stop(self)
to try and destroy the actors. Any ideas?
Edit: Here's a bit more to flesh out what I'm trying to do. The program opens up and spawns ten actors.
val system = ActorSystem("system")
(1 to 10) foreach { x =>
Entity.count += 1
system.actorOf(Props[Entity], name = Entity.count.toString())
}
Here's the code for the Entity:
class Entity () extends Actor {
Entity.entities += this
val id = Entity.count
import context.dispatcher
val tick = context.system.scheduler.schedule(0 millis, 100 millis, self, "update")
def receive = {
case "update" => {
Entity.entities.foreach(that => collide(that))
}
}
override def postStop() = tick.cancel()
def collide(that:Entity) {
if (!this.isBetterThan(that)) {
destroyMe()
spawnNew()
}
}
def isBetterThan() :Boolean = {
//computationally intensive logic
}
private def destroyMe(){
Entity.entities.remove(Entity.entities.indexOf(this))
self.tell(PoisonPill, self)
//context.stop(self)
}
private def spawnNew(){
val system = ActorSystem("system")
Entity.count += 1
system.actorOf(Props[Entity], name = Entity.count.toString())
}
}
object Entity {
val entities = new ListBuffer[Entity]()
var count = 0
}
Thanks #AmigoNico, you pointed me in the right direction. It turns out that neither
self.tell(PoisonPill, self)
nor
context.stop(self)
worked for timely Actor disposal; I switched the line to:
system.stop(self)
and everything works as expected.

Scala actors left hanging

I am spawning a small number of actors to fetch, process and save RSS feed items to a database. This is done through a main method of an object running on cron. I create these actors and dole out jobs to them as they complete the previous job assigned to them. My main class spawns a single actor, the one that doles out jobs to a pool of actors. Eventually the main method seems to hang. It doesn't exit, but execution halts on all the actors. My CTO believes the main is exiting before the actors complete their work and leaving them, but I am not convinced that's the case. I receive no success exit on main (no exit at all).
Essentially I'm wondering how to debug these actors, and what possible reason could cause this to happen. Will main exit before actors have completed their execution (and if it does, does that matter?) From what I can tell actors using receive are mapped 1-to-1 to threads, correct? Code is below. Please ask any follow-up questions, help is greatly appreciated. I know I may not have provided sufficient detail, I'm new to scala and actors and will update as needed.
object ActorTester {
val poolSize = 10
var pendingQueue :Set[RssFeed] = RssFeed.pendingQueue
def main(args :Array[String]) {
val manager = new SpinnerManager(poolSize, pendingQueue)
manager.start
}
}
case object Stop
class SpinnerManager(poolSize :Int = 1, var pendingQueue :Set[RssFeed]) extends Actor {
val pool = new Array[Spinner](poolSize)
override def start() :Actor = {
for (i <- 0 to (poolSize - 1)) {
val spinner = new Spinner(i)
spinner.start()
pool(i) = spinner
}
super.start
}
def act() {
for {
s <- pool
if (!pendingQueue.isEmpty)
} {
s ! pendingQueue.head
pendingQueue = pendingQueue.tail
}
while(true) {
receive {
case id :Int => {
if (!pendingQueue.isEmpty) {
pool(id) ! pendingQueue.head
pendingQueue = pendingQueue.tail
} else if ((true /: pool) { (done, s) => {
if (s.getState != Actor.State.Runnable) {
val exited = future {
s ! Stop
done && true
}
exited()
} else {
done && false
}
}}) {
exit
}
}
}
}
}
}
class Spinner(id :Int) extends Actor {
def act() {
while(true) {
receive {
case dbFeed :RssFeed => {
//process rss feed
//this has multiple network requests, to the original blogs, bing image api
//our instance of solr - some of these spawn their own actors
sender ! id
}
case Stop => exit
}
}
}
}
For one thing you're making a tiny but important mistake when you're folding left in order to determine whether all Spinner actors have "terminated" or not. What you should do is evaluate to done && true resp. done && false at the end of the if cases, but currently you just say true resp. false without respect to done.
For example, imagine having 4 Spinner actors where the first and second ones were Runnable, the third one not, and the fourth one Runnable again. In that case the result of your foldleft would be true in spite of the fact that the third actor hasn't finished yet. If you were using a logical &&, you'd get the correct result.
This is possibily also what causes your application to hang.
EDIT: There also was an issue wrt a race condition. The following code works now, hope it helps. Anyway, I was wondering, doesn't Scala's actor implementation automatically make use of worker threads?
import actors.Actor
import scala.collection.mutable.Queue
case class RssFeed()
case class Stop()
class Spinner(id: Int) extends Actor {
def act() {
loop {
react {
case dbFeed: RssFeed => {
// Process RSS feed
sender ! id
}
case Stop => exit()
}
}
}
}
class SpinnerManager(poolSize: Int, pendingQueue: Queue[RssFeed]) extends Actor {
val pool = Array.tabulate(poolSize)(new Spinner(_).start())
def act() {
for (s <- pool; if (!pendingQueue.isEmpty)) {
pendingQueue.synchronized {
s ! pendingQueue.dequeue()
}
}
loop {
react {
case id: Int => pendingQueue.synchronized {
if (!pendingQueue.isEmpty) {
Console println id
pool(id) ! pendingQueue.dequeue()
} else {
if (pool forall (_.getState != Actor.State.Runnable)) {
pool foreach (_ ! Stop)
exit()
}
}
}
}
}
}
}
object ActorTester {
def main(args: Array[String]) {
val poolSize = 10
val pendingQueue: Queue[RssFeed] = Queue.tabulate(100)(_ => RssFeed())
new SpinnerManager(poolSize, pendingQueue).start()
}
}
So after several days of debugging I've solved this issue. fotNelton's code suggestions were very helpful in doing so, so I've given him a vote. However, they didn't address the problem itself. What I've found is that if you are running this in a main method then if the parent actors exit before their child actors then the program will hang forever and never exit, still holding all of its memory. In the process of handling the RSS feed, a Fetcher would spawn actors and send them messages to do things involving network requests. These actors need to complete their work before the parent actor quits. Fetcher wouldn't wait for these actors to finish though, once he sent the message he would just move on. So he would tell manager he was finished before his child actors had finished all their work. To deal with this, one option would be to use futures and wait until the actors are done (pretty slow). My solution was to create services accessible via URL (POST to a service that has an actor waiting to react). The service would respond right away, and send a message to its own actor. Thus the actors can quit once they send the request to the service, and don't need to spawn any other actors.
object FeedFetcher {
val poolSize = 10
var pendingQueue :Queue[RssFeed] = RssFeed.pendingQueue
def main(args :Array[String]) {
new FetcherManager(poolSize, pendingQueue).start
}
}
case object Stop
class FetcherManager(poolSize :Int = 1, var pendingQueue :Queue[RssFeed]) extends Actor {
val pool = new Array[Fetcher](poolSize)
var numberProcessed = 0
override def start() :Actor = {
for (i <- 0 to (poolSize - 1)) {
val fetcher = new Fetcher(i)
fetcher.start()
pool(i) = fetcher
}
super.start
}
def act() {
for {
f <- pool
if (!pendingQueue.isEmpty)
} {
pendingQueue.synchronized {
f ! pendingQueue.dequeue
}
}
loop {
reactWithin(10000L) {
case id :Int => pendingQueue.synchronized {
numberProcessed = numberProcessed + 1
if (!pendingQueue.isEmpty) {
pool(id) ! pendingQueue.dequeue
} else if ((true /: pool) { (done, f) => {
if (f.getState == Actor.State.Suspended) {
f ! Stop
done && true
} else if (f.getState == Actor.State.Terminated) {
done && true
} else {
false
}
}}) {
pool foreach { f => {
println(f.getState)
}}
println("Processed " + numberProcessed + " feeds total.")
exit
}
}
case TIMEOUT => {
if (pendingQueue.isEmpty) {
println("Manager just woke up from timeout with all feeds assigned.")
pool foreach { f => {
if (f.getState == Actor.State.Suspended) {
println("Sending Stop to Fetcher " + f.id)
f ! Stop
}
}}
println("Checking state of all Fetchers for termination.")
if ((true /: pool) { (done, f) => {
done && (f.getState == Actor.State.Terminated)
}}) {
exit
}
}
}
}
}
}
}
class Fetcher(val id :Int) extends Actor {
var feedsIveDone = 0
def act() {
loop {
react {
case dbFeed :RssFeed => {
println("Fetcher " + id + " starting feed")
//process rss feed here
feedsIveDone = feedsIveDone + 1
sender ! id
}
case Stop => {
println(id + " exiting")
println(feedsIveDone)
exit
}
}
}
}