I am new to scala and actors. I need to implement such hypothetical situation:
Server wait for messages, if it does not get any in say 10s period of time, it sends message to the Client. Otherwise it receives messages incoming. If it is inside processing some message and another message comes, it needs to be queued (I suppose that is done automatically by scala actors).
The second problem I am encountering is Sleeping. I need the actor to sleep for some constant period of time when it receives the message. But on the other hand I can't block, as I want incoming messages to be queued for further processing.
How about this?
loop {
reactWithin(10000) {
case TIMEOUT => // send message to client
case work => // do work
}
}
Daniel has provided a better answer to the no-input condition part of the question. So I've edited out my inferior solution.
As to the delayed response part of the question, the message queue doesn't block while an actor sleeps. It can just sleep and messages will still accumulate.
However, if you want a fixed delay from when you receive a message to when you process it, you can, for example, create an actor that works immediately but wraps the message in a request for a delay:
case class Delay(when: Long, what: Any) { }
// Inside class DelayingActor(workingActor: Actor)
case msg => workingActor ! Delay(delayValue + System.currentTimeMillis , msg)
Then, the working actor would
case Delay(t,msg) =>
val t0 = System.currentTimeMillis
if (t>t0) Thread.sleep( t - t0 )
msg match {
// Handle message
}
Related
In my project, I created UserRepositoryActor which create their own router with 10 UserRepositoryWorkerActor instances as routee, see hierarchy below:
As you see, if any error occur while fetching data from database, it will occur at worker.
Once I want to fetch user from database, I send message to UserRepositoryActor with this command:
val resultFuture = userRepository ? FindUserById(1)
and I set 10 seconds for timeout.
In case of network connection has problem, UserRepositoryWorkerActor immediately get ConnectionException from underlying database driver and then (what I think) router will restart current worker and send FindUserById(1) command to other worker that available and resultFuture will get AskTimeoutException after 10 seconds passed. Then some time later, once connection back to normal, UserRepositoryWorkerActor successfully fetch data from database and then try to send result back to the caller and found that resultFuture was timed out.
I want to propagate error from UserRepositoryWorkerActor up to the caller immediately after exception occur, so that will prevent resultFuture to wait for 10 seconds and stop UserRepositoryWorkerActor to try to fetch data again and again.
How can I do that?
By the way, if you have any suggestions to my current design, please suggest me. I'm very new to Akka.
Your assumption about Router resending the message is wrong. Router has already passed the message to routee and it doesnt have it any more.
As far as ConnectionException is concerned, you could wrap in a scala.util.Try and send response to sender(). Something like,
Try(SomeDAO.getSomeObjectById(id)) match {
case Success(s) => sender() ! s
case Failure(e) => sender() ! e
}
You design looks correct. Having a router allows you to distribute work and also to limit number of concurrent workers accessing the database.
Option 1
You can make your router watch its children and act accordingly when they are terminated. For example (taken from here):
import akka.routing.{ ActorRefRoutee, RoundRobinRoutingLogic, Router }
class Master extends Actor {
var router = {
val routees = Vector.fill(5) {
val r = context.actorOf(Props[Worker])
context watch r
ActorRefRoutee(r)
}
Router(RoundRobinRoutingLogic(), routees)
}
def receive = {
case w: Work =>
router.route(w, sender())
case Terminated(a) =>
router = router.removeRoutee(a)
val r = context.actorOf(Props[Worker])
context watch r
router = router.addRoutee(r)
}
}
In your case you can send some sort of a failed message from the repository actor to the client. Repository actor can maintain a map of worker ref to request id to know which request failed when worker terminates. It can also record the time between the start of the request and actor termination to decide whether it's worth retrying it with another worker.
Option 2
Simply catch all non-fatal exceptions in your worker actor and reply with appropriate success/failed messages. This is much simpler but you might still want to restart the worker to make sure it's in a good state.
p.s. Router will not restart failed workers, neither it will try to resend messages to them by default. You can take a look at supervisor strategy and Option 1 above on how to achieve that.
In the receive method of a persistent actor, I receive a bunch a events I want to persist, and only after all events are persisted, update again my state. How can I do that?
def receive: Receive = {
...
case NewEvents(events) =>
persist(events) { singleEvent =>
// Update state using this single event
}
// After every events are persisted, do one more thing
}
Note that the persist() call is not blocking so I cannot put my code just after that.
Update: Why I need this
These new events come from an external web-service. My persistent actor needs to store in its state the last event id, which will be used for the subsequent ws call when it receives a command. The thing is that these commands may come concurrently, so I need some kind of locking system:
Received ws call command: stash next commands until this one finishes (that is, to sum up, a boolean)
Received responses from ws: store them, update the state and save the last id, execute another, single ws call for all commands that are in the stash (I'm keeping the command senders to be able to respond to them all once done) otherwise don't stash commands anymore.
I haven't tried defer yet, my initial solution was to send myself a PersistEventsDone message. It works because the persist method will stash all incoming messages until all the events handlers are executed. If another command came in the process, it doesn't really matter if it's before or after PersistEventsDone:
def receive: Receive = {
...
case PersistEventsDone =>
...
case NewEvents(events) =>
persist(events) { singleEvent =>
// Update state using this single event
}
self ! PersistEventsDone
}
defer is a bit weird in my case because it requires an event I don't need. But it still looks more natural than my solution.
I've made following code:
package com.star.wars
import akka.actor._
import akka.actor.SupervisorStrategy._
import akka.util.duration._
object Test extends App {
case object Kill
case object Create
class Luke extends Actor {
var x: Int = 0
println("Luke here")
Thread.sleep(1000)
println("Luke here2")
def receive = {
case Kill => 1/0// context.stop(self)
case msg: String => {
x += 1
println(x + msg)
}
}
}
class Vader extends Actor {
println("Vader here")
override val supervisorStrategy =
OneForOneStrategy(maxNrOfRetries = 10, withinTimeRange = 1 minute) {
case _: ArithmeticException => Restart
case _: NullPointerException => Restart
case _: IllegalArgumentException => Restart
case _: Exception => Restart
}
def receive = {
case Create => context.actorOf(Props(new Luke), name = "Luke")
case Kill => {
val luke = context.actorFor("/user/Vader/Luke")
luke ! "Pre hi there"
luke ! Kill
luke ! "Post hi there"
println("Pre -> Kill -> Post sent to Luke")
}
}
}
val system = ActorSystem("MySystem")
val vader = system.actorOf(Props(new Vader), name = "Vader")
vader ! Create
vader ! Kill
println("Create -> Kill sent to Vader")
}
Purpose of this code, is to prove, that while Luke is restarting, his dead letter box can receive messages, and when Luke comes online again, he can receive messages sent to him while he was absent.
Output seems ok. It is a proof in a way:
Create -> Kill sent to Vader
Vader here
Luke here
Pre -> Kill -> Post sent to Luke
Luke here2
1Pre hi there
[ERROR] [01/12/2015 00:32:02.74] [MySystem-akka.actor.default-dispatcher-3] [akka://MySystem/user/Vader/Luke] / by zero
java.lang.ArithmeticException: / by zero
at com.sconysoft.robocode.Test$Luke$$anonfun$receive$1.apply(test.scala:21)
at com.sconysoft.robocode.Test$Luke$$anonfun$receive$1.apply(test.scala:20)
at akka.actor.Actor$class.apply(Actor.scala:318)
at com.sconysoft.robocode.Test$Luke.apply(test.scala:12)
at akka.actor.ActorCell.invoke(ActorCell.scala:626)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:197)
at akka.dispatch.Mailbox.run(Mailbox.scala:179)
at akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(AbstractDispatcher.scala:516)
at akka.jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:259)
at akka.jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:975)
at akka.jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1479)
at akka.jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
Luke here
Luke here2
1Post hi there
However, I am not sure is it always truth. I can't find it in akka documentation, so is my reasoning ok ? Is dead letter box reusable in akka (after actor restart) ?
Btw. How can I handle Luke's context.stop(self) in Vader's supervisorStrategy ?
This is not connected to dead letters, see Message Delivery Reliability. In Akka, when using In-JVM-messaging - you have guarantee that message will be delivered with high-probability in most cases (restarting after exception is one of those). Messages that could not be delivered (like messages to voluntary stopped or never existed actor) are going to the DeadLetters box, but you should subscribe to them explicitly, which is not you've got here. You had just received the messages from your own actor's mailbox (as box wasn't removed during restarting - only actor's instance). You need to explicitly subscribe to the corresponding Event Stream to watch deadLetters.
You can't process context.stop(self) inside supervisorStrategy as it's voluntary termination (which actually will cause messages going to deadLetters) not exceptional situation (failure). So 1/0 and context.stop(self) are very different. For listening child's lifecycle see - What Lifecycle Monitoring Means
For example, let's see what if you really put context.stop(self) into the code instead of 1/0:
Luke here
Pre -> Kill -> Post sent to Luke
Luke here2
1Pre hi there
[INFO] [01/12/2015 09:20:37.325] [MySystem-akka.actor.default-dispatcher-4] [akka://MySystem/user/Vader/Luke] Message [java.lang.String] from Actor[akka://MySystem/user/Vader#-1749418461] to Actor[akka://MySystem/user/Vader/Luke#-1436540331] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
So that's where ("blabla was not delivered" in logging) deadLetters mailbox is used.
Anyway, any delivery (including dead-letters as it's just a synthetic actor) is based on best effort principle for in-JVM messaging, so there is no absolute guarantees:
The Akka test suite relies on not losing messages in the local context
(and for non-error condition tests also for remote deployment),
meaning that we actually do apply the best effort to keep our tests
stable. A local tell operation can however fail for the same reasons
as a normal method call can on the JVM:
StackOverflowError
OutOfMemoryError
other VirtualMachineError
In addition, local sends can fail in Akka-specific ways:
if the mailbox does not accept the message (e.g. full BoundedMailbox)
if the receiving actor fails while processing the message or is already terminated
While the first is clearly a matter of configuration the second
deserves some thought: the sender of a message does not get feedback
if there was an exception while processing, that notification goes to
the supervisor instead. This is in general not distinguishable from a
lost message for an outside observer.
In case of network-messaging you have no delivery guarantee at all. Luke could never know who is his father. Why? Because it's faster and actually Nobody Needs Reliable Messaging:
The only meaningful way for a sender to know whether an interaction
was successful is by receiving a business-level acknowledgement
message, which is not something Akka could make up on its own (neither
are we writing a “do what I mean” framework nor would you want us to).
A project I'm working on requires the reading of messages from SQS, and I decided to use Akka to distribute the processing of these messages.
Since SQS is support by Camel, and there is functionality built in for use in Akka in the Consumer class, I imagined it would be best to implement the endpoint and read messages this way, though I had not seen many examples of people doing so.
My problem is that I cannot poll my queue quickly enough to keep my queue empty, or near empty. What I originally thought was that I could get a Consumer to receive messages over Camel from SQS at a rate of X/s. From there, I could simply create more Consumers to get up to the rate at which I needed messages processed.
My Consumer:
import akka.camel.{CamelMessage, Consumer}
import akka.actor.{ActorRef, ActorPath}
class MyConsumer() extends Consumer {
def endpointUri = "aws-sqs://my_queue?delay=1&maxMessagesPerPoll=10&accessKey=myKey&secretKey=RAW(mySecret)"
var count = 0
def receive = {
case msg: CamelMessage => {
count += 1
}
case _ => {
println("Got something else")
}
}
override def postStop(){
println("Count for actor: " + count)
}
}
As shown, I've set delay=1 as well as &maxMessagesPerPoll=10 to improve the rate of messages, but I'm unable to spawn multiple consumers with the same endpoint.
I read in the docs that By default endpoints are assumed not to support multiple consumers. and I believe this holds true for SQS endpoints as well, as spawning multiple consumers will give me only one consumer where after running the system for a minute, the output message is Count for actor: x instead of the others which output Count for actor: 0.
If this is at all useful; I'm able to read approximately 33 messages/second with this current implementation on the single consumer.
Is this the proper way to be reading messages from an SQS queue in Akka? If so, is there way I can get this to scale outward so that I can increase my rate of message consumption closer to that of 900 messages/second?
Sadly Camel does not currently support parallel consumption of messages on SQS.
http://camel.465427.n5.nabble.com/Amazon-SQS-listener-as-multi-threaded-td5741541.html
To address this I've written my own Actor to poll batch messages SQS using the aws-java-sdk.
def receive = {
case BeginPolling => {
// re-queue sending asynchronously
self ! BeginPolling
// traverse the response
val deleteMessageList = new ArrayList[DeleteMessageBatchRequestEntry]
val messages = sqs.receiveMessage(receiveMessageRequest).getMessages
messages.toList.foreach {
node => {
deleteMessageList.add(new DeleteMessageBatchRequestEntry(node.getMessageId, node.getReceiptHandle))
//log.info("Node body: {}", node.getBody)
filterSupervisor ! node.getBody
}
}
if(deleteEntryList.size() > 0){
val deleteMessageBatchRequest = new DeleteMessageBatchRequest(queueName, deleteMessageList)
sqs.deleteMessageBatch(deleteMessageBatchRequest)
}
}
case _ => {
log.warning("Unknown message")
}
}
Though I'm not certain if this is the best implementation, and it could of course be improved upon so that requests are not constantly hitting an empty queue, it does suit my current needs of being able to poll messages from the same queue.
Getting about 133 (messages/second)/actor from SQS with this.
Camel 2.15 supports concurrentConsumers, though not sure how useful this is in that I don't know if akka camel support 2.15 and I do not know if having one Consumer actor makes a difference even if there are multiple consumers.
I need to write a client that continuously polls a web server for commands. A response from the server indicates that a command is available (in which case the response contains the command) or an instruction that no command is available, and you should fire off a new request for incoming commands.
I'm trying to figure out how to do it using spray-client and Akka, and I can think of ways to do it, but none of them look like they're the idiomatic way to get it done. So the question is:
what's the most sensible way to have a couple of threads poll the same web server for incoming commands and hand the commands off to an actor?
This example uses spray-client, scala futures, and Akka scheduler.
Implementation varies depending on desired behavior (execute many requests in parallel at the same time, execute in different intervals, send responses to one actor to process one response at a time, send responses to many actors to process in parallel... etc).
This particular example shows how execute many requests in parallel at the same time, and then do something with each result as it completes, without waiting for any other requests that were fired off at the same time to complete.
The code below will execute two HTTP requests every 5 seconds to 0.0.0.0:9000/helloWorld and 0.0.0.0:9000/goodbyeWorld in parallel.
Tested in Scala 2.10, Spray 1.1-M7, and Akka 2.1.2:
Actual scheduling code that handles periodic job execution:
// Schedule a periodic task to occur every 5 seconds, starting as soon
// as this schedule is registered
system.scheduler.schedule(initialDelay = 0 seconds, interval = 5 seconds) {
val paths = Seq("helloWorld", "goodbyeWorld")
// perform an HTTP request to 0.0.0.0:9000/helloWorld and
// 0.0.0.0:9000/goodbyeWorld
// in parallel (possibly, depending on available cpu and cores)
val retrievedData = Future.traverse(paths) { path =>
val response = fetch(path)
printResponse(response)
response
}
}
Helper methods / boilerplate setup:
// Helper method to fetch the body of an HTTP endpoint as a string
def fetch(path: String): Future[String] = {
pipeline(HttpRequest(method = GET, uri = s"/$path"))
}
// Helper method for printing a future'd string asynchronously
def printResponse(response: Future[String]) {
// Alternatively, do response.onComplete {...}
for (res <- response) {
println(res)
}
}
// Spray client boilerplate
val ioBridge = IOExtension(system).ioBridge()
val httpClient = system.actorOf(Props(new HttpClient(ioBridge)))
// Register a "gateway" to a particular host for HTTP requests
// (0.0.0.0:9000 in this case)
val conduit = system.actorOf(
props = Props(new HttpConduit(httpClient, "0.0.0.0", 9000)),
name = "http-conduit"
)
// Create a simple pipeline to deserialize the request body into a string
val pipeline: HttpRequest => Future[String] = {
sendReceive(conduit) ~> unmarshal[String]
}
Some notes:
Future.traverse is used for running futures in parallel (ignores order). Using a for comprehension on a list of futures will execute one future at a time, waiting for each to complete.
// Executes `oneThing`, executes `andThenAnother` when `oneThing` is complete,
// then executes `finally` when `andThenAnother` completes.
for {
oneThing <- future1
andThenAnother <- future2
finally <- future3
} yield (...)
system will need to be replaced with your actual Akka actor system.
system.scheduler.schedule in this case is executing an arbitrary block of code every 5 seconds -- there is also an overloaded version for scheduling messages to be sent to an actorRef.
system.scheduler.schedule(
initialDelay = 0 seconds,
frequency = 30 minutes,
receiver = rssPoller, // an actorRef
message = "doit" // the message to send to the actorRef
)
For your particular case, printResponse can be replaced with an actor send instead: anActorRef ! response.
The code sample doesn't take into account failures -- a good place to handle failures would be in the printResponse (or equivalent) method, by using a Future onComplete callback: response.onComplete {...}
Perhaps obvious, but spray-client can be replaced with another http client, just replace the fetch method and accompanying spray code.
Update: Full running code example is here:
git clone the repo, checkout the specified commit sha, $ sbt run, navigate to 0.0.0.0:9000, and watch the code in the console where sbt run was executed -- it should print Hello World!\n'Goodbye World! OR Goodbye World!\nHelloWorld! (order is potentially random because of parallel Future.traverse execution).
You can use HTML5 Server-Sent Events. It is implemented in many Scala frameworks. For example in xitrum code looks like:
class SSE extends Controller {
def sse = GET("/sse") {
addConnectionClosedListener {
// The connection has been closed
// Unsubscribe from events, release resources etc.
}
future {
respondEventSource("command1")
//...
respondEventSource("command2")
//...
}
}
SSE is pretty simple and can be used in any software not only in browser.
Akka integrated in xitrum and we use it in similar system. But it uses netty for async server it is also good for processing thousands of request in 10-15 threads.
So in this way your client will keep connection with server and reconnect when connection will be broken.