I'm new to Scala Actors. What I plan to build is an application that has several cartridges that each do a specific http call and retrieve+persist some info periodically. Robustness is what matters the most. So far these are the ways I've thought of:
Build the app around a TimerTask,
extend cartridges from Actor and
call their .act s periodically (or
should I send them messages instead?
what's the difference?)
Extend from Actor and use Timeouts
to periodically run them.
Can someone shed some light on the differences?
Scala Actors will be merged with Akka so take a look at http://akka.io,
You can use Akka's "Scheduler" to schedule messages to be sent to actors at certain intervals, it's all in the docs:
http://akka.io/docs/akka/1.1.3/
Hope this helps,
Cheers,
√
Related
I am wondering why there is no non-blocking support via simple callbacks or Java's CompletableFuture or Scala Futures in the Kafka Stream API.
I do understand that ordering in a partition needs to be maintained, but across partitions I do not see the reason to achieve ordering by blocking an expensive resource: a thread.
i.e. when I let my Kafka Streams app with a call to an external service, e.g. in mapValues run on 1 server and I have more than thousands of partitions, I will probably lock up the machine because all threads are blocked. Having some API method like mapValuesAsync() would be nice here, wouldn't it?
Also just imagine on Kafka Stream App with doing several blocking operations in it's flow one would need way less partitions per each topic to run into the problem. Wasting threads doesn't look like a nice API design here.
Is there any support planned for this? Or do I oversee something here?
Async processing is generally hard in stream processing. It's not just about ordering, but also about fault-tolerance, tracking progress etc.
It's not impossible to support though and in fact there is already a design proposal for it: https://cwiki.apache.org/confluence/display/KAFKA/KIP-408%3A+Add+Asynchronous+Processing+To+Kafka+Streams
Feel free to help building this feature!
We know Akka is one implementation of actor pattern. Without Akka, I usually implement a simple actor pattern using ThreadPool+BlockingQueue. So the message is offered into the queue, and the works(actors) take the message from the Queue, then do what they should do. Of course, this kind of implementation can be only in just ONE process.
So as to in one process,
What's the essential difference between these two(Akka vs.
ThreadPool+BlockingQueue)
Moreover, what's the difference between actor pattern and producer-consumer model?
Actor model is indeed quite similar to producer-consumer model (P-C).
However, if you use a blocking queue with P-C your application won't be completely non-blocking and asynchronous. The promise of actor model and Akka is that all messages are sent asynchronously and don't block the sender.
Another aspect of it is managing these queues gets quite cumbersome once you have many consumers and producers. With actors you simply send a message and don't have to think about these low level details. Under the hood Akka will keep a message queue aka mailbox per actor with a dispatcher assigning actors to the thread pool to process those messages.
It's much easier to use Akka to achieve highly performant and resilient application than coding it yourself. You get fault tolerance, resource management, location transparency, routing, distributed, async processing, hierarchical supervision out of the box. Not to mention other frameworks and libraries leveraging these features to give you even more (reactive streams, akka http, etc). There are lot's of patterns developed for you already there, so why bother with your own.
I would like to create an app who integrate our (enterprise) specific tools with Slack. For that I can foreseen some bots who send events to Slack and a few commands who trigger actions.
I would like to use Akka because it is a buzzword and seems nice but don't have any arguments in favor of it. (It is not a problem since I will develop this app alone on my freetime).
But I don't have any experience on how to create an "Actor based application". I already have 3 actors, two to collect Events and one to publish those Events to Slack. Each collector are triggered by a timer, they hold a reference to the publisher and send message to him... that works..
For the commands part I din't have anything but can imagine a listener (Play http controller) who convert Slack requests to message and send them to one Actor. Ideally, I would like to decouple this listener from the Actor who will handle the command.
I would like to have some advices on how to develop this kind of applications where I have actors to collect information on a time basis and other to react to messages.
Thanks.
For time based activity, i.e. "ticks" or "heartbeats", the documentation demonstrates a very good pattern for sending messages to Actors periodically.
For your other need, Actors responding to messages as they come in, this is exactly the purpose of the Actor's receive method. From the documentation:
Actors are objects which encapsulate state and behavior, they
communicate exclusively by exchanging messages which are placed into
the recipient’s mailbox. In a sense, actors are the most stringent
form of object-oriented programming, but it serves better to view them
as persons: while modeling a solution with actors, envision a group of
people and assign sub-tasks to them, arrange their functions into an
organizational structure and think about how to escalate failure (all
with the benefit of not actually dealing with people, which means that
we need not concern ourselves with their emotional state or moral
issues). The result can then serve as a mental scaffolding for
building the software implementation.
Using the event bus mechanism between actors in the same ActorSystem is straight-forward, but I was wondering if there was a sanctioned method for doing so between:
Actors in different ActorSystems in the same JVM
Actors in different JVMs (via remoting)
Assuming that I know the paths to the actors is fine, but if there was a commonly used mechanism to discover those kinds of things as well, I'd love to hear about it.
I think in this case you need to look for distributed publish-subscribe on a cluster, supposing you want to subscribe actors to events, without awareness of the location of the actors. This link may prove useful.
This is a note from the official Akka documentation:
The event stream is a local facility, meaning that it will not
distribute events to other nodes in a clustered environment (unless
you subscribe a Remote Actor to the stream explicitly). If you need to
broadcast events in an Akka cluster, without knowing your recipients
explicitly (i.e. obtaining their ActorRefs), you may want to look
into: Distributed Publish Subscribe in Cluster.
I have a back-end Scala application that needs to integrate with RabbitMQ. The back-end Scala app executes long running tasks asynchronously. Messages to execute the tasks are queued into RabbitMQ by a web client. The back-end application then consumes each of these messages, executing the corresponding long-running tasks.
Should Scala app directly consume the message from RabbitMQ and simply have the corresponding tasks be processed using Futures? Or is it better to use Akka Actors to receive these messages from RabbitMQ, and then execute the long-running tasks?
What are the pro's and con's of each approach?
Futures sound like a simpler approach for your use case combined with the RabbitMQ Java client.
My model for choosing actors v. futures is: prefer futures, switching to actors when I feel I have a good use case for them (see Good use case for Akka for some examples). For example, if you were trying to divide-and-conquer the batch workloads (as the linked answer states), actors may serve your purposes well.
Use the RabbitMQ Java examples below as a starting point, modifying to do work in futures so that the thread polling the work queue is not blocked. I included links to both work queue and RPC examples in case you need to return some response (RabbitMQ is good at this case as it has a concept of correlationId built in).
Java RabbitMQ examples:
Work Queues
Remote procedure call (RPC)