I have a master Actor responsible for initializing some worker actors (there are two types of worker actors, namely, ParamServer actor and DataShard actor). For example, if I initiated 20 datashard actors via ClusterSharding(system).start(_,_,_,_,_) and after that I want to send some message to all datashard actors (say case object ReadyToProcess). I read that I can send messages to entities in Akka Cluster Shard via local ShardRegion(system).shardRegion(_). Is local shardRegion(_) will send to all datashards or just one. How can I send msgs to all datashard actors?
The master class given be:
class Master(ports: Seq[String],
dataSet: Seq[Example],
dataPerReplica: Int,
layerDimensions: Seq[Int],
activation: ActivationFunction,
activationFunctionDer: ActivationFunction,
learningRate: Double) extends Actor with ActorLogging {
val dataShards = dataSet.grouped(dataPerReplica).toSeq
val numLayers = layerDimensions.size
var numShardsFinished = 0
ports foreach { port =>
val config = ConfigFactory.parseString("akka.remote.netty.tcp.port=" + port).withFallback(ConfigFactory.load())
val clusterSystem = ActorSystem("ClusterSystem", config)
val paramServerRegions: Array[ActorRef] = new Array[ActorRef](numLayers - 1)
for (i <- 0 to numLayers - 2) {
paramServerRegions(i) = ClusterSharding(clusterSystem).start(
typeName = ParamServer.shardName,
entityProps = ParamServer.props(i, dataShards.size, learningRate, NeuralNetworkOps.randomMatrix(layerDimensions(i + 1), layerDimensions(i) + 1)),
settings = ClusterShardingSettings(clusterSystem),
extractEntityId = ParamServer.extractEntityId,
extractShardId = ParamServer.extractShardId
)
}
//create actors for each data shard/replica. Each replica needs to know about all parameter shards because they will
//be reading from them and updating them
val dataShardRegions: Array[ActorRef] = new Array[ActorRef](dataShards.size)
for (i <- 0 to dataShards.size) {
dataShardRegions(i) = ClusterSharding(clusterSystem).start(
typeName = DataShard.shardName,
entityProps = DataShard.props(i, clusterSystem, dataShards(i), activation, activationFunctionDer, paramServerRegions),
settings = ClusterShardingSettings(clusterSystem),
extractEntityId = ParamServer.extractEntityId,
extractShardId = ParamServer.extractShardId
)
}
}
def receive: Receive = {
case Start => {
val shardRegionSender = ClusterSharding(context.system).shardRegion(DataShard.shardName)
println("Tomosha boshlandi")
shardRegionSender ! ReadyToProcess
}
case ShardDone(id) => {
numShardsFinished+=1
log.info("")
if (numShardsFinished == dataShards.size) {
context.parent ! JobDone
context.stop(self)
}
}
}
}
Related
I've been trying to set-up some instrumentation for Akka streams. Got it working, but, even though I named all my Flows that are part of the streams, I still get this sort of names in the metrics: flow-0-0-unknown-operation
A simple example of what I'm trying to do:
val myflow = Flow[String].named("myflow").map(println)
Source.via(myflow).to(Sink.ignore).run()
I basically want to see the metrics for the Actor that gets created for "myflow", with a proper name.
Is this even possible? Am I missing something?
I was having this challenge in my project and I solved by using Kamon + Prometheus. However I had to create an Akka Stream Flow which I can set its name metricName and export the metric values from it using val kamonThroughputGauge: Metric.Gauge.
class MonitorProcessingTimerFlow[T](interval: FiniteDuration)(metricName: String = "monitorFlow") extends GraphStage[FlowShape[T, T]] {
val in = Inlet[T]("MonitorProcessingTimerFlow.in")
val out = Outlet[T]("MonitorProcessingTimerFlow.out")
Kamon.init()
val kamonThroughputGauge: Metric.Gauge = Kamon.gauge("akka-stream-throughput-monitor")
override def createLogic(inheritedAttributes: Attributes): GraphStageLogic = new TimerGraphStageLogic(shape) {
// mutable state
var open = false
var count = 0
var start = System.nanoTime
setHandler(in, new InHandler {
override def onPush(): Unit = {
try {
push(out, grab(in))
count += 1
if (!open) {
open = true
scheduleOnce(None, interval)
}
} catch {
case e: Throwable => failStage(e)
}
}
})
setHandler(out, new OutHandler {
override def onPull(): Unit = {
pull(in)
}
})
override protected def onTimer(timerKey: Any): Unit = {
open = false
val duration = (System.nanoTime - start) / 1e9d
val throughput = count / duration
kamonThroughputGauge.withTag("name", metricName).update(throughput)
count = 0
start = System.nanoTime
}
}
override def shape: FlowShape[T, T] = FlowShape[T, T](in, out)
}
Then I created a simple stream that uses the MonitorProcessingTimerFlow to export the metrics:
implicit val system = ActorSystem("FirstStreamMonitoring")
val source = Source(Stream.from(1)).throttle(1, 1 second)
/** Simulating workload fluctuation: A Flow that expand the event to a random number of multiple events */
val flow = Flow[Int].extrapolate { element =>
Stream.continually(Random.nextInt(100)).take(Random.nextInt(100)).iterator
}
val monitorFlow = Flow.fromGraph(new MonitorProcessingTimerFlow[Int](5 seconds)("monitorFlow"))
val sink = Sink.foreach[Int](println)
val graph = source
.via(flow)
.via(monitorFlow)
.to(sink)
graph.run()
with a proper configuration at application.conf:
kamon.instrumentation.akka.filters {
actors.track {
includes = [ "FirstStreamMonitoring/user/*" ]
}
}
I can see the throughput metrics on prometheus console with the name name="monitorFlow":
When I try to set bounded-mailbox for routees of a pool (RoundRobinPool) in configuration file, somehow Akka ignores the mailbox configuration.
Here is the configuration I use:
bounded-mailbox {
mailbox-type = "akka.dispatch.BoundedMailbox"
mailbox-capacity = 1
mailbox-push-timeout-time = 1s
}
akka.actor.deployment {
/singletestactor {
mailbox = bounded-mailbox
}
/groupedtestactor {
mailbox = bounded-mailbox
router = round-robin-pool
nr-of-instances = 5
}
}
And here is the test code:
object MailboxTest {
def main(args: Array[String]): Unit = {
val actorSystem = ActorSystem()
val singleTestActor = actorSystem.actorOf(Props[TestActor], "singletestactor")
for (i <- 1 to 10) {
singleTestActor ! Hello(i)
}
val groupedTestActor = actorSystem.actorOf(Props[TestActor].withRouter(FromConfig, "groupedtestactor")
for (i <- 1 to 1000) {
groupedTestActor ! Hello(i)
}
}
}
class TestActor extends Actor {
def receive = {
case Hello(i) => {
println(s"Hello($i) - begin!")
Thread.sleep(10000)
println(s"Hello($i) - end!")
}
}
}
case class Hello(i: Int)
Am I doing something wrong, or there is no way to define mailbox for routees?
You need to add mailbox.requirements configuration in application.conf;
akka.actor.mailbox.requirements {
"akka.dispatch.BoundedMessageQueueSemantics" = bounded-mailbox
}
Then need to extend TestActor like this;
class TestActor extends Actor with RequiresMessageQueue[BoundedMessageQueueSemantics]
See the documentation here for mailbox configuration.
I also created round robin pool like this;
val groupedTestActor = actorSystem.actorOf(FromConfig.props(Props[TestActor]), "groupedtestactor")
I'm currently developing an application using Play-Scala framework and akka actor to send email when a CRUD action with database happen. I have a problem when restart server.
Any way to reload akka scheduler when Play framework restart or persist akka scheduler and when restart app it will run scheduler. This is my code for example.
#Singleton
class SendEmailSaveDraftActor #Inject()(system: ActorSystem,
config: Configuration,
mailService: MailService)(implicit exec: ExecutionContext) extends Actor {
val userSchedules: mutable.HashMap[String, Seq[Cancellable]] = mutable.HashMap()
private val emailAdmin = config.getString("email.admin").getOrElse("")
private val interval24Hours = config.getInt("batch.saveDarf24.extractIntervalHours").getOrElse(1)
private val interval72Hours = config.getInt("batch.saveDarf72.extractIntervalHours").getOrElse(1)
private val interval14Days = config.getInt("batch.saveDarf14.extractIntervalDays").getOrElse(1)
def receive = {
case s: SendEmailSaveDrafModel =>
userSchedules.get(s.userName) match {
case Some(schedules) =>
schedules.map(_.cancel())
userSchedules.update(s.userName, getSchedules(s))
case None =>
userSchedules += (s.userName -> getSchedules(s))
}
case userName: String =>
userSchedules.get(userName) match {
case Some(schedules) =>
schedules.map(_.cancel())
case None =>
}
}
def getSchedules(s: SendEmailSaveDrafModel): Seq[Cancellable] = {
val schedule1 = system.scheduler.scheduleOnce(5 minutes) {
mailService Send EmailSaveDraft24Hours(to = Seq(s.to), bccEmails = Seq(emailAdmin),id = s.id, orderNo = s.orderNo,designId = s.designId)
}
val schedule2 = system.scheduler.scheduleOnce(10 minutes) {
mailService Send EmailSaveDraft72Hours(to = Seq(s.to), bccEmails = Seq(emailAdmin),id = s.id, orderNo = s.orderNo,designId = s.designId)
}
val schedule3 = system.scheduler.scheduleOnce(15 minutes) {
mailService Send EmailSaveDraft14Days(to = Seq(s.to), bccEmails = Seq(emailAdmin),id = s.id, orderNo = s.orderNo,designId = s.designId)
}
Seq(schedule1, schedule2, schedule3)
}
}
I am using akka scheduler as shown in below code.
Each user can login to my Application and can create scheduler for background task.I am using Akka scheduler for this purpose.
public Cancellable buildScheduler(String actorName, SchedulerActorMessage message, long interval, TimeUnit timeUnit, long initialDelay, String actorMapKey) {
ActorRef daemonRef = actorSystem.actorOf(Props.create(SchedulerActor.class), actorName);
Cancellable cancellableActor = actorSystem.scheduler().schedule(FiniteDuration.apply(initialDelay, timeUnit),
FiniteDuration.apply(interval, timeUnit), daemonRef, message,
actorSystem.dispatcher(), ActorRef.noSender());
actorMap.put(actorMapKey, cancellableActor);
return cancellableActor;
}
I am storing the necessary information in the DB and when application start, I am staring the scheduler again in the global class.
I'm trying to write a simple matrix multiplication program with concurrent processing using Scala and Akka actors. I've not even written 10% of the code and I'm running into trouble. I created two actors - master and worker. I'm trying to communicate between them but its runs into an infinite loop. Any suggestions are really appreciated. As you can see, the code below does nothing, it prints 2 10X10 matrices in the master, after that the worker is called. But the worker's workDone message never comes back to the master. I also suspect this has to do something with a warning I'm getting:
patterns after a variable pattern cannot match (inside receive of master for case "masterSend")
import akka.actor.{ActorRef, Actor, ActorSystem, Props}
import scala.Array._
import scala.util.Random
case object masterSend
case object workSend
case object workDone
object MatrixMultiply {
val usage = """
Usage: MainStart <matrix-dimension> <high-value>
"""
def main(args: Array[String]) {
if (args.length != 2) {
println(usage)
System.exit(1)
}
val Dim = args(0).toInt
val Max = args(1).toInt
val system = ActorSystem("ComputeSystem")
val worker = system.actorOf(Props[Worker], name = "worker")
val master = system.actorOf(Props(new Master(Dim, Max, worker)), name = "master")
master ! masterSend
}
class Master(Dim: Int, Max: Int, worker : ActorRef) extends Actor {
def receive = {
case masterSend =>
val r = new Random(34636)
val matrixA = ofDim[Int](Dim,Dim)
val matrixB = ofDim[Int](Dim,Dim)
println("Matrix A: ")
for (i <- 0 to Dim - 1) {
for (j <- 0 to Dim - 1) {
matrixA(i)(j) = r.nextInt(Max)
print(matrixA(i)(j) + " ")
}
println()
}
r.setSeed(23535)
println("Matrix B: ")
for (i <- 0 to Dim - 1) {
for (j <- 0 to Dim - 1) {
matrixB(i)(j) = r.nextInt(Max)
print(matrixB(i)(j) + " ")
}
println()
}
worker ! workSend
case workDone =>
println("Work was done!!")
context.system.shutdown()
}
}
class Worker extends Actor {
def receive = {
case workSend =>
println("Work Done")
sender ! workDone
}
}
}
The problem is with pattern matching on objects you've created. It's matching inproperly. Do not bother yourself with objects. Use strings for example:
object A {
val masterSend = "masterSend"
val workSend = "workSend"
val workDone = "workDone"
}
object MatrixMultiply {
val usage = """
Usage: MainStart <matrix-dimension> <high-value>
"""
def main(args: Array[String]) {
val Dim = 3
val Max = 2
val system = ActorSystem("ComputeSystem")
val worker = system.actorOf(Props[Worker], name = "worker")
val master = system.actorOf(Props(new Master(Dim, Max, worker)), name = "master")
master ! A.masterSend
}
class Master(Dim: Int, Max: Int, worker : ActorRef) extends Actor {
def receive = {
case A.masterSend =>
println("Master sent")
worker ! A.workSend
case A.workDone =>
println("Work was done!!")
context.system.shutdown()
}
}
class Worker extends Actor {
def receive = {
case A.workSend =>
println("Work Done")
sender ! A.workDone
}
}
}
You've named your object from lower case letter.
object messageSend
But pattern matching consider it not as an object but as a some new variable instead.
case messageSend => messageSend - is a variable
You'd be able to write anything here case magicBall => will also compile.
I moved from Casbah to Reactive Mongo and from that moment I couldn't make work the test of my actor.
I have a dao for the persistence layer and tests for that tier. All the tests passed. So, the only thing that comes to my mind its a problem of synchronization.
" UserActor " should {
val socketActorProbe = new TestProbe(system)
val peyiProbe = new TestProbe(system)
val identifyId = 1
val emailCsr = "csr#gmail.com"
val emailPeyi = "peyi#gmail.com"
val point = new Point[LatLng](new LatLng(-31.4314041, -64.1670626))
" test preStart() " in new WithApplication {
db.createDB(id1, id2, id3)
val userActorRefCsr = TestActorRef[UserActor](Props(classOf[UserActor], emailCsr, socketActorProbe.ref))
val csr = userActorRefCsr.underlyingActor
val userActorRef = TestActorRef[UserActor](Props(classOf[UserActor], emailPeyi, socketActorProbe.ref))
val peyi = userActorRef.underlyingActor
peyi.receive(ActorIdentity(identifyId, Option(userActorRefCsr)))
db.clearDB()
}
Actor class.
class UserActor(email: String, upstream: ActorRef) extends Actor {
import UserActor._
val identifyId = 1
val usersFromDB = ReactiveMongoFactory.db.collection[BSONCollection]("users")
val userDao = new UserDao(usersFromDB)
val meFuture = userDao.findMeByEmail(email)
var friends: Map[String, ActorRef] = Map()
override def preStart() = {
meFuture onComplete { result =>
val emailsFriends: List[String] = userDao.getMyFriendsEmail(result.get.get)
println(emailsFriends)
for (email <- emailsFriends) {
println("sending msg to " + email)
context.actorSelection("/user/" + email) ! Identify(identifyId)
}
}
}
private def giveMyFriend(email: String): Option[ActorRef] = {
for(friend <- friends){
if (friend._1 == email) new Some(friend._2)
}
None
}
def active(another: ActorRef): Actor.Receive = {
case Terminated(`another`) => context.stop(self)
}
def receive = {
case ActorIdentity(`identifyId`, Some(actorRef)) =>
meFuture onComplete { result =>
println(" ... subscribing ... " + result.get.get.basicProfile.email)
actorRef ! Subscribe(result.get.get.basicProfile.email.get)
context.watch(actorRef)
context.become(active(actorRef))
}
case Subscribe(email) =>
friends += (email -> sender)
context watch sender
case Terminated(user) => {
for(friend <- friends){
if (friend._2 == user ) friends -= friend._1 //removing by key
}
}
case UserMoved(email, point) =>
upstream ! UserPosition(email, System.currentTimeMillis(), point.coordinates)
}
}
Im receiving the following output.
The exception is thrown in the following lines of code.
def findMeByEmail(email: String): Future[Option[User]] = {
val query = BSONDocument("email" -> email)
println( " .... finding user ..... email: " + email )
val cursor = users.find(query).cursor[BSONDocument]
val userFuture = cursor.headOption.map(
doc => Some(userReader.read(doc.get))
)
userFuture
}
If I run the test for that method, it's all ok.
describe("get my friends emails") {
it("returns a list of emails") {
val futureUser = userDao.findMeByEmail("csr#gmail.com")
ScalaFutures.whenReady(futureUser) { result =>
val friends = userDao.getMyFriendsEmail(result.get)
assert(friends.length == 2)
}
}
}
Basically, Im trying to look my friends (Other actor) and then register them in a map to have a reference. I couldn't find any good example which shows tests using Reactive Mongo with Actors.
I hope somebody can help me to understand whats going on here. Thanks in advance.