How to configure Akka to expose metrics to Prometheus using Kamon? - scala

Question

I am trying to configure my project which is based on Akka 2.6.10 to expose metric values to Prometheus. I saw this question which uses Kamon but I couldn't figure out what I am missing on my configuration. My build.sbt file has the following configuration:
name := """explore-akka"""
version := "1.1"
scalaVersion := "2.12.7"
val akkaVersion = "2.6.10"
lazy val kamonVersion = "2.1.9"
libraryDependencies ++= Seq(
// Akka basics
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"com.typesafe.akka" %% "akka-testkit" % akkaVersion,
// Metrics: Kamon + Prometheus
"io.kamon" %% "kamon-core" % kamonVersion,
"io.kamon" %% "kamon-akka" % kamonVersion,
"io.kamon" %% "kamon-prometheus" % kamonVersion
)
and the plugins.sbt:
resolvers += Resolver.bintrayRepo("kamon-io", "sbt-plugins")
addSbtPlugin("io.kamon" % "sbt-aspectj-runner" % "1.1.1")
I added on the application.conf:
kamon.instrumentation.akka.filters {
actors.track {
includes = [ "CounterSystem/user/Counter**" ]
}
}
then I start a MainClass which call the counterActor:
object MainClass extends App {
Kamon.registerModule("akka-test", new PrometheusReporter())
Kamon.init()
CounterActor.run()
}
import akka.actor.{Actor, ActorSystem, Props}
import kamon.Kamon
object CounterActor extends App {
run()
def run() = {
import Counter._
val actorSystem = ActorSystem("CounterSystem")
val countActor = actorSystem.actorOf(Props[Counter], "Counter")
(1 to 100).foreach { v =>
Thread.sleep(1000)
countActor ! Increment
}
(1 to 50).foreach { v =>
Thread.sleep(1000)
countActor ! Decrement
}
countActor ! Print
}
class Counter extends Actor {
import Counter._
val counter = Kamon.counter("my-counter")
var count = 0
override def receive: Receive = {
case Increment =>
count += 1
println(s"incrementing... $count")
counter.withoutTags().increment()
case Decrement =>
count -= 1
println(s"decrementing... $count")
counter.withoutTags().increment()
case Print =>
sender() ! count
println(s"[counter] current count is: $count")
}
}
object Counter {
case object Increment
case object Decrement
case object Print
}
}
I think that after this I could launch the application using sbt run and listen to the metrics on the Prometheus console (http://127.0.0.1:9090/graph), but I do not see any metrics related to my actor. My guess is that I have to configure the scrape_config at the prometheus file /etc/prometheus/prometheus.yml. Am I right? How should I configure it?

Answer 1

I had to configure Prometheus to scrape the Kamon web service through the config file
cat /etc/prometheus/prometheus.yml
global:
scrape_interval: 15s
scrape_configs:
- job_name: "kamon"
scrape_interval: "5s"
static_configs:
- targets: ['localhost:9095']
metrics_path: /
add these 2 Kamon libraries at build.sbt:
"io.kamon" %% "kamon-bundle" % "2.1.9",
"io.kamon" %% "kamon-prometheus" % "2.1.9",
add this configuration at application.conf:
kamon.instrumentation.akka.filters {
actors.track {
includes = [ "AkkaQuickStart/user/*" ]
# excludes = [ "AkkaQuickStart/system/**" ]
}
}
Start Kamon and call the counter:
Kamon.init()
val counterSendMsg = Kamon.counter("counter-send-msg")
counterSendMsg.withTag("whom", message.whom).increment()
Here is the full demo application from Akka quick start with Kamon configured to count messages:
import akka.actor.typed.scaladsl.Behaviors
import akka.actor.typed.{ActorRef, ActorSystem, Behavior}
import kamon.Kamon
import scala.util.Random
object Greeter {
val counterSendMsg = Kamon.counter("counter-send-msg")
def apply(): Behavior[Greet] = Behaviors.receive { (context, message) =>
context.log.info("Hello {}!", message.whom)
//#greeter-send-messages
message.replyTo ! Greeted(message.whom, context.self)
counterSendMsg.withTag("whom", message.whom).increment()
//#greeter-send-messages
Behaviors.same
}
final case class Greet(whom: String, replyTo: ActorRef[Greeted])
final case class Greeted(whom: String, from: ActorRef[Greet])
}
object GreeterBot {
def apply(max: Int): Behavior[Greeter.Greeted] = {
bot(0, max)
}
private def bot(greetingCounter: Int, max: Int): Behavior[Greeter.Greeted] =
Behaviors.receive { (context, message) =>
val n = greetingCounter + 1
context.log.info("Greeting {} for {}", n, message.whom)
if (n == max) {
Behaviors.stopped
} else {
message.from ! Greeter.Greet(message.whom, context.self)
bot(n, max)
}
}
}
object GreeterMain {
def apply(): Behavior[SayHello] =
Behaviors.setup { context =>
//#create-actors
val greeter = context.spawn(Greeter(), "greeter")
//#create-actors
Behaviors.receiveMessage { message =>
//#create-actors
val replyTo = context.spawn(GreeterBot(max = 3), message.name)
//#create-actors
greeter ! Greeter.Greet(message.name, replyTo)
Behaviors.same
}
}
final case class SayHello(name: String)
}
object AkkaQuickstart {
def main(args: Array[String]): Unit = {
run()
}
def run() = {
Kamon.init()
import GreeterMain._
val greeterMain: ActorSystem[GreeterMain.SayHello] = ActorSystem(GreeterMain(), "AkkaQuickStart")
val allPerson = List("Charles", "Bob", "Felipe", "Simone", "Fabio")
def randomPerson = allPerson(Random.nextInt(allPerson.length))
while (true) {
greeterMain ! SayHello(randomPerson)
Thread.sleep(1000)
}
}
}
and my Prometheus web console: