How to use storm's New Metrics Reporting API? - apache-kafka

Question

I use the storm V1.2.1. After setting up according to the official documentation, I want to get some metrics in the spout, the spout code is as follows, but there is no expected metric data in graphite-web.
Question 1：How to use the New Metrics Reporting API correctly？
Question 2：How do I get the ACK number metric in the storm-bound KafkaSpout by using Storm's Old or New Metrics API?
Using New API in the spout to get the number of the tuple：
public static class MyTestWordSpout extends BaseRichSpout {
public static Logger LOG = LoggerFactory.getLogger(TestWordSpout.class);
boolean _isDistributed;
SpoutOutputCollector _collector;
private Counter tupleCounter;
transient CountMetric ackcountMetric;
long msid=0;
public void open(Map conf, TopologyContext context, SpoutOutputCollector collector) {
_collector = collector;
this.tupleCounter = context.registerCounter("tupleCount");
ackcountMetric = new CountMetric();
context.registerMetric("ack_count", ackcountMetric, 5);
}
public void close() {
}
public void nextTuple() {
Utils.sleep(100);
final String[] words = new String[] {"nathan", "mike", "jackson", "golda", "bertels"};
final Random rand = new Random();
final String word = words[rand.nextInt(words.length)];
_collector.emit(new Values(word),msid++);
this.tupleCounter.inc();
}
public void ack(Object msgId) {
ackcountMetric.incr();
}
public void fail(Object msgId) {
}
public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("word"));
}
}
storm.yaml：
storm.metrics.reporters:
# Graphite Reporter
- class: "org.apache.storm.metrics2.reporters.GraphiteStormReporter"
daemons:
- "supervisor"
- "nimbus"
- "worker"
report.period: 1
report.period.units: "SECONDS"
graphite.host: "10.11.6.79"
graphite.port: 2003
- class: "org.apache.storm.metrics2.reporters.ConsoleStormReporter"
daemons:
- "worker"
report.period: 1
report.period.units: "SECONDS"
graphite browser:
graphite browser

Answer 1

you can use this library https://github.com/staslev/storm-metrics-reporter. Add this to your pom.xml
<dependency>
<groupId>com.github.staslev</groupId>
<artifactId>storm-metrics-reporter</artifactId>
<version>1.5.0</version>
</dependency>
Add this configuration to your topology:
config.put(YammerFacadeMetric.FACADE_METRIC_TIME_BUCKET_IN_SEC, 30);
config.put(SimpleGraphiteStormMetricProcessor.GRAPHITE_HOST, "127.0.0.1");
config.put(SimpleGraphiteStormMetricProcessor.GRAPHITE_PORT, 2003);
config.put(SimpleGraphiteStormMetricProcessor.REPORT_PERIOD_IN_SEC, 10);
config.put(Config.TOPOLOGY_NAME, YOUR-TOPOLOGY.class.getCanonicalName());
config.registerMetricsConsumer(MetricReporter.class,
new MetricReporterConfig(".*", SimpleGraphiteStormMetricProcessor.class.getCanonicalName()), 1);
And add the following call into the prepare method of your bolts:
public void prepare(Map stormConf, TopologyContext context, OutputCollector collector) {
StormYammerMetricsAdapter.configure(stormConf, context, new MetricsRegistry());
Then you can see at your browser if Graphite shows the metrics.