jmxtrans writes Kafka metrics about topics in one prefix of Graphite

[trider]

Hello guys!

I want to have a gradation of metrics on the topics of Kafka in Graphite, but jmxtrans merges identical metrics into one prefix:
INFO   | jvm 1    | 2018/11/12 19:26:33 | 2018-11-12 19:26:33 [jmxtrans-result-0] DEBUG c.g.j.model.output.GraphiteWriter - Query result: Result(attributeName=Count, className=com.yammer.metrics.reporting.JmxReporter$Meter, objDomain=kafka.server, typeName=type=BrokerTopi
cMetrics,name=TotalProduceRequestsPerSec,topic=my_topic, valuePath=[], value=311960, epoch=1542039993362, keyAlias=null)
INFO   | jvm 1    | 2018/11/12 19:26:33 | 2018-11-12 19:26:33 [jmxtrans-result-0] DEBUG c.g.j.model.output.GraphiteWriter - Graphite Message: portal.kafka.youla-mq1.127_0_0_1_9393.com_yammer_metrics_reporting_JmxReporter$Meter.Count 311960 1542039993
INFO   | jvm 1    | 2018/11/12 19:26:33 |
INFO   | jvm 1    | 2018/11/12 19:26:33 | 2018-11-12 19:26:33 [jmxtrans-result-0] DEBUG c.g.j.model.output.GraphiteWriter - Query result: Result(attributeName=Count, className=com.yammer.metrics.reporting.JmxReporter$Meter, objDomain=kafka.server, typeName=type=BrokerTopi
cMetrics,name=TotalProduceRequestsPerSec,topic=other_topic, valuePath=[], value=797574, epoch=1542039993363, keyAlias=null)
INFO   | jvm 1    | 2018/11/12 19:26:33 | 2018-11-12 19:26:33 [jmxtrans-result-0] DEBUG c.g.j.model.output.GraphiteWriter - Graphite Message: portal.kafka.youla-mq1.127_0_0_1_9393.com_yammer_metrics_reporting_JmxReporter$Meter.Count 797574 1542039993
INFO   | jvm 1    | 2018/11/12 19:26:33 |
INFO   | jvm 1    | 2018/11/12 19:26:33 | 2018-11-12 19:26:33 [jmxtrans-result-0] DEBUG c.g.j.model.output.GraphiteWriter - Query result: Result(attributeName=Count, className=com.yammer.metrics.reporting.JmxReporter$Meter, objDomain=kafka.server, typeName=type=BrokerTopi
cMetrics,name=TotalProduceRequestsPerSec,topic=one_more_topic, valuePath=[], value=0, epoch=1542039993364, keyAlias=null)
INFO   | jvm 1    | 2018/11/12 19:26:33 | 2018-11-12 19:26:33 [jmxtrans-result-0] DEBUG c.g.j.model.output.GraphiteWriter - Graphite Message: portal.kafka.youla-mq1.127_0_0_1_9393.com_yammer_metrics_reporting_JmxReporter$Meter.Count 0 1542039993

I.e. counters by TotalProduceRequestsPerSec for all topics merge in portal.kafka.youla-mq1.127_0_0_1_9393.com_yammer_metrics_reporting_JmxReporter$Meter.Count:

I need metrics for each topic. My current YAML config:

{
  "servers" : [ {
    "port" : "9393",
    "host" : "127.0.0.1",
    "queries" : [ {
      "obj" : "kafka.server:name=TotalProduceRequestsPerSec,topic=*,type=BrokerTopicMetrics",
      "attr" : [ "Count" ],
      "outputWriters" : [ {
        "@class" : "com.googlecode.jmxtrans.model.output.GraphiteWriter",
        "settings": {
           "port" : 2003,
           "host" : "10.255.12.2",
           "rootPrefix": "portal.kafka.youla-mq1"
        }
      } ]
    } ]
  } ]
}

[trider]

Help me please!

I have a lot of metrics in Kafka and I don't want jmxtrans to group them all. I want jmxtrans to loop through each bean and write all the attributes for each bean as is in Graphite.

It will take me several weeks to study all the beans of JMX and write the YAML configuration.

I want to get a universal YAML config where the program will get all the beans and write the attributes as is without grouping for each bean. Manipulations with the typeName do not give me an unambiguous result.

понедельник, 12 ноября 2018 г., 20:01:47 UTC+3 пользователь trider написал:

[trider]

Here's what I wanted to do:

{
  "servers" : [ {
    "port" : "9393",
    "host" : "127.0.0.1",
    "queries" : [ {

      "obj" : "",
      "attr" : [],
      "useAllTypeNames": true,
      "useObjDomainAsKey": true,
      "allowDottedKeys": true,

      "outputWriters" : [ {
        "@class" : "com.googlecode.jmxtrans.model.output.GraphiteWriter",
        "settings": {
           "port" : 2003,
           "host" : "10.255.12.2",
           "rootPrefix": "portal.kafka.youla-mq1"
        }
      } ]
    } ]
  } ]
}

I'm happy!!!

вторник, 13 ноября 2018 г., 13:12:11 UTC+3 пользователь trider написал: