Re: Native prometheus export for Apache Cassandra (and other coda hale users)

[Ben Kochie]

Fantastic, we've generally used the JMX exporter for monitoring cassandra[0], but having native metrics is much better.

[0]: https://github.com/prometheus/jmx_exporter/blob/master/example_configs/cassandra.yml

On Fri, Jun 24, 2016 at 6:39 PM, <sn...@snazy.de> wrote:

Hi,

upcoming 3.8 release of Apache Cassandra includes support to export metrics in prometheus native format. The relevant C*-ticket is https://issues.apache.org/jira/browse/CASSANDRA-11967 .

It requires to drop in a few jars, which are provided by a build of https://github.com/snazy/prometheus-metrics-exporter/ . This lib can export the metrics via HTTP using both the protobuf and text formats. From a coding perspective, all you need is to pass in the instance of the MetricsRegistry.
This means, that it's not necessary to use a graphite exporter. Frankly, I need to update the readme for this lib, but wanted to give you guys a heads up :)

Robert

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Robert Stupp]

Sorry guys,

it will be probably reverted. Maybe I have to go via the metrics-exporter-config project to support it.

—

Robert Stupp

@snazy

[Brian Brazil]

On 24 June 2016 at 18:15, Robert Stupp <sn...@snazy.de> wrote:

Sorry guys,

it will be probably reverted. Maybe I have to go via the metrics-exporter-config project to support it.

How do you handle labels with that?

The java client now includes a dropwizard collector, which may be of interest.

As far as I'm aware, the JMX exporter remains the best way to monitor Cassandra with Prometheus.

Brian

--

Brian Brazil

www.robustperception.io

[Johnny Miller]

Brian - not sure I agree on the JMX exporter with Cassanda being the best option.

I have found when there is a more than a few tables, the performance degrades considerably. Using the graphite exporter in Cassandra to push the metrics is significantly quicker.

--
Sent from my iPhone, apologies for any typos

--

Any views or opinions presented are solely those of the author and do not necessarily represent those of the company. digitalis.io is a trading name of Digitalis.io Ltd. Company Number: 98499457 Registered in England and Wales. Registered Office: Kemp House, 152 City Road, London, EC1V 2NX, United Kingddom

[Brian Brazil]

On 24 June 2016 at 23:03, Johnny Miller <joh...@digitalis.io> wrote:

Brian - not sure I agree on the JMX exporter with Cassanda being the best option.

I have found when there is a more than a few tables, the performance degrades considerably. Using the graphite exporter in Cassandra to push the metrics is significantly quicker.

JMX is unfortunately slow, but we get good metrics out of it. Even on a big instance, it's not slow enough to be a problem.

Brian 

--

Brian Brazil

www.robustperception.io

[Robert Stupp]

Well, JMX is definitely the slowest and most expensive alternative. Most expensive, because it is an RPC protocol - i.e. a lot of network roundtrips (although probably on localhost, but anyway), method dispatching, object allocations. It adds an extra layer between. JMX itself is also insecure - “by design” it allows execution or arbitrary code, which means, you should add authentication and encryption (if accessed remotely). See CVE-2015-0225.

prometheus’ native protobuf format is very resource efficient and eliminates the need to administrate another process (graphite or jmx exporter). There is no way to actually execute code.

I’ve updated the code and integrated it into metrics-reporter-config and opened a PR (https://github.com/addthis/metrics-reporter-config/pull/26) for this. The corresponding ticket for Apache Cassandra is https://issues.apache.org/jira/browse/CASSANDRA-12089.

—

Robert Stupp

@snazy

[Brian Brazil]

On 25 June 2016 at 09:52, Robert Stupp <sn...@snazy.de> wrote:

Well, JMX is definitely the slowest and most expensive alternative. Most expensive, because it is an RPC protocol - i.e. a lot of network roundtrips (although probably on localhost, but anyway), method dispatching, object allocations. It adds an extra layer between. JMX itself is also insecure - “by design” it allows execution or arbitrary code, which means, you should add authentication and encryption (if accessed remotely). See CVE-2015-0225.

prometheus’ native protobuf format is very resource efficient and eliminates the need to administrate another process (graphite or jmx exporter). There is no way to actually execute code.

It's recommended to use the JMX exporter as a java agent.

 

I’ve updated the code and integrated it into metrics-reporter-config and opened a PR (https://github.com/addthis/metrics-reporter-config/pull/26) for this. The corresponding ticket for Apache Cassandra is https://issues.apache.org/jira/browse/CASSANDRA-12089.

The apache issue tracker is being its usual self and timing out.

--

Brian Brazil

www.robustperception.io