Prometheus C++ Client Library

[jup...@gmail.com]

Hi,

I created a C++ client library at https://github.com/jupp0r/prometheus-cpp
It's mostly functional (although missing summary metrics for now) and I'd like to invite everybody to try it out, give me feedback and contribute.

Some highlights:

- uses modern C++ everywhere
- batteries included (embedded lightweight http server)
- full unit and integration test suite
- designed in a way that makes it easy to bring your own http stack, etc

Thanks and let me know what you think,

Jupp

[Brian Brazil]

On 18 October 2016 at 17:34, <jup...@gmail.com> wrote:

Hi,

I created a C++ client library at https://github.com/jupp0r/prometheus-cpp
It's mostly functional (although missing summary metrics for now) and I'd like to invite everybody to try it out, give me feedback and contribute.

Thanks for this. I'd suggest a read of https://prometheus.io/docs/instrumenting/writing_clientlibs/ as I notice several deviations from the guidelines from a quick peek such as:

- Allowing labels to be set across all exposed metrics

- 'add' rather than 'register' on the registry

- registry taking specific metrics types rather than a Collector

- registries being added to the exposition, rather than exposition using one registry (that'd make the exposer a 2nd type of registry)

- Label handling is doing something odd, as the readme example seems to have two counters with identical labels in a metric family with no label names

- Gauges rejecting increments by negative numbers (only counters should do that)

- Included counters not ending in _total

- Included latency metrics not being in seconds, and not including units in the metric name

- Default histogram buckets not matching other clients

- Supporting json, it's long deprecated

- Does not expose the Prometheus text format

I'd suggest looking at the Java client for inspiration of how to lay things out.

Brian

 

Some highlights:

- uses modern C++ everywhere
- batteries included (embedded lightweight http server)
- full unit and integration test suite
- designed in a way that makes it easy to bring your own http stack, etc

Thanks and let me know what you think,

Jupp

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.
To post to this group, send email to prometheus-developers@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/86166cf9-4cd8-4d0e-bb4e-6e6460a69709%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Brian Brazil

www.robustperception.io

[Jupp Müller]

Hey Brian,

thanks for taking the time to give feedback.

I'll address your points in the next weeks. While I agree with you mostly, I'd like to discuss some points in more detail.

> On 19 Oct 2016, at 00:53, Brian Brazil <brian....@robustperception.io> wrote:
> - Allowing labels to be set across all exposed metrics

I was under the impression that this is required:

A client library MUST allow for optionally specifying a list of label names at Gauge/Counter/Summary/Histogram creation time.

Are you referring to the constant labels optionally being added by registries?

> - registries being added to the exposition, rather than exposition using one registry (that'd make the exposer a 2nd type of registry)

I have a specific use case where an exposer has to scrape multiple registries (each originating from a component contained in a different library) and also each registry is scraped by different exposers (http and dumping metrics to log files). This seems to be a reasonable requirement and not very well modeled by the one registry to one exposition mapping you describe above.

> - Default histogram buckets not matching other clients

There are no defaults, the API forces you to specify them.

> - Supporting json, it's long deprecated

I don't see anything wrong with offering additional formats to the specified ones, what's your rationale for discouraging client libraries to do this?

> - Does not expose the Prometheus text format

Although preferable (and planned), this is not required by the spec:

Clients MUST implement one of the documented exposition formats.
Clients MAY implement more than one format. There SHOULD be a human readable format offered.

The library implements the protobuf format for actual scrapes, the current human readable format is provided by protobufs DebugString() output. This complies with the MUST and SHOULD, but not the MAY requirements.

> I'd suggest looking at the Java client for inspiration of how to lay things out.

I did, although I found the Go library to be closer to my use case.

Thanks again for your feedback,

Jupp

[Brian Brazil]

On 25 October 2016 at 10:30, Jupp Müller <jup...@gmail.com> wrote:

Hey Brian,

thanks for taking the time to give feedback.

I'll address your points in the next weeks. While I agree with you mostly, I'd like to discuss some points in more detail.

> On 19 Oct 2016, at 00:53, Brian Brazil <brian.brazil@robustperception.io> wrote:
> - Allowing labels to be set across all exposed metrics
I was under the impression that this is required:

A client library MUST allow for optionally specifying a list of label names at Gauge/Counter/Summary/Histogram creation time.

Are you referring to the constant labels optionally being added by registries?

 

Yes, that's a big no-no. https://www.robustperception.io/target-labels-are-for-life-not-just-for-christmas/

> - registries being added to the exposition, rather than exposition using one registry (that'd make the exposer a 2nd type of registry)
I have a specific use case where an exposer has to scrape multiple registries (each originating from a component contained in a different library) and also each registry is scraped by different exposers (http and dumping metrics to log files). This seems to be a reasonable requirement and not very well modeled by the one registry to one exposition mapping you describe above.

This is the first time I've come across a requirement like this, all libraries are meant to share the one default registry.

 

> - Default histogram buckets not matching other clients
There are no defaults, the API forces you to specify them.

There should be a default in line with the other libraries.

 

> - Supporting json, it's long deprecated
I don't see anything wrong with offering additional formats to the specified ones, what's your rationale for discouraging client libraries to do this?

It's misleading and may cause confusion as to what a Prometheus client library is.

 

> - Does not expose the Prometheus text format
Although preferable (and planned), this is not required by the spec:

Clients MUST implement one of the documented exposition formats.
Clients MAY implement more than one format. There SHOULD be a human readable format offered.

The library implements the protobuf format for actual scrapes, the current human readable format is provided by protobufs DebugString() output. This complies with the MUST and SHOULD, but not the MAY requirements.

This doesn't follow the SHOULD as it's not providing the Prometheus text format.

 

> I'd suggest looking at the Java client for inspiration of how to lay things out.
I did, although I found the Go library to be closer to my use case.

The Go client has many oddities for historical reasons, the Java client is a better one to follow generally.

Brian 

Thanks again for your feedback,

Jupp

--

Brian Brazil

www.robustperception.io

[Richard Hartmann]

On Tue, Oct 25, 2016 at 9:39 PM, Brian Brazil
<brian....@robustperception.io> wrote:
>> Clients MAY implement more than one format. There SHOULD be a human
>> readable format offered.

> This doesn't follow the SHOULD as it's not providing the Prometheus text
> format.

Quoth RFC2119:
3. SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a
particular item, but the full implications must be understood and
carefully weighed before choosing a different course.


I agree that the text format is a reasonable baseline to require as
that makes debugging easier. So maybe the text should be changed to
state that text format MUST be supported and protobuf SHOULD be.


Richard

[Jupp Müller]

The text could also be improved by stating that "a human readable format" explicitly refers to _the_ prometheus text format, not just _any_ human readable format (if that is the intended meaning, as Brian suggested).

Jupp

[Richard Hartmann]

On Tue, Oct 25, 2016 at 11:12 PM, Jupp Müller <jup...@gmail.com> wrote:

> The text could also be improved by stating that "a human readable format" explicitly refers to _the_ prometheus text format, not just _any_ human readable format (if that is the intended meaning, as Brian suggested).

Agreed.


Richard

[Björn Rabenstein]

On 25 October 2016 at 23:32, Richard Hartmann

Historical context: When the guidelines were written, there was still
the legacy Java client library that supported JSON as human-readable
format and protobuf as the one that is required for Prometheus to talk
to the library.

Since the legacy Java client library is gone by now, we could be
stricter about the human-readable format being the text format.

However, I don't think that's an obvious discussion. If a client
library uses protobuf (like in this case), the text representation of
protobuf comes for free as a human-readable text format (for certain
definitions of readable...). Requiring the Prometheus text format as
the human readable format is an additional barrier for client library
implementations in that case. We should be careful about how we set
incentives. If we have a relatively strong emphasis on supporting the
Prometheus text format, we might discourage implementers from
supporting the protobuf format at all.

--
Björn Rabenstein, Engineer
http://soundcloud.com/brabenstein

SoundCloud Ltd. | Rheinsberger Str. 76/77, 10115 Berlin, Germany
Managing Director: Alexander Ljung | Incorporated in England & Wales
with Company No. 6343600 | Local Branch Office | AG Charlottenburg |
HRB 110657B

[Björn Rabenstein]

On 25 October 2016 at 21:39, Brian Brazil

<brian....@robustperception.io> wrote:
>> > - registries being added to the exposition, rather than exposition using
>> > one registry (that'd make the exposer a 2nd type of registry)
>> I have a specific use case where an exposer has to scrape multiple
>> registries (each originating from a component contained in a different
>> library) and also each registry is scraped by different exposers (http and
>> dumping metrics to log files). This seems to be a reasonable requirement and
>> not very well modeled by the one registry to one exposition mapping you
>> describe above.
>
> This is the first time I've come across a requirement like this, all
> libraries are meant to share the one default registry.

Multiple registries were a long time requirement for the Go client –
and finally made it into v0.8 of the Go client. The main use case is
exposing different metrics at different endpoints, the most concrete
need for which showed up in Kubernetes instrumentation.

Jupp, you can check out how it was solved in the current Go client:
The registry implements to different interfaces, the Registerer and
the Gatherer. The former is for the registration, the latter for
handing over the gathered metrics to some kind of exposition mechanism
(an HTTP endpoint, or to a Graphite bridge, or for testing, or – as
you said – for dumping into a logfile). There is also a way to merge
different Gatherers into one Gatherer, which in turn doesn't implement
the Registerer parts. Not sure how directly this can translate into
C++, but it might serve as an inspiration.

>> > - Supporting json, it's long deprecated
>> I don't see anything wrong with offering additional formats to the
>> specified ones, what's your rationale for discouraging client libraries to
>> do this?
>
> It's misleading and may cause confusion as to what a Prometheus client
> library is.

We have also seen people creating makeshift tools that consume the
deprecated JSON format, or even do makeshift exposition using the JSON
format. Essentially, by not discouraging the JSON format, we are
indirectly encouraging the creation of a fragmented tool-set within
the community.

>> > I'd suggest looking at the Java client for inspiration of how to lay
>> > things out.
>> I did, although I found the Go library to be closer to my use case.
>
> The Go client has many oddities for historical reasons, the Java client is a
> better one to follow generally.

That's not generally true.

Historical oddities of the Go client are being wed out. (Note what
features are marked deprecated, and also have a look at the 0.9 and
0.10 milestones for imminent changes.)

The Java client provides weaker guarantees for metric consistency,
doesn't make full use of static typing, and doesn't support protobuf
(and has made it hard by design to ever do). Thus, for a client
library in a statically typed language and with the intention to
support protobuf, the Go client library is probably a better role
model.

[Brian Brazil]

On 26 October 2016 at 13:34, Björn Rabenstein <bjo...@soundcloud.com> wrote:

On 25 October 2016 at 21:39, Brian Brazil

<brian.brazil@robustperception.io> wrote:
>> > - registries being added to the exposition, rather than exposition using
>> > one registry (that'd make the exposer a 2nd type of registry)
>> I have a specific use case where an exposer has to scrape multiple
>> registries (each originating from a component contained in a different
>> library) and also each registry is scraped by different exposers (http and
>> dumping metrics to log files). This seems to be a reasonable requirement and
>> not very well modeled by the one registry to one exposition mapping you
>> describe above.
>
> This is the first time I've come across a requirement like this, all
> libraries are meant to share the one default registry.

Multiple registries were a long time requirement for the Go client –
and finally made it into v0.8 of the Go client. The main use case is
exposing different metrics at different endpoints, the most concrete
need for which showed up in Kubernetes instrumentation.

Jupp, you can check out how it was solved in the current Go client:
The registry implements to different interfaces, the Registerer and
the Gatherer. The former is for the registration, the latter for
handing over the gathered metrics to some kind of exposition mechanism
(an HTTP endpoint, or to a Graphite bridge, or for testing, or – as
you said – for dumping into a logfile). There is also a way to merge
different Gatherers into one Gatherer, which in turn doesn't implement
the Registerer parts. Not sure how directly this can translate into
C++, but it might serve as an inspiration.

Gathers are specific to the Go client, and I personally consider them to be unnecessary complexity. All other clients get away with just the Registry and a collect method or equivalent (for the ones that even support multiple registries).

You are the first person to want to have a registry to be able to collect from a registry, which indicates to me that you're probably not thinking about things in the right way.

Brian

--

Brian Brazil

www.robustperception.io

[vc....@gmail.com]

the client library gives a good example for counter. Where can I get a similar simple-yet-working example for simple histograms. Can anyone help here..
Vick

[vc....@gmail.com]

On Tuesday, October 18, 2016 at 10:04:39 PM UTC+5:30, jup...@gmail.com wrote:

Mr Jupp:

I am considering using it. Its organiztion is quite good. I was able to use it in minutes using your awesome simple counters example.
However struggling for hours to use histograms. I am able to BuildHistogram but how to specify buckets width and count ??? .Add() seems not working for me on the Built Histo family. can u please help here? finally I think .observe(value) will complete my program.
- Vicky

[vc....@gmail.com]

Can anyone help here?

Vick