GRPC endpoint for metrics

[Curtis Mahieu]

I was wondering if it was possible to make a GRPC service to expose Prometheus metrics.

I can configure Prometheus to pull the correct end point, but I'm stuck on what the "protobuf" message payload should actually be. 

According to the docs: 

https://prometheus.io/docs/instrumenting/exposition_formats/ 

It support an array of encoded Protobuf messages, but how would i specify this as a GRPC service response type.  

or is that not possible at this time. 

Thanks,

[Ben Kochie]

We have proto implementations for various languages already.  I highly recommend you include one of our libraries in your code to provide the metrics output.  What language is your app written in?

https://prometheus.io/docs/instrumenting/clientlibs/

You can also take a look at how we implement it in our libraries.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/333bba36-9e18-4fde-9555-daa42b709975%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Curtis Mahieu]

Thanks for the reply, the language i am working in is c++ I have taken a look at the un-official c++ sdk.  

However the main issue is that they all respond via HTTP with a special encoding of the Protobuf files (with the length prefix added before every message):

https://github.com/jupp0r/prometheus-cpp/blob/master/lib/protobuf_delimited_serializer.cc

GRPC is designed to have a single proto message type for the reply, something like this:

//import https://github.com/prometheus/client_model/blob/fa8ad6fec33561be4280a8f0514318c79d7f6cb6/metrics.proto

message Metrics {

    repeated MetricFamily metricFamily = 1;

}

On Wednesday, May 31, 2017 at 11:04:27 PM UTC-7, Ben Kochie wrote:

We have proto implementations for various languages already.  I highly recommend you include one of our libraries in your code to provide the metrics output.  What language is your app written in?

https://prometheus.io/docs/instrumenting/clientlibs/

You can also take a look at how we implement it in our libraries.

On Thu, Jun 1, 2017 at 12:55 AM, Curtis Mahieu <cur...@gmail.com> wrote:

I was wondering if it was possible to make a GRPC service to expose Prometheus metrics.

I can configure Prometheus to pull the correct end point, but I'm stuck on what the "protobuf" message payload should actually be. 

According to the docs: 

https://prometheus.io/docs/instrumenting/exposition_formats/ 

It support an array of encoded Protobuf messages, but how would i specify this as a GRPC service response type.  

or is that not possible at this time. 

Thanks,

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.

[Ben Kochie]

I'll have to defer to the C++ experts, as it's definitely not my area of expertise. :-)

It may also be simply easier to implement using the text protocol.  There is some perceived overhead for the text format, but in reality it's minimal, maybe a couple percent.  Compared to JSON which was much worse.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/dc00c08b-c01d-47b2-a1cf-4c209f5a837c%40googlegroups.com.

[Björn Rabenstein]

On 1 June 2017 at 09:12, Curtis Mahieu <cur...@gmail.com> wrote:
>
> GRPC is designed to have a single proto message type for the reply, something like this:

Hmm, interesting... Some time ago, I did some research about the
largest possible size of a proto message, with the disturbing result
that there is certainly a limit but it is not well defined,
essentially determined by implementation details. See various
discussions on stack overflow. If you ever create proto messages
larger than 64MiB, you are certainly asking for trouble.

Now the problem is that scrapes can, at times, become rather large.
That's an edge case, but it definitely happens. How does GRPC deal
with the issue? Is the maximum response size of a GRPC implicitly
limited in that way?

--
Björn Rabenstein, Engineer
http://soundcloud.com/brabenstein

SoundCloud Ltd. | Rheinsberger Str. 76/77, 10115 Berlin, Germany
Managing Director: Alexander Ljung | Incorporated in England & Wales
with Company No. 6343600 | Local Branch Office | AG Charlottenburg |
HRB 110657B

[Tristan Colgate]

You can use a streaming call (http://www.grpc.io/docs/guides/concepts.html#server-streaming-rpc), they work well in Go, I've no idea what the C++ API for them is like though.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CALSNGh%3DPMehPYst2%3Dc9u1dQU_fp5m8oW23A%3D0R3vA3zL3D-6Ww%40mail.gmail.com.

[Björn Rabenstein]

On 1 June 2017 at 14:44, Tristan Colgate <tcol...@gmail.com> wrote:
>
> You can use a streaming call
> (http://www.grpc.io/docs/guides/concepts.html#server-streaming-rpc),
> they work well in Go, I've no idea what the C++ API for them is like though.

If I understand correctly, then you do get many proto messages as the
response to one request. So the existing delimited protobuf solution
is somewhat similar to that in concept.

[Tristan Colgate]

Absolutely. ListFeatures in the getting started docs is a good example (http://www.grpc.io/docs/tutorials/basic/go.html#server-side-streaming-rpc)

We use this for real-time streaming of logs so I can state pretty categorically that streaming of independent messages works just fine.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CALSNGhkBH87BY6pSoGK8vwyUhXyP7X0myAoU%2B0LECbXnJz0YKw%40mail.gmail.com.

[Curtis Mahieu]

I assume support for streaming grcp scraping of endpoints would need to be added to the Prometheus server?

Is anything like this on the roadmap?

[Björn Rabenstein]

On 1 June 2017 at 17:08, Curtis Mahieu <cur...@gmail.com> wrote:
> I assume support for streaming grcp scraping of endpoints would need to be added to the Prometheus server?
>
> Is anything like this on the roadmap?

I guess grpc would be out if we tossed protobuf support altogether.

[Curtis Mahieu]

Prometheus is going to drop support for protobuf?

[Björn Rabenstein]

On 6 June 2017 at 18:54, Curtis Mahieu <cur...@gmail.com> wrote:

Prometheus is going to drop support for protobuf?

Nothing is decided yet, but it is considered.

[cpug...@gmail.com]

No, grpc can use other encoding formats.

Or you can encode it as a stream of bytes and still use a normal protobuf message to send it. 

[Björn Rabenstein]

On 24 August 2017 at 16:33, <cpug...@gmail.com> wrote:
>
> On Tuesday, June 6, 2017 at 12:48:04 PM UTC-4, Björn Rabenstein wrote:
>>
>> I guess grpc would be out if we tossed protobuf support altogether.
>

> No, grpc can use other encoding formats.
> Or you can encode it as a stream of bytes and still use a normal protobuf message to send it.

True. Technically, you can send any opaque payload via grpc. The
question is more if that would still make sense.
https://github.com/RichiH/OpenMetrics/blob/master/protobuf_vs_text.md#implications-for-grpc
is a short section about that issue (and the whole doc goes into more
details what the pros and cons of protobuf vs. text format are).

[Curtis Mahieu]

I posted in the GRPC forum trying to get some help implementing such a thing, but unfortunately have not had much luck figuring it out myself.

https://groups.google.com/forum/#!topic/grpc-io/2cNVbAQgBrU

[bigb...@gmail.com]

I am a user of gRPC and Prometheus. I believe we met a very similiar requirements.

The service server in our case is build with gRPC and in this service, we want to expose some metrics to prometheus. Currently, we start another jetty server but this is not a good design..

So we are wondering whether the prometheus could call the gRPC service with the path: "http://host:port/service_path/method_name" directly.

Or, we have to expose the metrics through another http server, no matter whether the format is plain/text or encoded protobuf.

On the other hand, we depends on the grpc-gatway( a tool to expose json and http interface based on gRPC), but the response from this service are always JSON. Is it possible that Prometheus could handle JSON format in the future?

Thanks

在 2017年6月1日星期四 UTC+8上午6:55:54，Curtis Mahieu写道：

[panda.n...@gmail.com]

Hi
Can we have a custom sidecar to the Prometheus POD which can make this conversion? And we can follow a Visitor Pattern(i.e. exposing a metric function) in every gRPC based ms.
Will this be an ideal solution?