Envoy Debug Logs

[Rama Rao]

Hi,

Is it possible to capture logs that envoy generates using any extensions to envoy or by any other means so that they can be shipped to our log indexers?

Thanks,

Rama

[Harvey Tuch]

If you want detailed debug logs from Envoy, you get these either on stderr by default, verbosity controlled by -l <log level>, e.g. -l trace. You can send these to a file with --log-path <path>. That is about the extent of the extensibility here.

If you want HTTP access logs, you get these via https://envoyproxy.github.io/envoy/configuration/http_conn_man/access_log.html?highlight=logs. There will be support in the v2 API for proto logging and statically registered access log sinks for HTTP (and probably TCP). But, this isn't a supported feature yet.

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.
To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/CAFfmCFHnSn%3DfmNZJ%3DDVA8T0Hhj-1osoLixaFeqa6TFs%2B05EKHw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Rama Rao]

Cool, thanks! What about the stats generated by Envoy? Is there a way to access them via code or some other means? Looks like envoy can push them
to Prometheus but I want to push it some other place, I need to access them programmatically. Is there an API that we can implement that Envoy can call periodically pushing its stats?

[Timothy Perrett]

You can access the stats via an admin server on the Envoy instance. The intention I believe was for human inspection, but I don't see why it wouldn't work programmatically. Still though, this is a hack, if you want metrics out of Envoy you can either use statsd push, or you can implement your own metrics collection sink

[Rama Rao]

Ok. I was thinking of admin end point - but is it structured enough to process it programmatically? Can we enhance it to output structured stuff like json if we pass in additional argument to it?
Can you please point me to docs on other two options like stated push or our own sink?

On Sun, 15 Oct 2017 at 2:46 AM, Timothy Perrett <tper...@gmail.com> wrote:

You can access the stats via an admin server on the Envoy instance. The intention I believe was for human inspection, but I don't see why it wouldn't work programmatically. Still though, this is a hack, if you want metrics out of Envoy you can either use statsd push, or you can implement your own metrics collection sink

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.
To post to this group, send email to envoy...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/28b520f1-4c8a-4962-b21f-f0d69a6de1ab%40googlegroups.com.

[Tim Perrett]

I'm not sure there are docs for the sink, its just in the code. Here's the statsd UDP sink: https://github.com/envoyproxy/envoy/blob/master/source/common/stats/statsd.h#L43

Altering the admin port to expose JSON feels like the wrong approach, IMO, but i'd let Matt or Harvey weigh in on that. It would help if you tried to explain more about what it is exactly you're looking to do...?

-- T

On 14 October 2017 at 21:26, Rama Rao <ramarao...@gmail.com> wrote:

Ok. I was thinking of admin end point - but is it structured enough to process it programmatically? Can we enhance it to output structured stuff like json if we pass in additional argument to it?
Can you please point me to docs on other two options like stated push or our own sink?

On Sun, 15 Oct 2017 at 2:46 AM, Timothy Perrett <tper...@gmail.com> wrote:

You can access the stats via an admin server on the Envoy instance. The intention I believe was for human inspection, but I don't see why it wouldn't work programmatically. Still though, this is a hack, if you want metrics out of Envoy you can either use statsd push, or you can implement your own metrics collection sink

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.

[Rama Rao]

Thanks. Will go through the docs. Regarding altering the admin, just want to be clear am suggesting to support additional behaviour of outputting Json based on additional request parameter so
mething like ‘?json=true’. Our use case is to get the stats and push it our own metrics database. We can certainly look at other options but it looks like it requires recompiling envoy if I am not wrong.

On Sun, 15 Oct 2017 at 10:32 AM, Tim Perrett <tper...@gmail.com> wrote:

I'm not sure there are docs for the sink, its just in the code. Here's the statsd UDP sink: https://github.com/envoyproxy/envoy/blob/master/source/common/stats/statsd.h#L43

Altering the admin port to expose JSON feels like the wrong approach, IMO, but i'd let Matt or Harvey weigh in on that. It would help if you tried to explain more about what it is exactly you're looking to do...?

-- T

On 14 October 2017 at 21:26, Rama Rao <ramarao...@gmail.com> wrote:

Ok. I was thinking of admin end point - but is it structured enough to process it programmatically? Can we enhance it to output structured stuff like json if we pass in additional argument to it?
Can you please point me to docs on other two options like stated push or our own sink?

On Sun, 15 Oct 2017 at 2:46 AM, Timothy Perrett <tper...@gmail.com> wrote:

You can access the stats via an admin server on the Envoy instance. The intention I believe was for human inspection, but I don't see why it wouldn't work programmatically. Still though, this is a hack, if you want metrics out of Envoy you can either use statsd push, or you can implement your own metrics collection sink

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/28b520f1-4c8a-4962-b21f-f0d69a6de1ab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/CAHrPNAZeCKpoi2t5sGB3qDOMMBezrnFePVVQ9bU9x32thVdprA%40mail.gmail.com.

[Matt Klein]

I don't have any opposition to different stats output formats via the admin handlers. IIRC Prometheus default mode is pull based, so we will probably support that directly at some point. Feel free to open a ticket and we can mark it help wanted.

On Sat, Oct 14, 2017 at 10:30 PM, Rama Rao <ramarao...@gmail.com> wrote:

Thanks. Will go through the docs. Regarding altering the admin, just want to be clear am suggesting to support additional behaviour of outputting Json based on additional request parameter so
mething like ‘?json=true’. Our use case is to get the stats and push it our own metrics database. We can certainly look at other options but it looks like it requires recompiling envoy if I am not wrong.

On Sun, 15 Oct 2017 at 10:32 AM, Tim Perrett <tper...@gmail.com> wrote:

I'm not sure there are docs for the sink, its just in the code. Here's the statsd UDP sink: https://github.com/envoyproxy/envoy/blob/master/source/common/stats/statsd.h#L43

Altering the admin port to expose JSON feels like the wrong approach, IMO, but i'd let Matt or Harvey weigh in on that. It would help if you tried to explain more about what it is exactly you're looking to do...?

-- T

On 14 October 2017 at 21:26, Rama Rao <ramarao...@gmail.com> wrote:

Ok. I was thinking of admin end point - but is it structured enough to process it programmatically? Can we enhance it to output structured stuff like json if we pass in additional argument to it?
Can you please point me to docs on other two options like stated push or our own sink?

On Sun, 15 Oct 2017 at 2:46 AM, Timothy Perrett <tper...@gmail.com> wrote:

You can access the stats via an admin server on the Envoy instance. The intention I believe was for human inspection, but I don't see why it wouldn't work programmatically. Still though, this is a hack, if you want metrics out of Envoy you can either use statsd push, or you can implement your own metrics collection sink

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/28b520f1-4c8a-4962-b21f-f0d69a6de1ab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/CAHrPNAZeCKpoi2t5sGB3qDOMMBezrnFePVVQ9bU9x32thVdprA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "envoy-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/CAFfmCFE_Bd5BCCSsAbPt7yb36d1ZnrRiL86zf7eec67CTGCqfg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Matt Klein

Software Engineer

mkl...@lyft.com

https://calendly.com/mattklein123

[Rama Rao]

Ok. Thanks Matt. Opened the ticket https://github.com/envoyproxy/envoy/issues/1862. So currently, Prometheus calls this admin handler at regular intervals to get the stats from envoy? 

On a related note, what do you think about - a generic data plane api for getting stats out of proxy?

Thanks,

Rama

[Timothy Perrett]

There's currently no support for Prometheus AFAIK, only the statsd and sink API I linked to earlier.

[Matt Klein]

There's currently no support for Prometheus AFAIK, only the statsd and sink API I linked to earlier.

Correct. Everyone using Prometheus today is using a statsd -> Prometheus push converter.

On a related note, what do you think about - a generic data plane api for getting stats out of proxy?

No objections. 

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/A96C81F7-83C4-4B75-ABFB-1767CC6073D3%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Rama Rao]

Thanks Matt.

 Let me take a stab at defining a data plane api for it. Will submit a PR and we can discuss on that.

Thanks,

Rama

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/CA%2BOFKguwbRaXX0OuRYYSsE%2B2PD_TV5j3i_zGMaVggwaE3L%2Bhsg%40mail.gmail.com.