Golang client: metrics internal-only access

[Oleksandr Matsak]

Hi!

I have REST API application and looking for solution for exposing metrics endpoint for internal only access.

I can see two options:

1. use same http.Server as application and protect endpoint with supported auth flow (basic, token, etc)
2. use different http.Server on internal port without any authentication

First has drawback of hardcoding auth credentials on both infrastructure and application sides, I want to avoid it.

While considering second, found that if just put promhttp.Handler to separate server I will loose instrumentation metrics.

So how do I see it: 

• put instrumentation metrics middleware to application server
• run another server on internal only port with metrics output handler

Any advise ? 

Thanks,

Alex

[Julius Volz]

Hi Alex,

both approaches are generally fine, based on your needs. With "losing instrumentation metrics", do you mean the "promhttp_*" metrics about the scrape itself? Those should still be present when just using "promhttp.Handler()" on a different http.ServeMux and starting a second server for that ServeMux. Could you share the relevant lines of your code and/or elaborate?

Regards,

Julius

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/54038168-acda-40f2-97f7-7fc1212e926cn%40googlegroups.com.

--

Julius Volz

PromLabs - promlabs.com

[Oleksandr Matsak]

Hi, Julius! 

Thanks for quick reply.

Yes, you are right, if I will use separate http.ServeMux on separate server I will loose "promhttp_*" metrics for my application(because they will be counted for separate server).

I have found the solution, but don't like it.

вторник, 16 августа 2022 г. в 21:05:14 UTC+3, juliu...@promlabs.com:

[Julius Volz]

Hi Oleksandr,

I'm a bit confused. The "promhttp_"-prefixed metrics are about the scrape requests themselves and they are still tracked fine when tying the promhttp.Handler() to another mux and server (because the scrape requests are now happening through that server, and the handler is also doing its tracking there now). Or are you talking about HTTP metrics for the rest of your application's HTTP handlers (not /metrics) that is running on the other server?

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/1766f79b-ab33-401c-8e0d-f2e7e935ff35n%40googlegroups.com.

[Oleksandr Matsak]

Look please at the code below

metricsMux := http.NewServeMux()
// here I want only to expose metrics outside, without any tracking
// just gather them and write response
metricsMux.Handle("/metrics", promhttp.Handler()) // <------
metricsSrv := &http.Server{Handler: metricsMux}

appMux := http.NewServeMux()
// here I want to add prometheus middleware with "promhttp_"-prefix tracking
// only tracking, without any output written

appSrv := &http.Server{Handler: appMux}

If simply add promhttp.Handler() handler to metrics server(as in my example), "promhttp_"-prefix metrics will contains info about requests to metrics server and it's useless information. I want them to contain metrics for my application.

Short: I'm trying to follow separation of concerns while tracking and exposing metrics.

среда, 17 августа 2022 г. в 13:38:17 UTC+3, juliu...@promlabs.com:

[Oleksandr Matsak]

Hi, Julius! 

Thanks for your time and I'm sorry, it's my misunderstanding nature of "promhttp_" metrics.

Currently reread documentation and now everything is clear.

среда, 17 августа 2022 г. в 14:29:38 UTC+3, Oleksandr Matsak: