Unauthorized message in GUI after setting basic auth.

[Danny de Waard]

For my prometheus  setup i put in basic auth.
This works fine and in grafana i can get the data when putting in the credentials.
But when i enter the GUI on http://myprometheushost:9090 it states Unauthorized

I would expect at least a sign in message.

What am i missing?

[Brian Candler]

Presumably it shows status code 401 Unauthorized, not 403 Forbidden?

Normally, when a browser gets a 401 Unauthorized, it prompts you for basic auth credentials.

Have you tried a different browser? It could be that the browser you're using is set to disable basic auth over unencrypted http, since this would expose the username and password on the network in clear text.

[Danny de Waard]

I did try Chrome and Edge.
I will see if another (non work browser) does the trick.

Op dinsdag 10 januari 2023 om 08:52:52 UTC+1 schreef Brian Candler:

[Danny de Waard]

Firefox portable did the trick.
My question now is, how do i get it working on "normal" edge or chrome?

Op dinsdag 10 januari 2023 om 09:21:05 UTC+1 schreef Danny de Waard:

[Julien Pivotto]

You could try to use http://user@myprometheushost:9090 in your browser
to force the authentication.

It would help to see what the debug tools of the browser have to say.

> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/46093d6e-906d-4cb6-a215-3eb9c871dc28n%40googlegroups.com.


--
Julien Pivotto
@roidelapluie

[Danny de Waard]

Since firefox portable did the trick that's a concern for later ;)
Now i face the issue that the prometheus data is not scraped by itself.
I get:

                "scrapePool": "prometheus",

                "scrapeUrl": "http://10.xx.x.2:9090/metrics",

                "globalUrl": "http://10.x.x.2:9090/metrics",

                "lastError": "server returned HTTP status 401 Unauthorized",

                "lastScrape": "2023-01-10T11:35:54.483356541+01:00",

                "lastScrapeDuration": 0.000457087,

                "health": "down",

                "scrapeInterval": "5s",

                "scrapeTimeout": "5s"

I did include basic_auth in the prometheus config.

scrape_configs:
  - job_name: 'prometheus'
    basic_auth:
      username: 'waardd'
      password: '$2a$12$A/andsometextwcqYNREaAIIill2'
    scrape_interval: 5s
    static_configs:
      - targets: ['10.x.x.2:9090']
        labels:
          host: 'lspe000397'
          srvrole: 'prometheus'
          env: 'management'

The return message is of course:  server returned HTTP status 401 Unauthorized

Op dinsdag 10 januari 2023 om 10:39:32 UTC+1 schreef Julien Pivotto:

[Brian Candler]

On Tuesday, 10 January 2023 at 10:50:13 UTC Danny de Waard wrote:

My question now is, how do i get it working on "normal" edge or chrome?

If your browser has been locked down by your employer to disable basic auth over http, then there's not much you can do.  Vanilla Chrome ought to be fine, although I don't use Windows.

Otherwise, you could run Prometheus with HTTPS, which involves setting it up with a DNS name and certificate (e.g. via LetsEncrypt).  Or maybe your browser would accept a self-signed certificate, again depending on what policies have been applied to it.

 

Now i face the issue that the prometheus data is not scraped by itself.
I get:

                "scrapePool": "prometheus",

                "scrapeUrl": "http://10.xx.x.2:9090/metrics",

                "globalUrl": "http://10.x.x.2:9090/metrics",

                "lastError": "server returned HTTP status 401 Unauthorized",

                "lastScrape": "2023-01-10T11:35:54.483356541+01:00",

                "lastScrapeDuration": 0.000457087,

                "health": "down",

                "scrapeInterval": "5s",

                "scrapeTimeout": "5s"

I did include basic_auth in the prometheus config.

scrape_configs:
  - job_name: 'prometheus'
    basic_auth:
      username: 'waardd'
      password: '$2a$12$A/andsometextwcqYNREaAIIill2'

The password is wrong.  What you have to put there is the actual plaintext password (which gets sent over the wire, the same as you would type into Firefox) - not the hash of the password.

The server side needs only the hash, because it takes the plaintext password received over the wire, hashes it, and compares it to the saved hash.

But if you *send* the hash over the wire, then it will take a hash of the hash, which is wrong.

[Danny de Waard]

Tnx for this help. I got a lot further.

For the prometheus server i could get a certificate with the servername and use that to get it running with https?
Just make the cert, hang it in prometheus, restart and go?

Then Basic Auth on HTTPS would also work?

Op dinsdag 10 januari 2023 om 12:58:17 UTC+1 schreef Brian Candler:

[Brian Candler]

On Tuesday, 10 January 2023 at 12:45:27 UTC Danny de Waard wrote:

For the prometheus server i could get a certificate with the servername and use that to get it running with https?
Just make the cert, hang it in prometheus, restart and go?

Yes.  I run prometheus behind a HTTPS reverse proxy, but as long as you have a reasonably recent version of prometheus, there is built-in functionality you can use:

https://prometheus.io/docs/prometheus/latest/configuration/https/

https://prometheus.io/docs/guides/tls-encryption/

https://inuits.eu/blog/prometheus-server-tls/

(looks like it was introduced in 2.24)

Then Basic Auth on HTTPS would also work?

Yes.

However, since the problem was not with prometheus but with your browser not performing basic auth, and we don't really know why, it's not *guaranteed* this will fix your browser problem.  (Conceivably, your browser has basic auth completely disabled. I've never seen a browser like that, but it's possible).  Try it and see.

[Danny de Waard]

Brian, tnx for all the ionfo. Going to look into the https thingy.
Both our edge and chrome browser are managed by the company and have their AuthSchema set to  ntlm,negotiate
So no basic here and i can not change this. So my guess would be that even with https this basic auth will fail.

Op dinsdag 10 januari 2023 om 15:21:01 UTC+1 schreef Brian Candler:

[Brian Candler]

On Wednesday, 11 January 2023 at 05:41:06 UTC Danny de Waard wrote:

Brian, tnx for all the ionfo. Going to look into the https thingy.
Both our edge and chrome browser are managed by the company and have their AuthSchema set to  ntlm,negotiate
So no basic here and i can not change this. So my guess would be that even with https this basic auth will fail.

Sounds likely.

You could put prometheus behind a reverse proxy with some other authentication mechanism, for example Apache with mod_auth_form.  Then the username/password request will be generated in a HTML form, authenticated, and a session cookie handed out that lets requests pass through.

But really, that's just an attempt to bypass your corporate security policy, which is "we don't allow you to connect to websites which require authentication".