Prometheus and Alertmanager behind Apache Proxy

380 views
Skip to first unread message

wpjo...@gmail.com

unread,
Oct 22, 2022, 8:46:12 AM10/22/22
to Prometheus Users
Hi, I have Prometheus and Alertmanager running behind an Apache Proxy.

My problem is that i cannot make Alertmanager accessible through
"https://my.url.com/alertmanager"
With my current config i only get an 404 for that site.

I start Alertmanager with following flags:

ExecStart=/etc/alertmanager/alertmanager --config.file=/etc/alertmanager/alertmanager.yml --storage.path=/etc/alertmanager/data --cluster.listen-address= --web.config.file=/etc/prometheus/web.yml --web.external-url=https://my.url.com/alertmanager --web.route-prefix=/alertmanager

and my Proxy looks like this:
<VirtualHost *:443>
...
ProxyPass           / http://localhost:9090/
ProxyPassReverse    / http://localhost:9090/
ProxyPass           /alertmanager http://localhost:9093/alertmanager/
ProxyPassReverse    /alertmanager http://localhost:9093/alertmanager/
</VirtualHost>

please send some help!!!

Michael Kogelman

unread,
Oct 22, 2022, 8:48:04 AM10/22/22
to wpjo...@gmail.com, Prometheus Users
Get rid of the /alertmanager in the proxypass statements and you should be good. 

From: promethe...@googlegroups.com <promethe...@googlegroups.com> on behalf of wpjo...@gmail.com <wpjo...@gmail.com>
Sent: Saturday, October 22, 2022 8:46:12 AM
To: Prometheus Users <promethe...@googlegroups.com>
Subject: [prometheus-users] Prometheus and Alertmanager behind Apache Proxy
 
--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/98a2def6-78ee-41b4-a192-10ba117888bcn%40googlegroups.com.

wpjo...@gmail.com

unread,
Oct 22, 2022, 8:54:41 AM10/22/22
to Prometheus Users
Wow thanks for the fast response!


It looks like his now:
ProxyPass           / http://localhost:9090/
ProxyPassReverse    / http://localhost:9090/
ProxyPass           / http://localhost:9093/
ProxyPassReverse    / http://localhost:9093/

but does not work either.

sayf.eddi...@gmail.com

unread,
Oct 22, 2022, 9:31:08 AM10/22/22
to Prometheus Users
I use the following jinja template to expose alertmanagers on `/alertmanager`

###########################################################

# Alertmanager

###########################################################

Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED

<Proxy balancer://alertmgrcluster>

{%- for member in members %}

BalancerMember http://{{ member }}:{{ port }} route={{ loop.index }} timeout=5

{%- endfor %}

ProxySet lbmethod=bybusyness

ProxySet stickysession=ROUTEID

</Proxy>

ProxyPass /alertmanager balancer://alertmgrcluster

ProxyPassReverse /alertmanager balancer://alertmgrcluster

wpjo...@gmail.com

unread,
Oct 22, 2022, 9:34:59 AM10/22/22
to Prometheus Users
Thanks,
i am running no cluster. 
Alertmanager and prometheus are on the same machine.

i am somhow not able to make this work with apache and SSL

best regards
Philipp

Brian Candler

unread,
Oct 22, 2022, 3:35:34 PM10/22/22
to Prometheus Users
I run it like that and It Works For Me™

Alertmanager options:

OPTIONS='--config.file=/etc/prometheus/alertmanager.yml --web.external-url=http://example.com/alertmanager'
(note: I am *not* using any other options, including --web.path-prefix)

Prometheus options:

OPTIONS='--config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/var/lib/prometheus/data/ --storage.tsdb.wal-compression --web.external-url=http://example.com/prometheus'

Apache config:

ProxyPass /prometheus http://127.0.0.1:9090/prometheus
ProxyPassReverse /prometheus http://127.0.0.1:9090/prometheus
ProxyPass /alertmanager http://127.0.0.1:9093/alertmanager
ProxyPassReverse /alertmanager http://127.0.0.1:9093/alertmanager

I think your problem might be that you also have "ProxyPass /" which routes the whole lot somewhere. I seem to remember it was very difficult to "punch a hole" in proxying like this, without using mod_rewrite and [P] (proxy) RewriteRules.  Try removing it.

wpjo...@gmail.com

unread,
Oct 22, 2022, 3:38:49 PM10/22/22
to Prometheus Users

Hey Brian,

this worked for me too!
"ProxyPass /" was the Problem after all. Now that both use a context path everything is fine.

Thanks for your help!
Reply all
Reply to author
Forward
0 new messages