How to add http_proxy environment variable to prometheus alertmanager.yml ?

[Pravin S]

Hi

Please can anyone suggest how to add the environment variable of http_proxy as i am facing issues with sending alert notifications to slack. 

I am able to see the alert coming Prometheus alert manager : local host:9093/#/alerts but they are not getting forwarded to slack due to firewall issues so had to use the proxy. I manually tried to export the proxy variables at OS level but they kind of  worked if I try the manual curl -X post command to send the alerts to slack but somehow alert manager not picking up the setting at os level so have to define it somewhere inside the yaml file to pick it up not sure if its going to prometheus or alertmanager.yml file.

Please can anyone suggest and advise of the exact syntax and correct yml file where this can be defined. I am not using the docker container rather doing on standalone linux machine but getting stuck at corporate firewall network .

thanks

pravin

[sayf eddine Hammemi]

For me it is easier to setup alert manager as a systemd service then add a drop-in directory to force proxy only on it with the file :

/usr/lib/systemd/system/alertmanager.service.d/proxy.conf that contains
[Service] Environment="HTTP_PROXY=http://proxy.example.com:80/" "NO_PROXY=localhost,127.0.0.1,<Prometheus IP>"

[Pravin S]

Thanks Sayf. 

I already have alertmanager.service file to start/stop the alertmanager.  Is there any way to add the file reference inside this alertmanager.service file ?

cat /usr/lib/systemd/system/alertmanager.service.d/proxy.conf

[Service] Environment="HTTP_PROXY=http://proxy.example.com:80/" "NO_PROXY=localhost,127.0.0.1,<Prometheus IP>"

$ cat alertmanager.service:   -- current file output

[Unit]
Description=AlertManager Server Service
Wants=network-online.target
After=network-online.target

[Service]
User=root
Group=root
Type=simple
ExecStart=/home/opc/software/prometheus/alertmanager/alertmanager  --config.file /home/opc/software/prometheus/alertmanager/alertmanager.yml

[Install]
WantedBy=multi-user.target

thanks

praveen

[Pravin S]

Thanks Sayf .

That solution worked by adding the reference of proxy.conf inside the alertmanager.service. but still notification are not going to slack.

Able to see the data output on alertmanager "http://localhost:9093/api/v1/alerts" when instance was shutdown but still notification not getting fwd to slack.  Below is config output where it has loaded the alertmanager.yml and picked the slack channel name but didn't send the alert.  Any idea ?

thanks

pravin

[Mubeen Nakade]

Hi Pravin S,

Please use below, its appropriate way and a 100% solution.

In file ---> alertmanager.yml

Under Global Section

global:

  http_config:

    proxy_url: 'http://your-proxy:3128'