prometheus alermanager not sending alert CC user list

118 views
Skip to first unread message

Prashant Singh

unread,
Jan 20, 2023, 7:38:24 AM1/20/23
to Prometheus Users
Dear team,

prometheus alertmanager CC is not working even as add TO_ and CC_ . 

TO:  it is working 
CC: it is not working.


- name: 'MoniDashboard'
  email_configs:
  - send_resolved: true
    to: 'vinoth.sundaram
    headers:
      cc: 'PrashantKumar.Singh1'
    require_tls: no

thanks
Prashant 

Julius Volz

unread,
Jan 20, 2023, 8:28:42 AM1/20/23
to Prashant Singh, Prometheus Users
Hi Prashant,

Looking at the email implementation in Alertmanager, "to" should be treated exactly as "cc" internally (just optionally supplied through a dedicated YAML field). It's just another header: https://github.com/prometheus/alertmanager/blob/f59460bfd4bf883ca66f4391e7094c0c1794d158/notify/email/email.go#L53-L70

So I would be surprised if the problem is in Alertmanager itself, rather than something in the email pipeline after it.

Kind regards,
Julius

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/9987573c-ed69-4fb2-94aa-4c37870c540dn%40googlegroups.com.


--
Julius Volz
PromLabs - promlabs.com

Julien Pivotto

unread,
Jan 20, 2023, 8:34:51 AM1/20/23
to Julius Volz, Prashant Singh, Prometheus Users
To send an email with a CC: in alertmanager, it is not sufficient to add a CC: header.

receivers:
- name: my-receiver
email_configs:
- to: 'o...@foo.com'
headers:
subject: 'my subject'
CC: 't...@bar.com'

You also need to add the CC: address to the to: field and explicitely add a to: field.

receivers:
- name: my-receiver
email_configs:
- to: 'o...@foo.com,t...@bar.com'
headers:
subject: 'my subject'
To: 'o...@foo.com'
CC: 't...@bar.com'

To send a mail in BCC:, overwrite the To: header and add the BCC addresses to the to: field:

receivers:
- name: my-receiver
email_configs:
- to: 'o...@foo.com,th...@foo.com'
headers:
subject: 'my subject'
To: 'o...@foo.com'
> > <https://groups.google.com/d/msgid/prometheus-users/9987573c-ed69-4fb2-94aa-4c37870c540dn%40googlegroups.com?utm_medium=email&utm_source=footer>
> > .
> >
>
>
> --
> Julius Volz
> PromLabs - promlabs.com
>
> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAObpH5wv7WVme5ysp720Cj815fJdwATria-%2B0yDW0sG46_82XA%40mail.gmail.com.

--
Julien Pivotto
@roidelapluie

Julius Volz

unread,
Jan 20, 2023, 9:03:08 AM1/20/23
to Julius Volz, Prashant Singh, Prometheus Users
Ah, thanks for that addition. Indeed, "to" is handled in a special way here: https://github.com/prometheus/alertmanager/blob/f59460bfd4bf883ca66f4391e7094c0c1794d158/notify/email/email.go#LL230C28-L230C28

Which makes sense given that in SMTP, there's always separate "RCPT TO: <...>" lines (which I guess this translates into) before the body of the email that contains the headers.

Prashant Singh

unread,
Jan 23, 2023, 1:17:05 AM1/23/23
to Prometheus Users
Hi ,

this is not working , i am not able to receive alerts  in having user  CC 

- name: 'MoniDashboard'
  email_configs:
  - send_resolved: true
    to: 'vinoth.sundaram@
    headers:
      subject: "{{ .CommonAnnotations.summary }}"
      to: 'vinoth.sundaram@
      CC: 'PrashantKumar.Singh1@
    require_tls: no


Thanks 
prashant 

Brian Candler

unread,
Jan 23, 2023, 3:10:16 AM1/23/23
to Prometheus Users
You've mangled the YAML so it definitely won't work with what you've shown.

You'll need something like this:

- name: 'MoniDashboard'
  email_configs:
  - send_resolved: true
    to: 'vinoth.sundaram@domain1,PrashantKumar.Singh1@domain2'
    headers:
      Subject: "{{ .CommonAnnotations.summary }}"
      To: 'vinoth.sundaram@domain1'
      Cc: 'PrashantKumar.Singh1@domain2'
    require_tls: no

The important thing is that you need the 'to' value to be a comma-separated list of recipient addresses.  See:

If it still doesn't work how you want, then you need to be more specific about your problem than "this is not working". For example:
- If the message isn't delivered to both receipients, you'll need to look at the log output of alertmanager and the logs from your E-mail relay (SMTP host)
- If the message is delivered to both receipients, but the "To" and "Cc" headers don't appear in the way that you want, then you'll need to show the actual message headers that you receive, and explain how they differ from what you expected.
Reply all
Reply to author
Forward
0 new messages