How to hide auth password writing in alertmanager for alerting

123 views
Skip to first unread message

mohd

unread,
Mar 12, 2020, 2:24:40 AM3/12/20
to Prometheus Users
Hello Team,

How to hide auth password writing in alertmanager for alerting.
I am running setup on docker

Version:      17.05.0-ce
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04


alertmanager config:
route:
  repeat_interval: 1d
  group_interval: 1d
  group_by: [Alertname]
  # Send all notifications to me.
  receiver: email-me
receivers:
- name: email-me
  email_configs:
  - to: receiver-emil-address
    from: sender-email-address
    smarthost: smtp.gmail.com:587
    auth_username: "sender-email-address"
    auth_identity: "sender-email-address"
    auth_password: "password"


Any help would be greatly appreciated.

Thank you.

Brian Candler

unread,
Mar 12, 2020, 4:44:38 AM3/12/20
to Prometheus Users
Hide it from whom?  Someone who uses the alertmanager web interface?  Someone who has direct access to the filesystem of the docker container?

mohd

unread,
Mar 12, 2020, 6:14:56 AM3/12/20
to Prometheus Users
Many thanks for your reply Brain Candler,

The one who has direct access to the filesystem of the docker container.
And also want to know at ubuntu level  for filesystem.

Thank you.

Brian Candler

unread,
Mar 12, 2020, 8:56:35 AM3/12/20
to Prometheus Users
On Thursday, 12 March 2020 10:14:56 UTC, mohd wrote:
The one who has direct access to the filesystem of the docker container.

You can't. You can have an encrypted filesystem, but your Ubuntu user will be able to read it as well.

You could:
- keep the config encrypted in a gpg file
- unpack it into a RAM disk (tmpfs)
- start alertmanager
- delete the tmpfs

You would have to go through these steps every time you (re)start alertmanager.

You could build your own separate SMTP relay, which doesn't need authentication, which then relays to your final SMTP server.  But obviously the password will be exposed in that SMTP relay's configuration.

Christian Hoffmann

unread,
Mar 14, 2020, 7:19:07 AM3/14/20
to mohd, Prometheus Users
On 3/12/20 11:14 AM, mohd wrote:
> The one who has direct access to the filesystem of the docker container.
> And also want to know at ubuntu level  for filesystem.

I fully agree with what Brian said, just want to add another opinion:

I commonly see this as a mis- or overinterpretation of security
guidelines, such as "You should not have plaintext secrets". I think
such statements are primarily targeted at user databases where you don't
have to have plaintext secrets at all. You can (and should) use hashing
or password derivation algorithms such as PBKDF2, scrypt and bcrypt.

However, that's a totally different use case to what Alertmanager has to
do here: Alertmanager behaves as an SMTP client. It therefore needs to
have a usable, unencrypted secret at some point. Every encryption or
obfuscation you add on top is just that -- obfuscation, e.g. Security by
obscurity. You can add multiple layers of complexity and encryption on
top. At some point, you will still have to store the
master-master-master decryption key somewhere and haven't reached any
real improvement at all.

There is a single (weak) argument that I buy regarding plaintext
passwords: They could be stolen by "shoulder surfing". There probably is
an easy remedy as well: Just make them long and cryptic enough. There is
no reason not to when talking about machine-to-machine authentication.

There is also a wide variety of "enterprise tools" which try to fool you
into believing that some magic method for solving this problem exists.
In practice, checking the "use encrypted passwords" checkbox usually
means encrypting passwords with some static key in the binary. In my
opinion, this just adds complexity. It does not add a security boundary.

There are two things which you *can* do to improve security:

- Use the principle of least privilege. In this case: Ensure that
alertmanager config file permissions are set in a way that it only
allows access to the user running alertmanager (e.g. chown
alertmanager:alertmanager, chmod 600). Also ensure that only those
people have super-user permissions on your machine who really need this.

- Use separation of duties. In this case: Use a password which is only
used for authentication against your SMTP server. As a drastic contrast:
Do not use the same password as for your root account or something like
that.


Kind regards,
Christian
Reply all
Reply to author
Forward
0 new messages