Bootstrapping a HA Vault setup and setting up certs and authentication

748 views
Skip to first unread message

Krishna Raman

unread,
Mar 31, 2017, 3:00:40 PM3/31/17
to Vault
This is a follow up to twitter conversation https://twitter.com/sethvargo/status/847858452785508353.

Im trying to setup Vault with a HA etcd3 or consul backend. Either case requires TLS certs and user/pass credentials.
Currently those are not password protected and some are even stored as plaintext in config files. 

If using a non-HA setup, as suggested by Seth, I can generate a self-signed cert and bring up vault with it, then swap out the certificates.
But when using HA, for example with etcd, I need additional certs to secure vault <--> etcd and etcd <--> etcd communication across multiple hosts.

How have you tackled this security bootstrap issue?

Thanks
Krishna

Jeff Mitchell

unread,
Mar 31, 2017, 5:15:12 PM3/31/17
to Vault
Hi Krishna,

We don't know of a super straightforward way to tackle this. There are lots of potential things you can do but they all have trade-offs.

For instance, one possibility is to stand up one Vault server using a self-signed cert and the file backend, then enable the PKI mount on it with a long-lived root CA, and use that to sign certificates for both etcd and a new Vault listener, which once it's set up can then create an intermediate PKI mount whose CA cert is signed by the root. But...not super fantastic.

Another possibility is to just use a fresh self-signed cert everywhere (including etcd3) until you get to a point where you can issue your own certificates, and then replace with those.

Often, people simply use an external CA to sign the etcd3/Vault listener certs, then use Vault to issue certs for everything else.

Best,
Jeff

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Vault" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vault-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/c9405eaf-4cf9-40ad-a987-e2c501060705%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Krishna Raman

unread,
Mar 31, 2017, 6:56:04 PM3/31/17
to Vault
Hi Jeff,

Thanks for the suggestions. It is possible that Im trying to use vault for something its not intended for. If so, please let me know :)
Unfortunately none of these options allow for good automation. 

(1) Have to keep the file-stores in sync in each node somehow and initializing nodes on reboot requires 2 vault unlocks on each node.
(2) This doesn't work great in a recovery scenario when etcd server is rebooting since the self-signed cert would not be accepted by its peers.
(3) this means that we have to store the certs using encrypted form** and require 2 sets of passwords, one to decrypt (on each node) and again to unlock vault.

** Has to be encrypted at rest

Have you seen anyone attempt to automate a HA vault setup?

Thanks
Krishna


On Friday, March 31, 2017 at 2:15:12 PM UTC-7, Jeff Mitchell wrote:
Hi Krishna,

We don't know of a super straightforward way to tackle this. There are lots of potential things you can do but they all have trade-offs.

For instance, one possibility is to stand up one Vault server using a self-signed cert and the file backend, then enable the PKI mount on it with a long-lived root CA, and use that to sign certificates for both etcd and a new Vault listener, which once it's set up can then create an intermediate PKI mount whose CA cert is signed by the root. But...not super fantastic.

Another possibility is to just use a fresh self-signed cert everywhere (including etcd3) until you get to a point where you can issue your own certificates, and then replace with those.

Often, people simply use an external CA to sign the etcd3/Vault listener certs, then use Vault to issue certs for everything else.

Best,
Jeff
On Fri, Mar 31, 2017 at 12:00 PM, Krishna Raman <kra...@gmail.com> wrote:
This is a follow up to twitter conversation https://twitter.com/sethvargo/status/847858452785508353.

Im trying to setup Vault with a HA etcd3 or consul backend. Either case requires TLS certs and user/pass credentials.
Currently those are not password protected and some are even stored as plaintext in config files. 

If using a non-HA setup, as suggested by Seth, I can generate a self-signed cert and bring up vault with it, then swap out the certificates.
But when using HA, for example with etcd, I need additional certs to secure vault <--> etcd and etcd <--> etcd communication across multiple hosts.

How have you tackled this security bootstrap issue?

Thanks
Krishna

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Vault" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vault-tool+...@googlegroups.com.

Jeff Mitchell

unread,
Mar 31, 2017, 9:41:00 PM3/31/17
to Vault
On Fri, Mar 31, 2017 at 3:56 PM, Krishna Raman <kra...@gmail.com> wrote:
Have you seen anyone attempt to automate a HA vault setup?

Sure, it happens all the time, just not using Vault to issue it and its storage backend's certificates. They just use self-signed certs for those or externally sourced certs.

Best,
Jeff
Reply all
Reply to author
Forward
0 new messages