Renewing root and intermediate certificates using Vault's PKI backend

[Francis Chuang]

Hey guys,

I am looking into deploying vault to serve as a secrets store and private CA to run my microservices infrastructure. In particular, I am interested in using PKI to issue certificates for all instances of my microservices (for https) and to also use the certificates to support tls client authentication between services.

Following the example in the docs and the blog post, I was able to get the root CA and intermediate CA up and running relatively easily on my machine. I noticed that the root certificate has a validity of 10 years and the intermediate CA has a validity of 6 months.

When it comes to the expiration of the root certificate and the intermediate certificate, what is the best practice for renewing or recreating them? Should I unmount /rootpki and /intermediatepki and recreate them? Obviously this will lead to downtime, so I should probably create /rootpki2 and /intermediatepki2 before the expiration, transition all the services to the new ca and intermediate. However, if I revoke the original /rootpki and /intermediatepki, the CRLs would still be at /rootpki and /intermediatepki and if I unmount the path, the CRLset will be lost.

Has anyone used this in production? What is your procedure? Also, I noticed that creating an intermediate CA is a pretty manual process. We need to generate the intermediate cert, save the CSR, sign it with the root cert, submit the signed cert back to vault and then start issuing certificates. Are there any plans to automate this further so that intermediate CAs can be created with just 1 command?

Also, I only plan to use the private CA to issue certs for microservices within the infrastructure. In that case, is it still useful to have an intermediate CA? If there is an intrusion, the root should be revoked and recreated anyway.

Cheers,

Francis

[Jeff Mitchell]

Hi Francis,

The general way that root and intermediate CAs are managed are:

* The root is never "renewed" -- once it expires you need a new root.
However, since this lifetime is typically extremely long, you can plan
for this well in advance and spin up your new root, say, two years
before the old one expires.

* The intermediate CAs are either swapped out fully or the private key
is reused with a new CSR so that the issuer's key ID stays the same.
In either case, you end up you generating a new intermediate CA
certificate and switch to using that for issuing certificates. You
would generally want to start issuing from the new certificate at a
duration before the previous expires that is equivalent to the maximum
duration of the certificates it's issuing. IOW, if the max lifetime of
any certificate you issue is 30 days, you want to start issuing from
the new intermediate CA at least, and ideally more than, 30 days
before the prior one expires. That way, by the time the old
intermediate CA expires, you can be sure that there will be no broken
trust chains.

BTW, something to note: prior to current master, issuing CAs from the
PKI backend generated leases, which meant that if your token lifetime
wasn't at least as long as the lifetime of your CA, it'd end up being
revoked. Usually people used root tokens so this wasn't a problem, but
in some cases it has ended up being problematic, so we've changed this
behavior in master and for the next release. Generating CA certs will
no longer generate associated leases, and when running a newer version
of Vault if revocation of a CA is attempted due to a lease expiring,
the lease is silently discarded instead.

Best,
Jeff

> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in violation
> of those guidelines may result in your removal from this mailing list.
>
> GitHub Issues: https://github.com/hashicorp/vault/issues
> IRC: #vault-tool on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Vault" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vault-tool+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/vault-tool/c4df0514-02a9-4fc1-81d1-22cba4dfe71c%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Francis Chuang]

Hey Jeff,

Thanks! That makes a lot of sense.

Francis

[Romain Buisson]

The intermediate CAs are either swapped out fully or the private key
is reused with a new CSR so that the issuer's key ID stays the same.
In either case, you end up you generating a new intermediate CA
certificate and switch to using that for issuing certificates

This means that one needs to create a new PKI backend for the new CA? or can the CA be "swapped" within the same backend somehow?