Vault + Bastion Host: Best Practices or Alternative?

2,153 views
Skip to first unread message

Josh Padnick

unread,
Apr 13, 2016, 4:34:12 PM4/13/16
to Vault
We're setting up a Vault cluster as a secret store to be used both by apps and individual developers; we also plan to use the SSH Secret Backend. I'd like the Vault cluster to be in private subnets, but since individual developers need access, they need a way to reach Vault to authenticate against it.

The standard solution here is a Bastion Host or VPN. In the interest of simplicity, it makes sense in this case for us to use a Bastion Host, but now I'm running into a chicken-and-egg problem:
  • Ideally, I'd like the users to authenticate against Vault, then SSH into the Bastion Host (using Vault's One-Time Password) to get where they need to go. But that requires access to Vault, which is behind the Bastion Host.
  • Alternatively, I expose Vault to the public, but now it's no longer in the private subnet.
One simple option would be to have a separate Bastion Host that independently implements its own method of authentication (e.g. GitHub oAuth) which is the same as Vault, but then I wonder if it's adding any value over just using Vault itself as the public-facing Bastion Host which both issues OTP's and allows SSH connections to be proxied?

I'm wondering how others have solved this problem, or if there's an alternative setup I'm missing?  Thanks for your input!

Josh

Timothy Kimball

unread,
Apr 14, 2016, 5:52:03 AM4/14/16
to Vault
Hi Josh -

I am sure Jim will have better ideas - and I am looking forward to hearing his response. This is me chucking my two cents into the pot.

Adding layers gives you multiple orthogonal means of securing your vault server (e.g. separate bastion versus not) - but increases complexity. A constant tradeoff in secure designs. 

With that said, I would suggest you use a VPN rather than a bastion:
- Use a VPN such as OpenVPN. This allows you to configure MFA via OpenVPN.
- Configure the VPN server to be able to access the private subnet where the Vault server resides. In AWS you would put the VPN server in the public subnet in a VPC and vault in a private one. Use terraform to create it for you :D
- Setup firewall and router rules so that only the VPN server can talk to the private vault server

Reasons:
- VPN easier to manage and configure
- You have a generic way to access resources in the public subnet - you do not need to setup port forward rules for new services you put there.
- You get MFA for free. Always nice :D

Hope that helps,
Tim

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Vault" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vault-tool+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/2f19362b-555e-4d75-a527-04413f6ab735%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Timothy Kimball

unread,
Apr 14, 2016, 5:53:11 AM4/14/16
to Vault
opps - meant 'private subnet' in the second to last bullet point :D

Josh Padnick

unread,
Apr 14, 2016, 4:44:45 PM4/14/16
to Vault
Hi Tim,

Thanks for your input. So you're proposing keeping Vault private and using a separate entrypoint into the VPC, but replacing the bastion host with a VPN.  I agree VPN is more robust, and it looks like OpenVPN allows you to write a PAM module to implement any authentication method you want.

I'm curious to hear how others have set this up. I suppose using Vault is an "enterprisey" thing so most Vault shops will already have VPN established. Any happy Vault Bastion Host users out there?

Josh

Jeff Mitchell

unread,
Apr 14, 2016, 5:07:31 PM4/14/16
to vault...@googlegroups.com
Hi Josh,

Based on my discussions with users, it seems like many of them
actually do have Vault running publicly, but also have public SSH
access to their machines; or, they have Vault in a private area but
their users also have access to that private area. That's the VPN
approach, where users VPN into the private network, then use Vault for
actual SSH access control to machines.

So basically: it seems that users generally like having Vault located
wherever access to your machine lies.

Best,
Jeff
> https://groups.google.com/d/msgid/vault-tool/46491067-e5a4-4d95-89cd-97ef2bea03bd%40googlegroups.com.

Josh Padnick

unread,
Apr 15, 2016, 5:40:56 PM4/15/16
to Vault
Hi Jeff,

That's very helpful information. What I conclude from the discussion so far is:
  • If you want to use Vault as the system of record for identity, it needs to be public (or proxied by something that's public).  In this case, you could use it as a bastion host directly, or it's still probably better to use it to gain access to a separate bastion host and then to a private instance.

  • If your system of record for identity is not Vault, then you can write your own custom Bastion AMI that, e.g. uses GitHub to authenticate users, or use VPN with your authn method of choice.

  • VPN is ideal if you can get it setup, but it's more complex than the Bastion Host. It's probably worth taking time to use a well-thought out system of record for identity for either Bastion Host or VPN beyond using their defaults.
Josh

Richard Mauri

unread,
Dec 6, 2018, 2:59:31 PM12/6/18
to Vault
I built a tool to perform ssh via vault signing ca ssh keys.
It helps reduce the temptation to store private keys on shared bastion hosts where ssh agent forwarding is not allowed.

Yan Michalevsky

unread,
Mar 1, 2019, 7:24:56 PM3/1/19
to Vault
Hi Josh,
One approach could be setting up the Vault instance that stores the SSH keys such that it is publicly available, however, protecting it by running it inside a secure enclave (like Intel SGX).
This would enable your users to connect to Vault through the legitimate API, but will protect it against any attacker that gets access to the host (even if they get root).
My company, Anjuna.io, has a tool that does exactly that.
Feel free to reach out in case you're interested and I'll give you access.

Best,
Yan


On Wednesday, April 13, 2016 at 1:34:12 PM UTC-7, Josh Padnick wrote:

Richard Mauri

unread,
Mar 1, 2019, 7:59:23 PM3/1/19
to vault...@googlegroups.com
Check out my vaultssh client for this! 

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to a topic in the Google Groups "Vault" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vault-tool/6Y5WIaL94d4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vault-tool+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/b2d5d3fe-0f98-4a48-b122-bd8813de5dd5%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages