My Vault Use Case / Questions
728 views
Skip to first unread message
Tim Broder
Feb 16, 2016, 11:02:30 AM2/16/16
to Vault
Hi All,
I am working on the encryption piece for our project and was hoping I could validate my use cases / Vault approach & questions with this list
We are building a web and native app that will contain PII. PII will include name, birthday, home address, and tokens representing savings/checking accounts that are given to us by our financial partner. (In the future we will also have tokenized credit card information but that will be in 2017)
We were originally going to be hosted on AWS and I was going to leverage their KMS/S3 encryption, specifically this setup mentioned in their security best practices document: Amazon S3 supports server-side encryption of user data. Server-side encryption is transparent side to the end user. AWS generates a unique encryption key for each object, and then encrypts the object using AES-256. The encryption key is then encrypted itself using AES-256-with a master key that is stored in a secure location. The master key is rotated on a regular basis.
We are moving hosting to a secure host: Armor (formerly firehost), so losing some AWS magic sauce. The whys are outside the scope of this thread but if anyone has experience with them I'd love to hear it!
I'd like to use Vault to encrypt my data and I really liked the idea of having separate keys per user.
The questions I have are:
- The transit backend sounds like what I need, correct?
- If using transit, I still need a separate data store for Vault to store it's keys, correct? (If yes, let's assume I'll use MySQL)
- Vault's MySQL store should be completely separate from the app's MySQL store, correct? (be on different servers)
- What is the best way to get a per-user key in Transmit? The way I think it's done is a) create a named 'master' key. b) create a new key per user using /transit/datakey/ But, how do I tell Vault which user I'm dealing with? Is that where the context comes into play? If so, I'm working through what the context should be if anyone has suggestions. My reading on the subject is starting here
- My thinking is that if MySQL is compromised this approach will make it harder to get all data. But, if Vault itself is compromised, there is minimal gain
- My current backend setup consists of 4 servers: 2x app, 1x DB, 1x Util (Vault, Redis, cron jobs) In this setup, where should I put Vault's store?
- Does this actually offer better protection then encrypting the data in the app using a strong method. I think it does, but also wary of overthinking it
Thanks and much appreciated
miguel...@kiwibank.co.nz
Feb 16, 2016, 9:52:40 PM2/16/16
to Vault
Hi Tim, it does look like transit is what you want, transit can generate a named key per secret and store it for you, all you have to provide is the encrypted data and the named key and will get clear text in return.
I see that your design has no high availability considerations, as there are multiple single points of failure, you might want to think about that.
Cheers
Miguel
I see that your design has no high availability considerations, as there are multiple single points of failure, you might want to think about that.
Cheers
Miguel
Chris Stevens
Feb 17, 2016, 8:23:08 AM2/17/16
to Vault
Tim,
+1 on Miguel's comments.
I think most Vault integrators will at least start out with Consul as the "second data store for Vault to store it's keys" that you mentioned. It provides the HA that Miguel mentioned and a wealth of other capabilities.
Tim Broder
Feb 17, 2016, 11:03:51 AM2/17/16
to vault...@googlegroups.com
Thanks Miguel and Chris,
Fully acknowledging this isn’t that much of a HA setup. The first few months will be validating the product/idea and then expanding our infrastructure
@Miguel - For creating a named key per user, are you referring to /transit/keys/ or /transit/datakey/? If the former, with that being a root protected endpoint, wouldn’t the app having permission at that level be a bad idea?
--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to a topic in the Google Groups "Vault" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vault-tool/AcZVZIoMb9c/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vault-tool+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/ac7b4b05-ecde-4c3d-9700-b96de9c3e23f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jeff Mitchell
Feb 18, 2016, 1:08:29 PM2/18/16
to vault...@googlegroups.com
Hi Tim,
The answers given above are great but I'll add a bit more. At a high
level, transit does indeed work very similarly to the AWS workflow you
described. I think generally people using transit don't use a separate
key per value, but you certainly can if you like, especially using the
backend's ability to upsert new keys on first access.
The questions I have are:
* Vault's MySQL store should be completely separate from the app's
MySQL store, correct? (be on different servers) My thinking is that if
tolerance for maintaining a single data store but possibly with
complications from load across your various apps, etc. The Vault data
itself is encrypted so disclosure to another app is probably not
really a concern.
* What is the best way to get a per-user key in Transmit? The way I
1) Use a single key (with key derivation turned on) and give each user
their own context value. This lets you handle rotation of the key and
the minimum allowed decryption version centrally.
2) Grant each user access to a specific named key. This would require
you to create specific policies for each user, though. It could be
templated but might be annoying. One thing that might eventually be
nice would be to be able to match some part of the client's token to a
path in ACLs, but that doesn't exist yet (and me hypothetically
suggesting it is *not* a guarantee that it will appear :-) )
3) Use data keys. The issue here is that the data keys are given to
the clients who perform encryption and decryption locally -- the
transit backend in this case is purely used to give you a high quality
set of bytes, and to encrypt/decrypt that set of bytes with the chosen
(non-data) key. So you generate a data key for each user and store the
ciphertext of the data keys, then give your users access to the
decryption endpoint. They can use that to get the real key back, and
use that locally.
* My current backend setup consists of 4 servers: 2x app, 1x DB, 1x
nice to have. Also keep in mind that only a few of the physical
backends are officially HC-maintained and supported. We certainly try
to help users with other backends, but mostly rely on community
fixes/patches.
* Does this actually offer better protection then encrypting the data
Encrypting the data in the app using a strong method is nice, but you
have to store the key somewhere. With transit (unless you use data
keys), you don't need to store the key -- you never have it to begin
with. You do still need your app to get a Vault token, but that's more
ephemeral.
* If the former, with that being a root protected endpoint, wouldn’t
we've been removing root protection on endpoints in favor of letting
the (deny-by-default) ACL system do its thing, especially with the new
capabilities model). I've updated the documentation on the web site --
sorry about that!
--Jeff
> You received this message because you are subscribed to the Google Groups
> "Vault" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> https://groups.google.com/d/msgid/vault-tool/etPan.56c499e3.538baed5.2c2%40Tims-MacBook-Pro.local.
The answers given above are great but I'll add a bit more. At a high
level, transit does indeed work very similarly to the AWS workflow you
described. I think generally people using transit don't use a separate
key per value, but you certainly can if you like, especially using the
backend's ability to upsert new keys on first access.
The questions I have are:
MySQL store, correct? (be on different servers) My thinking is that if
MySQL is compromised this approach will make it harder to get all
data. But, if Vault itself is compromised, there is minimal gain
That seems like a fine assessment to me. So it's up to you, and your
data. But, if Vault itself is compromised, there is minimal gain
tolerance for maintaining a single data store but possibly with
complications from load across your various apps, etc. The Vault data
itself is encrypted so disclosure to another app is probably not
really a concern.
* What is the best way to get a per-user key in Transmit? The way I
think it's done is a) create a named 'master' key. b) create a new key
per user using /transit/datakey/ But, how do I tell Vault which user
I'm dealing with? Is that where the context comes into play? If so,
I'm working through what the context should be if anyone has
suggestions. My reading on the subject is starting here
There are a couple of approaches you could take:
per user using /transit/datakey/ But, how do I tell Vault which user
I'm dealing with? Is that where the context comes into play? If so,
I'm working through what the context should be if anyone has
suggestions. My reading on the subject is starting here
1) Use a single key (with key derivation turned on) and give each user
their own context value. This lets you handle rotation of the key and
the minimum allowed decryption version centrally.
2) Grant each user access to a specific named key. This would require
you to create specific policies for each user, though. It could be
templated but might be annoying. One thing that might eventually be
nice would be to be able to match some part of the client's token to a
path in ACLs, but that doesn't exist yet (and me hypothetically
suggesting it is *not* a guarantee that it will appear :-) )
3) Use data keys. The issue here is that the data keys are given to
the clients who perform encryption and decryption locally -- the
transit backend in this case is purely used to give you a high quality
set of bytes, and to encrypt/decrypt that set of bytes with the chosen
(non-data) key. So you generate a data key for each user and store the
ciphertext of the data keys, then give your users access to the
decryption endpoint. They can use that to get the real key back, and
use that locally.
* My current backend setup consists of 4 servers: 2x app, 1x DB, 1x
Util (Vault, Redis, cron jobs) In this setup, where should I put
Vault's store?
On the DB seems fine, but as others have pointed out, an HA setup is
Vault's store?
nice to have. Also keep in mind that only a few of the physical
backends are officially HC-maintained and supported. We certainly try
to help users with other backends, but mostly rely on community
fixes/patches.
* Does this actually offer better protection then encrypting the data
in the app using a strong method. I think it does, but also wary of
overthinking it
What you get using Vault is management of the encryption keys.
overthinking it
Encrypting the data in the app using a strong method is nice, but you
have to store the key somewhere. With transit (unless you use data
keys), you don't need to store the key -- you never have it to begin
with. You do still need your app to get a Vault token, but that's more
ephemeral.
* If the former, with that being a root protected endpoint, wouldn’t
the app having permission at that level be a bad idea?
As of 0.5 these are actually not root-protected any more (in general
we've been removing root protection on endpoints in favor of letting
the (deny-by-default) ACL system do its thing, especially with the new
capabilities model). I've updated the documentation on the web site --
sorry about that!
--Jeff
> "Vault" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> https://groups.google.com/d/msgid/vault-tool/etPan.56c499e3.538baed5.2c2%40Tims-MacBook-Pro.local.
Tim Broder
Feb 19, 2016, 10:40:03 AM2/19/16
to Vault
Hi Jeff,
First off I want to thank you for such a detailed and lengthy response. I really appreciate it
I read through vault-rails to see how it was handling things. (I don't really know ruby but I think I got what I needed)
Combining that with your answer here:
1) Use a single key (with key derivation turned on) and give each user
their own context value. This lets you handle rotation of the key and
the minimum allowed decryption version centrally.
A few more questions:
a) Maybe user was the wrong term, I really meant key per row. vault-rails does this on a per-col level which I could easily do. I'm wondering about the pro/cons of the col approach. Is that because the name of the key can be easily derived based on the col name? I'm also assuming that key rotation would be difficult with a row-based approach
b) Does the context need to be difficult to guess? Or is a base64 of the user id sufficient?
c) on my dev machine, I have 2 databases, homestead for the app, and vault for vault. My HCL is configured as such:
backend "mysql" {
username="homestead"
password="secret"
}
I then mounted transit.
The way I had understood transit and reading through vault-rails is that the encrypted values for each field would be stored in my app DB. I'm also seeing them stored in the vault DB under a logical path: logical/e9906be3-e754-b97a-1fba-a55762a3c7a5/mysql/app/user123 (I added mysql/app as part of the path)
Is this expected? Or should transit be leaving that storage to me and I set something up wrong
Thanks again!
Jeff Mitchell
Feb 19, 2016, 12:01:32 PM2/19/16
to vault...@googlegroups.com
Hi Tim,
> First off I want to thank you for such a detailed and lengthy response. I
> really appreciate it
No problem!
> I read through vault-rails to see how it was handling things. (I don't
> really know ruby but I think I got what I needed)
> Combining that with your answer here:
> 1) Use a single key (with key derivation turned on) and give each user
> their own context value. This lets you handle rotation of the key and
> the minimum allowed decryption version centrally.
>
> A few more questions:
>
> a) Maybe user was the wrong term, I really meant key per row. vault-rails
> does this on a per-col level which I could easily do. I'm wondering about
> the pro/cons of the col approach. Is that because the name of the key can be
> easily derived based on the col name? I'm also assuming that key rotation
> would be difficult with a row-based approach
In that suggestion, what I meant was a key per column -- or even a key
per table/database -- with a context per row. If the context is kept
secret, this essentially gives you as many keys as you want --
per-column or per-row -- but allows rotation and minimum decryption
version to be managed for all of the derived keys at once. But any
approach can work!
> b) Does the context need to be difficult to guess? Or is a base64 of the
> user id sufficient?
Not having a difficult-to-guess context simply changes the security of
the approach. Using different contexts can still be valuable to
prevent bruce-force attacks against a data set if the easy-to-guess
context isn't part of the stolen data set. If it is, it doesn't
provide much benefit. If you want the context to be easy-to-guess, you
might find it an acceptable convenience/security tradeoff to simply
not use contexts at all.
> c) on my dev machine, I have 2 databases, homestead for the app, and vault
> for vault. My HCL is configured as such:
> backend "mysql" {
> username="homestead"
> password="secret"
> }
> I then mounted transit.
>
> The way I had understood transit and reading through vault-rails is that the
> encrypted values for each field would be stored in my app DB. I'm also
> seeing them stored in the vault DB under a logical path:
> logical/e9906be3-e754-b97a-1fba-a55762a3c7a5/mysql/app/user123 (I added
> mysql/app as part of the path)
> Is this expected? Or should transit be leaving that storage to me and I set
> something up wrong
It's hard to say since I don't know on your system which mount
corresponds to that UUID. My guess is that that is a value you wrote
to secret/user123, and that does not itself have anything to do with
transit. (Let me know if I'm right or wrong). Path names within
Vault's backend storage are not encrypted, only the values, so this
would be expected -- the actual value there should be opaque if you
read it out.
Best,
Jeff
> First off I want to thank you for such a detailed and lengthy response. I
> really appreciate it
> I read through vault-rails to see how it was handling things. (I don't
> really know ruby but I think I got what I needed)
> Combining that with your answer here:
> 1) Use a single key (with key derivation turned on) and give each user
> their own context value. This lets you handle rotation of the key and
> the minimum allowed decryption version centrally.
>
> A few more questions:
>
> a) Maybe user was the wrong term, I really meant key per row. vault-rails
> does this on a per-col level which I could easily do. I'm wondering about
> the pro/cons of the col approach. Is that because the name of the key can be
> easily derived based on the col name? I'm also assuming that key rotation
> would be difficult with a row-based approach
per table/database -- with a context per row. If the context is kept
secret, this essentially gives you as many keys as you want --
per-column or per-row -- but allows rotation and minimum decryption
version to be managed for all of the derived keys at once. But any
approach can work!
> b) Does the context need to be difficult to guess? Or is a base64 of the
> user id sufficient?
the approach. Using different contexts can still be valuable to
prevent bruce-force attacks against a data set if the easy-to-guess
context isn't part of the stolen data set. If it is, it doesn't
provide much benefit. If you want the context to be easy-to-guess, you
might find it an acceptable convenience/security tradeoff to simply
not use contexts at all.
> c) on my dev machine, I have 2 databases, homestead for the app, and vault
> for vault. My HCL is configured as such:
> backend "mysql" {
> username="homestead"
> password="secret"
> }
> I then mounted transit.
>
> The way I had understood transit and reading through vault-rails is that the
> encrypted values for each field would be stored in my app DB. I'm also
> seeing them stored in the vault DB under a logical path:
> logical/e9906be3-e754-b97a-1fba-a55762a3c7a5/mysql/app/user123 (I added
> mysql/app as part of the path)
> Is this expected? Or should transit be leaving that storage to me and I set
> something up wrong
corresponds to that UUID. My guess is that that is a value you wrote
to secret/user123, and that does not itself have anything to do with
transit. (Let me know if I'm right or wrong). Path names within
Vault's backend storage are not encrypted, only the values, so this
would be expected -- the actual value there should be opaque if you
read it out.
Best,
Jeff
Timothy Broder
Feb 19, 2016, 4:58:59 PM2/19/16
to vault...@googlegroups.com
Hi Jeff,
Thanks again. I think I have everything I need except some ideas on what makes a good context. I'll make a new thread for that after I've thought through it some more. It might be a beneficial discussion for others
I figured out my issue with C
Thanks!
Thanks again. I think I have everything I need except some ideas on what makes a good context. I'll make a new thread for that after I've thought through it some more. It might be a beneficial discussion for others
I figured out my issue with C
Thanks!
--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to a topic in the Google Groups "Vault" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vault-tool/AcZVZIoMb9c/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vault-tool+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/CAORe8GHzU%3D56r%2BThc1TwQRw1t7tDByj2fsa%2BeSSOZNXG%2BuVeXw%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages