Vault 0.4: when unsealing vault (3 keys required), third key throws error

[Jerry Walling]

We had to restart/unseal our vault server today. We require 3 keys to unseal. The first two keys are accepted fine:

   {"sealed":true,"t":3,"n":5,"progress":1}

   {"sealed":true,"t":3,"n":5,"progress":2}

But when we attempt to enter the third key, we receive the following error:

   {"errors":["Unseal failed, invalid key"]}

The progress then reverts back to 0:

   {"sealed": true,"t":3,"n":5,"progress": 0}

At first we thought one of the keys was incorrect, or entered incorrectly, so we tried 3 different ones. Same result. Then, in a final test we entered the keys in a different order with the same result. It seems that no matter what key is entered as the third key, vault throws this error and resets the seal progress, even though the key is valid. 

Any assistance in resolving this issue would be much appreciated.

Regards,
Jerry

[Jeff Mitchell]

Hi Jerry,

When the three keys are entered, Vault attempts to unseal itself using
the unseal keys. The error you're getting back is saying that the
reconstructed key from the unseal keys is not valid to decrypt the
master key. So it's not saying that whatever you are entering for the
third key is invalid, it's just that it doesn't actually attempt
decryption until it has three keys.

Likewise, the first two keys being accepted says nothing about their
validity, because decryption is not attempted until the required
number of unseal keys is provided.

I'm not sure how you are running the unseal, but I've definitely seen
this kind of behavior in copy+pasting scenario with hidden/unexpected
whitespace. In the CLI I believe that usually will give an error about
the unseal key being the wrong length, but I don't remember offhand if
the core will perform similar checking until it gets a quorum.

Another possibility is that Vault was rekeyed and you're trying to use
the old unseal keys.

I know that seems like it's grasping at straws a little bit, but this
is a very well-tested code path in Vault both in unit tests and the
real world. Without any more information, the answer is likely that
the data you're passing in is the culprit. You may also want to check
the server logs for anything that seems off, though.

Best,
Jeff

> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in violation
> of those guidelines may result in your removal from this mailing list.
>
> GitHub Issues: https://github.com/hashicorp/vault/issues
> IRC: #vault-tool on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Vault" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vault-tool+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/vault-tool/ee73bc50-af14-4167-9e3d-7cbaf7de9985%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.