AES Decryption: wrong key detection

[Ahewd.com]

I have an application which encrypts a file with AES in CBC mode,
using a password entered by user to generate a key. I'm using
FileSource with StreamTransformationFilter to do the job.
The problem is that if the user enters wrong password and decrypts the
file, FileSource bombs out (after the whole file is decrypted) with
exception: "StreamTransformationFilter: invalid PKCS #7 block padding
found".

What's the correct method to detect that a file to decrypt was
encrypted with a different key? Is it safe to catch that exception for
this purpose? I need to notify the user that the password entered is
wrong.
Ideally, this would have to be done without first having to decrypt
the entire file, which may take a long time when the file is large.

This question must already have been asked many times but i can't find
the answer...

[Geoff Beier]

Caveat: I can't think of an application I've done where I consider
rapid wrong-key detection a feature I want. Usually, I'm perfectly
happy for that to take as long as I can feasibly make it take. The
faster I tell someone about an incorrect key, the faster they can
brute-force my real key, as far as I see it.

That said, If I wanted to rapidly detect a wrong key, I'd concoct some
known or agreed-upon value, and use the password-derived decryption
key (or a hash thereof if it's too long to use directly) as the key
for an HMAC over the known or agreed-upon value. I'd store that with
the file and calculate it prior to attempting to decrypt the file to
see if my decryption key was valid.[1]

But let me reiterate... slow discovery of an invalid decryption key
sounds more like a feature than a bug to me, so I'd think very
carefully before implementing this kind of scheme. I suspect that this
is why you can't find the answer. Why do you want to make it faster to
attack your encryption scheme?

Geoff

[1] An authenticated mode like GCM might achieve the same goal here.
I'm not sure. My instinct is that this application would be an abuse
of such a mode and would provide neither security nor performance
advantages over what I describe, but I haven't spent much time
studying authenticated modes and could definitely be wrong about that
assertion.

> --
> You received this message because you are subscribed to the "Crypto++ Users" Google Group.
> To unsubscribe, send an email to cryptopp-user...@googlegroups.com.
> More information about Crypto++ and this group is available at http://www.cryptopp.com.

[Ahewd.com]

Slow detection is not such a problem. I agree that it's OK if it takes
long. What bothers me is the exception which is thrown when the key is
invalid. I can just catch it, but i would like to know it this is the
only exception which can occur and if it occurs each time or only with
some wrong keys, size of data or anything.

So to rephrase the question, is it possible that the file would
decrypt with a wrong key without throwing an exception? Or do i have
to append some checksum of original file and then compare it to
checksum of the decrypted file to detect if the key was good and the
result is not just some garbage?

[Geoff Beier]

On Tue, Apr 5, 2011 at 03:04, Ahewd.com <ahew...@gmail.com> wrote:
>
> So to rephrase the question, is it possible that the file would
> decrypt with a wrong key without throwing an exception? Or do i have
> to append some checksum of original file and then compare it to
> checksum of the decrypted file to detect if the key was good and the
> result is not just some garbage?
>

It's highly improbable, but I wouldn't be comfortable saying it's
impossible. Integrity isn't explicitly guaranteed by CBC. I'd expect a
wrong key, wrong IV or corrupt data stream to throw this exception for
CBC, and don't know of a means for distinguishing which caused it
without layering some other primitive. I'd suggest a signature and/or
an HMAC-based scheme for verifying message integrity if you need to do
that.

Geoff