CMS Decrypt returns wrong error message on mismatching private key after Bleichenbachers FIX

[Harakiri]

This commit:

http://git.openssl.org/gitweb/?p=openssl.git;a=commit;h=146b52edd122f55e2b2bfeb486dae8dbe96f739e

 

Introduced an error/new behavior, specifically this file

http://git.openssl.org/gitweb/?p=openssl.git;a=blobdiff;f=crypto/cms/cms_smime.c;h=8c56e3a8520d73802c7ea00f81e81c1d574bc49b;hp=a40307605bde5467e46f7cea4ca59a055e46196e;hb=146b52edd122f55e2b2bfeb486dae8dbe96f739e;hpb=13747c6fdabbba33cb187a133548b73d41ae282d

 

When you now call

openssl cms -decrypt -inkey mykey.pem -in encrypted_mail.txt -out openssl_decrypted.txt

where mykey.pem is the wrong private key the following error is returned:

digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:evp_enc.c:474:

Moreover, the outfile openssl_decrypted.txt is filled with 120 bytes of garbage.

Previous versions - correctly reported

CMS routines:CMS_decrypt_set1_pkey:no matching recipient:cms_smime.c:640:

To inform that the message has been encrypted to another recipient. Moreover, if decryption failed - not ever was something written to the -out file.

The code and comment makes no sense

+       /* If no cert and not debugging always return success */
+       if (!cert && !debug)
+               {
+               ERR_clear_error();
+               return 1;
+               }


Why would you always return a success ?
 If you change this line to to remove return 1 then the normal code handling happens

CMSerr(CMS_F_CMS_DECRYPT_SET1_PKEY, CMS_R_NO_MATCHING_RECIPIENT);
    return 0;

Moreover - with the undocumented hidden option (only found via grepping the sources) - you can fix this with adding the
-debug_decrypt option.

This option will tell you the real reason why decryption failed.

Please consider reverting/ or fixing this debug behavior - otherwise its hard to understand why automated smime gateways have issues decrypting messages. Otherwise update the documentation - that under no circumenstances the CMS_R_NO_MATCHING_RECIPIENT is ever returned - you might as well remove it from any header file.

Thanks

BTW: The 120 random byte in the outfile - is that the result of the failed decryption with a symmetric random key ? Regarding MMA - (Bleichenbacher's attack on PKCS #1 v1.5 RSA padding)