Crypto++ encryption between C++ and Python

[Dwight Kulkarni]

Hello,

I have two systems, one is running Python and one is running CryptoPP C++.

I create a common certificate that is .DER encoded. I am able to import the certificate into both systems. I use the certificate to encrypt and decrypt successfully on each system individually.

Then I generate an encrypted message with Crypto++ and send it to Python and try to decrypt and it gives an error that decryption failed.

Can you suggest any debugging steps ?

std::string encrypt_rsa(std::string message, CryptoPP::RSA::PublicKey key)

{

try{

message = b64encode(message);

CryptoPP::AutoSeededRandomPool rng;

CryptoPP::RSAES_OAEP_SHA_Encryptor encryptor(key);

std::string ciphertext;

CryptoPP::StringSource(message, true, new CryptoPP::PK_EncryptorFilter(rng, encryptor, new CryptoPP::StringSink(ciphertext)));

return ciphertext;

}

catch(...)

{

std::cout << "error encrypting RSA";

return "";

}

}

def encrypt_rsa(message, key):
    try:
        result = rsa.encrypt(base64.b64encode(message), key)
        return result
    except Exception as err:
        print("There was an error encryption RSA", err)
        return None

def decrypt_rsa(cipherbytes, key):
    try:
        base64_bytes = rsa.decrypt(cipherbytes, key)
        return base64.b64decode(base64_bytes)
    except Exception as err:
        print(err)
        return None

[Jeffrey Walton]

I did not consult the Python documentation, so I am shooting from the hip...

Crypto++ RSAES_OAEP_SHA_Encryptor uses SHA1 by default. See
https://www.cryptopp.com/wiki/RSA_Encryption_Schemes .

I'm guessing Python is using SHA-256 or SHA-512.

You should do something like:
typedef RSAES<OAEP<SHA256> >::Encryptor RSAES_OAEP_SHA256_Encryptor;
typedef RSAES<OAEP<SHA256> >::Decryptor RSAES_OAEP_SHA256_Decryptor;

and then use RSAES_OAEP_SHA256_Encryptor and RSAES_OAEP_SHA256_Decryptor.

Or, you can consult the Python documentation and see how to configure
RSA/OAEP to use SHA1.

Jeff