Facing issue with i2d_CMS_ContentInfo and d2i_CMS_ContentInfo in 3.x.x

[madhu bajpai]

Hello everyone,

We have CMS encryption and decryption implemented in two different functions and using i2d_CMS_ContentInfo and d2i_CMS_ContentInfo for CMS_ContentInfo object serialisation and deserialisation across functions. We are using our own PKCS11 engine.

Code was working fine with OpenSSL 1.1.1x but failing in 3.x.x.

Encryption Code Snippet:

    OSSL_LIB_CTX* libctx = OSSL_LIB_CTX_new(); 
    CMS_ContentInfo* pCMS = CMS_encrypt_ex(pCertStack, inData, EVP_aes_256_cbc(), CMS_BINARY, libctx, NULL);

    unsigned char *buf = NULL;
    int len = i2d_CMS_ContentInfo(pCMS, &buf);

Decryption Code Snippet:

    pkcs11_set_pin( [userPIN cStringUsingEncoding:NSASCIIStringEncoding] );

    CMS_ContentInfo* p7Env = d2i_CMS_ContentInfo(NULL, (unsigned char const**)&pEncryptedMessage, encryptedMessageLength);

  

    NSString* idString = [keyID toIdString];

    const char* szParams = [idString UTF8String];

    EVP_PKEY* key = ENGINE_load_private_key(engineID, szParams , UI_OpenSSL(), NULL);

    int error = CMS_decrypt(p7Env, key , user, NULL, outData, 0);

Additional Notes:

* Looks like with serialization/deserialization, it is loosing our engine context. 

* Tried i2d_CMS_bio & d2i_CMS_bio but no success.

* If I perform encryption & decryption within same function (and don’t use serialisation/deserialization), decryption works.

* We do have plans to move from engine to provider implementation but it may take some time.

Looking for your suggestions to fix this issue.

Regards,

Madhu