I'm having the same problem with version 1.0.2a. I wrote a small program to demonstrate the problem (or my mistake). Here is what my program outputs:
return from EVP_DecyrptUpdate: 1
After EVP_decryptFinal_ex
return from EVP_DecryptUpdate_ex: 0
len: 0
plaintext_len = 13
plainText: Hello World!
From my understanding, EVP_DecryptFinal_ex should return 0 on error and 1 on success. And if it does return 0, then you should not trust the decrypted result.
cipher[13] is what I created from the mirror image of the EVP_Encrypt* version of this program. I don't specify any padding which is supposed to default to on which means the EVP_DecryptFinal_ex is not supposed to fail with 0 from my understanding.
The problem is there isn't any data left to decrypt for EVP_DecryptFinal_ex, but I don't necessary know that when I call the EVP_DecryptFinal_ex, which is the whole point: it is supposed to keep track of whether there is left over encryption to finish up and remove any padding "according to the docs". But then I have seen others say GCM doesn't create any padding. (?)
What's the correct way to handle this?
Here is my source code example:
==================== START ====================
#include <stdio.h>
#include <stdlib.h> // EXIT_SUCCESS
#include <unistd.h> // _exit()
#include <openssl/evp.h>
#include <openssl/err.h>
int main(int argc, char *argv[])
{
unsigned char key[32] = { 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8,
0x9, 0xa, 0xb, 0xc, 0xd, 0xe, 0xf, 0x0,
0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8,
0x9, 0xa, 0xb, 0xc, 0xd, 0xe, 0xf, 0x0 };
unsigned char iv[12] = { 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8,
0x9, 0xa, 0xb, 0xc };
unsigned char cipher[13] = { 0x72, 0x1A, 0xFE, 0x88,
0x6D, 0x5C, 0x3E, 0x36,
0x43, 0xC1, 0x3D, 0x38,
0x9F };
EVP_CIPHER_CTX *ctxDecrypt;
int len;
int plaintext_len;
int ret;
unsigned char plainText[256];
unsigned cipher_len = 13;
if(!(ctxDecrypt = EVP_CIPHER_CTX_new())) {
fprintf(stderr,"EVP_CIPHER_CTX_new() failed.\n");
_exit(EXIT_FAILURE);
}
if(!EVP_DecryptInit_ex(ctxDecrypt, EVP_aes_256_gcm(), NULL, NULL, NULL)) {
fprintf(stderr,"EVP_DecryptInit_ex() failed.\n");
_exit(EXIT_FAILURE);
}
if(!EVP_DecryptInit_ex(ctxDecrypt, NULL, NULL, key, iv)) {
fprintf(stderr,"EVP_EncryptInit_ex() #2 failed.\n");
_exit(EXIT_FAILURE);
}
if (!EVP_CIPHER_CTX_set_key_length(ctxDecrypt, 32)) {
fprintf(stderr, "EVP_CIPHER_CTX_set_key_length failed.\n");
_exit(EXIT_FAILURE);
}
if (!EVP_CIPHER_CTX_ctrl(ctxDecrypt, EVP_CTRL_GCM_SET_IVLEN, 12, NULL)) {
fprintf(stderr, "EVP_CIPHER_CTX_ctrl failed.\n");
_exit(EXIT_FAILURE);
}
ret = EVP_DecryptUpdate(ctxDecrypt, plainText, &len, cipher, cipher_len);
printf("return from EVP_DecyrptUpdate: %d\n", ret);
if(ret != 1) {
fprintf(stderr, "EVP_DecryptUpdate failed.\n");
ERR_print_errors_fp(stderr);
_exit(EXIT_FAILURE);
}
plaintext_len = len;
ret = EVP_DecryptFinal_ex(ctxDecrypt, plainText + len, &len);
printf("After EVP_decryptFinal_ex\n");
printf(" return from EVP_DecryptUpdate_ex: %d\n", ret);
printf(" len: %d\n", len);
printf(" plaintext_len = %d\n", plaintext_len);
printf(" plainText: %s\n", plainText);
EVP_CIPHER_CTX_free(ctxDecrypt);
==================== END ====================