Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Trouble with Microsoft Enhanced RSA and AES Cryptographic Provider (Prototype)

590 views
Skip to first unread message

Andrew Fiddian-Green

unread,
Mar 24, 2002, 10:23:18 AM3/24/02
to
I am writing an application using the new Microsoft Enhanced RSA and AES Cryptographic Provider (Prototype), and am experiencing some odd behaviour.

I am using CALG_AES_128 session keys to encrypt and decrypt data schematically as follows:

1: if not CryptGenKey(fCryptoProvider, CALG_AES_128, CRYPT_EXPORTABLE, @fKey) then RaiseError;
2: if not CryptEncrypt(Key, 0, true, 0, pointer(aBuffer), @aBufferDataLength, aBufferMaxSize) then RaiseError;
3: if not CryptDecrypt(Key, 0, true, 0, pointer(aBuffer), @aBufferDataLength) then RaiseError;

The CryptEncrypt function in line 2 works 100% reliably.
However, whenever line 2 generates a cipher text containing a zero byte then the CryptDecrypt function in line 3 fails with NTE_BAD_DATA.
But whenever the cipher text contains all non zero bytes, then the CryptDecrypt function call suceeds!

The MS SDK help says that the NTE_BAD_DATA error can be returned when the padding is found to be invalid, so I guess that CryptDecrypt is confusing a zero byte as something to do with padding?

Andrew F-G


Andrew Fiddian-Green

unread,
Mar 24, 2002, 1:01:56 PM3/24/02
to
To be honsest, I was not telling the complete story about what I was doing. The full story is:

1: if not CryptGenKey(fCryptoProvider, CALG_AES_128, CRYPT_EXPORTABLE, @fKey) then RaiseError;

2: if not CryptEncrypt(fKey, 0, true, 0, pointer(aBuffer), @aBufferDataLength, aBufferMaxSize) then RaiseError;
if not CryptExportKey(fKey, aPublicKey, SIMPLEBLOB, 0, nil, @len) then RaiseError;
if not CryptExportKey(fKey, aPublicKey, SIMPLEBLOB, 0, pointer(aBigEnoughBlob), @len) then RaiseError;
if not CryptImportKey(fCryptoProvider, pointer(aBigEnoughBlob), len, aPublicKey, 0, @fOtherKey) then
3: if not CryptDecrypt(fOtherKey, 0, true, 0, pointer(aBuffer), @aBufferDataLength) then RaiseError;

To summarise:
- When I follow the steps 1,2,3 (below) the encrypt / decrypt works fine.
- When I insert the steps CryptExportKey / CryptImportKey (above) the decrypt fails with NTE_BAD_DATA but ONLY when the cipher text contains zeros. If there are no zeros in the cipher text, then the decrypt works fine.

In other words, the new fOtherKey does not accept zeros whereas the old fKey does. Or to put it even more specifically, the fOtherKey is NOT behaving identically to fKey.

Andrew F-G

"Andrew Fiddian-Green" <andr...@zugernet.ch> wrote in message news:uXtXGf00BHA.392@tkmsftngp03...

Prabagar Ramadasse

unread,
Mar 25, 2002, 3:50:27 PM3/25/02
to
Andrew,

Encrypted data should be handled as a binary data. Also, the key material
when exported/imported should be treated as a binary data. If you handle
them as strings in whatever language you use, the data you interpret at the
application level will be incorrect,

Thanks,
Prabagar

This posting is provided "AS IS" with no warranties, and confers no rights.

--------------------
> From: "Andrew Fiddian-Green" <andr...@zugernet.ch>
> References: <uXtXGf00BHA.392@tkmsftngp03>
> Subject: Re: Trouble with Microsoft Enhanced RSA and AES Cryptographic
Provider (Prototype)
> Date: Sun, 24 Mar 2002 19:01:56 +0100
> Lines: 62
> MIME-Version: 1.0
> Content-Type: text/plain;
> charset="iso-8859-1"
> Content-Transfer-Encoding: quoted-printable
> X-Priority: 3
> X-MSMail-Priority: Normal
> X-Newsreader: Microsoft Outlook Express 6.00.2600.0000
> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
> Message-ID: <uWhPv310BHA.2528@tkmsftngp05>
> Newsgroups: microsoft.public.platformsdk.security
> NNTP-Posting-Host: 212.4.75.135
> Path: cpmsftngxa07!cpmsftngxa09!tkmsftngp01!tkmsftngp05
> Xref: cpmsftngxa07 microsoft.public.platformsdk.security:4988
> X-Tomcat-NG: microsoft.public.platformsdk.security

Andrew Fiddian-Green

unread,
Mar 25, 2002, 5:04:10 PM3/25/02
to
Of course I am handling the data as binary data. Where do you the idea that I am handling it as "strings"? (Indeed what do you mean by "strings"?)

Andrew F-G

"Prabagar Ramadasse" <prabaga...@microsoft.com> wrote in message news:vdArm6D1BHA.1528@cpmsftngxa08...

Prabagar Ramadasse

unread,
Mar 26, 2002, 7:16:02 PM3/26/02
to
Andrew,

If your application code handles the data as binary with the appropriate
length passed as parameters to the Crypto API calls, everything should work
fine. If you have a zero (Zero if you dump the memory) in data and if your
application uses string variables or string functions, then you will
account only for bytes up to the null termination character. Based on the
symptoms you explained like having zero in the data or not, I guessed you
are handling them as strings.

You can place breakpoint in the debugger and analyze the parameters that
are passed by dumping the memory. The encrypted data buffer and length
returned from CryptEncrypt() should be identical to information passed to
CryptDecrypt(). Similarly, the data buffer and length returned from
CryptExportKey() should be identical to information passed to
CryptImportKey().

Below is a working C sample code for any binary data, for your reference.

/*** Encryption Sample Code ***/
#define _WIN32_WINNT 0x0400
#include <windows.h>
#include <stdio.h>

#pragma comment(lib, "advapi32.lib")

void PrintUsage();

const IN_BUFFER_SIZE = 2048;
// OUT_BUFFER_SIZE slightly larger than IN_BUFFER_SIZE
// encrypted data could end up being, at most, one ancrypted block
// larger than the data supplied by the input buffer
const OUT_BUFFER_SIZE = IN_BUFFER_SIZE + 64; // extra padding

int main(int argc, char **argv)
{
BYTE pbBuffer[OUT_BUFFER_SIZE];

HCRYPTPROV hProv = NULL;

LPBYTE pbSessionKeyBlob = NULL;

HCRYPTKEY hRSAKey = NULL;
HCRYPTKEY hSessionKey = NULL;

HANDLE hInFile = INVALID_HANDLE_VALUE;
HANDLE hOutFile = INVALID_HANDLE_VALUE;

LPSTR ContainerName = NULL;
LPSTR SourceFileName = NULL;
LPSTR EncryptedFileName = NULL;

BOOL fResult;
DWORD dwSessionKeyBlob;

DWORD dwByteCount, dwBytesWritten;
BOOL finished;

if (argc < 4)
{
PrintUsage();
return 0;
}

__try
{
ContainerName = argv[1];
SourceFileName = argv[2];
EncryptedFileName = argv[3];

// Acquire context for RSA key
fResult = CryptAcquireContext(&hProv, ContainerName,
MS_ENH_RSA_AES_PROV,
PROV_RSA_AES, 0);
if (!fResult)
{
if (GetLastError() == NTE_BAD_KEYSET)
{
// Create a key container if one does not exist.
fResult = CryptAcquireContext(&hProv,
ContainerName,
MS_ENH_RSA_AES_PROV,
PROV_RSA_AES,
CRYPT_NEWKEYSET);
if (!fResult)
{
printf("CryptAcquireContext (2) failed with %x\n",
GetLastError());
__leave;
}
}
else
{
printf("CryptAcquireContext (1) failed with %x\n",
GetLastError());
__leave;
}
}

// get the RSA key handle
fResult = CryptGetUserKey(hProv, AT_KEYEXCHANGE, &hRSAKey);
if (!fResult)
{
if (GetLastError() == NTE_NO_KEY)
{
// Create a key if one does not exist.
fResult = CryptGenKey(hProv,
AT_KEYEXCHANGE,
CRYPT_EXPORTABLE,
&hRSAKey);
if (!fResult)
{
printf("CryptGenKey failed with %x\n", GetLastError());
__leave;
}
}
else
{
printf("CryptGetUserKey failed with %x\n", GetLastError());
__leave;
}
}

// Open infile and create outfile.
hInFile = CreateFile(SourceFileName, GENERIC_READ, FILE_SHARE_READ,
NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
if (hInFile == INVALID_HANDLE_VALUE)
{
printf("CreateFile failed with %x\n", GetLastError());
__leave;
}

hOutFile = CreateFile(EncryptedFileName, GENERIC_WRITE,
FILE_SHARE_READ,
NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
if (hOutFile == INVALID_HANDLE_VALUE)
{
printf("CreateFile failed with %x\n", GetLastError());
__leave;
}

fResult = CryptGenKey(hProv, CALG_AES_128, CRYPT_EXPORTABLE,
&hSessionKey);
if (!fResult)
{
printf("CryptGenKey failed with %x\n", GetLastError());
__leave;
}

// The first call to ExportKey with NULL gets the key size.
dwSessionKeyBlob = 0;
fResult = CryptExportKey(hSessionKey, hRSAKey, SIMPLEBLOB, 0,
NULL, &dwSessionKeyBlob);
if (!fResult)
{
printf("CryptExportKey failed with %x\n", GetLastError());
__leave;
}

// Allocate memory for Encryted Session key blob
pbSessionKeyBlob = (LPBYTE)LocalAlloc(LPTR, dwSessionKeyBlob);
if (!pbSessionKeyBlob)
{
printf("Unable to allocate memory for session key\n");
__leave;
}

fResult = CryptExportKey(hSessionKey, hRSAKey, SIMPLEBLOB, 0,
pbSessionKeyBlob, &dwSessionKeyBlob);
if (!fResult)
{
printf("CryptExportKey failed with %x\n", GetLastError());
__leave;
}

// Write the size of key blob, then the key blob itself, to output
file.
fResult = WriteFile(hOutFile, &dwSessionKeyBlob,
sizeof(dwSessionKeyBlob),
&dwBytesWritten, NULL);
if (!fResult)
{
printf("WriteFile failed with %x\n", GetLastError());
__leave;
}
fResult = WriteFile(hOutFile, pbSessionKeyBlob,
dwSessionKeyBlob,
&dwBytesWritten, NULL);
if (!fResult)
{
printf("WriteFile failed with %x\n", GetLastError());
__leave;
}

// Now read data in, encrypt it, and write encrypted data to output.
do
{

fResult = ReadFile(hInFile, pbBuffer, IN_BUFFER_SIZE,
&dwByteCount, NULL);
if (!fResult)
{
printf("ReadFile failed with %x\n", GetLastError());
__leave;
}

finished = (dwByteCount < IN_BUFFER_SIZE);

fResult = CryptEncrypt(hSessionKey, 0, finished, 0, pbBuffer,
&dwByteCount,
OUT_BUFFER_SIZE);
if (!fResult)
{
printf("CryptEncrypt failed with %x\n", GetLastError());
__leave;
}

fResult = WriteFile(hOutFile, pbBuffer, dwByteCount,
&dwBytesWritten, NULL);
if (!fResult)
{
printf("WriteFile failed with %x\n", GetLastError());
__leave;
}

} while(!finished);

printf("%s File has been encrypted successfully\n", SourceFileName);

}
__finally
{
if (pbSessionKeyBlob != NULL)
LocalFree(pbSessionKeyBlob);

if (hRSAKey)
CryptDestroyKey(hRSAKey);
if (hSessionKey)
CryptDestroyKey(hSessionKey);

if (hProv)
CryptReleaseContext(hProv, 0);

if (hInFile != INVALID_HANDLE_VALUE)
CloseHandle(hInFile);
if (hOutFile != INVALID_HANDLE_VALUE)
CloseHandle(hOutFile);
}

return 0;
}

void PrintUsage()
{
printf("Usage: EncryptFile <ContainerName> <SourceFileName>
<EncryptedFileName>\n");
}
/*** Encryption Sample Code ***/

/*** Decryption Sample Code ***/
#include <windows.h>
#include <wincrypt.h>
#include <stdio.h>

#pragma comment(lib, "advapi32.lib")

void PrintUsage();

const BUFFER_SIZE = 2048; // needs to be multiple block size

int main(int argc, char **argv)
{
BYTE pbBuffer[BUFFER_SIZE];
BOOL finished, fResult;

HANDLE hInFile = INVALID_HANDLE_VALUE;
HANDLE hOutFile = INVALID_HANDLE_VALUE;

HCRYPTPROV hProv = NULL;
HCRYPTKEY hSessionKey = NULL;

DWORD dwByteCount, dwBytesWritten;

LPSTR KeyContainerName = NULL;
LPSTR EncryptedFileName = NULL;
LPSTR DecryptedFileName = NULL;

if (argc < 4)
{
PrintUsage();
return 0;
}

__try
{
KeyContainerName = argv[1];
EncryptedFileName = argv[2];
DecryptedFileName = argv[3];


// Get handle for the provider (use RSA encryption).
fResult = CryptAcquireContext(&hProv, KeyContainerName,
MS_ENH_RSA_AES_PROV,
PROV_RSA_AES, 0);
if (!fResult)
{
printf("CryptAcquireContext failed with %x\n", GetLastError());
__leave;
}

// Open infile and create outfile.
hInFile = CreateFile(EncryptedFileName, GENERIC_READ, FILE_SHARE_READ,
NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
if (hInFile == INVALID_HANDLE_VALUE)
{
printf("CreateFile failed with %x\n", GetLastError());
__leave;
}

hOutFile = CreateFile(DecryptedFileName, GENERIC_WRITE,
FILE_SHARE_READ,
NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);

if (hOutFile == INVALID_HANDLE_VALUE)
{
printf("CreateFile failed with %x\n", GetLastError());
__leave;
}

// Read in key block size, then key blob itself from input file.
fResult = ReadFile(hInFile, &dwByteCount, sizeof(dwByteCount),
&dwBytesWritten, NULL);
if (!fResult)
{
printf("ReadFile failed with %x\n", GetLastError());
__leave;
}

fResult = ReadFile(hInFile, pbBuffer, dwByteCount, &dwBytesWritten,
NULL);
if (!fResult)
{
printf("ReadFile failed with %x\n", GetLastError());
__leave;
}

// import key blob into "CSP"
fResult = CryptImportKey(hProv, pbBuffer, dwByteCount, 0, 0,
&hSessionKey);
if (!fResult)
{
printf("CryptImportKey failed with %x\n", GetLastError());
__leave;
}

// Now read data in, Decrypt it, and write Decrypted data to output.
do
{
fResult = ReadFile(hInFile, pbBuffer, BUFFER_SIZE, &dwByteCount,
NULL);
if (!fResult)
{
printf("ReadFile failed with %x\n", GetLastError());
__leave;
}

finished = (dwByteCount < BUFFER_SIZE);
fResult = CryptDecrypt(hSessionKey, 0, finished, 0, pbBuffer,
&dwByteCount);
if (!fResult)
{
printf("CryptDecrypt failed with %x\n", GetLastError());
__leave;
}

fResult = WriteFile(hOutFile, pbBuffer, dwByteCount,
&dwBytesWritten,
NULL);
if (!fResult)
{
printf("WriteFile failed with %x\n", GetLastError());
__leave;
}


} while (!finished);
}
__finally
{
// Clean up: release handles, close files.
if (hSessionKey)
CryptDestroyKey(hSessionKey);
if (hProv)
CryptReleaseContext(hProv, 0);

if (hInFile != INVALID_HANDLE_VALUE)
CloseHandle(hInFile);
if (hOutFile != INVALID_HANDLE_VALUE)
CloseHandle(hOutFile);
}
return(0);
}

void PrintUsage()
{
printf("Usage: DecryptFile <KeyContainerName> <EncryptedFileName>
<DecryptedFileName>\n");
}
/*** Decryption Sample Code ***/

Thanks,
Prabagar

This posting is provided "AS IS" with no warranties, and confers no rights.

--------------------
> From: "Andrew Fiddian-Green" <andr...@zugernet.ch>

> References: <uXtXGf00BHA.392@tkmsftngp03> <uWhPv310BHA.2528@tkmsftngp05>
<vdArm6D1BHA.1528@cpmsftngxa08>


> Subject: Re: Trouble with Microsoft Enhanced RSA and AES Cryptographic
Provider (Prototype)

> Date: Mon, 25 Mar 2002 23:04:10 +0100
> Lines: 110


> MIME-Version: 1.0
> Content-Type: text/plain;
> charset="iso-8859-1"
> Content-Transfer-Encoding: quoted-printable
> X-Priority: 3
> X-MSMail-Priority: Normal
> X-Newsreader: Microsoft Outlook Express 6.00.2600.0000
> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000

> Message-ID: <#gNAzjE1BHA.1796@tkmsftngp02>
> Newsgroups: microsoft.public.platformsdk.security
> NNTP-Posting-Host: 212.4.75.135
> Path: cpmsftngxa07!tkmsftngxs01!tkmsftngp01!tkmsftngp02
> Xref: cpmsftngxa07 microsoft.public.platformsdk.security:5002
> X-Tomcat-NG: microsoft.public.platformsdk.security

Andrew Fiddian-Green

unread,
Mar 28, 2002, 6:16:02 AM3/28/02
to
Prabagar,

Thanks for your message. My problem is an intermittent problem so it is
very difficult to debug. In essence my code has a back to back "sandwich"of
CryptEncrypt and CryptDecrypt calls:

plaintext => CryptEncrypt => ciphertext => CryptDecrypt => plaintext

In actuality my code is not quite as simple as the above, since on the left
of the sandwich I create a session key, use it in CryptEncrypt, export it
(enciphered with the public key), and finally destroy the handle. By analogy
on the right side of the sandwich, I create a new session key handle from
the exported session key data, and use this second key in the CryptDecrypt
call, (and finally destroy the second key handle).

I just wrote a short program to test a "clean" version of the sandwich
(keeping the same session key handle throughout), and I can confirm that the
encrypt / decrypt works 100%

However in the case where I use separate session key handles on the left and
right sides, I am finding is that in the (random) cases when CryptEncrypt
produces a CIPHERTEXT that contains one or more zero bytes, then
CryptDecrypt always fails with NTE_BAD_DATA. But in the (random) cases when
ciphertext does NOT contain any zero bytes, the call to CryptDecrypt always
succeeds.

Therefore I think that the problem does not lie directly with the "pure"
CryptEncrypt / CryptDecrypt pair, but rather it is a result of the
particular combination of CryptCreateKey / CryptEncrypt / CryptExportKey /
CryptDestroyKey / CryptImportKey / CryptDecrypt / CryptDestroyKey that I am
using in my code. I still did not find anything wrong with my own code
(yet), so I am suspecting a bug inside the AES Cryptographic Provider
(Prototype). However I still want to do some more debugging in order to
give more information about the exact causes.

I will keep you posted.

Regards,
Andrew F-G

"Prabagar Ramadasse" <prabaga...@microsoft.com> wrote in message

news:ItXlNSS1BHA.1988@cpmsftngxa07...


> Andrew,
>
> If your application code handles the data as binary with the appropriate
> length passed as parameters to the Crypto API calls, everything should
work
> fine. If you have a zero (Zero if you dump the memory) in data and if your
> application uses string variables or string functions, then you will
> account only for bytes up to the null termination character. Based on the
> symptoms you explained like having zero in the data or not, I guessed you
> are handling them as strings.
>
> You can place breakpoint in the debugger and analyze the parameters that
> are passed by dumping the memory. The encrypted data buffer and length
> returned from CryptEncrypt() should be identical to information passed to
> CryptDecrypt(). Similarly, the data buffer and length returned from
> CryptExportKey() should be identical to information passed to
> CryptImportKey().
>
> Below is a working C sample code for any binary data, for your reference.
>

...


Prabagar Ramadasse

unread,
Mar 28, 2002, 2:31:59 PM3/28/02
to
Andrew,

The C sample code that I posted, is similar to your scenario.

Encryption Application creates a random session key - CryptGenKey /
CryptExportKey / CryptEncrypt / CryptDestroyKey
Decryption Application does CryptImportKey / CryptDecrypt / CryptDestroyKey

Thanks,
Prabagar

This posting is provided "AS IS" with no warranties, and confers no rights.

--------------------
> From: "Andrew Fiddian-Green" <andr...@zugernet.ch>
> References: <uXtXGf00BHA.392@tkmsftngp03> <uWhPv310BHA.2528@tkmsftngp05>

<vdArm6D1BHA.1528@cpmsftngxa08> <#gNAzjE1BHA.1796@tkmsftngp02>
<ItXlNSS1BHA.1988@cpmsftngxa07>


> Subject: Re: Trouble with Microsoft Enhanced RSA and AES Cryptographic
Provider (Prototype)

> Date: Thu, 28 Mar 2002 12:16:02 +0100
> Lines: 67


> X-Priority: 3
> X-MSMail-Priority: Normal
> X-Newsreader: Microsoft Outlook Express 6.00.2600.0000
> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000

> Message-ID: <#dpMJnk1BHA.2212@tkmsftngp05>
> Newsgroups: microsoft.public.platformsdk.security
> NNTP-Posting-Host: inet01-i.zug.ch.abatos.com 194.6.178.1
> Path: cpmsftngxa07!tkmsftngp01!tkmsftngp05
> Xref: cpmsftngxa07 microsoft.public.platformsdk.security:5049
> X-Tomcat-NG: microsoft.public.platformsdk.security

0 new messages