Moving from cryptopp 4.2 to cryptopp 5.6.4

[edwin schriek]

Hi,

We are updating a project which uses the ancient cryptopp 4.2, to the latest version in order to have better cross platform support (4.2 suffers from undefined behaviour under platforms like Android/iOS).

In this project, we use AES to decrypt/encrypt some data, nothing to fancy.

Now, the problem when moving to the newer version lies in generation the IV.

The code used to generate our IV is as follows:

BOOL CCryptorAES101::Init(BYTE* pKey, DWORD dwKeySize)
{
 ASSERT(pKey);


 m_bInit = true;


 try
 {
 CryptoPP::SHA256         hash;
 CryptoPP::RandomPool     rng;
 CryptoPP::MySecByteBlock bufSeedIV(IV_SEED_SIZE);


 m_bufSeed.Resize(hash.DigestSize());
 m_bufIV  .Resize(IV_SIZE);


 hash.CalculateDigest(m_bufSeed,pKey,dwKeySize);
 rng.Put(pKey,dwKeySize);
 rng.GenerateBlock(bufSeedIV,bufSeedIV.Size());


 CryptoPP::MySecByteBlock bufHash(hash.DigestSize());


 hash.CalculateDigest(bufHash,bufSeedIV.Begin(),bufSeedIV.Size());


 ASSERT(2 * m_bufIV.Size() == bufHash.Size());


 for (int i = 0; i < m_bufIV.Size(); i++)
 m_bufIV[i] = bufHash[i] ^ bufHash[i + 16];


 }
 catch (CryptoPP::Exception const& e)
 {
 UNUSED_ALWAYS(e);


 m_bInit = false;
 
 return false;
 }


 return true;
}

The following code does the actual decryption:

BOOL CCryptorAES101::Decrypt(BYTE* pData, DWORD dwSize)
{
 if (!m_bInit)
 return FALSE;


 try
 {
#if CRYPTO_VER == 42
 CryptoPP::AESEncryption aes(m_bufSeed,m_bufSeed.Size());
 CryptoPP::CFBDecryption decryptor(aes,m_bufIV);
#else
 CryptoPP::AES::Encryption                     aes(m_bufSeed,m_bufSeed.size());
 CryptoPP::CFB_Mode_ExternalCipher::Decryption decryptor(aes,m_bufIV);
#endif
 decryptor.ProcessString(pData,dwSize);
 }
 catch (CryptoPP::Exception const& e)
 {
 UNUSED_ALWAYS(e);


 return FALSE;
 }


 return TRUE;
}

Under cryptopp 4.2, bufSeedIV is always the same, resulting in a bufHash and m_bufIV which are always the same. 

Under cryptopp 5.6.4 however, bufSeedIV is always different, this is our problem.

As far as I can figure, the RandomPool implementation changed over time, but I could be wrong.

Any guidance would be appreciated!

Kind regards,

Edwin

[edwin schriek]

Apparently the random pool is the cause of our problems. In 5.6.4 it uses time and AES instead of no time and MDC<SHA> in cryptopp 4.2.

Is there anyway to replicate the old randpool behaviour, or are there other possible solutions?

Op woensdag 21 september 2016 11:06:52 UTC+2 schreef edwin schriek:

[Jean-Pierre Münch]

The issue you seem to have is that the RNG is, well, actually producing random results.

If you want deterministic behavior, you should avoid RNGs if possible.

To clarify, I'm trying to draw a picture of the old IV generation:

Key -> SHA256 -> RandomPool -> SHA256 -> XOR "compression" -> IV
  |                             |
  +---------------------+

From what I can see, your implementation always tries to produce a static IV, which is only dependent on the key. This is only fine if you never re-use the key, e.g. you never instantiate (and use) a second instance with the same key (for encryption). Otherwise this is an IV re-use which is horribly broken and allows for really easy attacks.

Another issue that I see in this code is the complete absence of authentication. If an attacker manipulates the cipher text you've no chance of (cryptographically reliably) detecting it. The fix would be to use a mode like GCM or EAX.

To fix your current construction (and get the same dangerous behavior back), you can replace RandomPool with HMAC<SHA256> and feed the key in as the key and the hashed key as the message.

To actually fix the problems, you should use

1. Authenticated encryption
2. Explicitly or implicitly set a new IV / nonce for each message (either by using a deterministic counter or by generating the IV / nonce at random)

Here are the related wiki pages for EAX and GCM

• https://cryptopp.com/wiki/EAX_Mode
• https://cryptopp.com/wiki/GCM_Mode
  

BR

JPM

--
--
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-user...@googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.
---
You received this message because you are subscribed to the Google Groups "Crypto++ Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cryptopp-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[edwin schriek]

Thanks for your answer,

Your statement about security makes sense, the thing is that this code was written 20 years ago by some developer and I do not know his intention/philosophy behind this encryption/decryption mechanism.

With that being said, fixing the current construction, which is considered dangerous according to your story, is sufficient. At least for now, since my only 'task' is to have our project running on various platforms (Android/iOS).

One final request; could you help me write the code for the HMAC? I barely know the cryptopp api and I know even less about cryptography.

I figured I need something like the following,

CryptoPP::HMAC<CryptoPP::SHA256> hmac(pKey, dwKeySize);

CryptoPP::StringSource ss(m_bufSeed, true,

  new CryptoPP::HashFilter(hmac,

  new CryptoPP::StringSink(hmac)

      )

);

Op woensdag 21 september 2016 15:00:04 UTC+2 schreef jean-pierre.muench:

[Jean-Pierre Münch]

After HMAC is keyed (which you did in the constructor and can also do via SetKey() ) it behaves like a normal hash function, so Update() and Final() or CalculateDigest() are your methods of choice.

So it would be something like:

HMAC<SHA256> hmac(key,keySize); // replace with your parameters
hmac.CalculateDigest(target,input,inputSize); // replace with your parameters

If you want to stay with the old style.
For the fancy Filter-Source based style, look in the following wiki page and replace SHA256 with HMAC<SHA256> and key it in the constructor.
https://cryptopp.com/wiki/HashFilter

BR

JPM

[edwin schriek]

I came up with the following based on your remarks, instantiated the HMAC with the key and fed the hashest key as the message.

However, output does not coincide with that of the old code.

CryptoPP::SHA256 hash;
   
 m_bufSeed.Resize(hash.DigestSize());
 m_bufIV.Resize(IV_SIZE); //16

 hash.CalculateDigest(m_bufSeed, pKey, dwKeySize);

 CryptoPP::HMAC<CryptoPP::SHA256> hmac(pKey, dwKeySize);

 CryptoPP::MySecByteBlock bufHmac(hmac.DigestSize());

 hmac.CalculateDigest(bufHmac, m_bufSeed, m_bufSeed.Size());
       
 for(int i = 0; i < m_bufIV.Size(); i++) {
  m_bufIV[i] = bufHmac[i] ^ bufHmac[i + 16];


  TRACE("%d ", m_bufIV[i]);
 }

Op woensdag 21 september 2016 16:05:46 UTC+2 schreef jean-pierre.muench:

[Jean-Pierre Münch]

The change I proposed tried to maintain the same flow as closely as possible, making HMAC the obvious choice here. If you wanted to exact same results as from 4.2, you'll either have to inspect the 4.2 code and try and reproduce the same results (shouldn't be too hard) or you just copy and paste the implementation from 4.2 into your new project, rename it and be done with it.

BR

JPM

[edwin schriek]

I decided to go with the quick and dirty solution of copying the old randpool implementation, which works.

Anyhow, thanks for your insight, we will probably make stuff more secure based on your remarks in the near future. 

Op woensdag 21 september 2016 16:51:48 UTC+2 schreef jean-pierre.muench:

[Jeffrey Walton]

Under cryptopp 4.2, bufSeedIV is always the same, resulting in a bufHash and m_bufIV which are always the same. 

Under cryptopp 5.6.4 however, bufSeedIV is always different, this is our problem.

As far as I can figure, the RandomPool implementation changed over time, but I could be wrong.

Yeah, it changed around May 2007. Here's the [imported] git commit: https://github.com/weidai11/cryptopp/commit/f41245df6fb9b85574260eca9cd32777e8ab5136 .

I'll get the documentation updated. I dont think the sources or Doxygen have a treatment. There's just a blurb on the wiki at https://cryptopp.com/wiki/RandomNumberGenerator#RandomPool.

Any guidance would be appreciated!

You are not the first person who has experienced an issue. I seem to recall another person was quite pissed off at the change.

I'm wondering if we should provide a legacy generator to give folks a path for a migrations. Its easy enough to provide RandomPool_MDC_SHA and say "don't use this". Or keep it out of the library and provide it on the Patch Page at https://cryptopp.com/wiki/RandomNumberGenerator#RandomPool .

SHA as a PRF should be OK for those willing to accept it. The SHA cracks are due to collisions and the birthday attack. I don't know about MDC, however.

Until this issue is disposed, maybe the following will help you:

    $ git clone https://github.com/weidai11/cryptopp cryptopp-ancient
    $ cryptopp-ancient

    # Checkout the RandomPool change
    $ git checkout f41245df6fb9b85574260eca9cd32777e8ab5136
   
    # Go back one more
    git checkout HEAD~1

    $ grep 'MDC<SHA>' *.h *.cpp
    randpool.cpp:typedef MDC<SHA> RandomPoolCipher;

And there's the old generator.

Jeff

[Jeffrey Walton]

On Wednesday, September 21, 2016 at 11:30:54 AM UTC-4, Jeffrey Walton wrote:

Under cryptopp 4.2, bufSeedIV is always the same, resulting in a bufHash and m_bufIV which are always the same. 

Under cryptopp 5.6.4 however, bufSeedIV is always different, this is our problem.

As far as I can figure, the RandomPool implementation changed over time, but I could be wrong.

Yeah, it changed around May 2007. Here's the [imported] git commit: https://github.com/weidai11/cryptopp/commit/f41245df6fb9b85574260eca9cd32777e8ab5136 .

I'll get the documentation updated. I dont think the sources or Doxygen have a treatment. There's just a blurb on the wiki at https://cryptopp.com/wiki/RandomNumberGenerator#RandomPool.

The documentation changes were committed at https://github.com/weidai11/cryptopp/commit/caf8aef8fe91229d9df370596519e8e80eba6eb7 . Its now documented in the source code, documented in the Manual, and documented at the wiki.

Jeff

[edwin schriek]

Thanks, Jeff!

Op woensdag 21 september 2016 17:59:29 UTC+2 schreef Jeffrey Walton:

[Jeffrey Walton]

On Wednesday, September 21, 2016 at 8:13:16 AM UTC-4, edwin schriek wrote:

Apparently the random pool is the cause of our problems. In 5.6.4 it uses time and AES instead of no time and MDC<SHA> in cryptopp 4.2.

Is there anyway to replicate the old randpool behaviour, or are there other possible solutions?

Sorry to dig up an old thread. This topic came up again.

I hacked together a wiki page on the subject. its available at https://www.cryptopp.com/wiki/Old_RandomPool.

Please let me know if it needs errors or omissions addressed.

Jeff

[Jeffrey Walton]

Now open in the bug tracker: "Add OldRandomPool for pre-Crypto++ 5.5 compatibility", https://github.com/weidai11/cryptopp/issues/452.

Jeff

[Jeffrey Walton]

The changes were check in at:

* https://github.com/weidai11/cryptopp/commit/02e3a794443a
* https://github.com/weidai11/cryptopp/commit/5fbbc5311cea

The docs were updated:

* https://www.cryptopp.com/docs/ref/class_old_random_pool.html

Jeff