Re: Threefish-1024 IV specification issue/doumentaion...or bug?

[Jeffrey Walton]

On Thursday, November 16, 2017 at 8:32:59 PM UTC-5, El Ray wrote:

...

Has anyone successfully used threefish-1024 (or -512 which I surmise would have the same issue)?

This looks like a bug on our (my?) part. I say "my" because I cut it in.

Looking at our test vectors, we could only get a hold of ECB mode. We exercise a Tweak but not an IV. Cf., https://github.com/weidai11/cryptopp/blob/master/TestVectors/threefish.txt . The last time I checked Botan is missing Threefish-1024, so I could not produce them from Botan, either.

Looking at the source code it looks like IV_Length=32 is wrong for Threefish-1024. It is the default IV length, and it is paired with the default key length. Later, when a key is set, IVLength() should return the correct length under a key. That's how it is supposed to work.

The wrinkle is, we may not be getting dynamic dispatch such that IVLength() and Blocksize() are being called, so we stay latched on the initial value. This is part of a bigger engineering discomfort/problem when trying to add variable block sizes to the library (which happened in May 2017). Here's a couple of bug reports we used to track them: "Add support for variable block sizes", https://github.com/weidai11/cryptopp/issues/408 and "Add Threefish block cipher" https://github.com/weidai11/cryptopp/issues/422.

After we cut-in variable block sizes I felt a little uneasy about them. They felt like they were a little forced. At this point I think we should back pedal a bit and walk away from variable block ciphers using the "wait until a key is set to take shape" pattern. I think we should just provide Threefish_256, Threefish_512 and Threefish_1024 classes. We can rework them using a common base so base class pointers and references work as expected.

Give us a day or two for Threefish_256, Threefish_512 and Threefish_1024. It should be a quick cut-over.

We also need to get a hold of more test vectors. Does anyone happen to know where additional test vectors are available for Threefish block cipher? (Skein vectors are readily available. Its Threefish block cipher that's missing them).

Jeff

[El Ray]

Sorry..deleted that just before your reply appeared.  On further consideration I also realized that although I just got some code to work, I shouldn't assume it's a non-issue till the cross-validation you discussed was done.  But apologies in advanve if this is a non-issue.

So first, my original post, followed by new code that appears to work:



Threefish-1024 IV specification issue/doumentaion...or bug?

There appears to be a problem setting the longer IV required for
threefish-1024.  My code (modified from code on the wiki) works
for -256 but nothing I've tried works for -1024.

The problem might be in documentation of something specific to
threefish-1024, though it might be related to

threefish.h:
41:   CRYPTOPP_CONSTANT(IV_LENGTH=32)

which is described as NOT_RESYNCHRONIZABLE [though I don't know
whether this is in the sense pertaining to IV length] and is not
varied for different cases as the keylength is nearby.

But the benchmark suggests there's a way around this if it's not
misreporting.

Crypto++ 6.0.0 Benchmarks
Threefish/CTR (1024-bit key)    61    25.2    0.918    1469

Here's what I get with similar code for threefish-256 and
threefish-1024:

$ ./threefish256try
IV_LENGTH: 32
Algorithm:
  Threefish/CTR
Maximum Key Size:
  128 bytes

Plain Text (12 bytes)
  'Hello World.'

Recovered Text:
  'Hello World.'


$ ./threefish1024try
IV_LENGTH: 128
terminate called after throwing an instance of 'CryptoPP::InvalidArgument'
  what():  Threefish/CTR: IV length 32 is less than the minimum of 128
Aborted (core dumped)

==============================

code for 1024:

// threefish1024try.cpp
//based on
// https://web.archive.org/web/20150315102342/http://www.cryptopp.com/fom-serve/cache/79.html
//2006-Jan-21 11:08am jason entry

// g++ -g3 -O2 -Wall -Wextra  threefish1024try.cpp -o threefish1024try -I/usr/local/include/cryptopp -L/usr/local/lib -lcryptopp


#include <iostream>
//#include <iomanip>

// Crypto++ Includes
#include "cryptlib.h"
#include "modes.h"      // xxx_Mode< >
#include "filters.h"    // StringSource and
#include "threefish.h"

#define CIPHER_MODE CTR_Mode
#define CIPHER Threefish

#define BLOCKSIZE 128
#define IV_LENGTH 128

// using namespace CryptoPP;

int main(int argc, char* argv[]) {
    std::cout << "IV_LENGTH: " <<  IV_LENGTH << std::endl;
   
    CryptoPP::byte key[ CryptoPP::CIPHER::MAX_KEYLENGTH ],
                    iv[ IV_LENGTH ];

    ::memset( key, 0x01, CryptoPP::CIPHER::MAX_KEYLENGTH );
    ::memset(  iv, 0x01, IV_LENGTH );
   
    // Message M
    std::string PlainText = "Hello World.";

    // Cipher Text Sink
    std::string CipherText;

    // Encryptor
    CryptoPP::CTR_Mode<CryptoPP::Threefish >::Encryption
        Encryptor( key, CryptoPP::CIPHER::MAX_KEYLENGTH, iv );


    // Encryption
    CryptoPP::StringSource( PlainText, true,
        new CryptoPP::StreamTransformationFilter( Encryptor,
            new CryptoPP::StringSink( CipherText )
        ) // StreamTransformationFilter
    ); // StringSource


    ///////////////////////////////////////
    //                DMZ                //
    ///////////////////////////////////////

    // Recovered Text Sink
    std::string RecoveredText;

    // Decryptor
    CryptoPP::CIPHER_MODE<CryptoPP::CIPHER >::Decryption
        Decryptor( key, CryptoPP::CIPHER::MAX_KEYLENGTH, iv );

    // Decryption
    CryptoPP::StringSource( CipherText, true,
        new CryptoPP::StreamTransformationFilter( Decryptor,
            new CryptoPP::StringSink( RecoveredText )
        ) // StreamTransformationFilter
    ); // StringSource

    //////////////////////////////////////////
    //                Output                //
    //////////////////////////////////////////

    std::cout << "Algorithm:" << std::endl;
    std::cout << "  " << Encryptor.AlgorithmName() << std::endl;
    std::cout << "Maximum Key Size:" << std::endl;
    std::cout << "  " << Encryptor.MaxKeyLength() << " bytes" << std::endl;
    std::cout << std::endl;


    std::cout << "Plain Text (" << PlainText.length() << " bytes)" << std::endl;
    std::cout << "  '" << PlainText << "'" << std::endl;
    std::cout << std::endl;

    std::cout << "Recovered Text:" << std::endl;
    std::cout << "  '" << RecoveredText << "'" << std::endl;
    std::cout << std::endl;

    return 0;
}



==============================
==============================

Result for new code:
$ ./threefish1024try2IV_LENGTH: 128
Algorithm:
  Threefish/CTR
Maximum Key Size:
  128 bytes

Plain Text (12 bytes)
  'Hello World.'

Recovered Text:
  'Hello World.'


==============================

new code:

// threefish1024try2.cpp
//based on
// https://web.archive.org/web/20150315102342/http://www.cryptopp.com/fom-serve/cache/79.html
//2006-Jan-21 11:08am jason entry

// g++ -g3 -O2 -Wall -Wextra  threefish1024try2.cpp -o threefish1024try2 -I/usr/local/include/cryptopp -L/usr/local/lib -lcryptopp


#include <iostream>
//#include <iomanip>

// Crypto++ Includes
#include "cryptlib.h"
#include "modes.h"      // xxx_Mode< >
#include "filters.h"    // StringSource and
#include "threefish.h"

#define CIPHER_MODE CTR_Mode
#define CIPHER Threefish

#define BLOCKSIZE 128
#define KEY_LENGTH 128
#define IV_LENGTH 128

// using namespace CryptoPP;

int main(int argc, char* argv[]) {
    std::cout << "IV_LENGTH: " <<  IV_LENGTH << std::endl;
   
    CryptoPP::byte key[ CryptoPP::CIPHER::MAX_KEYLENGTH ],
                    iv[ IV_LENGTH ];

    ::memset( key, 0x01, CryptoPP::CIPHER::MAX_KEYLENGTH );
    ::memset(  iv, 0x01, IV_LENGTH );
   
    // Message M
    std::string PlainText = "Hello World.";

    // Cipher Text Sink
    std::string CipherText;

    // Encryptor
    CryptoPP::CTR_Mode<CryptoPP::Threefish >::Encryption
        Encryptor; //( key, CryptoPP::CIPHER::MAX_KEYLENGTH, iv, IV_LENGTH );

    Encryptor.SetKeyWithIV(key, KEY_LENGTH, iv, IV_LENGTH);
//     Encryptor.SetKeyWithIV(key, key.size(), iv, iv.size());

    // Encryption
    CryptoPP::StringSource( PlainText, true,
        new CryptoPP::StreamTransformationFilter( Encryptor,
            new CryptoPP::StringSink( CipherText )
        ) // StreamTransformationFilter
    ); // StringSource


    ///////////////////////////////////////
    //                DMZ                //
    ///////////////////////////////////////

    // Recovered Text Sink
    std::string RecoveredText;

    // Decryptor
    CryptoPP::CIPHER_MODE<CryptoPP::CIPHER >::Decryption
        Decryptor;//( key, CryptoPP::CIPHER::MAX_KEYLENGTH, iv, IV_LENGTH );
   
    Decryptor.SetKeyWithIV(key, KEY_LENGTH, iv, IV_LENGTH);

    // Decryption
    CryptoPP::StringSource( CipherText, true,
        new CryptoPP::StreamTransformationFilter( Decryptor,
            new CryptoPP::StringSink( RecoveredText )
        ) // StreamTransformationFilter
    ); // StringSource

    //////////////////////////////////////////
    //                Output                //
    //////////////////////////////////////////

    std::cout << "Algorithm:" << std::endl;
    std::cout << "  " << Encryptor.AlgorithmName() << std::endl;
    std::cout << "Maximum Key Size:" << std::endl;
    std::cout << "  " << Encryptor.MaxKeyLength() << " bytes" << std::endl;
    std::cout << std::endl;


    std::cout << "Plain Text (" << PlainText.length() << " bytes)" << std::endl;
    std::cout << "  '" << PlainText << "'" << std::endl;
    std::cout << std::endl;

    std::cout << "Recovered Text:" << std::endl;
    std::cout << "  '" << RecoveredText << "'" << std::endl;
    std::cout << std::endl;

    return 0;
}

[Jeffrey Walton]

On Thursday, November 16, 2017 at 9:47:22 PM UTC-5, El Ray wrote:

Sorry..deleted that just before your reply appeared.  On further consideration I also realized that although I just got some code to work, I shouldn't assume it's a non-issue till the cross-validation you discussed was done.  But apologies in advanve if this is a non-issue.

Nah, don't worry about it.

The intentions were good and most of the design was good for variable block sizes. We had a few pain points on the cut-in because of the high degree of template parameterization, and depending on virtual functions to correct some hard coded values.

If the library is hard to use correctly, then that points to failures in our design. I think it is more important to correct the design problems then applying band-aides to each incorrect usage. Consider, you used the block cipher the same way block ciphers have been used for the last 25 years, but you could not arrive at the correct result. That's our problem, not your problem.

We are tracking the variable block cipher removal at https://github.com/weidai11/cryptopp/issues/535.

Jeff

[Jeffrey Walton]

On Thursday, November 16, 2017 at 9:36:43 PM UTC-5, Jeffrey Walton wrote:

On Thursday, November 16, 2017 at 8:32:59 PM UTC-5, El Ray wrote:

...
Has anyone successfully used threefish-1024 (or -512 which I surmise would have the same issue)?

...

Give us a day or two for Threefish_256, Threefish_512 and Threefish_1024. It should be a quick cut-over.

We also need to get a hold of more test vectors. Does anyone happen to know where additional test vectors are available for Threefish block cipher? (Skein vectors are readily available. Its Threefish block cipher that's missing them).

Threefish was cut-in. It is being CI'd on a testing branch. OS X and Solaris tested OK on my local test machines. Also see https://github.com/noloader/cryptopp/commit/e52d06070403.

If all goes well Threefish will be cut-in on Master in the next hour or so.

Jeff

[Jeffrey Walton]

The change tested good on the CI branch. It was committed on Master at https://github.com/weidai11/cryptopp/commit/b9bd51f7a61b.

We still need additional test vectors. If you come across a source that can provide them for CTR and CBC mode, then please ping me.

Jeff