AES-NI and march=native

[Siyuan Ren]

It seems that without being compiled with the option `-march=native`, crypto++ does not have AES-NI enabled. `-march=native`, however, generate builds that possibly cannot be run on other CPUs. Could Crypto++ always compile with AES-NI in, and selects whichever implementation available at runtime?

[Jeffrey Walton]

On Tuesday, May 3, 2016 at 5:09:12 AM UTC-4, Siyuan Ren wrote:

It seems that without being compiled with the option `-march=native`, crypto++ does not have AES-NI enabled. `-march=native`, however, generate builds that possibly cannot be run on other CPUs. Could Crypto++ always compile with AES-NI in, and selects whichever implementation available at runtime?

At this point in time, no it cannot.

If GCC supports AES, then you can use `-march=native -maes` to unconditionally enable AES-NI. However, the bigger problem is the interface in the header (H file) potentially changes. That's because when AES is available, AES::Decryption adds additional symbols:

#if CRYPTOPP_BOOL_AESNI_INTRINSICS_AVAILABLE
  size_t AdvancedProcessBlocks(const byte *inBlocks, const byte *xorBlocks, byte *outBlocks, size_t length, word32 flags) const;
#endif

We really need to provide a consistent header for AES, and make changes in the source (CPP file). We also need the source to provide multiple implementations (CXX and AES-NI). Then at runtime, pick the fastest implementation possible.

We identified this as a potential gap, and used a different strategy with BLAKE2. See the use of s_pfn in blake2.cpp. s_pfn is a static function pointer, and it selects the fastest BLAKE2 compression function at runtime. Here's the initialization of s_pfn for the 64-bit compression function (http://github.com/weidai11/cryptopp/blob/master/blake2.cpp):

pfnCompress64 InitializeCompress64Fn()
{
#if CRYPTOPP_BOOL_SSE4_INTRINSICS_AVAILABLE
    if (HasSSE4())
        return &BLAKE2_SSE4_Compress64;
    else
#endif
#if CRYPTOPP_BOOL_SSE2_INTRINSICS_AVAILABLE
    if (HasSSE2())
        return &BLAKE2_SSE2_Compress64;
    else
#endif
    return &BLAKE2_CXX_Compress64;
}

None of this switching bleeds into a header file.

Jeff

[Jeffrey Walton]

On Tue, May 3, 2016 at 10:29 PM, Siyuan Ren <nethe...@gmail.com> wrote:
> So will we and when will we see AES and GCM implemented in the same way?

Fair question.

The earliest we could do something like that is Crypto++ 6.0 because
it removes symbols. Removing symbols requires a major version bump.
Also see http://cryptopp.com/wiki/Release_Versioning.

*IF* the change will be made is a different debate. Personally, I'd
like to see the change made for both AES and SHA. If you look (which I
think you have), AES is some of the hairiest code in the library
because things are so intertwined. It does not allow us to change
things easily because the code paths lack clear delineations or
isolation. And testing changes is non-trivial even with the test
scripts that automate things.

The change will also help with tickets like "Support for CPU
instructions for hardware-accelerated SHA"
(http://github.com/weidai11/cryptopp/issues/139). Moving
implementations to CCP files will allow us to more-easily support
additional platforms, like ARM/NEON. The "hairiness" is the reason I'm
afraid to add code to AES and SHA - I'm concerned I may
unintentionally break something.

*WHEN* the change will be made is yest another path. We have limited
dev cycles, so a change like this needs to be prioritized. I might be
able to peel something off and put it on an experimental branch. The
branch may (or may not) be merged back to mainline.

My commitment for the dev cycles at the moment is to finish Windows
Store support, finish OCB mode, finish Poly1305, finish deterministic
signatures and then push 5.6.4 because of CVE-2016-3995. Also see
"Issue 146: Timing Attack Counter Measure AES
(http://github.com/weidai11/cryptopp/issues/146).

After 5.6.4, we need the interface for the new KDFs, so that will
likely be a 5.7 or 6.0 release. I also want to get a couple new EC
fields and their operations added so we can support the safer curves,
like curve25519.

Jeff

> On Wed, May 4, 2016 at 10:03 AM, Jeffrey Walton <nolo...@gmail.com> wrote:
>>
>>
>> On Tuesday, May 3, 2016 at 5:09:12 AM UTC-4, Siyuan Ren wrote:
>>>
>>> It seems that without being compiled with the option `-march=native`,
>>> crypto++ does not have AES-NI enabled. `-march=native`, however, generate
>>> builds that possibly cannot be run on other CPUs. Could Crypto++ always
>>> compile with AES-NI in, and selects whichever implementation available at
>>> runtime?
>>
>>
>> At this point in time, no it cannot.
>>

>> ...