Crypto++ 5.6.3 Release Candidate available (RC1)

[Jeffrey Walton]

Hi Everyone,

We have a new release candidate available at https://www.cryptopp.com/cryptopp563rc1.zip. Here are the checksums:

  $ md5sum cryptopp563rc1.zip
  a33dd656de4567e02e64704b92820f9f

  $ sha1sum cryptopp563rc1.zip
  dd6f54fb68d4133f23dff85d0b106a1053058ad8

  $ sha256sum cryptopp563rc1.zip
  5628e248291a8f238f0d929d7d956fb4ccf836d65958172de11488b9cd1abc4a

RC1 performs more warning cleanup under Visual Studio. It also clears some issues raised by Enterprise Analysis (/analyze). Finally, RC1 clears a Multiarch/Fat binary issue under OS X.

There's a diff in the ZIP describing precisely what changed from Crypto++ 5.6.2 (version 5.6.2 was SVN revision 541).

Please report any problems you encounter.

[Jean-Pierre Münch]

Am 07.09.2015 um 17:24 schrieb Jeffrey Walton:

Hi Everyone,

We have a new release candidate available at https://www.cryptopp.com/cryptopp563rc1.zip. Here are the checksums:

  $ md5sum cryptopp563rc1.zip
  a33dd656de4567e02e64704b92820f9f

  $ sha1sum cryptopp563rc1.zip
  dd6f54fb68d4133f23dff85d0b106a1053058ad8

  $ sha256sum cryptopp563rc1.zip
  5628e248291a8f238f0d929d7d956fb4ccf836d65958172de11488b9cd1abc4a

RC1 performs more warning cleanup under Visual Studio.

I just compiled it and was *very* impressed. The number of warnings reduced from 3.5k+ down to 5/9 :) Really good job there. Although it looks like you just suppressed them ;)

It also clears some issues raised by Enterprise Analysis (/analyze). Finally, RC1 clears a Multiarch/Fat binary issue under OS X.

There's a diff in the ZIP describing precisely what changed from Crypto++ 5.6.2 (version 5.6.2 was SVN revision 541).

Please report any problems you encounter.

We still don't get a clean compile on VS 2015 though.

We get an C4589 on wait.h line 140, which complains about CryptoPP::WaitObjectsTracer not calling the constructor CryptoPP::Tracer four times, I don't know how to fix it though.

We still need to clean those LNK4221s which can be done by simply adding the required line (exclude from build) to the paths in cryptlib.vcproj for all configurations of strciphr.cpp, simple.cpp, polynomi.cpp, algebra.cpp, like already done for eccrypto.cpp. It doesn't work for me doing this with an editor though so we'd have to use something like VS2005 (the least version we still support) to set the flags.

We also get a strange warning on algparam.h line 35 (C4459) "declaration of 'cryptopp_assert_35' hides global declaration", there seems to be no official MS documentation about this one though. I've no clue about to fix this one as it references StringSource and fipstest.cpp line 35 which is a CRYPTOPP_COMPILE_ASSERT.

I'm sad to have to tell you that the 64-bit builds of *all* projects except cryptlib (=cryptest, cryptopp) are broken and result in a bunch of "error LNK2001: unresolved external symbol ...". I suspect that dlltest works because it doesn't link any of the affected functions.

The affected functions seem to be: Baseline_Add, Baseline_Sub, X86_SHA256_HashBlocks, Rijndael_Enc_AdvancedProcessBlocks, Panama_SSE2_Pull, Salsa20_OperateKeystream, GCM_AuthenticateBlocks_2K, GCM_AuthenticateBlocks_64K, Sosemanuk_OperateKeystream

These may be functions that reside in the .asm files which are de facto excluded from the build by "does not participate in build" which means the assembler is not there to assemble the necessary code. Strange. IIRC past cryptopp manually called masm.exe or ml64.exe or whatever with the corresponding file...

Sorry that I couldn't provide more fixes :(

BR

JPM

--
--
You received this message because you are subscribed to the "Crypto++ Users" Google Group.
To unsubscribe, send an email to cryptopp-user...@googlegroups.com.
More information about Crypto++ and this group is available at http://www.cryptopp.com.
---
You received this message because you are subscribed to the Google Groups "Crypto++ Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cryptopp-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jeffrey Walton]

We have a new release candidate available at https://www.cryptopp.com/cryptopp563rc1.zip. Here are the checksums:

  $ md5sum cryptopp563rc1.zip
  a33dd656de4567e02e64704b92820f9f

  $ sha1sum cryptopp563rc1.zip
  dd6f54fb68d4133f23dff85d0b106a1053058ad8

  $ sha256sum cryptopp563rc1.zip
  5628e248291a8f238f0d929d7d956fb4ccf836d65958172de11488b9cd1abc4a

RC1 performs more warning cleanup under Visual Studio.

I just compiled it and was *very* impressed. The number of warnings reduced from 3.5k+ down to 5/9 :) Really good job there. Although it looks like you just suppressed them ;)

It also clears some issues raised by Enterprise Analysis (/analyze). Finally, RC1 clears a Multiarch/Fat binary issue under OS X.

There's a diff in the ZIP describing precisely what changed from Crypto++ 5.6.2 (version 5.6.2 was SVN revision 541).

Please report any problems you encounter.

We still don't get a clean compile on VS 2015 though.

Here's what the release notes have to say:

      * cleared most MSVC warnings with /W4

 We don't claim we cleared all of them.

We still need to clean those LNK4221s which can be done by simply adding the required line (exclude from build) to the paths in cryptlib.vcproj for all configurations of strciphr.cpp, simple.cpp, polynomi.cpp, algebra.cpp, like already done for eccrypto.cpp. It doesn't work for me doing this with an editor though so we'd have to use something like VS2005 (the least version we still support) to set the flags.

This issue finds its roots in C++ compiler workarounds in the 1990s. I recall this issue because I was doing my CompSci undergrad work at the time, and I used Visual Studio 5.0. There was an MSDN article that stated to include the implementation in the header so the compiler could find the bodies/implementations:

    // eccrypto.h
    class SomeECCryptoClass
    {
        ...
    }

    #include eccrytpo.cpp

Then, in eccrypto.cpp::

    // The implementation
    SomeECCryptoClass::SomeECCryptoClass() { ... }
 

We also get a strange warning on algparam.h line 35 (C4459) "declaration of 'cryptopp_assert_35' hides global declaration", there seems to be no official MS documentation about this one though. I've no clue about to fix this one as it references StringSource and fipstest.cpp line 35 which is a CRYPTOPP_COMPILE_ASSERT.

Yeah, that one is kind of odd. It seems to be a one-off, so its a lower priority at the moment.
 

I'm sad to have to tell you that the 64-bit builds of *all* projects except cryptlib (=cryptest, cryptopp) are broken and result in a bunch of "error LNK2001: unresolved external symbol ...". I suspect that dlltest works because it doesn't link any of the affected functions.

From the README:

*** MSVC-Specific Information ***

    On Windows, Crypto++ can be compiled into 3 forms: a static library
    including all algorithms, a DLL with only FIPS Approved algorithms, and
    a static library with only algorithms not in the DLL.
    (FIPS Approved means Approved according to the FIPS 140-2 standard.)
    The DLL may be used by itself, or it may be used together with the second
    form of the static library. MSVC project files are included to build
    all three forms, and sample applications using each of the three forms
    are also included.

    To compile Crypto++ with MSVC, open the "cryptest.dsw" (for MSVC 6 and MSVC .NET
    2003) or "cryptest.sln" (for MSVC 2005 - 2010) workspace file and build one or
   more of the following projects:

    cryptopp - This builds the DLL. Please note that if you wish to use Crypto++
      as a FIPS validated module, you must use a pre-built DLL that has undergone
      the FIPS validation process instead of building your own.
    dlltest - This builds a sample application that only uses the DLL.
      cryptest Non-DLL-Import Configuration - This builds the full static library
      along with a full test driver.
    cryptest DLL-Import Configuration - This builds a static library containing
      only algorithms not in the DLL, along with a full test driver that uses
      both the DLL and the static library.

I'm not really going to spend any time on the DLL projects because its an evolutionary dead end. If you want a DLL, then wrap the static library after changing /MT or /MTd to /MD or /MDd in the Cryptlib project.

Jeff

[ra...@teamviewer.com]

Am Montag, 7. September 2015 17:24:41 UTC+2 schrieb Jeffrey Walton:

RC1 performs more warning cleanup under Visual Studio. It also clears some issues raised by Enterprise Analysis (/analyze). Finally, RC1 clears a Multiarch/Fat binary issue under OS X.

There's a diff in the ZIP describing precisely what changed from Crypto++ 5.6.2 (version 5.6.2 was SVN revision 541).

Please report any problems you encounter.

Everything works for me. But can't https://github.com/weidai11/cryptopp/commit/a0390f1fd725e37a201cc47ba47204182ebc1ccf make it into 5.6.3? I find the Visual C++ warning about unreferenced local functions to be the most annoying one as it appears in every compilation unit including misc.h and can only be suppressed for the whole compilation unit in user code, not only for the Crypto++ headers.

[ra...@teamviewer.com]

Am Montag, 7. September 2015 17:24:41 UTC+2 schrieb Jeffrey Walton:

Please report any problems you encounter.

In filters.h, there is an "#if CRYPTOPP_MSC_VERSION" before config.h is included, so if filters.h is the first include, there is a #pragma warning(pop) at the end of the file without a #pragma warning(push).

[ra...@teamviewer.com]

Am Dienstag, 8. September 2015 14:31:00 UTC+2 schrieb ra...@teamviewer.com:

In filters.h, there is an "#if CRYPTOPP_MSC_VERSION" before config.h is included, so if filters.h is the first include, there is a #pragma warning(pop) at the end of the file without a #pragma warning(push).

Oh, and there's the same problem at least in pubkey.h.

[Jeffrey Walton]

Everything works for me. But can't https://github.com/weidai11/cryptopp/commit/a0390f1fd725e37a201cc47ba47204182ebc1ccf make it into 5.6.3? I find the Visual C++ warning about unreferenced local functions to be the most annoying one as it appears in every compilation unit including misc.h and can only be suppressed for the whole compilation unit in user code, not only for the Crypto++ headers.

 That change breaks versioning because it changes the function signature. We'll have to wait until Crypto++ 6.0.

Jeff

[William Elliott]

Tried to open in VC++ 2010:

Project: F:\CRYPTSRC\cryptopp563rc1\cryptlib.vcproj

Filename	Status	Errors	Warnings
		1	1
Conversion Report - : Cannot load the project due to a corrupt project file. The following error has occurred during XML parsing: File: F:\CRYPTSRC\cryptopp563rc1\cryptlib.vcproj Line: 102 Column: 5 Error Message: System error: -2147154677. The file 'F:\CRYPTSRC\cryptopp563rc1\cryptlib.vcproj' has failed to load.
1 file	Converted: 0 Not converted: 1

what am I doing wrong?

[Jeffrey Walton]

Line 102 is the closing tag of:

            <Tool
                Name="VCPreBuildEventTool"
            />
 

what am I doing wrong?

I can't duplicate it on VS2010

I suspect there's a problem with your Visual Studio installation. Can you perform a repair?

Also see "Visual Studio 2010 C++ Project Upgrade Guide," http://blogs.msdn.com/b/vcblog/archive/2010/03/02/visual-studio-2010-c-project-upgrade-guide.aspx. It mentions that particular error.

Jeff

[Jeffrey Walton]

OK, thanks. I was trying to side step it through either a pch.h include, a cryptlib.h include and/or precompiled headers. But I think you are right - we have to perform the include.

Jeff

[Jeffrey Walton]

We came to a compromise....

Crypto++ 5.6.3 RC4 can dispatch the warning, but you have to omit/disable/delete CRYPTOPP_MAINTAIN_BACKWARDS_COMPATIBILITY_562 from config.h. Its enabled by default due to versioning requirements. If you don't have to worry about it, then nuke it because its generally no good. (And it won't be present in Crypto++ 6.0).

Also see https://groups.google.com/d/msg/cryptopp-users/-aD0A-2gBb8/dJw-uPOUBwAJ and https://cryptopp.com/wiki/Config.h#Avoid_MAINTAIN_BACKWARDS_COMPATIBILITY.

Jeff

[Jeffrey Walton]

I'm sad to have to tell you that the 64-bit builds of *all* projects except cryptlib (=cryptest, cryptopp) are broken and result in a bunch of "error LNK2001: unresolved external symbol ...". I suspect that dlltest works because it doesn't link any of the affected functions.

OK, so I had a chance to spend some time with this....

Microsoft documented some of the changes to the underlying MSBuild engine at http://blogs.msdn.com/b/vcblog/archive/2010/03/02/visual-studio-2010-c-project-upgrade-guide.aspx. They created a lot of problems.

*** TargetName and TargetExt ***

VS 2010 and above depend upon TargetName and TargetExt. Microsoft stated they would not set it based on OutputDirectory and OutputFile. What they don't say is VS2005/VS2008 *lack* the setting, so there's no way for use to fix it before the upgrade (and they refuse to set it after the upgrade).

*** Intermediate Directory Path ***

Over half the Intermediate Directories were simply wrong after the upgrade.

*** Name Mangling ***

It appears the algorithm to produce mangled names changed. We have at least one:

    error LNK2019: unresolved external symbol "unsigned __int64 * CryptoPP::rdtable::Te"
    (?Te@rdtable@CryptoPP@@3PA_KA) referenced in function
    Rijndael_Enc_AdvancedProcessBlocks
    x64\DLL_Output\Release\cryptopp.dll : fatal error LNK1120: 1 unresolved externals

I'll have to investigate this further.

*** Incorrect Architectures ***

I'm seeing a fair amount of this:

    cryptopp.lib(cryptopp.dll) : fatal error LNK1112: module machine type 'X86' conflicts
    with target machine type 'x64'

Even with the Intermediate directories fixed, its still managing to produce the wrong intermediate code, or place them in the wrong directories.

**********

its going to take time to find solutions to all the problems Microsoft created.

Jeff

[Jean-Pierre Münch]

Am 11.10.2015 um 02:50 schrieb Jeffrey Walton:

I'm sad to have to tell you that the 64-bit builds of *all* projects except cryptlib (=cryptest, cryptopp) are broken and result in a bunch of "error LNK2001: unresolved external symbol ...". I suspect that dlltest works because it doesn't link any of the affected functions.

OK, so I had a chance to spend some time with this....

Well, ... I already found the issue.

It's a project conversion error that removes the custom build steps for our .asm files. This makes the .asm not being assembled and integrated into the library, resulting in a linker failure as the necessary code is missing. This only affects x64 as x86 uses inline assembly. And it only occurred when migrating to VS2015, 2010 and 2012 worked just fine.

The workaround is to manually re-add the necessary commands.
It may also be the case that my system is the issue as it's often enough behaving abnormally.

Microsoft documented some of the changes to the underlying MSBuild engine at http://blogs.msdn.com/b/vcblog/archive/2010/03/02/visual-studio-2010-c-project-upgrade-guide.aspx. They created a lot of problems.

*** TargetName and TargetExt ***

VS 2010 and above depend upon TargetName and TargetExt. Microsoft stated they would not set it based on OutputDirectory and OutputFile. What they don't say is VS2005/VS2008 *lack* the setting, so there's no way for use to fix it before the upgrade (and they refuse to set it after the upgrade).

What are we actually doing that would bother us here? I think the conversion actually sets the correct entries for us here.

*** Intermediate Directory Path ***

Over half the Intermediate Directories were simply wrong after the upgrade.

Which? The x64 ones? I personally never experienced problems in that direction...

*** Name Mangling ***

It appears the algorithm to produce mangled names changed. We have at least one:

    error LNK2019: unresolved external symbol "unsigned __int64 * CryptoPP::rdtable::Te"
    (?Te@rdtable@CryptoPP@@3PA_KA) referenced in function
    Rijndael_Enc_AdvancedProcessBlocks
    x64\DLL_Output\Release\cryptopp.dll : fatal error LNK1120: 1 unresolved externals

I'll have to investigate this further.

That one looks really strange.

*** Incorrect Architectures ***

I'm seeing a fair amount of this:

    cryptopp.lib(cryptopp.dll) : fatal error LNK1112: module machine type 'X86' conflicts
    with target machine type 'x64'

Even with the Intermediate directories fixed, its still managing to produce the wrong intermediate code, or place them in the wrong directories.

so here the "strange" directories start breaking things? Interesting. I'll also investigate this one soon(tm).

BR

JPM

**********

its going to take time to find solutions to all the problems Microsoft created.

Jeff

[Jeffrey Walton]

I'm sad to have to tell you that the 64-bit builds of *all* projects except cryptlib (=cryptest, cryptopp) are broken and result in a bunch of "error LNK2001: unresolved external symbol ...". I suspect that dlltest works because it doesn't link any of the affected functions.

OK, so I had a chance to spend some time with this....

Well, ... I already found the issue.

It's a project conversion error that removes the custom build steps for our .asm files. This makes the .asm not being assembled and integrated into the library, resulting in a linker failure as the necessary code is missing. This only affects x64 as x86 uses inline assembly. And it only occurred when migrating to VS2015, 2010 and 2012 worked just fine.

The workaround is to manually re-add the necessary commands.
It may also be the case that my system is the issue as it's often enough behaving abnormally.

Have you tested it against the 22 configurations provided in the solution (including the non-x64 ones)?

Jeff

[Jean-Pierre Münch]

Am 11.10.2015 um 21:10 schrieb Jeffrey Walton:

I'm sad to have to tell you that the 64-bit builds of *all* projects except cryptlib (=cryptest, cryptopp) are broken and result in a bunch of "error LNK2001: unresolved external symbol ...". I suspect that dlltest works because it doesn't link any of the affected functions.

OK, so I had a chance to spend some time with this....

Well, ... I already found the issue.

It's a project conversion error that removes the custom build steps for our .asm files. This makes the .asm not being assembled and integrated into the library, resulting in a linker failure as the necessary code is missing. This only affects x64 as x86 uses inline assembly. And it only occurred when migrating to VS2015, 2010 and 2012 worked just fine.

it also affected the adhoc.cpp file (which gets renamed to adhoc.cpp from adhoc.cpp.proto at some point). Normally this file is missing and will cause a build error on cryptest.

The workaround is to manually re-add the necessary commands.
It may also be the case that my system is the issue as it's often enough behaving abnormally.

Have you tested it against the 22 configurations provided in the solution (including the non-x64 ones)?

I tested the build of the most important parts, which are {cryptest, cryptlib} on {Debug, Release} on {x86, x64}. dlltest already worked fine (or did afterwards) and the dll build was broken anyway. Cryptopp x86 works fine, DLLtest works fine.
Cryptopp x64 is more tricky as it looks like the reference it uses is wrong. It references to Cryptest (our test application) instead of Cryptlib (our library project).
So the only stuff untested by this are those DLL-Import configuration (we have 8 of them).

BR

JPM

[Jeffrey Walton]

On Sunday, October 11, 2015 at 12:45:12 PM UTC-4, jean-pierre.muench wrote:

Well, ... I already found the issue.

It's a project conversion error that removes the custom build steps for our .asm files. This makes the .asm not being assembled and integrated into the library, resulting in a linker failure as the necessary code is missing. This only affects x64 as x86 uses inline assembly. And it only occurred when migrating to VS2015, 2010 and 2012 worked just fine.

You'll have to forgive my ignorance here....

Below is what I see after a VCUpgrade and subsequent build. It appears the custom build steps are present for x64.

When I check for the symbol using dumpbin:

C:\Program Files (x86)\Microsoft Visual Studio 11.0>cd C:\...\cryptopp-5.6.3-vs2012\x64\Output\Debug

C:\...\cryptopp-5.6.3-vs2012\x64\Output\Debug>dumpbin /SYMBOLS cryptlib.lib | findstr "DEFAULT_CHANNEL"
1874 00000000 SECT71D notype       External    | ?DEFAULT_CHANNEL@CryptoPP@@3V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@B (class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const CryptoPP::DEFAULT_CHANNEL)
...

C:\...\cryptopp-5.6.3-vs2012\x64\Output\Debug>dumpbin /SYMBOLS cryptlib.lib | findstr "rdtable"
00E 00000000 UNDEF  notype       External     | ?Te@rdtable@CryptoPP@@3PA_KA (unsigned __int64 * CryptoPP::rdtable::Te)

How exactly, did you resolve the issue?

Jeff

----------
1>------ Build started: Project: cryptlib, Configuration: Debug x64 ------
1>Build started 10/12/2015 12:10:40 AM.
1>PrepareForBuild:
1>  Creating directory "x64\Output\Debug\".
1>InitializeBuildStatus:
1>  Creating "x64\Output\Temp\Debug\cryptlib.unsuccessfulbuild" because "AlwaysCreate" was specified.
1>CustomBuild:
1>  Performing Custom Build Tools
1>  Performing Custom Build Tools
1>   Assembling: C:\Users\...\Desktop\cryptopp-5.6.3-vs2012\x64dll.asm
1>  Performing Custom Build Tools
1>   Assembling: C:\Users\...\Desktop\cryptopp-5.6.3-vs2012\x64masm.asm
...
1>Lib:
1>  cryptlib.vcxproj -> C:\Users\...\Desktop\cryptopp-5.6.3-vs2012\x64\Output\Debug\cryptlib.lib
1>FinalizeBuildStatus:
1>  Deleting file "x64\Output\Temp\Debug\cryptlib.unsuccessfulbuild".
1>  Touching "x64\Output\Temp\Debug\cryptlib.lastbuildstate".
1>
1>Build succeeded.

2>------ Build started: Project: cryptest, Configuration: Debug x64 ------
2>Build started 10/12/2015 12:11:23 AM.
2>InitializeBuildStatus:
...
2>  Generating Code...
2>cryptlib.lib(x64dll.obj) : error LNK2019: unresolved external symbol "unsigned __int64 * CryptoPP::rdtable::Te" (?Te@rdtable@CryptoPP@@3PA_KA) referenced in function Rijndael_Enc_AdvancedProcessBlocks
2>x64\Output\Debug\cryptest.exe : fatal error LNK1120: 1 unresolved externals

[Jean-Pierre Münch]

Am 12.10.2015 um 06:46 schrieb Jeffrey Walton:

On Sunday, October 11, 2015 at 12:45:12 PM UTC-4, jean-pierre.muench wrote:

Well, ... I already found the issue.

It's a project conversion error that removes the custom build steps for our .asm files. This makes the .asm not being assembled and integrated into the library, resulting in a linker failure as the necessary code is missing. This only affects x64 as x86 uses inline assembly. And it only occurred when migrating to VS2015, 2010 and 2012 worked just fine.

You'll have to forgive my ignorance here....

Below is what I see after a VCUpgrade and subsequent build. It appears the custom build steps are present for x64.

They weren't for me after my update VS 2005 -> 2015 which caused my errors.

When I check for the symbol using dumpbin:

C:\Program Files (x86)\Microsoft Visual Studio 11.0>cd C:\...\cryptopp-5.6.3-vs2012\x64\Output\Debug

C:\...\cryptopp-5.6.3-vs2012\x64\Output\Debug>dumpbin /SYMBOLS cryptlib.lib | findstr "DEFAULT_CHANNEL"
1874 00000000 SECT71D notype       External    | ?DEFAULT_CHANNEL@CryptoPP@@3V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@B (class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const CryptoPP::DEFAULT_CHANNEL)
...

C:\...\cryptopp-5.6.3-vs2012\x64\Output\Debug>dumpbin /SYMBOLS cryptlib.lib | findstr "rdtable"
00E 00000000 UNDEF  notype       External     | ?Te@rdtable@CryptoPP@@3PA_KA (unsigned __int64 * CryptoPP::rdtable::Te)

How exactly, did you resolve the issue?

I did not solve this issue as I didn't experience it. I had the (different) error that the custom build tools got lost on conversion causing linker errors as ASM code was missing.

BR

JPM