Crypto++ 5.6.5 and Valgrind

[Ingo Naumann]

Dear All,

I've just upgraded from an older version to Crypto++ 5.6.5 and my
Valgrind is seriously unhappy about it. Even the "cryptest.exe v"
command crashes with "Illegal instruction" (details on request) when
calling it with

valgrind --tool=memcheck --leak-check=full --show-reachable=yes
./cryptest.exe v

as suggested here:

https://www.cryptopp.com/wiki/Profiling

Could anybody confirm or is that just me seeing that?

System is Debian 4.9.0-3-686-pae, gcc (Debian 6.3.0-18) 6.3.0 20170516
in a Virtual Machine (VMWare).

Cheers,

Ingo

[Jeffrey Walton]

On Tuesday, November 28, 2017 at 11:49:17 AM UTC-5, Ingo Naumann wrote:

Dear All,

I've just upgraded from an older version to Crypto++ 5.6.5 and my
Valgrind is seriously unhappy about it. Even the "cryptest.exe v"
command crashes with "Illegal instruction" (details on request) when
calling it with

Crypto++ 5.6.5 and 6.0 and both Valgrind clean. We disabled a few algorithms ASM impl due to Valgrind findings and we are working to re-enable them. I'm working on re-enabling some of them right now. See, for example, https://stackoverflow.com/q/47533475/608639.

We hit those illegal instructions often during testing. The problem is Debian and Fedora provide a Valgrind that is a tad bit old. New processors give Valgrind a lot of problems because the VEX decoder can't handle a lot of code being generated by GCC 6 and 7.

We work around it by building Valgrind from sources. Its fairly easy once you install the packages "automake autoconf libtool". Also see http://valgrind.org/downloads/repository.html.

Also note that last time I checked even Valgrind-Git had some missing VEX code, but it is not as bad as a distro's version. Usually Fedora catches an illegal instruction, but Debian is OK.

Jeff

[Ingo Naumann]

Hi,

Thanks a lot for your answer.

I've just upgraded from an older version to Crypto++ 5.6.5 and my
Valgrind is seriously unhappy about it. Even the "cryptest.exe v"
command crashes with "Illegal instruction" (details on request) when
calling it with

[....]

We work around it by building Valgrind from sources. Its fairly easy once you install the packages "automake autoconf libtool". Also see http://valgrind.org/downloads/repository.html.

Hmmm..... I had built valgrind from the "current" source, ie version 3.13 from June 2017.

So now I've upgraded to valgrind-3.14.0.GIT (cloned ten minutes ago) but it yields the same result. I'm hitting illegal instructions in both "cryptest.exe" as well as my own software (during ECDSA key generation).

But if nobody else can confirm this I'll probably either re-install and start from a scratch and/or perform a memory check with another tool and see what comes up.

Ingo

[Jeffrey Walton]

On Wed, Nov 29, 2017 at 9:16 AM, Ingo Naumann <cyber...@gmail.com> wrote:
> ...

> We work around it by building Valgrind from sources. Its fairly easy once
> you install the packages "automake autoconf libtool". Also see
> http://valgrind.org/downloads/repository.html.
>
> Hmmm..... I had built valgrind from the "current" source, ie version 3.13
> from June 2017.
>
> So now I've upgraded to valgrind-3.14.0.GIT (cloned ten minutes ago) but it
> yields the same result. I'm hitting illegal instructions in both
> "cryptest.exe" as well as my own software (during ECDSA key generation).
>
> But if nobody else can confirm this I'll probably either re-install and
> start from a scratch and/or perform a memory check with another tool and see
> what comes up.

I'm fairly certain a reinstall is going to be a waste of your time,
but I could be wrong. You might try searching for the opcodes that are
causing the problem. They have probably already been reported (I
reported about a half dozen or so myself). Also see
https://bugs.kde.org/buglist.cgi?bug_status=__all__&content=illegal%20instruction

You might be able to work around it by removing the `-march=native`
from CXXFLAGS. Open GNUmakefile and delete the line that adds it.
However, you will loose a fair amount of performance because the
source files depend on some of the options included in
`-march=native`.

You best bet at this point is probably work from Master. After 5.6.5
we changed the model to Base+SIMD. Base C++ files are fine without an
ISA, and SIMD files get an ISA added when needed. As far as I know,
lack of `-march=native` no longer has a material affect on
performance. Base+SIMD also had the effect of side-stepping most the
missing VEX codecs. I don't recall experiencing the problem since we
switched to Base+SIMD.

Jeff

[Jeffrey Walton]

On Wed, Nov 29, 2017 at 9:34 AM, Jeffrey Walton <nolo...@gmail.com> wrote:
> On Wed, Nov 29, 2017 at 9:16 AM, Ingo Naumann <cyber...@gmail.com> wrote:
>> ...

>> ...

>> So now I've upgraded to valgrind-3.14.0.GIT (cloned ten minutes ago) but it
>> yields the same result. I'm hitting illegal instructions in both
>> "cryptest.exe" as well as my own software (during ECDSA key generation).
>>
>> But if nobody else can confirm this I'll probably either re-install and
>> start from a scratch and/or perform a memory check with another tool and see
>> what comes up.
>
> I'm fairly certain a reinstall is going to be a waste of your time,
> but I could be wrong. You might try searching for the opcodes that are
> causing the problem. They have probably already been reported (I
> reported about a half dozen or so myself). Also see
> https://bugs.kde.org/buglist.cgi?bug_status=__all__&content=illegal%20instruction

My bad... Here's a better query. It returns results just for Valgrind.
https://bugs.kde.org/buglist.cgi?bug_status=__all__&content=illegal%20instruction&no_redirect=1&product=valgrind

If you start running searches on your own, be sure to select All
issues (open + closed). The team sometimes closes a report for missing
opcodes as "won't fix".

Jeff

[Ingo Naumann]

Hi,

>> You might be able to work around it by removing the `-march=native`
>> from CXXFLAGS. Open GNUmakefile and delete the line that adds it.
>> However, you will loose a fair amount of performance because the
>> source files depend on some of the options included in
>> `-march=native`.

Wo-ho! This one saves my day! Valgrind's smiling at me again. :)

I.