Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
[openssl.org #2789] SHA-256 bug - Big endian arch
75 views
Skip to first unread message
Pavel Stanek via RT
Apr 12, 2012, 11:16:19 AM4/12/12
to
Hi all,
I think that there is a bug in SHA-256 implementation.
I have tried to compile least Openssl of 0.9.8 branch - 0.9.8u
but make test failed on SHA-256 test.
../util/shlib_wrap.sh ./shatest
test 1 ok
test 2 ok
test 3 ok
../util/shlib_wrap.sh ./sha1test
test 1 ok
test 2 ok
test 3 ok
../util/shlib_wrap.sh ./sha256t
Testing SHA-256
TEST 1 of 3 failed.
make[1]: *** [test_sha] Error 1
I also notice that previous version of OpenSSL also calculate wrong SHA-256 hash of data.
Compared to little endian machines where the hash is correct.
Platform is Renesas SH3E in BIG-endian mode
( used compiler sh3eb-linux-gcc)
OpenSSL self-test report:
OpenSSL version: 0.9.8u
Last change: Fix MMA (Bleichenbacher's attack on PKCS #1 v1.5 RSA pa...
Options: --prefix=/usr enable-zlib-dynamic -DB_ENDIAN no-camellia no-capieng no-cms no-gmp no-hw no-jpake no-krb5 no-mdc2 no-montasm no-rc5 no-rfc3779
no-seed no-shared
OS (uname): Linux 10.1.5.102 2.6.15-29 #2 Tue Jan 10 15:23:09 CET 2012 sh3eb unknown
OS (config): sh3eb-whatever-linux2
Target (default): linux-generic32
Target: linux-generic32
Compiler: Configured with: ../configure --prefix=/usr --mandir=/usr/man --target=sh3eb-linux --host=sh3eb-linux --build=i686-pc-linux-gnu --enable-langu
ages=c,c++,java --disable-checking --with-x=no --enable-threads=posix --enable-long-long
Thread model: posix
gcc version 3.0.4
Thanks for any advance.
Pavel Stanek
______________________________________________________________________
OpenSSL Project http://www.openssl.org
Development Mailing List opens...@openssl.org
Automated List Manager majo...@openssl.org
Andy Polyakov
Apr 13, 2012, 5:32:01 AM4/13/12
to
Paul,
> OpenSSL is just one of many open-source packages whose authors have
> assumed that all IA32 implementations are little endian.
No-no-no. IA is reference to processor architecture and all IA32
implementations *are* little-endian. You emulate big-endian execution
environment on little-endian processor, but it doesn't make processor
big-endian. This is unlike several families of CPUs that actually can
operate in different endiannesses. Some can switch under software
control, e.g. PowerPC, some are soldered to specific endianness, e.g.
MIPS, when endianness is determined when you place processor on PCB and
arrange specific pin to be connected to ground or not (or whatever
equivalent). I'd guess SH is of latter kind, i.e. you can build either
endian system with it. Then there also are processors that can access
data [and data only] in specific byte order, e.g. Itanium, ARMv7, but
it's different story.
And do you mean that, unlike "authors of open-source packages",
developers of commercial software are not prone to make such
assumptions? :-) :-) :-)
> I discovered
> the same problem several years ago when we ported OpenSSL to the
> Stratus VOS operating system, which is also big-endian on IA32.
What is it you're trying to assert exactly? There that is bug in plain C
SHA256 that has everything to do with endianness? But your own
experience contradicts it. Indeed, if there was bug, you would suffer
from it too. VOS-specific changes simply switch off inline assembler and
assumptions of little endianness, but don't change a line of
computationally significant code. Basically it's very same source after
pre-processing that is fed to your and originator's compiler.
> This
> issue is so pervasive that when a numeric package fails on my system,
> I immediately start looking for C macros of the form "__i386".
I believe that we've touched suitability of __i386 on your platform. I'd
argue that it shouldn't be even defined on your platform. You could have
avoided all the problems (at least with badly authored open-source
packages ;-) altogether by pre-defining say __i386be instead...
> OpenSSL is just one of many open-source packages whose authors have
> assumed that all IA32 implementations are little endian.
No-no-no. IA is reference to processor architecture and all IA32
implementations *are* little-endian. You emulate big-endian execution
environment on little-endian processor, but it doesn't make processor
big-endian. This is unlike several families of CPUs that actually can
operate in different endiannesses. Some can switch under software
control, e.g. PowerPC, some are soldered to specific endianness, e.g.
MIPS, when endianness is determined when you place processor on PCB and
arrange specific pin to be connected to ground or not (or whatever
equivalent). I'd guess SH is of latter kind, i.e. you can build either
endian system with it. Then there also are processors that can access
data [and data only] in specific byte order, e.g. Itanium, ARMv7, but
it's different story.
And do you mean that, unlike "authors of open-source packages",
developers of commercial software are not prone to make such
assumptions? :-) :-) :-)
> I discovered
> the same problem several years ago when we ported OpenSSL to the
> Stratus VOS operating system, which is also big-endian on IA32.
What is it you're trying to assert exactly? There that is bug in plain C
SHA256 that has everything to do with endianness? But your own
experience contradicts it. Indeed, if there was bug, you would suffer
from it too. VOS-specific changes simply switch off inline assembler and
assumptions of little endianness, but don't change a line of
computationally significant code. Basically it's very same source after
pre-processing that is fed to your and originator's compiler.
> This
> issue is so pervasive that when a numeric package fails on my system,
> I immediately start looking for C macros of the form "__i386".
I believe that we've touched suitability of __i386 on your platform. I'd
argue that it shouldn't be even defined on your platform. You could have
avoided all the problems (at least with badly authored open-source
packages ;-) altogether by pre-defining say __i386be instead...
Andy Polyakov via RT
Apr 13, 2012, 5:38:36 AM4/13/12
to
> I think that there is a bug in SHA-256 implementation.
>
> I have tried to compile least Openssl of 0.9.8 branch - 0.9.8u
> but make test failed on SHA-256 test.
>
> ../util/shlib_wrap.sh ./shatest
> test 1 ok
> test 2 ok
> test 3 ok
> ../util/shlib_wrap.sh ./sha1test
> test 1 ok
> test 2 ok
> test 3 ok
> ../util/shlib_wrap.sh ./sha256t
> Testing SHA-256
> TEST 1 of 3 failed.
> make[1]: *** [test_sha] Error 1
>
>
> I also notice that previous version of OpenSSL also calculate wrong SHA-256 hash of data.
> Compared to little endian machines where the hash is correct.
What exactly do you mean by that? Code is maintained and was observed to
>
> I have tried to compile least Openssl of 0.9.8 branch - 0.9.8u
> but make test failed on SHA-256 test.
>
> ../util/shlib_wrap.sh ./shatest
> test 1 ok
> test 2 ok
> test 3 ok
> ../util/shlib_wrap.sh ./sha1test
> test 1 ok
> test 2 ok
> test 3 ok
> ../util/shlib_wrap.sh ./sha256t
> Testing SHA-256
> TEST 1 of 3 failed.
> make[1]: *** [test_sha] Error 1
>
>
> I also notice that previous version of OpenSSL also calculate wrong SHA-256 hash of data.
> Compared to little endian machines where the hash is correct.
work correctly on either endian system. There are two possibilities:
system is not believed endianness or compiler is buggy. If first was
case, then even SHA1 test would fail too, so the failure sounds more
like compiler bug. Try to drop optimization level: edit Makefile in text
editor and replace -O3 with -O0, remove crypto/sha/sha256.o and 'make'.
This, dropping optimization level, is suggested in README.
> Platform is Renesas SH3E in BIG-endian mode
> ( used compiler sh3eb-linux-gcc)
Andy Polyakov
Apr 13, 2012, 11:35:12 AM4/13/12
to
> I will take up the idea of __i386be with our compiler folks. I have
> not run across that macro before.
I didn't mean that you'd do anything about it. Omitting __i386 in favor
of replacement pre-defined macro was appropriate in the very beginning,
but now it's kind of too late, it's just reality to deal with. I mean
when I wrote "you *could*" I was referring to *missed* opportunity.
> not run across that macro before.
I didn't mean that you'd do anything about it. Omitting __i386 in favor
of replacement pre-defined macro was appropriate in the very beginning,
but now it's kind of too late, it's just reality to deal with. I mean
when I wrote "you *could*" I was referring to *missed* opportunity.
Pavel Stanek via RT
Apr 13, 2012, 3:04:28 PM4/13/12
to
HI,
We have been using older version of Openssl (0.9.8c) and this one has same problem.
Used compiler is not perfect and often has prolem with O3 so i already tried Os (doesn't work) and O2 - thats work ok. So I suppose that optimalisations are not the issue.
I tested both SMALL FOOTPRINT and normal code and both has same problem.
I also have own SHA256 implementation ( implemented as defined in standard ) from another project. So when I replace the core of SHA256 computation
(sha256 hash update ) with my implementation it works ok.
Then all tests (make tests) passed without errors. So it's not exactly endian problem, but SHA256 implementation on SH architecture.
Our implementation is using rotation as defined in standart but Openssl uses opposite.
SH3 architecture in Big endian is not so common so there could be some unexpected bahaviour in compiler.
What we can say is that problem is not in padding but only in hash update.
SH3E - this is renesas SuperH architrecture with DSP also so there could be some instructions to modify only part of word etc.
I found suspected code in OpenSSL - union of long and unsigned char - to detect endiannes.
This could be one of problem, but when I tried to force edian into code it still does not work.
If I will have more time i'll try to check where could be problem (but on SMALL footprint code).
Best regards
Pavel
______________________________________________________________
> Od: "Andy Polyakov via RT"
> Komu:
> Datum: 13.04.2012 11:38
> P?edm?t: Re: [openssl.org #2789] SHA-256 bug - Big endian arch
Andy Polyakov via RT
Apr 14, 2012, 4:41:38 AM4/14/12
to
> We have been using older version of Openssl (0.9.8c) and this one has
> same problem.
Even though 0.9.8c is more sensitive to endianness (sha256_block
> same problem.
function accepts additional 'host' flag, which caller can get wrong), it
should have worked as long as -DB_ENDIAN is passed. Later versions work
even without [LB]_ENDIAN. But in either case endianness was always taken
into account and code was actually tested on both big- and little-endian
platforms.
> Used compiler is not perfect and often has prolem with O3 so i
> already tried Os (doesn't work) and O2 - thats work ok. So I suppose
> that optimalisations are not the issue.
> I tested both SMALL FOOTPRINT and normal code and both has same
> problem.
>
> I also have own SHA256 implementation ( implemented as defined in
> standard ) from another project. So when I replace the core of SHA256
> computation (sha256 hash update ) with my implementation it works ok.
> Then all tests (make tests) passed without errors. So it's not
> exactly endian problem, but SHA256 implementation on SH architecture.
problem with OpenSSL source code?
> Our implementation is using rotation as defined in standart but
> Openssl uses opposite.
> SH3 architecture in Big endian is not so
> common so there could be some unexpected bahaviour in compiler.
formally you can't say "SH *architecture* is big-endian", you can only
say "this particular SH-based system is big-endian." As far as I
understand endianness is determined by logical level fed to specific CPU
pin at power-up.
Pavel Stanek via RT
Apr 15, 2012, 7:57:46 AM4/15/12
to
HI,
You are right, the optimalisations are the issue.
When I compile it with O0 then it's OK.
I also found that problem occurs in macros Siga0,1 Maj, ...
Wth O0 there are correct valus but with O2 there are incorrect values.
Compiler must have some problem there.
( I also test to modify variables declaration to volatile and that helps too. )
So definetly we could close this issue with result "broken compiler".
( which really makes me unhappy)
Many thanks for your time.
Best regars
Pavel
______________________________________________________________
> Od: "Andy Polyakov via RT"
> Komu:
> Datum: 14.04.2012 10:41
______________________________________________________________
> Od: "Andy Polyakov via RT"
> Komu:
0 new messages