Unittests failing on armel architecture

[Iustin Pop]

Hi,

I'm one of the Debian maintainers for protobuf and 2.3.0 fails during
unittests on the armel architecture (only on it). This might be a host
issue or a CFLAGS issue (we had something like this for sh4, where
'-mieee' is needed, for example).

In any case, here are the failues we see in the logs:

[ RUN ] CustomOptions.OptionLocations
google/protobuf/descriptor_unittest.cc:1979: Failure
Value of: service->options().GetExtension(protobuf_unittest::service_opt1)
Actual: -8589934594
Expected: -9876543210LL
Which is: -9876543210
[ FAILED ] CustomOptions.OptionLocations (4 ms)
[ RUN ] CustomOptions.OptionTypes
google/protobuf/descriptor_unittest.cc:1999: Failure
Value of: options->GetExtension(protobuf_unittest::sint64_opt)
Actual: -9223372035781033984
Expected: kint64min
Which is: -9223372036854775808
google/protobuf/descriptor_unittest.cc:2013: Failure
Value of: options->GetExtension(protobuf_unittest::sint64_opt)
Actual: 9223372035781033983
Expected: kint64max
Which is: 9223372036854775807
[ FAILED ] CustomOptions.OptionTypes (2 ms)

[ RUN ] ExtensionSetTest.Parsing
google/protobuf/test_util.cc:1127: Failure
Value of: message.GetExtension(unittest::optional_sint64_extension )
Actual: 0
Expected: 106
google/protobuf/test_util.cc:1184: Failure
Value of: message.GetExtension(unittest::repeated_sint64_extension , 0)
Actual: 0
Expected: 206
google/protobuf/test_util.cc:1212: Failure
Value of: message.GetExtension(unittest::repeated_sint64_extension , 1)
Actual: 0
Expected: 306
google/protobuf/test_util.cc:1265: Failure
Value of: message.GetExtension(unittest::default_sint64_extension )
Actual: 0
Expected: 406
[ FAILED ] ExtensionSetTest.Parsing (4 ms)

There are also some other tests failing, but this is probably because
some dependencies are missing:

libprotobuf FATAL google/protobuf/test_util_lite.cc:950] CHECK failed: (106) == (message.GetExt
/bin/bash: line 4: 26978 Aborted ${dir}$tst
FAIL: protobuf-lite-test

Can you confirm that we need 'unzip' and 'jar' to be available during
unittests? Or why does protobuf-lite-test 'FAIL' hard (instead of
exiting with no error if these are just optional deps?)

In any case, the full build log is at
https://buildd.debian.org/fetch.cgi?pkg=protobuf&arch=armel&ver=2.3.0-1&stamp=1264814475&file=log,
if you're interested.

Thanks in advance,
iustin

[Kenton Varda]

$ python

>>> "%x" % 9223372036854775808

'8000000000000000'

>>> "%x" % 9223372035781033984

'7fffffffc0000000'

So it looks like somehow the bits are getting cut off.  I suspect a compiler bug of some sort.  Perhaps the auto-generated copy construct for struct ExtensionSet::Extension is not copying the union correctly?  Unfortunately I can't really help debug as I don't have access to this platform.

 

[ RUN      ] ExtensionSetTest.Parsing
google/protobuf/test_util.cc:1127: Failure
Value of: message.GetExtension(unittest::optional_sint64_extension )
 Actual: 0
Expected: 106
google/protobuf/test_util.cc:1184: Failure
Value of: message.GetExtension(unittest::repeated_sint64_extension , 0)
 Actual: 0
Expected: 206
google/protobuf/test_util.cc:1212: Failure
Value of: message.GetExtension(unittest::repeated_sint64_extension , 1)
 Actual: 0
Expected: 306
google/protobuf/test_util.cc:1265: Failure
Value of: message.GetExtension(unittest::default_sint64_extension )
 Actual: 0
Expected: 406
[  FAILED  ] ExtensionSetTest.Parsing (4 ms)

Same bug, I think.  The lower 30 bits are being zeroed, which completely clobbers these small numbers.

 

There are also some other tests failing, but this is probably because
some dependencies are missing:

libprotobuf FATAL google/protobuf/test_util_lite.cc:950] CHECK failed: (106) == (message.GetExt
/bin/bash: line 4: 26978 Aborted                 ${dir}$tst
FAIL: protobuf-lite-test

This again looks like the same bug, but this time in protobuf-lite-test, whose output is not as nice as protobuf-test as it doesn't use the Google Test framework.

 

Can you confirm that we need 'unzip' and 'jar' to be available during
unittests?  Or why does protobuf-lite-test 'FAIL' hard (instead of

exiting with no error if these are just optional deps?)

protobuf-lite-test does not require these.  The error messages about these are being printed by zip_output_unittest, which is run after protobuf-lite-test.  zip_output_unittest is designed to skip the tests that use unzip and jar if the tools are not available, which is why the build log says it passed despite the errors.  protobuf-lite-test is failing for other reasons.

[Iustin Pop]

Thanks for the hint. Will see what we can do about it (at least check if there
are no known issues with gcc on that platform).

> > [ RUN ] ExtensionSetTest.Parsing
> > google/protobuf/test_util.cc:1127: Failure
> > Value of: message.GetExtension(unittest::optional_sint64_extension )
> > Actual: 0
> > Expected: 106
> > google/protobuf/test_util.cc:1184: Failure
> > Value of: message.GetExtension(unittest::repeated_sint64_extension , 0)
> > Actual: 0
> > Expected: 206
> > google/protobuf/test_util.cc:1212: Failure
> > Value of: message.GetExtension(unittest::repeated_sint64_extension , 1)
> > Actual: 0
> > Expected: 306
> > google/protobuf/test_util.cc:1265: Failure
> > Value of: message.GetExtension(unittest::default_sint64_extension )
> > Actual: 0
> > Expected: 406
> > [ FAILED ] ExtensionSetTest.Parsing (4 ms)
> >
>
> Same bug, I think. The lower 30 bits are being zeroed, which completely
> clobbers these small numbers.

ACK.

> > There are also some other tests failing, but this is probably because
> > some dependencies are missing:
> >
> > libprotobuf FATAL google/protobuf/test_util_lite.cc:950] CHECK failed:
> > (106) == (message.GetExt
> > /bin/bash: line 4: 26978 Aborted ${dir}$tst
> > FAIL: protobuf-lite-test
> >
>
> This again looks like the same bug, but this time in protobuf-lite-test,
> whose output is not as nice as protobuf-test as it doesn't use the Google
> Test framework.

ACK.

>
> > Can you confirm that we need 'unzip' and 'jar' to be available during
> > unittests? Or why does protobuf-lite-test 'FAIL' hard (instead of
> > exiting with no error if these are just optional deps?)
> >
>
> protobuf-lite-test does not require these. The error messages about these
> are being printed by zip_output_unittest, which is run after
> protobuf-lite-test. zip_output_unittest is designed to skip the tests that
> use unzip and jar if the tools are not available, which is why the build log
> says it passed despite the errors. protobuf-lite-test is failing for other
> reasons.

Ah OK, I see, sorry for misreading the logs.

thanks for debugging this!
iustin

[Kenton Varda]

Oops, I used the wrong numbers here.  The interesting numbers are:

>>> print "\n%x != %x\n%x != %x" % tuple((x & 0xffffffffffffffff for x in (-9223372035781033984, -9223372036854775808, 9223372035781033983, 9223372036854775807)))

8000000040000000 != 8000000000000000

7fffffffbfffffff != 7fffffffffffffff

So it looks like what we're actually seeing here is one bit of corruption.

I still strongly suspect a compiler bug.  Probably the compiler's emulation of 64-bit arithmetic is broken in some way.

This reminds me of another bug report we had awhile back, but I don't think they are exactly the same bug (look at message #19 in the thread):

http://groups.google.com/group/protobuf/browse_frm/thread/e2cb856aad6c81f6/b32e3460f10af3aa