cross compile blackfin: problems linking

Rob

unread,

Oct 21, 2009, 1:42:43 PM10/21/09

to Protocol Buffers

Hi,

This is my build script to cross compile protobuf 2.2.0:
================
#!/bin/sh

FLAGS="--disable-shared --disable-crypto-auth --without-gnutls --
without-ssl --without-zlib \
--without-libssh2 --disable-ipv6 --disable-manual --disable-telnet
--disable-tftp \
--disable-ldap --disable-file --disable-ftp"

cd $1
make distclean
mkdir host
cd host
make distclean
../configure $FLAGS --prefix=`pwd`/install && make -j4 && make install
cd ..
./configure --with-protoc=host/src/protoc --host=bfin-uclinux $FLAGS --
prefix=`pwd`/bfin CPPFLAGS="-DUCLINUX" LDFLAGS="-Wl,-elf2flt" &&
make -j4 && make install
================

When I go to link something against it I get all kinds of errors like
"undefined reference to `_pthread_once'"

The configure script output regarding pthreads looks like this:
t@godbox:~/work/blackfin/trunk/lib/protobuf$ ./configure --with-
protoc=host/src/protoc --host=bfin-linux-uclibc --disable-shared --
disable-crypto-auth --without-gnutls --without-ssl --without-zlib
--without-libssh2 --disable-ipv6 --disable-manual --disable-telnet --
disable-tftp --disable-ldap --disable-file --disable-ftp --
prefix=`pwd`/bfin CPPFLAGS="-DUCLINUX" LDFLAGS="-Wl,-elf2flt" 2>&1 |
grep -i pthread
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to check for GCC pthread/shared inconsistencies...
yes
checking whether -pthread is sufficient with -shared... yes
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to check for GCC pthread/shared inconsistencies...
yes
checking whether -pthread is sufficient with -shared... yes

I'm somewhat confused what the issue is - I'm assuming the configure
script isn't working quit right with the bfin toolchain? Here is the
full configure results:
http://pastie.org/private/prkft9dz5v2t6harphyw

Anything else I can do to help figure out what's going on?

Thanks!

Kenton Varda

unread,

Oct 21, 2009, 2:13:18 PM10/21/09

to Rob, Protocol Buffers

When you compile a program against libprotobuf, you need to use pkg-config to find out what additional cflags and libs are necessary. Example:

g++ -c my_prog.cc `pkg-config --cflags protobuf`

g++ -o my_prog my_prog.o `pkg-config --libs protobuf`

Your specific problem is probably that you are not linking against libpthread.

Rob

unread,

Oct 21, 2009, 3:29:15 PM10/21/09

to Protocol Buffers

Hmm... not sure if my last msg got eaten or not, but the issue was
specifying '-pthread' instead of '-lpthread'. Not sure why the
protobuf requires this when nothing else does... but regardless,
thanks for the pointer to pkg-config... it helped solve the problem
for me! :)

Thanks again!

-Rob

Kenton Varda

unread,

Oct 21, 2009, 3:32:50 PM10/21/09

to Rob, Protocol Buffers

It seems there's some ambiguity as to whether -pthread implies -lpthread. Are you able to compile and run the tests?

Kenton Varda

unread,

Oct 21, 2009, 8:07:43 PM10/21/09

to Rob, Protocol Buffers

From your output, it appears that the tests are, in fact, crashing mid-run (hence "Abort"). You may want to run in a debugger and find out what is happening.

But what I was expecting was that you wouldn't be able to compile the test in the first place due to the -pthread/-lpthread issue. I'm pretty sure the makefiles pass the same linker args as pkg-config --libs reports when compiling the tests.

On Wed, Oct 21, 2009 at 4:51 PM, Rob <r...@rmdashr.com> wrote:

Hi Kenton,

I've crosscompiled the tests and ran them on the target board, here are the results... everything appears to have passed! I'm assuming some of those tests verify the threading (pthreads)? Is there anything else I should run?

-Rob

root:~> ./protobuf-lite-test
PASS


And...
root:~> ./protobuf-lazy-descriptor-test
Running main() from gtest_main.cc
[==========] Running 34 tests from 3 test cases.
[----------] Global test environment set-up.
[----------] 30 tests from GeneratedMessageTest
[ RUN      ] GeneratedMessageTest.Defaults
[       OK ] GeneratedMessageTest.Defaults (1 ms)
[ RUN      ] GeneratedMessageTest.FloatingPointDefaults
[       OK ] GeneratedMessageTest.FloatingPointDefaults (0 ms)
[ RUN      ] GeneratedMessageTest.Accessors
[       OK ] GeneratedMessageTest.Accessors (1 ms)
[ RUN      ] GeneratedMessageTest.MutableStringDefault
[       OK ] GeneratedMessageTest.MutableStringDefault (1 ms)
[ RUN      ] GeneratedMessageTest.Clear
[       OK ] GeneratedMessageTest.Clear (0 ms)
[ RUN      ] GeneratedMessageTest.EmbeddedNullsInBytesCharStar
[       OK ] GeneratedMessageTest.EmbeddedNullsInBytesCharStar (0 ms)
[ RUN      ] GeneratedMessageTest.ClearOneField
[       OK ] GeneratedMessageTest.ClearOneField (1 ms)
[ RUN      ] GeneratedMessageTest.StringCharStarLength
[       OK ] GeneratedMessageTest.StringCharStarLength (0 ms)
[ RUN      ] GeneratedMessageTest.CopyFrom
[       OK ] GeneratedMessageTest.CopyFrom (1 ms)
[ RUN      ] GeneratedMessageTest.SwapWithEmpty
[       OK ] GeneratedMessageTest.SwapWithEmpty (1 ms)
[ RUN      ] GeneratedMessageTest.SwapWithSelf
[       OK ] GeneratedMessageTest.SwapWithSelf (1 ms)
[ RUN      ] GeneratedMessageTest.SwapWithOther
[       OK ] GeneratedMessageTest.SwapWithOther (1 ms)
[ RUN      ] GeneratedMessageTest.CopyConstructor
[       OK ] GeneratedMessageTest.CopyConstructor (1 ms)
[ RUN      ] GeneratedMessageTest.CopyAssignmentOperator
[       OK ] GeneratedMessageTest.CopyAssignmentOperator (1 ms)
[ RUN      ] GeneratedMessageTest.UpcastCopyFrom
[       OK ] GeneratedMessageTest.UpcastCopyFrom (1 ms)
[ RUN      ] GeneratedMessageTest.NonEmptyMergeFrom
[       OK ] GeneratedMessageTest.NonEmptyMergeFrom (1 ms)
[ RUN      ] GeneratedMessageTest.MergeFromSelf

[WARNING] src/gtest-death-test.cc:743: Death tests use fork(), which is unsafe .
Abort

And Finally...
root:~> ./protobuf-test
Running main() from gtest_main.cc
[==========] Running 684 tests from 94 test cases.
[----------] Global test environment set-up.
[----------] 1 test from GeneratedDescriptorTest
[ RUN      ] GeneratedDescriptorTest.IdenticalDescriptors
[       OK ] GeneratedDescriptorTest.IdenticalDescriptors (258 ms)
[----------] 1 test from GeneratedDescriptorTest (258 ms total)

[----------] 34 tests from GeneratedMessageTest
[ RUN      ] GeneratedMessageTest.Defaults
[       OK ] GeneratedMessageTest.Defaults (1 ms)
[ RUN      ] GeneratedMessageTest.FloatingPointDefaults
[       OK ] GeneratedMessageTest.FloatingPointDefaults (0 ms)
[ RUN      ] GeneratedMessageTest.Accessors
[       OK ] GeneratedMessageTest.Accessors (1 ms)
[ RUN      ] GeneratedMessageTest.MutableStringDefault
[       OK ] GeneratedMessageTest.MutableStringDefault (0 ms)
[ RUN      ] GeneratedMessageTest.Clear
[       OK ] GeneratedMessageTest.Clear (1 ms)
[ RUN      ] GeneratedMessageTest.EmbeddedNullsInBytesCharStar
[       OK ] GeneratedMessageTest.EmbeddedNullsInBytesCharStar (0 ms)
[ RUN      ] GeneratedMessageTest.ClearOneField
[       OK ] GeneratedMessageTest.ClearOneField (1 ms)
[ RUN      ] GeneratedMessageTest.StringCharStarLength
[       OK ] GeneratedMessageTest.StringCharStarLength (0 ms)
[ RUN      ] GeneratedMessageTest.CopyFrom
[       OK ] GeneratedMessageTest.CopyFrom (1 ms)
[ RUN      ] GeneratedMessageTest.SwapWithEmpty
[       OK ] GeneratedMessageTest.SwapWithEmpty (2 ms)
[ RUN      ] GeneratedMessageTest.SwapWithSelf
[       OK ] GeneratedMessageTest.SwapWithSelf (1 ms)
[ RUN      ] GeneratedMessageTest.SwapWithOther
[       OK ] GeneratedMessageTest.SwapWithOther (1 ms)
[ RUN      ] GeneratedMessageTest.CopyConstructor
[       OK ] GeneratedMessageTest.CopyConstructor (1 ms)
[ RUN      ] GeneratedMessageTest.CopyAssignmentOperator
[       OK ] GeneratedMessageTest.CopyAssignmentOperator (1 ms)
[ RUN      ] GeneratedMessageTest.UpcastCopyFrom
[       OK ] GeneratedMessageTest.UpcastCopyFrom (0 ms)
[ RUN      ] GeneratedMessageTest.DynamicMessageCopyFrom
[       OK ] GeneratedMessageTest.DynamicMessageCopyFrom (2 ms)
[ RUN      ] GeneratedMessageTest.NonEmptyMergeFrom
[       OK ] GeneratedMessageTest.NonEmptyMergeFrom (1 ms)
[ RUN      ] GeneratedMessageTest.MergeFromSelf

[WARNING] src/gtest-death-test.cc:743: Death tests use fork(), which is unsafe particularly in a threaded context. For thi.
Abort

Rob

unread,

Oct 21, 2009, 7:51:52 PM10/21/09

to Kenton Varda, Protocol Buffers

Rob

unread,

Oct 21, 2009, 8:10:47 PM10/21/09

to Protocol Buffers

I just wanted to clarify to anyone who runs into this - I had to use '-
pthread' because '-lpthread' did not work (linker flags) for my target
(blackfin).

Rob

unread,

Oct 21, 2009, 8:19:23 PM10/21/09

to Kenton Varda, Protocol Buffers

Oh, I didn't even see the 'Abort'. It must be related to the uClinux's lack of 'fork()'. Maybe use 'vfork()' as a drop in replacement?

http://docs.blackfin.uclinux.org/doku.php?id=living_without_forks

Would you recommend sticking to the 'lite' interface? That test actually passed!

Kenton Varda

unread,

Oct 21, 2009, 8:27:43 PM10/21/09

to Rob, Protocol Buffers

The lite test does not actually test all the features of the lite runtime. It only tests that the lite runtime works stand-alone. The full tests verify other features (many of which are shared between the two implementations). So you want to run the full tests.

The problem you seem to have is actually in gtest, not in protobuf. Gtest has a feature called "death tests" which lets you verify that some code crashes. It is implemented using fork(). vfork() would not work since the fork() is not followed by exec().

The solution is to edit the gtest headers such that they do not #define GTEST_HAS_DEATH_TEST on your platform -- then all death tests will be omitted. You can find the gtest source code in the "gtest" subdirectory of the protobuf package. You might want to submit a patch back to them, which will then be picked up in future protobuf versions.

Back on the original topic, I think I might have misunderstood you before. You're saying that -lpthread *didn't* work and -pthread *did*? Then that explains why the tests compile fine. I thought you were saying th opposite.

Rob

unread,

Oct 22, 2009, 2:01:31 PM10/22/09

to Kenton Varda, Protocol Buffers

Hi Kenton,

-pthread : did work
-lpthread : did not work - although I don't understand why

Thanks for the pointer to GTEST_HAS_DEATH_TEST - I was able to run the full tests and sadly they all didn't pass. Here are the results - you think it's safe to use on the blackfin? The other tests passed (./protobuf-lazy-descriptor-test and ./protobuf-lite-test)

Full results are here:
http://pastie.org/665409

[----------] Global test environment tear-down
[==========] 677 tests from 94 test cases ran. (30793 ms total)
[ PASSED ] 664 tests.
[ FAILED ] 13 tests, listed below:
[ FAILED ] ParseDecriptorDebugTest.TestAllDescriptorTypes
[ FAILED ] TextFormatTest.PrintExotic
[ FAILED ] TextFormatTest.PrintFloatPrecision
[ FAILED ] TextFormatTest.ParseExotic
[ FAILED ] DynamicMessageTest.Descriptor
[ FAILED ] DynamicMessageTest.OnePrototype
[ FAILED ] DynamicMessageTest.Defaults
[ FAILED ] DynamicMessageTest.IndependentOffsets
[ FAILED ] DynamicMessageTest.Extensions
[ FAILED ] DynamicMessageTest.PackedFields
[ FAILED ] DynamicMessageTest.SpaceUsed
[ FAILED ] MiscTest.DefaultValues
[ FAILED ] DatabaseBackedPoolTest.UnittestProto

Some Actual errors:

[ RUN      ] ParseDecriptorDebugTest.TestAllDescriptorTypes
google/protobuf/compiler/parser_unittest.cc:1215: Failure
Value of: error_collector_.text_
Actual: "102:54: Expected integer.
370:64: Expected integer.

[ RUN      ] TextFormatTest.PrintExotic
google/protobuf/text_format_unittest.cc:423: Failure
Value of: RemoveRedundantZeros(message.DebugString())
Actual: "repeated_int64: -42949672960

Kenton Varda

unread,

Oct 22, 2009, 4:55:16 PM10/22/09

to Rob, Protocol Buffers

The TextFormat failures suggest that either your printf() or your strtod() implementation is poor, but can probably be ignored if you don't care about TextFormat printing floating points nicely.

The "couldn't parse default value" errors suggest that your system is having problems printing negative 64-bit numbers -- it appears the error only happens in cases where a negative 64-bit number has been printed and then re-parsed. This is odd, though, since protocol buffers does not rely on the system libraries for this -- it has its own printing code.

This does look like a serious problem for you, though.

Could you change these lines in descriptor.cc:

if (proto.default_value().empty() || *end_pos != '\0') {

AddError(result->full_name(), proto,

DescriptorPool::ErrorCollector::DEFAULT_VALUE,

"Couldn't parse default value.");

}

To this:

if (proto.default_value().empty() || *end_pos != '\0') {

AddError(result->full_name(), proto,

DescriptorPool::ErrorCollector::DEFAULT_VALUE,

"Couldn't parse default value for field: " + proto.DebugString());

}

Then re-run the tests and send the new results? This might shed a little more light on the subject.

Rob

unread,

Oct 22, 2009, 5:21:32 PM10/22/09

to Kenton Varda, Protocol Buffers

Updated output here:
http://pastie.org/665809

I've also attached the results.

Let me know if there is anything else you'd like me to try.

Thanks for looking into this for me! I hope we can resolve the issue...

-Rob

protobuf.results.txt

Kenton Varda

unread,

Oct 22, 2009, 5:29:10 PM10/22/09

to Rob, Protocol Buffers

It looks like in this log you included only the stdout, not stderr, so the error messages I was looking for are not there.

Rob

unread,

Oct 22, 2009, 6:06:36 PM10/22/09

to Kenton Varda, Protocol Buffers

My apologies...

How's this: http://pastie.org/665912

Kenton Varda

unread,

Oct 22, 2009, 6:29:12 PM10/22/09

to Rob, Protocol Buffers

Neat. On your system, "-50" is apparently being written as "-(*".

The code doing the writing should be FastInt64ToBuffer() in strutil.cc. Apparently it only breaks on negative numbers. Perhaps you can debug? The code looks right to me, and works on every other platform. It looks to me like your platform or compiler may have some scary arithmetic bugs.

Rob

unread,

Oct 22, 2009, 7:46:03 PM10/22/09

to Kenton Varda, Protocol Buffers

Hi Kenton,

I was just looking through the anomaly list for our chip (bf548) and found this (and yes, I currently have rev 0.1):

76. 05000448 - DDR Clock Duty Cycle Spec Violation (tCH, tCL):
   DESCRIPTION:
   DDR clock duty cycle specifications tCH and tCL are not met. This can result in DDR read/write failures.
   WORKAROUND:
   None
   APPLIES TO REVISION(S):
   0.1

So, I'm going to assume your code is correct and a hardware bug is being triggered. I will retest when we get new hardware in. Prior to this, I have not experienced any weird memory issues (that I know of). Thanks for looking into this (and the -pthread thing) - I appreciate it.

-Rob

Kenton Varda

unread,

Oct 22, 2009, 7:57:07 PM10/22/09

to Rob, Protocol Buffers

I don't see how that issue could be causing this problem. "DDR clock duty cycles" being off would presumably cause random memory corruption, not a reproducible arithmetic error in a very specific piece of code. You should probably copy the FastInt64ToBuffer() function into a simple example program that demonstrates the problem and file a bug report against either the compiler or the chip. If your hardware uses an unusual instruction set then I would actually suspect the compiler first.

In any case, strutil.cc is not part of the "lite" runtime, so if you stick to that you should be OK.

Rob

unread,

Oct 22, 2009, 10:22:44 PM10/22/09

to Kenton Varda, Protocol Buffers

When I execute this code:
http://pastie.org/666166

I get the correct result:

root:~> ./bug
str: -50

I didn't really look, but how do you have int64 defined? I did this 'typedef long long int64;'

So, when isolated, the bug doesn't present itself.... very strange. What do you think?

Kenton Varda

unread,

Oct 22, 2009, 11:16:22 PM10/22/09

to Rob, Protocol Buffers

I dunno.

If you want to reproduce the problem in context, all you need to do is declare a message like:

message Foo {

optional sfixed64 foo = 1 [default = -50];

}

Then use this C++ code:

google::protobuf::DescriptorProto proto;

Foo::descriptor()->CopyTo(&proto);

cout << proto.DebugString() << endl;

And you should see the bogus default value in the printed text.

I guess I can't really debug it myself since I don't have access to your platform, so it's up to you.

Rob

unread,

Oct 23, 2009, 8:16:03 PM10/23/09

to Kenton Varda, Protocol Buffers

Hi Kenton,

I was able to reproduce the problem with the code you provided below. I have also hooked up a remote debugger and stepped through the code.

Apparently the line causing the problem is in FastInt64ToBuffer() as you said before. It's specifically: i = -i (see diff below);

i=-40 before that statement
i=-176093659096 after the statement

I changed that line to to: i = abs(i); (this works because the number is always going to be negative here)

and now all the tests regarding DynamicMessageTest pass... very very odd.

I have no clue why this is broken on my target. :/ I tried to get FastInt64ToBuffer() to fail without linking in protobuf - but it only fails when used in context.

Here is the latest output from ./protobuf-test:
http://pastie.org/667359

Here is the diff:
--- src/google/protobuf/stubs/strutil.cc    (revision 26)
+++ src/google/protobuf/stubs/strutil.cc    (working copy)
@@ -569,7 +569,9 @@
     } else {
       // Make sure we aren't at MIN_INT, in which case we can't say i = -i
       i = i + 10;
-      i = -i;
+      //i = -i;
+      i = abs((long int)i);
       *p-- = '0' + i % 10;
       // Undo what we did a moment ago
       i = i / 10 + 1;

On Thu, Oct 22, 2009 at 8:16 PM, Kenton Varda <ken...@google.com> wrote:

I dunno.

If you want to reproduce the problem in context, all you need to do is declare a message like:

message Foo {
optional sfixed64 foo = 1 [default = -50];

}

Then use this C++ code:

google::protobuf::DescriptorProto proto;
Foo::descriptor()->CopyTo(&proto);
cout << proto.DebugString() << endl;

And you should see the bogus default value in the printed text.

I guess I can't really debug it myself since I don't have access to your platform, so it's up to you.

On Thu, Oct 22, 2009 at 7:22 PM, Rob <r...@rmdashr.com> wrote:

When I execute this code:

http://pastie.org/66616to the target and can step through the code using DDD.6

Kenton Varda

unread,

Oct 23, 2009, 8:33:19 PM10/23/09

to Rob, Protocol Buffers

On Fri, Oct 23, 2009 at 5:16 PM, Rob <r...@rmdashr.com> wrote:

Hi Kenton,

I was able to reproduce the problem with the code you provided below. I have also hooked up a remote debugger and stepped through the code.

Apparently the line causing the problem is in FastInt64ToBuffer() as you said before. It's specifically: i = -i (see diff below);

i=-40 before that statement
i=-176093659096 after the statement

-40 = -0x28 = 0xffffffffffffffd8 (two's complement)

-176093659096 = -0x28ffffffd8 = 0xffffffd700000028 (two's complement)

Note also:

~0xffffffd7 = 0x00000028

My guess is that the compiler is implementing 64-bit integers in terms of two 32-bit integers, since I'm guessing your platform is 32-bit. It looks like somewhere in the process it is mixing up its registers, and somehow the upper 32 bits get polluted with data from the lower 32 bits. Anyway, you should definitely file a bug report against the compiler.

Reply all

Reply to author

Forward

cross compile blackfin: problems linking - pthreads

Rob

Kenton Varda

Rob

Kenton Varda

Kenton Varda

Rob

Rob

Rob

Kenton Varda

Rob

Kenton Varda

Rob

Kenton Varda

Rob

Kenton Varda

Rob

Kenton Varda

Rob

Kenton Varda

Rob

Kenton Varda