Segfault on Vine example
dan
I am running into some problems running Vine. I followed the manual
for the setup, and when I try to run the example, I get this:
dan:/opt/bitblaze/vine-1.0> ./trace_utils/trace_reader -trace examples/
five.trace
42075911: Segmentation fault
dan:/opt/bitblaze/vine-1.0> ./trace_utils/appreplay -trace examples/
five.trace -stp-out five.stp -ir-out five.il -wp-out five.wp
Setting deend_multi to false
Getting trace
Segmentation fault
I might have broken something though, since when I ran make I got the
following error:
/usr/bin/ld: cannot find -lstdc++
So I ran
sudo ln -s /usr/lib/libstdc++.so.6 /usr/lib/libstdc++.so
and then Vine (seemed to) build correctly. I'm running a 32-bit Ubuntu
9.04:
dan:/opt/bitblaze/vine-1.0> uname -a
Linux dan 2.6.28-15-generic #49-Ubuntu SMP Tue Aug 18 18:40:08 UTC
2009 i686 GNU/Linux
Any help would be totally appreciated ^^
cheers,
dan
Stephen McCamant
encountered them:
>>>>> "DR" == dan <reynaud...@gmail.com> writes:
DR> I might have broken something though, since when I ran make I got the
DR> following error:
DR> /usr/bin/ld: cannot find -lstdc++
This sounds to me like an issue between your C++ compiler and
libraries that may be independent of Vine. Do you have a similar
problem if you try to compile a little "Hello, world" program that
uses the C++ standard library, like:
#include <iostream>
using namespace std;
int main(int argc, char **argv) {
cout << "Hello, world!" << endl;
}
On the system we used for testing the Vine release, the only C++
compiler and libraries installed are g++-4.3 and libstdc++6-4.3.
Perhaps your system has either an earlier or later version of g++
installed without the corresponding library? You can see what you have
installed with:
% env COLUMNS=100 dpkg -l 'libstdc++*'
% env COLUMNS=100 dpkg -l 'g++*'
DR> So I ran
DR> sudo ln -s /usr/lib/libstdc++.so.6 /usr/lib/libstdc++.so
This strikes me as dangerous. I don't think you need an unversioned
symlink for libstdc++, since it's closely tied in with the compiler,
and the compiler should know which version it works with. For
instance, our test machine has only /usr/lib/libstdc++.so.6 ->
libstdc++.so.6.0.10, with no "libstdc++.so". It seems like this could
have caused the problems later on if it induces g++ to link with a
library version that's incompatible.
DR> I am running into some problems running Vine. I followed the manual
DR> for the setup, and when I try to run the example, I get this:
DR> dan:/opt/bitblaze/vine-1.0> ./trace_utils/trace_reader -trace examples/
DR> five.trace
DR> 42075911: Segmentation fault
DR> dan:/opt/bitblaze/vine-1.0> ./trace_utils/appreplay -trace examples/
DR> five.trace -stp-out five.stp -ir-out five.il -wp-out five.wp
DR> Setting deend_multi to false
DR> Getting trace
DR> Segmentation fault
The points about the C++ compiler setup are what I would try
investigating first, but if those don't help I can make some general
suggestions about debugging Vine segfaults:
* Make sure you don't have a too-small stack size limit. This is
unlikely to be the problem on these smaller examples, but it can
cause segfaults on larger examples; there's some more detail in the
Troubleshooting section of the manual.
* Try using the ".dbg" version of the programs with the same
options. They run slower, but can give better error messages. Though
this tends to be more important for failures in OCaml code, while
this sounds like it might more likely be a problem in the C++ code,
since there was no OCaml error message in the output you quoted.
* You can get a stack backtrace from a segmentation fault by running
the programs under gdb. Put "gdb --args" on the front of your
program command line, as in
% gdb --args ./trace_utils/trace_reader -trace examples/five.trace
then say "run" at the (gdb) prompt, and "backtrace" after it crashes.
Hope this helps,
-- Stephen
dan
> This sounds to me like an issue between your C++ compiler and
> libraries that may be independent of Vine. Do you have a similar
> problem if you try to compile a little "Hello, world" program that
> uses the C++ standard library
dan:~/test> g++ hello.cpp
dan:~/test> ./a.out
Hello, world!
> On the system we used for testing the Vine release, the only C++
> compiler and libraries installed are g++-4.3 and libstdc++6-4.3.
> Perhaps your system has either an earlier or later version of g++
> installed without the corresponding library? You can see what you have
> installed with:
>
> % env COLUMNS=100 dpkg -l 'libstdc++*'
> % env COLUMNS=100 dpkg -l 'g++*'
dan:~/test> env COLUMNS=100 dpkg -l 'libstdc++*'
[snip]
ii libstdc++6 4.3.3-5ubuntu4 The GNU Standard C++
Library v3
[snip]
ii libstdc++6-4.3-dev 4.3.3-5ubuntu4 The GNU Standard C++
Library v3 (development files)
dan:~/test> env COLUMNS=100 dpkg -l 'g++*'
[snip]
ii g++ 4:4.3.3-1ubuntu1 The GNU C++ compiler
[snip]
ii g++-4.3 4.3.3-5ubuntu4 The GNU C++ compiler
> * Try using the ".dbg" version of the programs with the same
> options. They run slower, but can give better error messages. Though
> this tends to be more important for failures in OCaml code, while
> this sounds like it might more likely be a problem in the C++ code,
> since there was no OCaml error message in the output you quoted.
the same
> * You can get a stack backtrace from a segmentation fault by running
> the programs under gdb. Put "gdb --args" on the front of your
> program command line, as in
>
> % gdb --args ./trace_utils/trace_reader -trace examples/five.trace
>
> then say "run" at the (gdb) prompt, and "backtrace" after it crashes.
dan:/opt/bitblaze/vine-1.0> gdb --args ./trace_utils/trace_reader -
trace examples/five.trace
This GDB was configured as "i486-linux-gnu"...
(gdb) run
Starting program: /opt/bitblaze/vine-1.0/trace_utils/trace_reader -
trace examples/five.trace
42075911:
Program received signal SIGSEGV, Segmentation fault.
0xb7a07896 in memcpy () from /lib/tls/i686/cmov/libc.so.6
(gdb) bt
#0 0xb7a07896 in memcpy () from /lib/tls/i686/cmov/libc.so.6
#1 0x42075911 in ?? ()
#2 0x080e7d8a in print_disasm_rawbytes (arch=bfd_arch_i386,
addr=1107777809, buf=0x18f97508 "\017�", size=15)
at ./ir/translate.cpp:196
#3 0x080acde2 in camlidl_libasmir_print_disasm_rawbytes (_v_arch=19,
_v_addr=418944444, _v_buf=418940552)
at libasmir_stubs.c:2094
#4 0x0807330d in camlAsmir__fun_870 ()
#5 0x080550c8 in camlTrace_reader__print_inst_344 ()
#6 0x18f88688 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb)
The same with the .dbg version:
dan:/opt/bitblaze/vine-1.0> gdb --args ./trace_utils/trace_reader.dbg -
trace examples/five.trace
This GDB was configured as "i486-linux-gnu"...
(gdb) run
Starting program: /opt/bitblaze/vine-1.0/trace_utils/trace_reader.dbg -
trace examples/five.trace
[Thread debugging using libthread_db enabled]
42075911: [New Thread 0xb78fb6d0 (LWP 14061)]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb78fb6d0 (LWP 14061)]
0x0806f6d1 in init_disasm_info (abfd=0x19c66810,
disasm_info=0x19c6677c) at ./disasm/asm_program.cpp:422
422 disasm_info->flavour = bfd_get_flavour (abfd);
Current language: auto; currently c++
(gdb) bt
#0 0x0806f6d1 in init_disasm_info (abfd=0x19c66810,
disasm_info=0x19c6677c) at ./disasm/asm_program.cpp:422
#1 0x0806f77c in init_disasm_info (prog=0x19c66728) at ./disasm/
asm_program.cpp:443
#2 0x080a9f2d in fake_prog_for_arch (arch=bfd_arch_i386) at ./ir/
translate.cpp:38
#3 0x08096792 in byte_insn_to_asmp (arch=bfd_arch_i386,
addr=1107777809, bb_bytes=0x19c3e588 "\017�", len=15)
at ./vex/irtoir_c.cpp:80
#4 0x080a9c9a in print_disasm_rawbytes (arch=bfd_arch_i386,
addr=1107777809, buf=0x19c3e588 "\017�", size=15)
at ./ir/translate.cpp:196
#5 0x0806ecf6 in camlidl_libasmir_print_disasm_rawbytes (_v_arch=19,
_v_addr=432344968, _v_buf=432340168)
at libasmir_stubs.c:2094
#6 0x081a018d in caml_interprete ()
#7 0x0818ffed in caml_main ()
#8 0x081900ac in main ()
And with appreplay:
dan:/opt/bitblaze/vine-1.0> gdb --args ./trace_utils/appreplay -trace
examples/five.trace -stp-out five.stp -ir-out five.il -wp-out five.wp
This GDB was configured as "i486-linux-gnu"...
(gdb) run
Starting program: /opt/bitblaze/vine-1.0/trace_utils/appreplay -trace
examples/five.trace -stp-out five.stp -ir-out five.il -wp-out five.wp
Getting trace
0xb7ba3896 in memcpy () from /lib/tls/i686/cmov/libc.so.6
(gdb) bt
#0 0xb7ba3896 in memcpy () from /lib/tls/i686/cmov/libc.so.6
#1 0x42075911 in ?? ()
#2 0x08130594 in camlidl_libasmir_byte_insn_to_asmp (_v_arch=19,
_v_addr=440961732, _v_bb_bytes=440957840)
at libasmir_stubs.c:2125
#3 0x080905d7 in camlAsmir__asm_bytes_to_vine_673 ()
#4 0x00000013 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb) q
The program is running. Exit anyway? (y or n) y
dan:/opt/bitblaze/vine-1.0> gdb --args ./trace_utils/appreplay.dbg -
trace examples/five.trace -stp-out five.stp -ir-out five.il -wp-out
five.wp
This GDB was configured as "i486-linux-gnu"...
(gdb) run
Starting program: /opt/bitblaze/vine-1.0/trace_utils/appreplay.dbg -
trace examples/five.trace -stp-out five.stp -ir-out five.il -wp-out
five.wp
[Thread debugging using libthread_db enabled]
Getting trace
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7a1a6d0 (LWP 28534)]
0x080a49e1 in init_disasm_info (abfd=0x18f1d848,
disasm_info=0x18f1d7b4) at ./disasm/asm_program.cpp:422
422 disasm_info->flavour = bfd_get_flavour (abfd);
Current language: auto; currently c++
(gdb) bt
#0 0x080a49e1 in init_disasm_info (abfd=0x18f1d848,
disasm_info=0x18f1d7b4) at ./disasm/asm_program.cpp:422
#1 0x080a4a8c in init_disasm_info (prog=0x18f1d760) at ./disasm/
asm_program.cpp:443
#2 0x080df23d in fake_prog_for_arch (arch=bfd_arch_i386) at ./ir/
translate.cpp:38
#3 0x080cbaa2 in byte_insn_to_asmp (arch=bfd_arch_i386,
addr=1107777809, bb_bytes=0x18f1d738 "\017�", len=15)
at ./vex/irtoir_c.cpp:80
#4 0x080a40ec in camlidl_libasmir_byte_insn_to_asmp (_v_arch=19,
_v_addr=418459320, _v_bb_bytes=418454520)
at libasmir_stubs.c:2125
#5 0x08234e5d in caml_interprete ()
#6 0x08224cbd in caml_main ()
#7 0x08224d7c in main ()
(gdb)
cheers,
dan
Stephen McCamant
DR> Hi Stephen, thanks for your support!
SMcC> This sounds to me like an issue between your C++ compiler and
SMcC> libraries that may be independent of Vine. Do you have a similar
SMcC> problem if you try to compile a little "Hello, world" program
SMcC> that uses the C++ standard library
DR> nope, it works fine
DR> dan:~/test> g++ hello.cpp
DR> dan:~/test> ./a.out
DR> Hello, world!
Oops, I should have asked more specifically, is that only with the
addition of the /usr/lib/libstdc++.so symlink, or does it also work
fine without it? (As I'd mentioned in the last message, if that
symlink is necessary to get programs to compile, I think that
represents a problem in your g++ setup.)
SMcC> On the system we used for testing the Vine release, the only C++
SMcC> compiler and libraries installed are g++-4.3 and libstdc++6-4.3.
SMcC> Perhaps your system has either an earlier or later version of g++
SMcC> installed without the corresponding library? You can see what you have
SMcC> installed with:
SMcC>
SMcC> % env COLUMNS=100 dpkg -l 'libstdc++*'
SMcC> % env COLUMNS=100 dpkg -l 'g++*'
DR> here's what I get:
DR> ii libstdc++6 4.3.3-5ubuntu4 The GNU Standard C++
DR> ii libstdc++6-4.3-dev 4.3.3-5ubuntu4 The GNU Standard C++
DR> ii g++ 4:4.3.3-1ubuntu1 The GNU C++ compiler
DR> ii g++-4.3 4.3.3-5ubuntu4 The GNU C++ compiler
Hmm. That exactly matches our test system. Could you also check that
that's the version of g++ in your path, e.g. there isn't another one
in /usr/local or your home directory, and "g++ --version" prints "g++
(Ubuntu 4.3.3-5ubuntu4) 4.3.3"?
The "cannot find -lstdc++" error you mentioned earlier still sounds to
me like it might be our best lead. Can you show me at what point in
the build process it occurs?
SMcC> * You can get a stack backtrace from a segmentation fault by running
DR> dan:/opt/bitblaze/vine-1.0> gdb --args ./trace_utils/trace_reader -
DR> trace examples/five.trace
DR> 0xb7a07896 in memcpy () from /lib/tls/i686/cmov/libc.so.6
DR> (gdb) bt
DR> #0 0xb7a07896 in memcpy () from /lib/tls/i686/cmov/libc.so.6
DR> #1 0x42075911 in ?? ()
DR> #2 0x080e7d8a in print_disasm_rawbytes (arch=bfd_arch_i386,
DR> addr=1107777809, buf=0x18f97508 "\017\357\277\275", size=15)
DR> at ./ir/translate.cpp:196
DR> #3 0x080acde2 in camlidl_libasmir_print_disasm_rawbytes (_v_arch=19,
DR> _v_addr=418944444, _v_buf=418940552)
DR> at libasmir_stubs.c:2094
DR> #4 0x0807330d in camlAsmir__fun_870 ()
DR> #5 0x080550c8 in camlTrace_reader__print_inst_344 ()
DR> The same with the .dbg version:
DR> dan:/opt/bitblaze/vine-1.0> gdb --args ./trace_utils/trace_reader.dbg -
DR> trace examples/five.trace
DR> 0x0806f6d1 in init_disasm_info (abfd=0x19c66810,
DR> disasm_info=0x19c6677c) at ./disasm/asm_program.cpp:422
DR> 422 disasm_info->flavour = bfd_get_flavour (abfd);
DR> Current language: auto; currently c++
DR> (gdb) bt
DR> #0 0x0806f6d1 in init_disasm_info (abfd=0x19c66810,
DR> disasm_info=0x19c6677c) at ./disasm/asm_program.cpp:422
DR> #1 0x0806f77c in init_disasm_info (prog=0x19c66728) at ./disasm/
DR> asm_program.cpp:443
DR> #2 0x080a9f2d in fake_prog_for_arch (arch=bfd_arch_i386) at ./ir/
DR> translate.cpp:38
DR> #3 0x08096792 in byte_insn_to_asmp (arch=bfd_arch_i386,
DR> addr=1107777809, bb_bytes=0x19c3e588 "\017\357\277\275", len=15)
DR> at ./vex/irtoir_c.cpp:80
DR> #4 0x080a9c9a in print_disasm_rawbytes (arch=bfd_arch_i386,
DR> addr=1107777809, buf=0x19c3e588 "\017\357\277\275", size=15)
DR> at ./ir/translate.cpp:196
DR> #5 0x0806ecf6 in camlidl_libasmir_print_disasm_rawbytes (_v_arch=19,
DR> _v_addr=432344968, _v_buf=432340168)
DR> at libasmir_stubs.c:2094
DR> #6 0x081a018d in caml_interprete ()
DR> #7 0x0818ffed in caml_main ()
DR> #8 0x081900ac in main ()
[similar backtraces with appreplay]
The regular and .dbg versions aren't quite the same backtrace, but
they do seem to be crashing at around the same place, which looks like
the first time the program invokes the disassembler from the GNU
Binutils. Could you check that the version of binutils and
binutils-dev you have installed is 2.19.1-0ubunutu3? Another
experiment you could try to see if this is the cause is to add the
"-eip" option to the trace_reader command line, which will cause it to
skip the disassembly (and so, if this is the cause, not crash).
The instruction addresses I see in your backtrace look different from
the ones I get when I run the same program in on our test virtual
machine, which also seems to confirm that there's something different
about your build environment. Perhaps we can try exchanging binaries
(since if everything is supposed to be the same, they should be
compatible).
You can download the version that works on our test system from
http://www.cs.berkeley.edu/~smcc/unadv/trace_reader.gz
By the way, here's what "ldd" on it says on that system (the addresses
will vary from run to run because of ASLR):
linux-gate.so.1 => (0xb7fcc000)
libopcodes-2.19.1.so => /usr/lib/libopcodes-2.19.1.so (0xb7f34000)
libbfd-2.19.1.so => /usr/lib/libbfd-2.19.1.so (0xb7e48000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7d58000)
libgdome.so.0 => /usr/lib/libgdome.so.0 (0xb7d20000)
libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0xb7c68000)
libxml2.so.2 => /usr/lib/libxml2.so.2 (0xb7b2c000)
libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7b06000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7b02000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7af2000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb798f000)
libz.so.1 => /lib/libz.so.1 (0xb7979000)
/lib/ld-linux.so.2 (0xb7fcd000)
libpcre.so.3 => /lib/libpcre.so.3 (0xb7947000)
dan
> Oops, I should have asked more specifically, is that only with the
> addition of the /usr/lib/libstdc++.so symlink, or does it also work
> fine without it? (As I'd mentioned in the last message, if that
> symlink is necessary to get programs to compile, I think that
> represents a problem in your g++ setup.)
compiles and runs without it
> Hmm. That exactly matches our test system. Could you also check that
> that's the version of g++ in your path, e.g. there isn't another one
> in /usr/local or your home directory, and "g++ --version" prints "g++
> (Ubuntu 4.3.3-5ubuntu4) 4.3.3"?
/usr/bin/g++
dan:~/test> g++ --version
g++ (Ubuntu 4.3.3-5ubuntu4) 4.3.3
> The "cannot find -lstdc++" error you mentioned earlier still sounds to
> me like it might be our best lead. Can you show me at what point in
> the build process it occurs?
dan:/opt/bitblaze/vine-1.0> make
Making all in VEX
make[1]: entrant dans le répertoire « /opt/bitblaze/vine-1.0/VEX »
make[1]: Rien à faire pour « all ».
make[1]: quittant le répertoire « /opt/bitblaze/vine-1.0/VEX »
Making all in libasmir
make[1]: entrant dans le répertoire « /opt/bitblaze/vine-1.0/libasmir
»
make all-recursive
make[2]: entrant dans le répertoire « /opt/bitblaze/vine-1.0/libasmir
»
Making all in src
make[3]: entrant dans le répertoire « /opt/bitblaze/vine-1.0/libasmir/
src »
make[3]: Rien à faire pour « all ».
make[3]: quittant le répertoire « /opt/bitblaze/vine-1.0/libasmir/src
»
Making all in apps
make[3]: entrant dans le répertoire « /opt/bitblaze/vine-1.0/libasmir/
apps »
make[3]: Rien à faire pour « all ».
make[3]: quittant le répertoire « /opt/bitblaze/vine-1.0/libasmir/apps
»
make[3]: entrant dans le répertoire « /opt/bitblaze/vine-1.0/libasmir
»
make[3]: Rien à faire pour « all-am ».
make[3]: quittant le répertoire « /opt/bitblaze/vine-1.0/libasmir »
make[2]: quittant le répertoire « /opt/bitblaze/vine-1.0/libasmir »
make[1]: quittant le répertoire « /opt/bitblaze/vine-1.0/libasmir »
Making all in stp/ocaml
make[1]: entrant dans le répertoire « /opt/bitblaze/vine-1.0/stp/ocaml
»
make[2]: entrant dans le répertoire « /opt/bitblaze/vine-1.0/stp/ocaml
»
make[2]: « libstpvc_stubs.a » est à jour.
ocamlmklib -L.. \
-o stpvc_stubs libstp_stubs.o libstp_regerrorhandler.o -lstp -lstdc
++ \
/usr/bin/ld: cannot find -lstdc++
collect2: ld returned 1 exit status
make[2]: *** [dllstpvc_stubs.so] Erreur 2
make[2]: quittant le répertoire « /opt/bitblaze/vine-1.0/stp/ocaml »
make[1]: *** [debug-code-library] Erreur 2
make[1]: quittant le répertoire « /opt/bitblaze/vine-1.0/stp/ocaml »
make: *** [all-recursive] Erreur 1
> The regular and .dbg versions aren't quite the same backtrace, but
> they do seem to be crashing at around the same place, which looks like
> the first time the program invokes the disassembler from the GNU
> Binutils. Could you check that the version of binutils and
> binutils-dev you have installed is 2.19.1-0ubunutu3?
ii binutils 2.19.1-0ubuntu3 The GNU assembler, linker
and binary utilities
ii binutils-dev 2.19.1-0ubuntu3 The GNU binary utilities
(BFD development files)
ii binutils-static 2.19.1-0ubuntu3 statically linked binutils
tools
> Another
> experiment you could try to see if this is the cause is to add the
> "-eip" option to the trace_reader command line, which will cause it to
> skip the disassembly (and so, if this is the cause, not crash).
$ ./trace_utils/trace_reader -eip -trace examples/five.trace | tail
4000b04d
4000b04e
4000b1b0
4000b1b1
4000b1b2
4000b1b5
420b4d60
420b4d62
420b4d66
> Perhaps we can try exchanging binaries
> (since if everything is supposed to be the same, they should be
> compatible).
dan:/opt/bitblaze/vine-1.0> ./trace_utils/trace_reader.stephen -eip -
trace examples/five.trace | tail
4000b04d
4000b04e
4000b1b0
4000b1b1
4000b1b2
4000b1b5
420b4d60
420b4d62
420b4d66
dan:/opt/bitblaze/vine-1.0> ./trace_utils/trace_reader.stephen -trace
examples/five.trace | tail
4000b04c: pop %edi R@edi[0x420b4d60][4] T0 M@0xbffff74c[0x00000000]
[4] T0
4000b04d: pop %ebp R@ebp[0xbffff750][4] T0 M@0xbffff750[0xbffff798]
[4] T0
4000b04e: ret M@0xbffff754[0x4000b1b0][4] T0
4000b1b0: pop %edx R@edx[0x421308b4][4] T0 M@0xbffff758[0x00000000]
[4] T0
4000b1b1: pop %ecx R@ecx[0x00000000][4] T0 M@0xbffff75c[0x00000000]
[4] T0
4000b1b2: xchg %eax,(%esp) R@eax[0x420b4d60][4] T0 M@0xbffff760
[0x00000000][4] T0
4000b1b5: ret $0x8 I@0x00000000[0x00000008][2] T0 M@0xbffff760
[0x420b4d60][4] T0
420b4d60: mov %ebx,%edx R@ebx[0x4213030c][4] T0 R@edx[0x00000000]
[4] T0
420b4d62: mov 0x4(%esp),%ebx R@ebx[0x4213030c][4] T0 M@0xbffff770
[0x00000000][4] T0
420b4d66: mov $0x1,%eax I@0x00000000[0x00000001][4] T0 R@eax
[0x00000000][4] T0
> By the way, here's what "ldd" on it says on that system (the addresses
> will vary from run to run because of ASLR):
>
> linux-gate.so.1 => (0xb7fcc000)
> libopcodes-2.19.1.so => /usr/lib/libopcodes-2.19.1.so (0xb7f34000)
> libbfd-2.19.1.so => /usr/lib/libbfd-2.19.1.so (0xb7e48000)
> libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7d58000)
> libgdome.so.0 => /usr/lib/libgdome.so.0 (0xb7d20000)
> libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0xb7c68000)
> libxml2.so.2 => /usr/lib/libxml2.so.2 (0xb7b2c000)
> libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7b06000)
> libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7b02000)
> libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7af2000)
> libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb798f000)
> libz.so.1 => /lib/libz.so.1 (0xb7979000)
> /lib/ld-linux.so.2 (0xb7fcd000)
> libpcre.so.3 => /lib/libpcre.so.3 (0xb7947000)
difference:
dan:/opt/bitblaze/vine-1.0> ldd trace_utils/trace_reader
linux-gate.so.1 => (0xb7ef2000)
libbfd-2.19.1.so => /usr/lib/libbfd-2.19.1.so (0xb7d67000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7c77000)
libxml2.so.2 => /usr/lib/libxml2.so.2 (0xb7a4b000)
libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7a25000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7a21000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb78ae000)
libz.so.1 => /lib/libz.so.1 (0xb7898000)
/lib/ld-linux.so.2 (0xb7ef3000)
libpcre.so.3 => /lib/libpcre.so.3 (0xb7866000)
Stephen McCamant
DR> I removed the symlink before compiling the example, so the hello
DR> world compiles and runs without it
SMcC> Hmm. That exactly matches our test system. Could you also check
SMcC> that that's the version of g++ in your path, e.g. there isn't
SMcC> another one in /usr/local or your home directory, and "g++
SMcC> --version" prints "g++ (Ubuntu 4.3.3-5ubuntu4) 4.3.3"?
DR> dan:~/test> which g++
DR> /usr/bin/g++
DR> dan:~/test> g++ --version
DR> g++ (Ubuntu 4.3.3-5ubuntu4) 4.3.3
Okay. Based on some further investigation below, I think the program
that's actually relevant to OCaml compilation is "gcc" rather than
"g++", so could you repeat those version checks for it too? (They
should all be the same.) And can you compile the Hello World program
by using "gcc" and "-lstdc++"?
I think relevant symlink for finding libstdc++ should be
/usr/lib/gcc/i486-linux-gnu/4.3.3/libstdc++.so
On our test system, it's a symlink to ../../../libstdc++.so.6, so it
should have the same effect as the one you added manually. (But
putting it in this directory ensures that a different version of GCC
won't use it.)
SMcC> The "cannot find -lstdc++" error you mentioned earlier still
SMcC> sounds to me like it might be our best lead. Can you show me at
SMcC> what point in the build process it occurs?
[in vine-1.0/stp/ocaml:]
DR> ocamlmklib -L..-o stpvc_stubs libstp_stubs.o libstp_regerrorhandler.o -lstp -lstdc++
Ocamlmklib is a wrapper around "gcc", which is in turn a wrapper
around "ld" here, so you can get an idea of what's going on by running
ocamlmklib under "strace -f -eexecve" to see the "gcc" command line
it's using, and then running gcc with "-v" to see its ld (really,
collect2) command line.
On our system, the ocamlmklib translates fairly straightforwardly into
a gcc invocation:
gcc -v -shared -o ./dllstpvc_stubs.so libstp_regerrorhandler.o libstp_stubs.o -L.. -lstp -lstdc++
SMcC> Perhaps we can try exchanging binaries (since if everything is
SMcC> supposed to be the same, they should be compatible).
DR> It works with your binary:
Okay, so we've narrowed it down that something about the build process
on your machine is causing you to build binaries that crash when they
try to access the Binutils library. My theory about the "-lstdc++"
linking failure is still that you're somehow using an incompatible
version of gcc. Another way of checking whether something like that
might be going on is too look at the version identifier strings that
GCC leaves in each object file it compiles. You can do this with a
command like:
% strings -a trace_utils/trace_reader | fgrep GCC: | sort | uniq -c
On our 9.04 test system, I see:
1 GCC: (Ubuntu 4.3.0-4ubuntu1) 4.3.1 20080509 (prerelease)
128 GCC: (Ubuntu 4.3.1-1ubuntu2) 4.3.1
51 GCC: (Ubuntu 4.3.3-5ubuntu4) 4.3.3
If your results are different at all, that represents an unexpected
difference in your build environment. And if any of the versions you
see are 4.2.x, 4.4.x, or something further afield, that could be a
cause of breakage.
It's not obvious how that would cause a Binutils crash, though it
seems well within the realm of possibility. If you want to get me a
copy of your binary (put it up on the web somewhere, email it to me
privately, etc.) I could try debugging it directly.
-- Stephen
dan
> Okay, so we've narrowed it down that something about the build process
> on your machine is causing you to build binaries that crash when they
> try to access the Binutils library. My theory about the "-lstdc++"
> linking failure is still that you're somehow using an incompatible
> version of gcc. Another way of checking whether something like that
> might be going on is too look at the version identifier strings that
> GCC leaves in each object file it compiles. You can do this with a
> command like:
>
> % strings -a trace_utils/trace_reader | fgrep GCC: | sort | uniq -c
>
> On our 9.04 test system, I see:
>
> 1 GCC: (Ubuntu 4.3.0-4ubuntu1) 4.3.1 20080509 (prerelease)
> 128 GCC: (Ubuntu 4.3.1-1ubuntu2) 4.3.1
> 51 GCC: (Ubuntu 4.3.3-5ubuntu4) 4.3.3
>
> If your results are different at all, that represents an unexpected
> difference in your build environment. And if any of the versions you
> see are 4.2.x, 4.4.x, or something further afield, that could be a
> cause of breakage.
yielded version 4.2.4. Indeed, /usr/local/bin/gcc was a symlink to /
usr/bin/gcc-4.2, I probably did that manually a year ago when I wanted
to install VMware.
So now the symlink points to /usr/bin/gcc-4.3, I removed the libstdc+
+.so symlink, and a clean build works (I no longer have the -lstdc++
error).
After that clean build, the "strings -a trace_utils/trace_reader |
fgrep GCC: | sort | uniq -c" yields exactly your output.
Unfortunately, when I try to run the example I still get the segfault
at the same place.
Back to the hello world, with gcc-4.3 this now works:
dan:~/test> gcc -lstdc++ hello.cpp
Hello, world!
cheers,
dan
Stephen McCamant
DR> Indeed, /usr/local/bin/gcc was a symlink to /usr/bin/gcc-4.2, I
DR> probably did that manually a year ago when I wanted to install
DR> VMware.
DR> Unfortunately, when I try to run the example I still get the
DR> segfault at the same place.
DR> I'm sending you my binary by mail.
From looking at the binary you sent, it seems like you might have a
similar problem of out-of-sync package versions with your Binutils
libraries. Your binary was compiled with the wrong type for the second
argument of bfd_alloc, which is a typedef bfd_size_type: your binary
thinks that type is "unsigned long" (32 bit), whereas the Ubunutu
version of the library thinks it's "unsigned long long" (64 bit). Vine
is trying to allocate 15 bytes, so in the binary you compiled it
pushes a 32-bit 0x0000000f on the stack, but the system's library is
expecting a 64-bit value, so it reads another 32 bits and gets a value
like 0xf79db4760000000f. But then it decides that's way too big to
allocate so it sets an error and returns a null pointer; Vine fails to
check for this error and tries to memcpy() into the null pointer.
This is probably caused by the compilation of the BFD-using code
getting the wrong version of the "bfd.h" header file. To track this
down, you can look at the compilation (for instance) of
libasmir/src/vex/irtoir_c.cpp by going to the vine-1.0/libasmir/src
directory and giving the command
g++ -E -DHAVE_CONFIG_H -I. -I../../libasmir -I../../VEX/pub \
-I./include vex/irtoir_c.cpp
which will pre-process that file as it would for the build and send
the results to standard output. On our clean test machine, the
"#include <bfd.h>" on line 12 resolves to /usr/include/bfd.h in the
binutils-dev package (2.19.1-0ubuntu3, the same version you said you
had), whose MD5 checksum starts with "909d32ea". The relevant parts
are "#define BFD_HOST_U_64_BIT unsigned long long" on line 88 and
"typedef BFD_HOST_U_64_BIT bfd_size_type" on line 135.
dan
> "#include <bfd.h>" on line 12 resolves to /usr/include/bfd.h in the
> binutils-dev package (2.19.1-0ubuntu3, the same version you said you
> had), whose MD5 checksum starts with "909d32ea". The relevant parts
> are "#define BFD_HOST_U_64_BIT unsigned long long" on line 88 and
> "typedef BFD_HOST_U_64_BIT bfd_size_type" on line 135.
I had a different version of bfd.h in /usr/local/include that was used
during the compilation. Removing it solved the problem.
Thanks *a lot* for your help :)
dan