Building n linux (Tumbleweed) - resulting binary crashes

[Lance Corrimal]

Hey all,

so I managed to get the viewer to build successfully on openSUSE Tumbleweed
now, but the resulting binary crashes on start, so early that there aren't any
log files... Where do I even start to look for the cause?


Cheers
LC

[Alex]

Run it under gdb - though if you compiled without debug symbols the
stack trace might not be so helpful.

[Lance Corrimal]

Am Montag, 17. Mai 2021, 07:37:34 CEST schrieb 'Alex' via opensource-dev:
> Run it under gdb - though if you compiled without debug symbols the
> stack trace might not be so helpful.

Here's one very interesting bit:

I'm building to make rpm packages. If I install the package that was built for
opensuse 15.3 on tumbleweed it works just fine - but the binary in the package
specifically built for tumbleweed crashes...

anyway i'm going to see what i can find with gdb.

cheers
LC

[Lance Corrimal]

Am Montag, 17. Mai 2021, 08:14:55 CEST schrieb Lance Corrimal:
> Am Montag, 17. Mai 2021, 07:37:34 CEST schrieb 'Alex' via opensource-dev:
> > Run it under gdb - though if you compiled without debug symbols the
> > stack trace might not be so helpful.
>
> Here's one very interesting bit:
>
> I'm building to make rpm packages. If I install the package that was built
> for opensuse 15.3 on tumbleweed it works just fine - but the binary in the
> package specifically built for tumbleweed crashes...
>
> anyway i'm going to see what i can find with gdb.

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6bbef7e in std::local_Rb_tree_decrement (__x=0x35287b0
<gSavedPerAccountSettings+48>) at ../../../../../libstdc++-v3/src/c++98/
tree.cc:98
98 && __x->_M_parent->_M_parent == __x)


So that tells me that the crash happens while reading the per account settings
file, right?
Well, if I rename the per user directory and force the viewer to create a new
one the same crash happens, with NO new directory created.

and I already tried using a newer GCC than GCC 7, it builds fine tith GCC 9
but the same crash happens.

Cheers
LC

[Alex]

Is there anything else running on the system like selinux?

[Lance Corrimal]

Am Montag, 17. Mai 2021, 10:23:46 CEST schrieb 'Alex' via opensource-dev:
> Is there anything else running on the system like selinux?
>

AppArmor is on in the default settings that openSUSE uses, and in Tumbleweed
there's also selinux active, and it's set to permissive.

I just disabled both completely, and i get the same crash.

Also, wouldn't that affect any viewer binary? I mean, I build one on openSUSE
Leap 15.3 and copy that to my tumbleweed machine, and that one works fine...

Cheers
LC

[Henri Beauchamp]

On Mon, 17 May 2021 09:29:24 +0200, Lance Corrimal wrote:

> Program received signal SIGSEGV, Segmentation fault.
> 0x00007ffff6bbef7e in std::local_Rb_tree_decrement (__x=0x35287b0
> <gSavedPerAccountSettings+48>) at ../../../../../libstdc++-v3/src/c++98/
> tree.cc:98
> 98 && __x->_M_parent->_M_parent == __x)

Type 'bt' at the gdb prompt to get the stack trace...

Henri.

[Lance Corrimal]

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.

0x00007ffff706af7e in std::local_Rb_tree_decrement (__x=0x33a5910) at
../../../../../libstdc++-v3/src/c++98/tree.cc:98

98 && __x->_M_parent->_M_parent == __x)

(gdb) bt
#0 0x00007ffff706af7e in std::local_Rb_tree_decrement (__x=0x33a5910) at
../../../../../libstdc++-v3/src/c++98/tree.cc:98
#1 std::_Rb_tree_decrement(std::_Rb_tree_node_base*) (__x=0x33a5910) at
../../../../../libstdc++-v3/src/c++98/tree.cc:123
#2 0x00000000014c87ea in ()
#3 0x0000000001e6e7ad in ()
#4 0x0000000001470700 in ()
#5 0x0000000000525c65 in ()
#6 0x000000000058fae7 in ()
#7 0x00000000025be66d in ()
#8 0x00007ffff6c95ab1 in __libc_start_main (main=0x53a690, argc=1,
argv=0x7fffffffd8c8, init=
0x25be620, fini=<optimized out>, rtld_fini=<optimized out>,
stack_end=0x7fffffffd8b8) at ../csu/libc-start.c:279
#9 0x000000000059251e in ()


Not very helpful ... I'm sure being on tumbleweed, which as a rolling release
gets majorly updated several times per week, doesn't help either.

How do I tell autobuild to leave the debuginfo untouched again...

Cheers
LC

[Henri Beauchamp]

On Mon, 17 May 2021 15:20:29 +0200, Lance Corrimal wrote:
>
> Not very helpful ...

You need to load the symbol files into gdb, or to build the viewer
with the RelWithDebInfo target.

Henri.

[Lance Corrimal]

even with debug info I don't get that much useful info:

Program received signal SIGSEGV, Segmentation fault.

0x00007ffff6bbef7e in std::local_Rb_tree_decrement (__x=0x3307630) at

../../../../../libstdc++-v3/src/c++98/tree.cc:98
98 && __x->_M_parent->_M_parent == __x)
(gdb) bt

#0 0x00007ffff6bbef7e in std::local_Rb_tree_decrement (__x=0x3307630) at
../../../../../libstdc++-v3/src/c++98/tree.cc:98
#1 std::_Rb_tree_decrement(std::_Rb_tree_node_base*) (__x=0x3307630) at
../../../../../libstdc++-v3/src/c++98/tree.cc:123
#2 0x0000000001444717 in ()
#3 0x00000000019c8846 in ()
#4 0x0000000001186842 in ()
#5 0x0000000000524bf0 in ()
#6 0x000000000058d8d7 in ()
#7 0x000000000252e06d in ()
#8 0x00007ffff67e9ab1 in __libc_start_main (main=0x5487c0, argc=1,
argv=0x7fffffffd788, init=
0x252e020, fini=<optimized out>, rtld_fini=<optimized out>,
stack_end=0x7fffffffd778) at ../csu/libc-start.c:279
#9 0x00000000005903ce in ()



At this point my guess is one of the many prebuilt 3p packages...
..but which one?
and why does a viewer that has been built with the same prebuilts, and the
same gcc version, but on and for openSUSE Leap 15.3, work fine?

*headdesk*.

LC

[Nicky Dasmijn]

First it is best to make sure the settings files the viewer tries to load are all there, you can use something like strace for that.

Or compare with the package that runs on the same machine.

For GDB/symbols there will always be a file with symbols, it is one directory above the packaged directory and should have the name secondlife-bin for SLV

If you are not sure how to load this into GDB for symbols you can just manually copy it over packaged/bin/don-not-run-directory.... and GDB will load the symbols

automatically.

Best regards,

   Nicky 

[Lance Corrimal]

I need a beer. a big one.

So I've built a viewer on a tumbleweed virtual machine, using exactly the same
packages and commands that I use in the spec file for the rpm i'm building -
and guess what, that binary *works*. The one in the RPM does not.

ARGH.
someone shoot me.

[Henri Beauchamp]

On Tue, 18 May 2021 12:21:16 +0200, Lance Corrimal wrote:

> So I've built a viewer on a tumbleweed virtual machine, using exactly
> the same packages and commands that I use in the spec file for the rpm
> i'm building - and guess what, that binary *works*. The one in the RPM
> does not.

I see references to libstdc++-v3/src/c++98/tree.cc in your crash dump,
i.e. the crash happens in libstdc++ itself (the c++98 part is strange
too since the viewer is now C++11 or even C++14, but this could be
"normal" and the result of the libstdc++/gcc includes layout); check
that the RPM macros do not force an other (older) ABI for C++ than the
one that got used to build the pre-built libraries:

grep -r _GLIBCXX_USE_CXX11_ABI /usr/lib/rpm/*
grep -r _GLIBCXX_USE_CXX11_ABI /etc/rpm/*

The ABI change happened with gcc v5, and that _GLIBCXX_USE_CXX11_ABI
define was added to libstdc++ (and used by some distros) to keep the
compatibility with libraries compiled with gcc v4. Not sure how OpenSuse
managed the transition...

Henri.

[Lance Corrimal]

I have "-D_GLIBCXX_USE_CXX11_ABI:BOOL=OFF" in my autobuild configure...
I'm building with gcc 7 right now - the source has no love for gcc 10, and
with gcc 9 the linker crashes with out of memory...

Cheers
LC

[Henri Beauchamp]

On Tue, 18 May 2021 17:25:26 +0200, Lance Corrimal wrote:

> I have "-D_GLIBCXX_USE_CXX11_ABI:BOOL=OFF" in my autobuild configure...

This could perfectly be overridden by the RPM macros... In particular,
check the "optflags".

If you can build a working viewer on a system out of rpmbuild and
the same sources produce a crashing binary with the latter, then
obviously it's something to do with the build parameters passed to
cmake or make by the RPM build system. Those parameters are (normally)
stored in files present in /usr/lib/rpm/, /etc/rpm/, maybe in
/etc/rpmrc, and locally (per-Linux account) in ~/.rpmrc

You could also try to replace the %cmake and/or %make macros in the
rpm *.spec file used for the viewer with cmake or make (so that the
rpm macro flags do not get injected).

Henri.

[Lance Corrimal]

Solved, of sorts. On Tumbleweed I'm now building with GCC9 instead of GCC7
which has the drawback that the build VM needs three times as much memory, but
at least the resulting binary works...

Cheers
LC