Symbol Tables and Backtraces in Linux
TLOlczyk
wrote:
>TLOlczyk <olczy...@yahoo.com> wrote in message news:<v7lqf0h53jmgf4sri...@4ax.com>...
>> I have a shared object for which I want to print (stack) backtraces
>> at certain locations. I've tried backtrace and backtrace_symbols_fd.
>>
Since then I've printed to the file using backtrace_symbols. Same
result.
>> The thing is that I don't quite get the output I want.
>> Many lines are printed simple as:
>> shared_object_file_name(offset)
>>
>> sometimes as
>>
>> shared_object_file_name(function_name+offset)
>>
>> I've compiled with the -g option.
>> The shared object is third party with it's own make system,
>> so I might be missing something in the build. If so, what should
>> I look for?
>
The funny thing is that if I use ddd, put in a sleep(10) and "attach
to process", I can see the lines of source in ddd. So the symbol
information must be in there.
>I recently went through much the same exercise trying to produce a
>stack trace for my executable. I wasn't able to get actual function
>names printed until I started using the linker's -E (--export-dynamic)
>option. To pass it through the compiler to the linker, you use
>"-Wl,-E".
>
Finding the source in the debugger is OK if I use -E or if I don't.
IAnyone give me suggestions?
The reply-to email address is olczy...@yahoo.com.
This is an address I ignore.
To reply via email, remove 2002 and change yahoo to
interaccess,
**
Thaddeus L. Olczyk, PhD
There is a difference between
*thinking* you know something,
and *knowing* you know something.
John Reiser
>>>at certain locations. I've tried backtrace and backtrace_symbols_fd.
>>>
>
> Since then I've printed to the file using backtrace_symbols. Same
> result.
So far, your posts have omitted nearly all the information
that is required for anyone else to help you.
Construct a near-minimal example which demonstrates the problem:
something like a shared.c file with a dozen lines or so, and
a main.c file with a dozen lines or so. Post the complete source,
and a transcript of the compile + link + run of both pieces.
--
Andrei Voropaev
Sorry, I'm not the OP, but I thought I'll put together small testing
package to illustrate what OP is talking about (or I guess he was
talking about :)
========= test.h ===============
#ifndef TEST_H
#define TEST_H
int setup_signal_handler(void);
char * copy_str(char *dst, const char *src);
#endif
======== libtest.c ============
#include <string.h>
#include <execinfo.h>
#include <signal.h>
#include <stdio.h>
#include "test.h"
char *
copy_str(char *dst, const char *src)
{
strcpy(dst, src);
return dst + strlen(src);
}
static void
report_segfault(int signo, siginfo_t * sinf, void * arg)
{
void * btrace[50];
size_t size;
int fd = fileno(stderr);
size = backtrace(btrace, 50);
backtrace_symbols_fd(btrace, size, fd);
}
int
setup_signal_handler(void)
{
struct sigaction act;
act.sa_flags = SA_ONESHOT | SA_SIGINFO;
act.sa_sigaction = report_segfault;
sigfillset(&act.sa_mask);
if(sigaction(SIGSEGV, &act, NULL)) return -1;
act.sa_handler = SIG_IGN;
sigemptyset(&act.sa_mask);
act.sa_flags = SA_RESETHAND|SA_RESTART;
if(sigaction(SIGPIPE, &act, NULL)) return -1;
return 0;
}
=========== test.c =================
#include "test.h"
#include <stdio.h>
int
main(void)
{
char *src = "abc";
char dst[100];
setup_signal_handler();
copy_str(dst, src);
copy_str(NULL, src);
return 0;
}
================ Makefile ===================
CFLAGS=-g -Wall
LDFLAGS=-L.
.PHONY: clean test
all: libtest.so.1 test
libtest.so.1: libtest.o
$(LINK.o) $^ $(LDLIBS) -o $@
ln -s $@ libtest.so
libtest.so.1: LDFLAGS+=-shared -nostartfiles -Wl,-soname,libtest.so.1
test.o: CPPFLAGS+=-fpic
test: test.o
$(LINK.o) $^ $(LDLIBS) -ltest -Wl,-rpath,. -o $@
================= test output ===============
andrei@avorop:~/test/stacktrace$ make
cc -g -Wall -c -o libtest.o libtest.c
cc -L. -shared -nostartfiles -Wl,-soname,libtest.so.1 libtest.o -o libtest.so.1
ln -s libtest.so.1 libtest.so
cc -g -Wall -fpic -c -o test.o test.c
cc -L. test.o -ltest -Wl,-rpath,. -o test
andrei@avorop:~/test/stacktrace$ ./test
./libtest.so.1[0x4001657b]
/lib/libc.so.6[0x4004d280]
./libtest.so.1(copy_str+0x16)[0x40016526]
./test[0x8048534]
/lib/libc.so.6(__libc_start_main+0xbd)[0x4003c17d]
./test(setup_signal_handler+0x35)[0x8048431]
Segmentation fault (core dumped)
John Reiser
> package to illustrate what OP is talking about (or I guess he was
> talking about :)
If I "make" the example and run it under gdb, then I see
-----
$ gdb ./test
(gdb) run
Starting program: ./test
Program received signal SIGSEGV, Segmentation fault.
0x4207a246 in strcpy () from /lib/tls/libc.so.6
(gdb) bt
#0 0x4207a246 in strcpy () from /lib/tls/libc.so.6
#1 0x40017574 in copy_str (dst=0x0, src=0x804852c "abc") at libtest.c:10
#2 0x08048470 in main () at test.c:14
#3 0x42015704 in __libc_start_main () from /lib/tls/libc.so.6
-----
which is the kind of traceback I presume you would like to get
from calling backtrace() or backtrace_symbols_fd(). Instead I see
(please search for the interspersed comments marked with '##'):
-----
(gdb) b report_segfault
Breakpoint 1 at 0x40017593: file libtest.c, line 19.
(gdb) c
Continuing.
Breakpoint 1, report_segfault (signo=11, sinf=0xbfffef38, arg=0xbfffefb8) at libtest.c:19
19 int fd = fileno(stderr);
(gdb) bt
#0 report_segfault (signo=11, sinf=0xbfffd6b8, arg=0xbfffd738) at libtest.c:19
#1 <signal handler called> ## gdb identifies this important info
#2 0x4207a246 in strcpy () from /lib/tls/libc.so.6
#3 0x40017574 in copy_str (dst=0x0, src=0x804852c "abc") at libtest.c:10
#4 0x08048470 in main () at test.c:14
#5 0x42015704 in __libc_start_main () from /lib/tls/libc.so.6
(gdb) x/4x $ebp ## manual backtrace of stack frames; notice the pc values
## in __this__ column
## pc:
0xbfffd6a4: 0xbfffda28 0x420277b0 0x0000000b 0xbfffd6b8
(gdb) x/4x 0xbfffda28
0xbfffda28: 0xbfffda48 0x40017574 0x00000000 0x0804852c
(gdb) x/4x 0xbfffda48
0xbfffda48: 0xbfffdae8 0x08048470 0x00000000 0x0804852c
(gdb) c
(gdb) n
21 size = backtrace(btrace, 50);
(gdb) x/6i $pc ## notice the return address from the "call"
0x400175aa <report_segfault+32>: sub $0x8,%esp
0x400175ad <report_segfault+35>: push $0x32
0x400175af <report_segfault+37>: lea 0xffffff28(%ebp),%eax
0x400175b5 <report_segfault+43>: push %eax
0x400175b6 <report_segfault+44>: call 0x420f1e10 <backtrace>
0x400175bb <report_segfault+49>: add $0x10,%esp
(gdb) c
Continuing.
./libtest.so.1[0x400175bb] ## matches return from call to backtrace()
/lib/tls/libc.so.6[0x420277b0] ## matches first pc in manual backtrace
./libtest.so.1(copy_str+0x14)[0x40017574] ## matches second pc in manual backtrace
./test[0x8048470] ## matches third pc in manual backtrace
/lib/tls/libc.so.6(__libc_start_main+0xe4)[0x42015704]
./test(setup_signal_handler+0x31)[0x8048391]
-----
Now, why the differences? First, gdb identifies the "#1 <signal handler called>"
but the backtrace* routines do not. Detecting a signal handler frame takes
a moderate amount of extra work for each frame: you must inspect the code
pointed to by the return address. backtrace*() does not do this: doing so
is more architecture-dependent than walking back through an ordinary frame.
So you might request an enhancement of backtrace*().
Second, gdb supplies filename and linenumber, while backtrace* does not.
This is because the debug info resides only in the file, and not in the
address space of the process during execution. gdb can afford the extra
effort. backtrace*() has decided to omit the tens or hundreds of kilobytes
of code that is required [namely: dlopen(NULL, ), followed by dl_iterate_phdr()
and use of struct link_map <link.h> to run through all the loaded files,
with open()+mmap()+inspect the debug info of each file.] You can see
that the debug info is not mapped by doing "readelf --headers libtest.so":
-----
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .hash HASH 00000094 000094 0000dc 04 A 2 0 4
[ 2] .dynsym DYNSYM 00000170 000170 000240 10 A 3 14 4
[ 3] .dynstr STRTAB 000003b0 0003b0 0000dd 00 A 0 0 1
[ 4] .gnu.version VERSYM 0000048e 00048e 000048 02 A 2 0 2
[ 5] .gnu.version_r VERNEED 000004d8 0004d8 000030 00 A 3 1 4
[ 6] .rel.dyn REL 00000508 000508 000058 08 A 2 0 4
[ 7] .text PROGBITS 00000560 000560 000137 00 AX 0 0 4
[ 8] .data PROGBITS 00001698 000698 000000 00 WA 0 0 4
[ 9] .dynamic DYNAMIC 00001698 000698 0000a0 08 WA 3 0 4
[10] .got PROGBITS 00001738 000738 00000c 04 WA 0 0 4
[11] .bss NOBITS 00001744 000744 000000 00 WA 0 0 4
[12] .comment PROGBITS 00000000 000744 000033 00 0 0 1
[13] .debug_aranges PROGBITS 00000000 000777 000020 00 0 0 1 ## not mapped
[14] .debug_pubnames PROGBITS 00000000 000797 000038 00 0 0 1 ##
[15] .debug_info PROGBITS 00000000 0007cf 001833 00 0 0 1 ##
[16] .debug_abbrev PROGBITS 00000000 002002 000224 00 0 0 1 ##
[17] .debug_line PROGBITS 00000000 002226 000188 00 0 0 1 ##
[18] .debug_frame PROGBITS 00000000 0023b0 000068 00 0 0 4 ##
[19] .debug_str PROGBITS 00000000 002418 000fb8 01 MS 0 0 1 ##
[20] .shstrtab STRTAB 00000000 0033d0 0000db 00 0 0 1
[21] .symtab SYMTAB 00000000 003844 000290 10 22 19 4
[22] .strtab STRTAB 00000000 003ad4 00012f 00 0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings)
I (info), L (link order), G (group), x (unknown)
O (extra OS processing required) o (OS specific), p (processor specific)
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x000000 0x00000000 0x00000000 0x00697 0x00697 R E 0x1000 ## omits .debug_*
LOAD 0x000698 0x00001698 0x00001698 0x000ac 0x000ac RW 0x1000 ##
DYNAMIC 0x000698 0x00001698 0x00001698 0x000a0 0x000a0 RW 0x4
Section to Segment mapping: ## notice no .debgu_*
Segment Sections...
00 .hash .dynsym .dynstr .gnu.version .gnu.version_r .rel.dyn .text
01 .dynamic .got
02 .dynamic
-----
So, what should one do? Turn to http://groups.google.com , do an Advanced
Groups Search on "just-in-time debugging group:*linux*", and read the post
by Dan Kegel
http://groups.google.com/groups?hl=en&lr=lang_en&ie=UTF-8&selm=linux.kernel.3AEB2B18.F1EC31DE%40kegel.com
-----
void dump_stack(int signum)
{
(void) signum;
char s[160];
// The right command to execute depends on the program. Adjust to taste.
system("echo 'info threads\nbt\nthread 3\nbt\nthread 4\nbt\nthread 5\nbt\n' > gdbcmd");
sprintf(s, "gdb -batch -x gdbcmd %s %d", argv0, (int) getpid());
printf("Crashed! Starting debugger to get stack dump. You may need to kill -9 this process afterwards.\n");
system(s);
exit(1);
}
-----
It seems to me that one could avoid creating a separate file for commands
by using /dev/stdin, and also return to execution by using the gdb commands
'detach' and then 'quit'. (The obvious alternatives are 'kill' then 'quit',
or just 'quit'.) Something like:
-----
sprintf(s, "echo 'bt\ndetach\nquit\n' | gdb -batch -x /dev/stdin %s %d",
argv[0], (int)getpid());
system(s);
-----
HTH,
--
John Reiser
#include <stdlib.h>
#include <unistd.h>
char str[100+4096];
char path[4096];
path[readlink("/proc/self/exe", path, -1+ sizeof(path))] = '\0';
sprintf(str, "echo 'bt\ndetach\nquit\n' | gdb -batch -x /dev/stdin %s %d\n",
path, (int)getpid() );
system(str);
--
Måns Rullgård
> #include <stdio.h>
> #include <stdlib.h>
> #include <unistd.h>
>
> char str[100+4096];
> char path[4096];
> path[readlink("/proc/self/exe", path, -1+ sizeof(path))] = '\0';
Don't ever do that. readlink() can return -1, which will cause you to
write outside the allocated buffer.
--
Måns Rullgård
m...@kth.se
John Reiser
> John Reiser <jre...@BitWagon.com> writes:
>> path[readlink("/proc/self/exe", path, -1+ sizeof(path))] = '\0';
>
>
> Don't ever do that. readlink() can return -1, which will cause you to
> write outside the allocated buffer.
>
Put your code where your mouth is.
Andrei Voropaev
Perfect, thanks.
Andrei