Debugging core in different environment

sunil

unread,

Mar 23, 2007, 3:19:59 AM3/23/07

to sunil.k...@wipro.com

Hi,
I am having certain queries in debugging the core in different
environment.
Suppose the core is generated in the production environment.
Is it feasible to debug that core in development environment
If we are able to debug the core in the dev environment what and
all the precaution measures we have to take.
what and all the constraints in the environment?
Like where the shared libraries should have to be present ?
and is there any environmental issues?
Is any primary things we have to do before debugging the core?

If anybody knows about these please help me out at the same time if
anybody debugged in the different environment what and all the
problems you are faced ,reply back to me as soon as possible.

Regards
Sunil

ajay kumar

unread,

Mar 28, 2007, 10:02:59 AM3/28/07

to

Of course, you can debug the core generated in the field, in your own
development/testing environment. In general the executable which
dumped the core would be an optimized/stripped version which could
make debugging a bit tough so the simplest way is to get the steps to
recreate the problem and use your own debug executable to recreate the
problem and then debug it. But if you do not know the steps to
recreate the problem then only choice you have is to debug the core in-
house. As you are trying to debug the core, I assume you have the
sources and also the exact binary which led to the core and you can
setup the environment (libraries, etc) which is required by this
binary. Essentially, debugging the core from field is no different
than debugging a core generated while developing/testing the program.

- Ajay

Gunvant Patil

unread,

Apr 4, 2007, 9:27:09 AM4/4/07

to

If you are able to reproduce core in your developement enviorment (You
are damn lucky!!!)
Most of the times its really hard to reproduce core for complex server
side applications.

As binaries in field are stripped you need to analyze that core file
it with your
non stripped binary if you get your stack trace intact (again start
feeling lucky :))
otherwise you need to follow following tricks and need to build stack
trace manually

Get the register information for the process
(gdb) info reg
eax 0x37393100 926494976
ecx 0x1 1
edx 0x90b3d08 151731464
ebx 0xa0ff748 168818504
esp 0xbfe4eb40 0xbfe4eb40
ebp 0xbfe4eb58 0xbfe4eb58
esi 0xbfe53af4 -1075496204
edi 0xbfe53a80 -1075496320
eip 0x81260a0 0x81260a0
eflags 0x10206 66054
cs 0x73 115
ss 0x7b 123
ds 0xc02d007b -1070792581
es 0x7b 123
fs 0x0 0
gs 0x33 51

here following regs we are interested in

%esp : top of the stack
%ebp : current stack frame
%eip : Instruction pointer

Using this stack dump we know that the stack frame address in the %ebp
register is the stack frame
for the current instruction in the %eip register. Using the %ebp and
%eip registers as a starting point
we can build the first line in our backtrace rebuild

StackFrame | Instruction
Pointer | Pointer
------------------------
0xbfe4eb58 | 0x81260a0

When program control branches to a new function a stack frame is
stacked and the callee function's
last address is stored at the new frame's address %ebp + 4.
In order to get the caller's stack frame and instruction pointer just
look at %ebp and %ebp+4:

(gdb) info reg ebp
ebp 0xbfe4eb58 0xbfe4eb58
(gdb) x/12h *0xbfe4eb58
ebp-> 0xbfe4f738: 0xf7a8 0xbfe4 0x5a4f 0x080d 0x2bf8 0x09e6
0x0000 0x0000
0xbfe4f748: 0x8658 0x094a 0x8527 0x092c
(gdb) x/12h *(0xbfe4f738 + 4)
eip-> 0x80d5a4f : 0x4589 0xebec 0x8b6f 0xd045 0xc883
0x8940 0x2444 0xc714
0x80d5a5f : 0x2444 0x0110 0x0000 0x8b00
(gdb) x/12h *0xbfe4f738
ebp-> 0xbfe4f7a8: 0xf7d8 0xbfe4 0xd25f 0x0805 0x82a0 0x092c
0xe799 0x0822
0xbfe4f7b8: 0xf7d8 0xbfe4 0x54a2 0x0017
(gdb) x/12h *(0xbfe4f7a8 + 4)
eip-> 0x805d25f : 0x4588 0xe9ff 0x02e4 0x0000 0x458b 0x8908
0x2404 0x21e8
0x805d26f : 0x0789 0x8800 0xff45 0xd1e9

Keep doing this until we have a full back trace.
You'll know you've reached the bottom of the stack when the previous
stack frame pointer is 0x00000000

Manually contructed stack trace will look like

Now get a non stripped version of your binary and run objdump on it
objdump -tD binary > textfile

lookup this disassembly (textfile) for above EIP's and you will get
your stack trace with function calls and go hunting root cause of
core.

-Cheers,
Gunvant
~~~~~~~~~
No trees were killed in the sending of this message. However a large
number of electrons were terribly inconvenienced.