Issues using KGDB on a Linux machine to debug NetBSD

[Israel Jacquez]

Hello,

I'll make this short. I can't seem to get debugging support working
even when following the guide:
<http://www.netbsd.org/docs/kernel/kgdb.html>.

Target: NetBSD 5.1.2 on the i386 port
Remote: Debian GNU/Linux
Kernel on target: NetBSD-CURRENT

I have enabled the following options in the kernel config file:

options DDB
options DDB_HISTORY_SIZE=512
options KGDB
options "KGDB_DEVNAME=\"com\"",KGDB_DEVADDR=0x3f8,KGDB_DEVRATE=115200
makeoptions DEBUG="-g"

After compiling the kernel on the remote machine (Debian GNU/Linux), I
copy the new kernel to / on the target machine and I see:

When I invoke dmesg(8): dmesg | grep -E '^com', I get the following:
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com0: console
com0: kgdb

I then reboot and at the boot loader, I invoke:
boot -d

Immediately, I get dropped into DDB. I can only see this through the
serial console as the target machine is running headless. After, I
invoke i486--netbsd-gdb that (should) read .gdbinit:
file sys/arch/i386/compile/obj/GENERIC-DEBUG/netbsd.gdb
set remote interrupt-sequence Ctrl-C
set remotebaud 115200
target remote /dev/ttyUSB0

After in GDB, all I see is:
warning: unrecognized item "qSupported:xm" in "qSupported" response
Ignoring packet error, continuing...
Ignoring packet error, continuing...
..

Any help? Tips?

[Paul_...@dell.com]

On Jun 8, 2012, at 4:35 PM, Israel Jacquez wrote:

> Hello,
>
> I'll make this short. I can't seem to get debugging support working
> even when following the guide:
> <http://www.netbsd.org/docs/kernel/kgdb.html>.

> ...

> Immediately, I get dropped into DDB. I can only see this through the
> serial console as the target machine is running headless. After, I
> invoke i486--netbsd-gdb that (should) read .gdbinit:
> file sys/arch/i386/compile/obj/GENERIC-DEBUG/netbsd.gdb
> set remote interrupt-sequence Ctrl-C
> set remotebaud 115200
> target remote /dev/ttyUSB0
>
> After in GDB, all I see is:
> warning: unrecognized item "qSupported:xm" in "qSupported" response
> Ignoring packet error, continuing...
> Ignoring packet error, continuing...

> ...
>
> Any help? Tips?

To see exactly what's going on, use the gdb command "set debug remote 1" before the "target remote" command. That will show the dialog between gdb and the system being debugged.

It looks like the system is sendng something that your gdb doesn't understand. Perhaps it's too old; what version are you using?

paul

[Israel Jacquez]

The version that was built (i486--netbsd-gdb) is GNU gdb (GDB) 7.3.1.

I'll try again. Here is the output. (/boot.cfg is set to automatically
"boot netbsd -d"):

GNU gdb (GDB) 7.3.1
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-unknown-linux-gnu
--target=i486--netbsdelf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Sending packet: :xmlRegisters=i386;qRelocInsn+#25...Sending packet:
:xmlRegisters=i386;qRelocInsn+#25...putpkt: Junk:
>> NetBSD/x86 BIOS Boot, Revision 5.2 (from NetBSD 5.1.2)
>> Memory: 638/1037184 k

1. Boot normally
2. Boot single user
3. Disable ACPI
4. Disable ACPI and SMP
5. Drop to boot prompt

Choose an option; RETURN for default; SPACE to stop countdown.
Option 1 will be chosen in 0 seconds.
booting hd0a:netbsd (howto 0x40)
/
Nak
Sending packet: :xmlRegisters=i386;qRelocInsn+#25...putpkt: Junk:/
Nak
Sending packet: :xmlRegisters=i386;qRelocInsn+#25...putpkt: Junk:9820128/
Nak
Timeout in mid-packet, retrying
Timed out.
Timed out.
Ignoring packet error, continuing...
Packet qSupported (supported-packets) is supported
warning: unrecognized item "ebp
db{0}> e" in "qSupported" response
Sending packet: #df...Nak
Sending packet: #df...Ack
Packet received: Hg0
Sending packet: 0#3f...Packet instead of Ack, ignoring it
Ack
Packet received: ?
Sending packet: -1#09...Ack
Packet received: Hc-1
Sending packet: #b4...Ack
Packet received: qC
Sending packet: #8f...Ack
Packet received: qAttached
Packet qAttached (query-attached) is supported
Sending packet: #4b...Ack
Packet received: qOffsets
gdbinit:5: Error in sourced command file:
Malformed response to offset query, qOffsets
(gdb)

>        paul
>

[Greg Troxel]

For what it's worth, some of my colleagues are successfully using the
gdb from netbsd-5 (and I'm 99% sure netbsd-6) with the corresponding
systems (all i386) on serial ports.

Your output reports make me wonder if gdb changed the protocol. I'd
look at the gdb 7 sources to see if there is an old-mode switch.

[Israel Jacquez]

On Fri, Jun 8, 2012 at 2:10 PM, Greg Troxel <g...@ir.bbn.com> wrote:
>
> For what it's worth, some of my colleagues are successfully using the
> gdb from netbsd-5 (and I'm 99% sure netbsd-6) with the corresponding
> systems (all i386) on serial ports.

I'll try to build an older version of GDB, possibly from NetBSD-5.0 as well.

> Your output reports make me wonder if gdb changed the protocol.  I'd
> look at the gdb 7 sources to see if there is an old-mode switch.
>

Hopefully I won't have to, but I'll look into it.

[Israel Jacquez]

On Fri, Jun 8, 2012 at 2:08 PM, <Paul_...@dell.com> wrote:

>>>> Memory: 638/1037184 k ...
>
> It looks like you connected while the boot output was still coming out.  Gdb doesn't seem to like that very much at all.  I have a modified gdb that treats anything unrecognized as "console output" but without that change, you're going to have problems if your target mixes program output with gdb protocol data.

Maybe I don't understand the "work flow". Should I then wait until I
get to the login prompt in order to fire up GDB? What if it crashes
halfway through? The problem is that it's headless so I don't know
what's going on. I guess I could use the serial console until it
crashes and then fire up GDB.

Many thanks for your help.

>        paul
>
>
>

[Paul_...@dell.com]

On Jun 8, 2012, at 5:14 PM, Israel Jacquez wrote:

> ...

> Maybe I don't understand the "work flow". Should I then wait until I
> get to the login prompt in order to fire up GDB? What if it crashes
> halfway through? The problem is that it's headless so I don't know
> what's going on. I guess I could use the serial console until it
> crashes and then fire up GDB.
>
> Many thanks for your help.

That's what I normally do.

paul

[matthew green]

> options DDB

try removing this one.


mrg.

[Eduardo Horvath]

On Fri, 8 Jun 2012, Israel Jacquez wrote:

> Hello,
>
> I'll make this short. I can't seem to get debugging support working
> even when following the guide:
> <http://www.netbsd.org/docs/kernel/kgdb.html>.
>
> Target: NetBSD 5.1.2 on the i386 port
> Remote: Debian GNU/Linux
> Kernel on target: NetBSD-CURRENT
>
> I have enabled the following options in the kernel config file:
>
> options DDB
> options DDB_HISTORY_SIZE=512
> options KGDB
> options "KGDB_DEVNAME=\"com\"",KGDB_DEVADDR=0x3f8,KGDB_DEVRATE=115200
> makeoptions DEBUG="-g"
>
> After compiling the kernel on the remote machine (Debian GNU/Linux), I
> copy the new kernel to / on the target machine and I see:
>
> When I invoke dmesg(8): dmesg | grep -E '^com', I get the following:
> com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
> com0: console
> com0: kgdb
>
> I then reboot and at the boot loader, I invoke:
> boot -d
>
> Immediately, I get dropped into DDB. I can only see this through the
> serial console as the target machine is running headless. After, I

If you have both DDB and KGDB enabled you need to give DDB a specific
command to drop into KGDB. Only then can you connect the remote gdb.

Eduardo

[Paul_...@dell.com]

On Jun 8, 2012, at 4:53 PM, Israel Jacquez wrote:

>>> Memory: 638/1037184 k ...

It looks like you connected while the boot output was still coming out. Gdb doesn't seem to like that very much at all. I have a modified gdb that treats anything unrecognized as "console output" but without that change, you're going to have problems if your target mixes program output with gdb protocol data.

paul

[Israel Jacquez]

On Fri, Jun 8, 2012 at 2:54 PM, matthew green <m...@eterna.com.au> wrote:
>
>> options DDB
>
> try removing this one.

What I found odd is that if I actually do remove it, I don't get the
output: "com0: kgdb". But I'll recompile the kernel anyway.

>
> .mrg.

[Israel Jacquez]

Hi Eduardo,

Would that command simply happen to be "kgdb"?

> Eduardo

[Eduardo Horvath]

On Fri, 8 Jun 2012, Israel Jacquez wrote:

> Hi Eduardo,
>
> Would that command simply happen to be "kgdb"?

Sounds familliar. It's been a while since I've done this.

Eduardo

[Reinoud Zandijk]

hi folks,

sorry i'm so late :)

On Fri, Jun 08, 2012 at 01:35:44PM -0700, Israel Jacquez wrote:
> When I invoke dmesg(8): dmesg | grep -E '^com', I get the following:
> com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
> com0: console
> com0: kgdb

and thats the issue: you are multiplexing the com0 port. For a reliable
testbed you need a second com port for the kgdb, say com1. I've used this
setup quite extensively and it works like a charm, even though 115200 baud is
still slowish for kgdb.

An relatively new alternative is to use NetBSD/usermode (i386 and amd64 only
for now). You can then attach a gdb to the usermode process just like kgdb.
Of couse it doesn't give you debugging for PCI stuff yet etc. but for more
generic kernel work, i'd recommend it.

With regards,
Reinoud