Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

netisr panic?

3 views
Skip to first unread message

Ian FREISLICH

unread,
Nov 17, 2012, 10:07:54 AM11/17/12
to
Hi

I have this consistently with:

FreeBSD firewall2.jnb1.gp-online.net 10.0-CURRENT FreeBSD 10.0-CURRENT #30 r243156: Fri Nov 16 20:12:33 SAST 2012 ia...@firewall2.jnb1.gp-online.net:/usr/obj/usr/src/sys/FIREWALL amd64


Fatal trap 12: page fault while in kernel mode
cpuid = 4; apic id = 04
fault virtual address = 0xc
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff8050f534
stack pointer = 0x28:0xffffff846384e9c0
frame pointer = 0x28:0xffffff846384ea00
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 11 (irq266: igb1:que 0)
trap number = 12
panic: page fault
cpuid = 4
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
panic() at panic+0x1ce
trap_fatal() at trap_fatal+0x290
trap_pfault() at trap_pfault+0x21f
trap() at trap+0x2b4
calltrap() at calltrap+0x8
--- trap 0xc, rip = 0xffffffff8050f534, rsp = 0xffffff846384e9c0, rbp = 0xffffff846384ea00 ---
ether_nh_input() at ether_nh_input+0x94
netisr_dispatch_src() at netisr_dispatch_src+0x212
igb_rxeof() at igb_rxeof+0x3f0
igb_msix_que() at igb_msix_que+0xfa
intr_event_execute_handlers() at intr_event_execute_handlers+0xfd
ithread_loop() at ithread_loop+0x9e
fork_exit() at fork_exit+0x11e
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff846384ecb0, rbp = 0 ---
Uptime: 2h2m15s
Dumping 1241 out of 16368 MB:..2%..11%..21%..31%..42%..51%..61%..71%..82%..91%

#0 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:266
266 if (textdump && textdump_pending) {
(kgdb) #0 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:266
#1 0xffffffff8044af04 in kern_reboot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:449
#2 0xffffffff8044b487 in panic (fmt=0x1 <Address 0x1 out of bounds>)
at /usr/src/sys/kern/kern_shutdown.c:637
#3 0xffffffff80605bd0 in trap_fatal (frame=0xc, eva=<value optimized out>)
at /usr/src/sys/amd64/amd64/trap.c:872
#4 0xffffffff80605f3f in trap_pfault (frame=0xffffff846384e910, usermode=0)
at /usr/src/sys/amd64/amd64/trap.c:789
#5 0xffffffff806062f4 in trap (frame=0xffffff846384e910)
at /usr/src/sys/amd64/amd64/trap.c:463
#6 0xffffffff805eff6f in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:228
#7 0xffffffff8050f534 in ether_nh_input (m=0xfffffe012521e700)
at /usr/src/sys/net/if_ethersubr.c:484
#8 0xffffffff8051a602 in netisr_dispatch_src (proto=9,
source=<value optimized out>, m=<value optimized out>)
at /usr/src/sys/net/netisr.c:1013
#9 0xffffffff803188b0 in igb_rxeof (que=0xfffffe000a183800, count=499,
done=0x0) at /usr/src/sys/dev/e1000/if_igb.c:4688
#10 0xffffffff803218da in igb_msix_que (arg=<value optimized out>)
at /usr/src/sys/dev/e1000/if_igb.c:1596
#11 0xffffffff804208cd in intr_event_execute_handlers (
p=<value optimized out>, ie=0xfffffe000a19f100)
at /usr/src/sys/kern/kern_intr.c:1272
#12 0xffffffff804220fe in ithread_loop (arg=0xfffffe000a1c6660)
at /usr/src/sys/kern/kern_intr.c:1285
#13 0xffffffff8041d52e in fork_exit (
callout=0xffffffff80422060 <ithread_loop>, arg=0xfffffe000a1c6660,
frame=0xffffff846384ec00) at /usr/src/sys/kern/kern_fork.c:995
#14 0xffffffff805f042e in fork_trampoline ()
at /usr/src/sys/amd64/amd64/exception.S:602
#15 0x0000000000000000 in ?? ()



--
Meditating Guru
Ian Freislich
_______________________________________________
freebsd...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-curre...@freebsd.org"

Adrian Chadd

unread,
Nov 17, 2012, 11:26:58 AM11/17/12
to
It's a NULL ponter deref. This is my line 484 in if_ethersubr.c:

eh = mtod(m, struct ether_header *);


.. if that's yours, see if eh is NULL?




adrian

Ian FREISLICH

unread,
Nov 17, 2012, 1:13:22 PM11/17/12
to
Adrian Chadd wrote:
> It's a NULL ponter deref. This is my line 484 in if_ethersubr.c:
>
> eh = mtod(m, struct ether_header *);
>
>
> .. if that's yours, see if eh is NULL?

(kgdb) frame 7
#7 0xffffffff8050f534 in ether_nh_input (m=0xfffffe012521e700)
at /usr/src/sys/net/if_ethersubr.c:484
484 eh = mtod(m, struct ether_header *);
(kgdb) print eh
No symbol "eh" in current context.
(kgdb) print *m
$2 = {m_hdr = {mh_next = 0x100000000000000, mh_nextpkt = 0x10000000000,
mh_data = 0x0, mh_len = 60, mh_flags = 4259842, mh_type = 0,
pad = "\000\000\000\000\000"}, M_dat = {MH = {MH_pkthdr = {
rcvif = 0xfffffe000a1c2000, header = 0xffffffff, len = 60, flowid = 0,
csum_flags = 3840, csum_data = 65535, tso_segsz = 0, PH_vt = {
vt_vtag = 4, vt_nrecs = 4}, tags = {slh_first = 0x3c000000}},
MH_dat = {MH_ext = {
ext_buf = 0x69e5498600000000 <Address 0x69e5498600000000 out of bounds>, ext_free = 0x10602, ext_arg1 = 0xc000000070000, ext_arg2 = 0x100,
ext_size = 2048, ref_cnt = 0xfffffe0125236d8c, ext_type = 6},
MH_databuf = "\000\000\000\000\206I�i\002\006\001\000\000\000\000\000\000\000\a\000\000\000\f\000\000\001\000\000\000\000\000\000\000\b\000\000\000\000\000\000\214m#%\001���\006", '\0' <repeats 118 times>}},
M_databuf = "\000 \034\n\000�������\000\000\000\000<\000\000\000\000\000\000\000\000\017\000\000��\000\000\000\000\004\000\000\000\000\000\000\000\000<\000\000\000\000\000\000\000\000\206I�i\002\006\001\000\000\000\000\000\000\000\a\000\000\000\f\000\000\001\000\000\000\000\000\000\000\b\000\000\000\000\000\000\214m#%\001���\006", '\0' <repeats 118 times>}}


Ian

--
Ian Freislich
Message has been deleted

Robert Watson

unread,
Nov 17, 2012, 4:31:18 PM11/17/12
to
Panics along these lines often occur if there is a concurrency bug in a device
driver such that it modifies an mbuf after dispatching to the network stack.
E.g., by freeing it, reusing it, an errant dereference, etc. Not guaranteed,
but that is where I'd start.

Robert

On Sat, 17 Nov 2012, Adrian Chadd wrote:

> Check what mtod() is doing.
>
> mbuf.h:#define mtod(m, t) ((t)((m)->m_data))
>
> .. so if m->m_data is NULL, bam.
>
> The question is why is m_data NULL here. Someone mbuf cluey is going
> to have to answer that. I don't know whether the MH_dat stuff is being
> treated as valid but m_data isn't being updated, or something.
>
>
> Adrian
>
> On 17 November 2012 10:13, Ian FREISLICH <ia...@clue.co.za> wrote:
>> Adrian Chadd wrote:
>>> It's a NULL ponter deref. This is my line 484 in if_ethersubr.c:
>>>
>>> eh = mtod(m, struct ether_header *);
>>>
>>>
>>> .. if that's yours, see if eh is NULL?
>>
>> (kgdb) frame 7
>> #7 0xffffffff8050f534 in ether_nh_input (m=0xfffffe012521e700)
>> at /usr/src/sys/net/if_ethersubr.c:484
>> 484 eh = mtod(m, struct ether_header *);
>> (kgdb) print eh
>> No symbol "eh" in current context.
>> (kgdb) print *m
>> $2 = {m_hdr = {mh_next = 0x100000000000000, mh_nextpkt = 0x10000000000,
>> mh_data = 0x0, mh_len = 60, mh_flags = 4259842, mh_type = 0,
>> pad = "\000\000\000\000\000"}, M_dat = {MH = {MH_pkthdr = {
>> rcvif = 0xfffffe000a1c2000, header = 0xffffffff, len = 60, flowid = 0,
>> csum_flags = 3840, csum_data = 65535, tso_segsz = 0, PH_vt = {
>> vt_vtag = 4, vt_nrecs = 4}, tags = {slh_first = 0x3c000000}},
>> MH_dat = {MH_ext = {
>> ext_buf = 0x69e5498600000000 <Address 0x69e5498600000000 out of bounds>, ext_free = 0x10602, ext_arg1 = 0xc000000070000, ext_arg2 = 0x100,
>> ext_size = 2048, ref_cnt = 0xfffffe0125236d8c, ext_type = 6},
>> MH_databuf = "\000\000\000\000\206Iеi\002\006\001\000\000\000\000\000\000\000\a\000\000\000\f\000\000\001\000\000\000\000\000\000\000\b\000\000\000\000\000\000\214m#%\001юяя\006", '\0' <repeats 118 times>}},
>> M_databuf = "\000 \034\n\000юяяяяяя\000\000\000\000<\000\000\000\000\000\000\000\000\017\000\000яя\000\000\000\000\004\000\000\000\000\000\000\000\000<\000\000\000\000\000\000\000\000\206Iеi\002\006\001\000\000\000\000\000\000\000\a\000\000\000\f\000\000\001\000\000\000\000\000\000\000\b\000\000\000\000\000\000\214m#%\001юяя\006", '\0' <repeats 118 times>}}
>>
>>
>> Ian
>>
>> --
>> Ian Freislich
>>

Gleb Smirnoff

unread,
Nov 17, 2012, 4:37:54 PM11/17/12
to
On Sat, Nov 17, 2012 at 05:07:54PM +0200, Ian FREISLICH wrote:
I> I have this consistently with:
I>
I> FreeBSD firewall2.jnb1.gp-online.net 10.0-CURRENT FreeBSD 10.0-CURRENT #30 r243156: Fri Nov 16 20:12:33 SAST 2012 ia...@firewall2.jnb1.gp-online.net:/usr/obj/usr/src/sys/FIREWALL amd64

Pretty sure this is a new version of wrong byte order panic, which
no longer can happen in HEAD.

Can you please try this patch?


--
Totus tuus, Glebius.
if_igb.c.no-relock.diff

Ian FREISLICH

unread,
Nov 19, 2012, 12:32:08 AM11/19/12
to
It survived the night, which it hasn't managed before. I'll keep you posted.

Ian

--

Ian FREISLICH

unread,
Nov 19, 2012, 9:29:46 AM11/19/12
to
Ian FREISLICH wrote:
> Gleb Smirnoff wrote:
> > On Sat, Nov 17, 2012 at 05:07:54PM +0200, Ian FREISLICH wrote:
> > I> I have this consistently with:
> > I>
> > I> FreeBSD firewall2.jnb1.gp-online.net 10.0-CURRENT FreeBSD 10.0-CURRENT #
30
> r243156: Fri Nov 16 20:12:33 SAST 2012 ia...@firewall2.jnb1.gp-online.net
:/
> usr/obj/usr/src/sys/FIREWALL amd64
> >
> > Pretty sure this is a new version of wrong byte order panic, which
> > no longer can happen in HEAD.
> >
> > Can you please try this patch?
>
> It survived the night, which it hasn't managed before. I'll keep you posted.

Jubilation short lived:

Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address = 0xc
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff8050f494
stack pointer = 0x28:0xffffff84637a19d0

frame pointer = 0x28:0xffffff84637a1a10

code segment = base 0x0, limit 0xfffff, type 0x1b
Fatal trap 12: page fault while in kernel mode
= DPL 0, pres 1, long 1, def32 0, gran 1
cpuid = 7; apic id = 07
processor eflags = fault virtual address = 0xc
interrupt enabled, fault code = supervisor read data, page not present
resume, IOPL = 0
instruction pointer = 0x20:0xffffffff8050f494
stack pointer = 0x28:0xffffff846386c9d0
current process = 11 (irq261: igb0:que 0)
frame pointer = 0x28:0xffffff846386ca10
trap number = 12
code segment = base 0x0, limit 0xfffff, type 0x1b
panic: page fault
cpuid = 0
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
panic() at panic+0x1ce
trap_fatal() at trap_fatal+0x290
trap_pfault() at trap_pfault+0x21f
trap() at trap+0x2b4
calltrap() at calltrap+0x8
--- trap 0xc, rip = 0xffffffff8050f494, rsp = 0xffffff84637a19d0, rbp = 0xffffff84637a1a10 ---
ether_nh_input() at ether_nh_input+0x94
netisr_dispatch_src() at netisr_dispatch_src+0x212
igb_rxeof() at igb_rxeof+0x384
igb_msix_que() at igb_msix_que+0xfa
intr_event_execute_handlers() at intr_event_execute_handlers+0xfd
ithread_loop() at ithread_loop+0x9e
fork_exit() at fork_exit+0x11e
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff84637a1cb0, rbp = 0 ---
Uptime: 19h5m45s
Dumping 2654 out of 16368 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%

#0 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:266
266 if (textdump && textdump_pending) {
(kgdb) #0 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:266
#1 0xffffffff8044ae64 in kern_reboot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:449
#2 0xffffffff8044b3e7 in panic (fmt=0x1 <Address 0x1 out of bounds>)
at /usr/src/sys/kern/kern_shutdown.c:637
#3 0xffffffff80605b30 in trap_fatal (frame=0xc, eva=<value optimized out>)
at /usr/src/sys/amd64/amd64/trap.c:872
#4 0xffffffff80605e9f in trap_pfault (frame=0xffffff84637a1920, usermode=0)
at /usr/src/sys/amd64/amd64/trap.c:789
#5 0xffffffff80606254 in trap (frame=0xffffff84637a1920)
at /usr/src/sys/amd64/amd64/trap.c:463
#6 0xffffffff805efecf in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:228
#7 0xffffffff8050f494 in ether_nh_input (m=0xfffffe004f3bde00)
at /usr/src/sys/net/if_ethersubr.c:484
#8 0xffffffff8051a562 in netisr_dispatch_src (proto=9,
source=<value optimized out>, m=<value optimized out>)
at /usr/src/sys/net/netisr.c:1013
#9 0xffffffff80318844 in igb_rxeof (que=0xfffffe000a183a00, count=499,
done=0x0) at /usr/src/sys/dev/e1000/if_igb.c:4688
#10 0xffffffff8032183a in igb_msix_que (arg=<value optimized out>)
at /usr/src/sys/dev/e1000/if_igb.c:1596
#11 0xffffffff8042082d in intr_event_execute_handlers (
p=<value optimized out>, ie=0xfffffe000a109e00)
at /usr/src/sys/kern/kern_intr.c:1272
#12 0xffffffff8042205e in ithread_loop (arg=0xfffffe000a1a16e0)
at /usr/src/sys/kern/kern_intr.c:1285
#13 0xffffffff8041d48e in fork_exit (
callout=0xffffffff80421fc0 <ithread_loop>, arg=0xfffffe000a1a16e0,
frame=0xffffff84637a1c00) at /usr/src/sys/kern/kern_fork.c:995
#14 0xffffffff805f038e in fork_trampoline ()
at /usr/src/sys/amd64/amd64/exception.S:602
#15 0x0000000000000000 in ?? ()


--
Meditating Guru
Message has been deleted
0 new messages