Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

b44 driver causes panic when using swiotlb

152 views
Skip to first unread message

Chuck Ebbert

unread,
Jan 31, 2011, 11:00:02 AM1/31/11
to
The b44 driver is triggering this panic in swiotlb_map_page():

if (!dma_capable(dev, dev_addr, size))
panic("map_single: bounce buffer is not DMA'ble");

The kernel log says the bounce buffers are at 0xdb400000, but b44 can
only do DMA to the first 1GB of memory:

PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Placing 64MB software IO TLB between ffff8800db400000 - ffff8800df400000
software IO TLB at phys 0xdb400000 - 0xdf400000
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Andi Kleen

unread,
Jan 31, 2011, 11:40:02 AM1/31/11
to
On Mon, Jan 31, 2011 at 10:54:12AM -0500, Chuck Ebbert wrote:
> The b44 driver is triggering this panic in swiotlb_map_page():
>
> if (!dma_capable(dev, dev_addr, size))
> panic("map_single: bounce buffer is not DMA'ble");
>
> The kernel log says the bounce buffers are at 0xdb400000, but b44 can
> only do DMA to the first 1GB of memory:

b44 needs to use GFP_DMA then and do its own custom bouncing.
The standard pci_map_* bounce buffering is only designed for at least
32bit capable devices.

BTW I'm pretty sure this worked at some point, must have regressed
somehow.

-Andi

--
a...@linux.intel.com -- Speaking for myself only

Robert Hancock

unread,
Jan 31, 2011, 8:00:02 PM1/31/11
to
On 01/31/2011 10:36 AM, Andi Kleen wrote:
> On Mon, Jan 31, 2011 at 10:54:12AM -0500, Chuck Ebbert wrote:
>> The b44 driver is triggering this panic in swiotlb_map_page():
>>
>> if (!dma_capable(dev, dev_addr, size))
>> panic("map_single: bounce buffer is not DMA'ble");
>>
>> The kernel log says the bounce buffers are at 0xdb400000, but b44 can
>> only do DMA to the first 1GB of memory:
>
> b44 needs to use GFP_DMA then and do its own custom bouncing.
> The standard pci_map_* bounce buffering is only designed for at least
> 32bit capable devices.

That seems wrong - it's a documented API and that restriction isn't
documented. Either it should comply with the request or return a failure
if it can't accomodate it, not just blow up internally. There's no
reason the driver should have to deal with this on its own.

In this case the DMA mapping code should really be falling back to
GFP_DMA automatically if the IOMMU aperture is outside the DMA mask of
the device.

>
> BTW I'm pretty sure this worked at some point, must have regressed
> somehow.

--

FUJITA Tomonori

unread,
Jan 31, 2011, 8:30:01 PM1/31/11
to
On Mon, 31 Jan 2011 18:54:21 -0600
Robert Hancock <hanco...@gmail.com> wrote:

> On 01/31/2011 10:36 AM, Andi Kleen wrote:
> > On Mon, Jan 31, 2011 at 10:54:12AM -0500, Chuck Ebbert wrote:
> >> The b44 driver is triggering this panic in swiotlb_map_page():
> >>
> >> if (!dma_capable(dev, dev_addr, size))
> >> panic("map_single: bounce buffer is not DMA'ble");
> >>
> >> The kernel log says the bounce buffers are at 0xdb400000, but b44 can
> >> only do DMA to the first 1GB of memory:
> >
> > b44 needs to use GFP_DMA then and do its own custom bouncing.
> > The standard pci_map_* bounce buffering is only designed for at least
> > 32bit capable devices.
>
> That seems wrong - it's a documented API and that restriction isn't
> documented. Either it should comply with the request or return a failure
> if it can't accomodate it, not just blow up internally. There's no
> reason the driver should have to deal with this on its own.
>
> In this case the DMA mapping code should really be falling back to
> GFP_DMA automatically if the IOMMU aperture is outside the DMA mask of
> the device.

swiotlb allocates the bounce buffer when a system boots up. We can't
allocate much in GFP_DMA. swiotlb uses somewhere under 4GB. So it
can't help devices that have odd dma_mask (that is, except for 4GB).

Unfortunately, Such device needs to do own custom bouncing or needs
their subsystem to does that.

Some ideas to implement something that works for such device were
discussed. Seems that the conclusion is that it's doesn't worth making
the common code complicated for such minor and insane devices.

Robert Hancock

unread,
Jan 31, 2011, 10:30:01 PM1/31/11
to

I don't think this is the only device that has sub-32-bit DMA
restrictions, this will just lead to a bunch of duplicated code. In
particular, how is LPC DMA supposed to work?

At the very least we should be allowing the driver to deal with the
failure instead of panicing the system. Otherwise we are just leaving a
land mine for people to trip over.

Larry Finger

unread,
Feb 1, 2011, 12:00:02 AM2/1/11
to
On 01/31/2011 10:41 PM, Larry Finger wrote:
> From: Robert Hancock
> Date: Mon Jan 31 2011 - 22:22:32 EST
>
> * Next message: Amit Shah: "Re: [PATCH 1/3] tty: move hvc drivers to
> drivers/tty/hvc/"
> * Previous message: Grant Likely: "Re: [RFC][PATCH] Power domains for
> platform bus type"
> * In reply to: FUJITA Tomonori: "Re: b44 driver causes panic when using swiotlb"
> * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Some devices driven by b43legacy and b43 only support 30-bit DMA, which is what
I suspect b44 handles. The b43* drivers use a bounce buffer. If a generic
mechanism were created, those 2 would use it.

Larry

FUJITA Tomonori

unread,
Feb 1, 2011, 12:30:02 AM2/1/11
to
On Mon, 31 Jan 2011 21:22:23 -0600
Robert Hancock <hanco...@gmail.com> wrote:

> > Some ideas to implement something that works for such device were
> > discussed. Seems that the conclusion is that it's doesn't worth making
> > the common code complicated for such minor and insane devices.
>
> I don't think this is the only device that has sub-32-bit DMA
> restrictions, this will just lead to a bunch of duplicated code.

Yeah, not only device but not many.

The block layer has the own bouncing mechanism. Some network drivers
have the similar bouncing code. I don't know if there are other kinds
of drivers that have the own bouncing code.

I thought that we can make mm/bounce.c (used for block drivers now)
work any drivers without complicating it. We could make swiotlb to do
but it's too complicated and it doesn't worth.


> In
> particular, how is LPC DMA supposed to work?

LPC DMA can't do 32bit dma?


> At the very least we should be allowing the driver to deal with the
> failure instead of panicing the system. Otherwise we are just leaving a
> land mine for people to trip over.

Agreed. swiotlb shouldn't panic in this case. I'll take care of it.

FUJITA Tomonori

unread,
Feb 1, 2011, 9:00:01 AM2/1/11
to
On Mon, 31 Jan 2011 10:54:12 -0500
Chuck Ebbert <ceb...@redhat.com> wrote:

> The b44 driver is triggering this panic in swiotlb_map_page():
>
> if (!dma_capable(dev, dev_addr, size))
> panic("map_single: bounce buffer is not DMA'ble");
>
> The kernel log says the bounce buffers are at 0xdb400000, but b44 can
> only do DMA to the first 1GB of memory:

Can you try this? b44 has the own bouncing mechanism so if swiotlb
returns an error, the driver can allocate an appropriate buffer.

diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index c47bbe1..93ca08b 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -686,8 +686,10 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
/*
* Ensure that the address returned is DMA'ble
*/
- if (!dma_capable(dev, dev_addr, size))
- panic("map_single: bounce buffer is not DMA'ble");
+ if (!dma_capable(dev, dev_addr, size)) {
+ swiotlb_tbl_unmap_single(dev, map, size, dir);
+ dev_addr = swiotlb_virt_to_bus(dev, io_tlb_overflow_buffer);
+ }

return dev_addr;

Andi Kleen

unread,
Feb 1, 2011, 1:10:02 PM2/1/11
to
On Mon, Jan 31, 2011 at 06:54:21PM -0600, Robert Hancock wrote:
> On 01/31/2011 10:36 AM, Andi Kleen wrote:
> >On Mon, Jan 31, 2011 at 10:54:12AM -0500, Chuck Ebbert wrote:
> >>The b44 driver is triggering this panic in swiotlb_map_page():
> >>
> >> if (!dma_capable(dev, dev_addr, size))
> >> panic("map_single: bounce buffer is not DMA'ble");
> >>
> >>The kernel log says the bounce buffers are at 0xdb400000, but b44 can
> >>only do DMA to the first 1GB of memory:
> >
> >b44 needs to use GFP_DMA then and do its own custom bouncing.
> >The standard pci_map_* bounce buffering is only designed for at least
> >32bit capable devices.
>
> That seems wrong - it's a documented API and that restriction isn'

Please read the documentation. When PCI DMA cannot handle it
it returns failure. That is what happened here.

Besides historically PCI-DMA never handled < 4GB.

> documented. Either it should comply with the request or return a
> failure if it can't accomodate it, not just blow up internally.

It does return failure, that is what breaks the driver because
it doesn't handle it.

> There's no reason the driver should have to deal with this on its
> own.

Even if swiotlb could handle it, other implementations of PCI-DMA
can not

> In this case the DMA mapping code should really be falling back to
> GFP_DMA automatically if the IOMMU aperture is outside the DMA mask
> of the device.

No, no -- you cannot allocate memory in DMA API. It's forbidden.
Some paths that use it cannot tolerate it.

-Andi

--
a...@linux.intel.com -- Speaking for myself only

Chuck Ebbert

unread,
Feb 1, 2011, 4:30:03 PM2/1/11
to
On Tue, 1 Feb 2011 10:28:00 +0900
FUJITA Tomonori <fujita....@lab.ntt.co.jp> wrote:

>
> swiotlb allocates the bounce buffer when a system boots up. We can't
> allocate much in GFP_DMA. swiotlb uses somewhere under 4GB. So it
> can't help devices that have odd dma_mask (that is, except for 4GB).
>
> Unfortunately, Such device needs to do own custom bouncing or needs
> their subsystem to does that.

I think we're chasing the wrong problem here.

swiotlb uses alloc_bootmem_low_pages() to try to get buffers as low
in memory as possible. I asked someone who is hitting this bug to
try 2.6.36 and he reports the buffers really are low there:

2.6.36: 5c00000
2.6.37: db600000

So something happened very early in the 2.6.37-rc cycle that changed
this behavior. I tried looking at the bootmem code but could not see
the problem. The only related option I could find in .config was this:

# CONFIG_NO_BOOTMEM is not set

It was set this way in both .36 and .37.

Robert Hancock

unread,
Feb 1, 2011, 4:50:01 PM2/1/11
to
On Tue, Feb 1, 2011 at 12:07 PM, Andi Kleen <a...@linux.intel.com> wrote:
> On Mon, Jan 31, 2011 at 06:54:21PM -0600, Robert Hancock wrote:
>> On 01/31/2011 10:36 AM, Andi Kleen wrote:
>> >On Mon, Jan 31, 2011 at 10:54:12AM -0500, Chuck Ebbert wrote:
>> >>The b44 driver is triggering this panic in swiotlb_map_page():
>> >>
>> >>         if (!dma_capable(dev, dev_addr, size))
>> >>                 panic("map_single: bounce buffer is not DMA'ble");
>> >>
>> >>The kernel log says the bounce buffers are at 0xdb400000, but b44 can
>> >>only do DMA to the first 1GB of memory:
>> >
>> >b44 needs to use GFP_DMA then and do its own custom bouncing.
>> >The standard pci_map_* bounce buffering is only designed for at least
>> >32bit capable devices.
>>
>> That seems wrong - it's a documented API and that restriction isn'
>
> Please read the documentation. When PCI DMA cannot handle it
> it returns failure. That is what happened here.

Apparently not, given that it hit a panic inside the swiotlb code..

FUJITA Tomonori

unread,
Feb 1, 2011, 7:10:02 PM2/1/11
to
On Tue, 1 Feb 2011 16:18:59 -0500
Chuck Ebbert <ceb...@redhat.com> wrote:

> > swiotlb allocates the bounce buffer when a system boots up. We can't
> > allocate much in GFP_DMA. swiotlb uses somewhere under 4GB. So it
> > can't help devices that have odd dma_mask (that is, except for 4GB).
> >
> > Unfortunately, Such device needs to do own custom bouncing or needs
> > their subsystem to does that.
>
> I think we're chasing the wrong problem here.
>
> swiotlb uses alloc_bootmem_low_pages() to try to get buffers as low
> in memory as possible. I asked someone who is hitting this bug to
> try 2.6.36 and he reports the buffers really are low there:
>
> 2.6.36: 5c00000
> 2.6.37: db600000
>
> So something happened very early in the 2.6.37-rc cycle that changed
> this behavior. I tried looking at the bootmem code but could not see
> the problem. The only related option I could find in .config was this:
>
> # CONFIG_NO_BOOTMEM is not set
>
> It was set this way in both .36 and .37.

I don't think this matters because we can't guarantee that swiotlb
can't allocate ZONE_DMA. Obviously, the default swiotlb bouncing
buffer size, 64MB, is too large for ZONE_DMA.

If you are lucky, you sometimes can allocate lower memory if swiotlb
allocates lower memory at the boot time. But you can't assume
that. swiotlb isn't designed to handle such.

Can you try the patch that I sent you? As I wrote, b44 has the own
bouncing mechanism, so we should be fine if swiotlb returns an
appropriate error instead of just panicking.

FUJITA Tomonori

unread,
Feb 1, 2011, 7:20:01 PM2/1/11
to
On Wed, 2 Feb 2011 09:06:34 +0900
FUJITA Tomonori <fujita....@lab.ntt.co.jp> wrote:

> On Tue, 1 Feb 2011 16:18:59 -0500
> Chuck Ebbert <ceb...@redhat.com> wrote:
>
> > > swiotlb allocates the bounce buffer when a system boots up. We can't
> > > allocate much in GFP_DMA. swiotlb uses somewhere under 4GB. So it
> > > can't help devices that have odd dma_mask (that is, except for 4GB).
> > >
> > > Unfortunately, Such device needs to do own custom bouncing or needs
> > > their subsystem to does that.
> >
> > I think we're chasing the wrong problem here.
> >
> > swiotlb uses alloc_bootmem_low_pages() to try to get buffers as low
> > in memory as possible. I asked someone who is hitting this bug to
> > try 2.6.36 and he reports the buffers really are low there:
> >
> > 2.6.36: 5c00000
> > 2.6.37: db600000
> >
> > So something happened very early in the 2.6.37-rc cycle that changed
> > this behavior. I tried looking at the bootmem code but could not see
> > the problem. The only related option I could find in .config was this:
> >
> > # CONFIG_NO_BOOTMEM is not set
> >
> > It was set this way in both .36 and .37.
>
> I don't think this matters because we can't guarantee that swiotlb
> can't allocate ZONE_DMA.

Oops, I meant that, we can't guarantee that swiotlb allocates

FUJITA Tomonori

unread,
Feb 14, 2011, 7:10:03 AM2/14/11
to

Ping, any chance to try this?


Thanks,

Arkadiusz Miśkiewicz

unread,
Feb 14, 2011, 11:50:02 AM2/14/11
to
FUJITA Tomonori wrote:

Friend machine oopsed on network with b44 on 2.6.37, so we tried this patch
and network card was detected nicely and was actually usable. The oops
started to happen after he upgraded memory from 2GB to 4GB in his dell
vostro 1500 notebook.

03:00.0 Ethernet controller: Broadcom Corporation BCM4401-B0 100Base-TX (rev
02)

# grep b44 dmesg.log
[ 8.524297] b44 0000:03:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[ 8.576838] b44: b44.c:v2.0
[ 8.593920] b44 ssb1:0: eth0: Broadcom 44xx/47xx 10/100BaseT Ethernet
00:1c:23:9b:3f:5f
[ 55.350228] b44 ssb1:0: eth0: Link is up at 100 Mbps, full duplex
[ 55.350233] b44 ssb1:0: eth0: Flow control is off for TX and off for RX


> Thanks,

--
Arkadiusz Miśkiewicz PLD/Linux Team
arekm / maven.pl http://ftp.pld-linux.org/

FUJITA Tomonori

unread,
Feb 14, 2011, 7:50:02 PM2/14/11
to

Thanks for the confirmation.

> started to happen after he upgraded memory from 2GB to 4GB in his dell
> vostro 1500 notebook.

Yeah, a system with 2GB memory doesn't use swiotlb so you don't hit
this bug.

Thanks,

Robert Hancock

unread,
Feb 14, 2011, 7:50:02 PM2/14/11
to
On 01/31/2011 11:22 PM, FUJITA Tomonori wrote:
> On Mon, 31 Jan 2011 21:22:23 -0600
> Robert Hancock<hanco...@gmail.com> wrote:
>
>>> Some ideas to implement something that works for such device were
>>> discussed. Seems that the conclusion is that it's doesn't worth making
>>> the common code complicated for such minor and insane devices.
>>
>> I don't think this is the only device that has sub-32-bit DMA
>> restrictions, this will just lead to a bunch of duplicated code.
>
> Yeah, not only device but not many.
>
> The block layer has the own bouncing mechanism. Some network drivers
> have the similar bouncing code. I don't know if there are other kinds
> of drivers that have the own bouncing code.
>
> I thought that we can make mm/bounce.c (used for block drivers now)
> work any drivers without complicating it. We could make swiotlb to do
> but it's too complicated and it doesn't worth.
>
>
>> In
>> particular, how is LPC DMA supposed to work?
>
> LPC DMA can't do 32bit dma?

At least not if it's using ISA-style 3rd-party DMA. Some devices may do
bus-mastering and be able to do 32-bit DMA, but I've never seen one, at
least.

Chuck Ebbert

unread,
Feb 15, 2011, 5:10:02 PM2/15/11
to
On Mon, 14 Feb 2011 20:59:43 +0900
FUJITA Tomonori <fujita....@lab.ntt.co.jp> wrote:

> > Can you try this? b44 has the own bouncing mechanism so if swiotlb
> > returns an error, the driver can allocate an appropriate buffer.
> >
> > diff --git a/lib/swiotlb.c b/lib/swiotlb.c
> > index c47bbe1..93ca08b 100644
> > --- a/lib/swiotlb.c
> > +++ b/lib/swiotlb.c
> > @@ -686,8 +686,10 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
> > /*
> > * Ensure that the address returned is DMA'ble
> > */
> > - if (!dma_capable(dev, dev_addr, size))
> > - panic("map_single: bounce buffer is not DMA'ble");
> > + if (!dma_capable(dev, dev_addr, size)) {
> > + swiotlb_tbl_unmap_single(dev, map, size, dir);
> > + dev_addr = swiotlb_virt_to_bus(dev, io_tlb_overflow_buffer);
> > + }
> >
> > return dev_addr;
> > }
>
> Ping, any chance to try this?

The user who reported the bug tried it and says that it works.

0 new messages