Call for bge(4) testers
Pyun YongHyeon
Hi,
I had been working on fixing bus_dma(9) bugs and adding TSO
capability to bge(4). Now TSO is supported for BCM5755 or newer
controllers. Actually some pre-BCM5755 controllers also support
TSO with the help of special firmware but the license issue and
lower performance of firmware based TSO as well as TSO bug I
intentionally excluded TSO support for pre-BCM5755 controllers.
You can get the patch form the following URL. The diff was
generated against latest HEAD.
http://people.freebsd.org/~yongari/bge/bge.tso.1111.diff
or
http://people.freebsd.org/~yongari/bge/if_bge.c
http://people.freebsd.org/~yongari/bge/if_bgereg.h
The diff also has a lot of functional changes which may affect
other bge(4) controllers. Specifically the diff
- Added workarounds for known DMA bug. This workaround shall make
bge(4) use bounce buffers so this fix requires all previous
changes made in HEAD. This may slow down overall performance if
bounce buffers are used but I think it's better than leaving the
bug.
- Tuned interrupt coalescing parameters.
- Use taskqeue to get better performance on PCIe controllers.
- TSO support for BCM5755 or newer controllers.
- Tx path cleaned up.
- Interrupt handler now clears status word as it does in polling
case.
I'm interesting in performance changes before/after the diff and
any regressions from the diff. Please test I'll commit the diff
next week unless I get regressions.
Thanks.
_______________________________________________
freebsd...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-curre...@freebsd.org"
Pyun YongHyeon
>
> Hi,
>
> I had been working on fixing bus_dma(9) bugs and adding TSO
> capability to bge(4). Now TSO is supported for BCM5755 or newer
> controllers. Actually some pre-BCM5755 controllers also support
> TSO with the help of special firmware but the license issue and
> lower performance of firmware based TSO as well as TSO bug I
> intentionally excluded TSO support for pre-BCM5755 controllers.
> You can get the patch form the following URL. The diff was
> generated against latest HEAD.
>
> http://people.freebsd.org/~yongari/bge/bge.tso.1111.diff
Eh, there was a typo so I regenerated the diff.
http://people.freebsd.org/~yongari/bge/bge.tso.1111-1.diff
Gonzalo Nemmi
> On Wed, Nov 11, 2009 at 02:37:51PM -0800, Pyun YongHyeon wrote:
> > Hi,
> >
> > I had been working on fixing bus_dma(9) bugs and adding TSO
> > capability to bge(4). Now TSO is supported for BCM5755 or newer
> > controllers. Actually some pre-BCM5755 controllers also support
> > TSO with the help of special firmware but the license issue and
> > lower performance of firmware based TSO as well as TSO bug I
> > intentionally excluded TSO support for pre-BCM5755 controllers.
> > You can get the patch form the following URL. The diff was
> > generated against latest HEAD.
> >
> > http://people.freebsd.org/~yongari/bge/bge.tso.1111.diff
>
> Eh, there was a typo so I regenerated the diff.
> http://people.freebsd.org/~yongari/bge/bge.tso.1111-1.diff
Hi
Just wanted to know before getting on to it, will your patch help to
resolve kern/136876?
Best Regards
Gonzalo Nemmi
Pyun YongHyeon
> On Thursday 12 November 2009 1:47:49 am Pyun YongHyeon wrote:
> > On Wed, Nov 11, 2009 at 02:37:51PM -0800, Pyun YongHyeon wrote:
> > > Hi,
> > >
> > > I had been working on fixing bus_dma(9) bugs and adding TSO
> > > capability to bge(4). Now TSO is supported for BCM5755 or newer
> > > controllers. Actually some pre-BCM5755 controllers also support
> > > TSO with the help of special firmware but the license issue and
> > > lower performance of firmware based TSO as well as TSO bug I
> > > intentionally excluded TSO support for pre-BCM5755 controllers.
> > > You can get the patch form the following URL. The diff was
> > > generated against latest HEAD.
> > >
> > > http://people.freebsd.org/~yongari/bge/bge.tso.1111.diff
> >
> > Eh, there was a typo so I regenerated the diff.
> > http://people.freebsd.org/~yongari/bge/bge.tso.1111-1.diff
>
> Hi
> Just wanted to know before getting on to it, will your patch help to
> resolve kern/136876?
>
My diff includes a fix for assuming PCIe device control register
and MSI control registers would be reside in fixed address. And
from the pciconf output I see the your MSI control register is
located at different address. However bge(4) does not touch that
register for BCM5906 so I guess my diff may not fix the resume
issue.
Gonzalo Nemmi
Thanks a lot for your prompt, clear and straight answer.
Xin LI
Hash: SHA1
Hi, Pyun,
Pyun YongHyeon wrote:
> I'm interesting in performance changes before/after the diff and
> any regressions from the diff. Please test I'll commit the diff
> next week unless I get regressions.
So far I have not seen regressions after bge.tso.1111-1.diff . I have
not done any benchmarks though, never imagined my laptop is equipped
with TSO :)
[delphij@delta] /usr/src/sys/net> ifconfig bge0
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
Cheers,
- --
Xin LI <del...@delphij.net> http://www.delphij.net/
FreeBSD - The Power to Serve! Live free or die
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.13 (FreeBSD)
iEYEARECAAYFAkr91UQACgkQi+vbBBjt66DBOgCffvF/8g5ATJMzEIa6RPHvzsBK
2fYAn18bCgVDOL/r0TP0Vg0Q8cUogga/
=OIds
-----END PGP SIGNATURE-----
Pyun YongHyeon
[...]
> > I'm interesting in performance changes before/after the diff and
> > any regressions from the diff. Please test I'll commit the diff
> > next week unless I get regressions.
>
> So far I have not seen regressions after bge.tso.1111-1.diff . I have
> not done any benchmarks though, never imagined my laptop is equipped
> with TSO :)
>
> [delphij@delta] /usr/src/sys/net> ifconfig bge0
> bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
>
AFAIK bge(4) was the only driver that lacks TSO support for TSO
capable hardwares.(I'm not sure whether et(4) controllers also have
TSO capability though) You may notice better TCP bulk transfer
performance as well as lower CPU load.
Thanks for testing!
Fabian Keil
> I had been working on fixing bus_dma(9) bugs and adding TSO
> capability to bge(4). Now TSO is supported for BCM5755 or newer
> controllers. Actually some pre-BCM5755 controllers also support
> TSO with the help of special firmware but the license issue and
> lower performance of firmware based TSO as well as TSO bug I
> intentionally excluded TSO support for pre-BCM5755 controllers.
> I'm interesting in performance changes before/after the diff and
> any regressions from the diff. Please test I'll commit the diff
> next week unless I get regressions.
No regressions noticed so far with:
bge0@pci0:4:0:0: class=0x020000 card=0x20d517aa chip=0x169314e4 rev=0x02 hdr=0x00
vendor = 'Broadcom Corporation'
device = 'Ethernet Controller Broadcom Netlink Gigabit (BCM5787A)'
Fabian
Gonzalo Nemmi
Hi Pyun!
Sorry for the delay, I was out of town and just got back.
I'm downloading RC3 as of now. Then I will install:
edit make.conf
edit src.conf
buildworld
buildkernel
installkernel
reboot
mergemaster -p
make installworld
reboot
cp bge.diff bge.patch
cd /usr/src/sys/dev/bge && patch < /path/to/patch
make
make install clean
kldunload if_bge
kldload if_bge
pciconf -lcvb
ifconfig bge0 up
acpiconf -s3
.. and hpefully .. resume from S3 ..
Is that ok with you or would you like me to do it in another way?
try some more stuff?
Some test in particular?
Best Regards and thanks for the patch
Pyun YongHyeon
Not sure you removed bge in GENERIC kernel configuration file.
> kldload if_bge
> pciconf -lcvb
> ifconfig bge0 up
> acpiconf -s3
>
> ... and hpefully .. resume from S3 ..
>
> Is that ok with you or would you like me to do it in another way?
That's ok. At first I wanted to add WOL to wake up bge(4) with
magic packet but bge(4) seems to require a lot of workaround for
each controller and it's too complex to implement at this time.
Just want to know whether bge(4) can resume from suspend.
Gonzalo Nemmi
Oh! Sorry for not being clear on that point. Due to the problems I've
had with bge(4), I do remove it from GENERIC .. Actually .. most of my
kernels are named NOBGE ;)
> > kldload if_bge
> > pciconf -lcvb
> > ifconfig bge0 up
> > acpiconf -s3
> >
> > ... and hpefully .. resume from S3 ..
> >
> > Is that ok with you or would you like me to do it in another way?
>
> That's ok. At first I wanted to add WOL to wake up bge(4) with
> magic packet but bge(4) seems to require a lot of workaround for
> each controller and it's too complex to implement at this time.
> Just want to know whether bge(4) can resume from suspend.
Perfetc then, I'll proceed as described above and mail you back as soon
as I have some results to let you know :)
Once Again, thanks for the patch Pyun :)
Best Regards
Gonzalo Nemmi
Gonzalo Nemmi
Well, I proceeded as I told you I would, applied your patch and even if
it didn't solve the problem (bge still doesn't resume) at least it
improved the previous situation given that now it doesn't loop timing
out once and again as before.
You can find a tail from /var/log/messages in here:
http://pastebin.com/f643555f7
As you can see, the first 3 lines corresponds to "kldload if_bge"
Line number 4 is "acpiconf -s3"
At line number 17 bge0 finally fails and let's the machine wake up at
line 18.
Then, as soon as I could I issued a "ifconfig bge0" to see what I could
get .. that's what you can see from line 20 to line 41. (as you can see
in there, I found there are problems resuming umass devices as
well ... )
Line 42 to 53 correspond to "kldnuload if_bge" .. which, by the way,
once unloaded, I could load it again but bge0 never showed
on "ifconfig".
Line 54 till the end correspond to "umount_msdos /dev/ad0s1" ... which
ended with pulling the pendrive out as the system coldn't umount it ...
I also found I get wpi0 messages on resume .. it spouts:
wpi0 could not lock memory
wpi0 could not lock memory
wpi0 could not lock memory
wpi0 could not lock memory
wpi0 could not lock memory
wpi0 could not lock memory
wpi0 could not lock memory
and then it let's the system resume.
All in all .. there's _a_lot_of_problems_on_resume_
Pyun, if you'd like me to try a new patch or to do some tests or
whatever, just let me know. I'm here awaiting orders :)
Best Regards
Pyun YongHyeon
[...]
> Well, I proceeded as I told you I would, applied your patch and even if
> it didn't solve the problem (bge still doesn't resume) at least it
> improved the previous situation given that now it doesn't loop timing
> out once and again as before.
>
> You can find a tail from /var/log/messages in here:
> http://pastebin.com/f643555f7
>
> As you can see, the first 3 lines corresponds to "kldload if_bge"
> Line number 4 is "acpiconf -s3"
>
> At line number 17 bge0 finally fails and let's the machine wake up at
> line 18.
>
> Then, as soon as I could I issued a "ifconfig bge0" to see what I could
> get .. that's what you can see from line 20 to line 41. (as you can see
Ok, would you try this one? I guess bge(4) register access method
could be in uninitialized state after resume.
> in there, I found there are problems resuming umass devices as
> well ... )
>
Probably Hans can help you on USB issues.
> Line 42 to 53 correspond to "kldnuload if_bge" .. which, by the way,
> once unloaded, I could load it again but bge0 never showed
> on "ifconfig".
>
> Line 54 till the end correspond to "umount_msdos /dev/ad0s1" ... which
> ended with pulling the pendrive out as the system coldn't umount it ...
>
> I also found I get wpi0 messages on resume .. it spouts:
> wpi0 could not lock memory
> wpi0 could not lock memory
> wpi0 could not lock memory
> wpi0 could not lock memory
> wpi0 could not lock memory
> wpi0 could not lock memory
> wpi0 could not lock memory
> and then it let's the system resume.
>
Don't know what it really means. You'd be better to ask wpi(4)
author Benjamin Close(benjsc@).
Gonzalo Nemmi
> On Tue, Nov 17, 2009 at 04:50:06PM -0200, Gonzalo Nemmi wrote:
>
> [...]
>
> > Well, I proceeded as I told you I would, applied your patch and
> > even if it didn't solve the problem (bge still doesn't resume) at
> > least it improved the previous situation given that now it doesn't
> > loop timing out once and again as before.
> >
> > You can find a tail from /var/log/messages in here:
> > http://pastebin.com/f643555f7
> >
> > As you can see, the first 3 lines corresponds to "kldload if_bge"
> > Line number 4 is "acpiconf -s3"
> >
> > At line number 17 bge0 finally fails and let's the machine wake up
> > at line 18.
> >
> > Then, as soon as I could I issued a "ifconfig bge0" to see what I
> > could get .. that's what you can see from line 20 to line 41. (as
> > you can see
>
> Ok, would you try this one? I guess bge(4) register access method
> could be in uninitialized state after resume.
Just did .. same result .. you�ll find the messages here:
http://pastebin.com/f38369b3c
1.- root login
2 to 9.- "kldload if_bge"
10.- "acpiconf -s3"
11 to 13.- wpi0 messages before suspending
14 to 23.- bge0 messages upon resume
24.- wakeup
BTW: is there an easy way to unroll the patches so I get a pristine copy
back and apply the patch over it again? Im asking because the 3rd chunk
of your patch ( bge_reset(sc); ) didn't apply cleanly and I had to edit
if_bge.c by hand to add that line in the right place.
Still willing to keep on trying and awaiting orders :)
Best Regards
Gonzalo Nemmi
--
Blessings
Gonzalo Nemmi
> Can you remove wpi(4) in kernel configuration and try again?
> I don't think wpi(4) can interfere bge(4) resume but just want to
> clear it.
>
>
Just did .. got rid of wpi, wpifw, firmware, recompile, reinstalled and got
the same result:
http://pastebin.com/f5933192d
> > BTW: is there an easy way to unroll the patches so I get a pristine copy
> > back and apply the patch over it again? Im asking because the 3rd chunk
> > of your patch ( bge_reset(sc); ) didn't apply cleanly and I had to edit
> > if_bge.c by hand to add that line in the right place.
> >
>
> There is option -R in patch(1) so I think you can safely undo
> applied patches if you have previous patch in your box.
>
>
Did that .. cd /usr/src && patch -R < /path/to/patch
It apllied like a charm :)
> Still have no clue yet. This kind of debugging really need hardware
> access to experiment various things but let's try narrow down
> possible cause of problem.
>
I might be able to fix that if you really care about fixing this bug .. I
can set you up a ssh account and give you root acces to this machine if you
need to :)
Just let me know ;)
Best Regards
Gonzalo Nemmi
Gonzalo Nemmi
> On Wed, Nov 18, 2009 at 01:40:24AM -0300, Gonzalo Nemmi wrote:
>
> [...]
>
> > I just tried ...
> > echo 'hw.bge.allow_asf="1"' >> /boot/loader.conf
> > reboot
> > load if_bge
> > acpiconf -s3
> > same results :(
> >
>
> Ok, here is new try.
>
Let's get on to it then !
Well now, the situation has improved .. here's what I got:
http://pastebin.com/f2d152f91
lines 2 to 8 == kldload if_bge
line 9 == acpiconf -s3
lines 10 to 18 == resume (notice there are only 2 messages now: lines 12 and
13!)
lines 19 to 36 == ifconfig bge0
lines 37 to 42 == me mounting the pendrive to get the messages from
/var/log/messages =P
lines 43 to 46 == kldunload if_bge0
I think you narrowed it down quite a lot this time !
I have to warn you though, that this time, my kernel was compiled using a
.. certainly modified /etc/make.conf file ... just in case you need to know
how it looks like, you'll fin it in here:
http://pastebin.com/f42e356d2
If you think that make.conf config may interfere with your pourposes or
tests, just let me know and I'll use a default one.
The good thing about that make.conf is that it saves me quite a time on
every recompile ;)
> im at your service .. tell me what to do and I'll do it :)
> >
>
> Thanks a lot for your patience and continuous support to fixing
> bugs.
>
Thank _YOU_ for keeping the good work up and for trying to solve a really
nasty bug that makes every bge(4) user (think of it
_as_every_dell_notebook_owner_) unable to get his laptop to resume .. or
even use FreeBSD altoghether just because of this.
Dear Pyung, rest assured that as long as you remain commited to fix this, or
any other bug that I can help you with, you can count on me to do everything
that may be within the reach of my hand. As long as you remain commited,
I'll be there, commited just as well :)
Awaiting further orders
Yours
Gonzalo Nemmi
Pyun YongHyeon
> On Wed, Nov 18, 2009 at 8:32 PM, Pyun YongHyeon <pyu...@gmail.com> wrote:
>
> > On Wed, Nov 18, 2009 at 01:40:24AM -0300, Gonzalo Nemmi wrote:
> >
> > [...]
> >
> > > I just tried ...
> > > echo 'hw.bge.allow_asf="1"' >> /boot/loader.conf
> > > reboot
> > > load if_bge
> > > acpiconf -s3
> > > same results :(
> > >
> >
> > Ok, here is new try.
> >
>
> Let's get on to it then !
>
> Well now, the situation has improved .. here's what I got:
> http://pastebin.com/f2d152f91
>
> lines 2 to 8 == kldload if_bge
> line 9 == acpiconf -s3
> lines 10 to 18 == resume (notice there are only 2 messages now: lines 12 and
> 13!)
> lines 19 to 36 == ifconfig bge0
> lines 37 to 42 == me mounting the pendrive to get the messages from
> /var/log/messages =P
> lines 43 to 46 == kldunload if_bge0
>
> I think you narrowed it down quite a lot this time !
>
Hmm, not actually. I guess it just removed unnecessary operation
for PHY but it still failed to get functional state. :-(
> I have to warn you though, that this time, my kernel was compiled using a
> ... certainly modified /etc/make.conf file ... just in case you need to know
> how it looks like, you'll fin it in here:
> http://pastebin.com/f42e356d2
>
> If you think that make.conf config may interfere with your pourposes or
> tests, just let me know and I'll use a default one.
I think that's ok.
> The good thing about that make.conf is that it saves me quite a time on
> every recompile ;)
>
> > im at your service .. tell me what to do and I'll do it :)
> > >
> >
> > Thanks a lot for your patience and continuous support to fixing
> > bugs.
> >
>
> Thank _YOU_ for keeping the good work up and for trying to solve a really
> nasty bug that makes every bge(4) user (think of it
> _as_every_dell_notebook_owner_) unable to get his laptop to resume .. or
> even use FreeBSD altoghether just because of this.
>
> Dear Pyung, rest assured that as long as you remain commited to fix this, or
> any other bug that I can help you with, you can count on me to do everything
> that may be within the reach of my hand. As long as you remain commited,
> I'll be there, commited just as well :)
>
Thanks a lot! This really helps me a lot to fix the bug.
Here is 4-th try. I added a couple of debug messages to see what
register contents it have after resume.
Thanks in advance.
Gonzalo Nemmi
Pyung, I have problems applying your patch. 4 out 13 hunks fail.
Take a look in here.
http://pastebin.com/f7eab233f
I tried to recompile anyways but buildkernel failed at bge ...
Can you tell me what to do to apply it cleanly and recompile?
Best Regards
Pyun YongHyeon
Hmm, I guess it was caused by the difference between HEAD and 8.0.
I've uploaded complete bge(4) files to the following URL.
http://people.freebsd.org/~yongari/bge/resume/if_bge.c
http://people.freebsd.org/~yongari/bge/resume/if_bgereg.h
Save your original bge(4) files in sys/dev/bge directory and
download above files and then rebuild.
Hope this helps.
Gonzalo Nemmi
> http://people.freebsd.org/~yongari/bge/resume/if_bge.c<http://people.freebsd.org/%7Eyongari/bge/resume/if_bge.c>
> http://people.freebsd.org/~yongari/bge/resume/if_bgereg.h<http://people.freebsd.org/%7Eyongari/bge/resume/if_bgereg.h>
>
> Save your original bge(4) files in sys/dev/bge directory and
> download above files and then rebuild.
>
> Hope this helps.
>
Dear Pyung:
Did as instructed but the result was negative .. so much so that now the
system doesn't even wake up .. upon resume it just shows the usb messages (
mouse ) and then it freezes .. no more messages ( bge(4) or anything else ),
no coredumps, no nothing .. it just stays there.
The only way to get the system back is to do a hard reset :(
As usuall, awaiting further instructions :)
Pyun YongHyeon
Sorry, I didn't expect such a regression. Here is updated one, hope
this one wouldn't freeze your box.
http://people.freebsd.org/~yongari/bge/resume/if_bge.c
http://people.freebsd.org/~yongari/bge/resume/if_bgereg.h
Gonzalo Nemmi
Don't worry ! No harm done !
I just recompiled with the files you sent me and here's the output:
http://pastebin.com/f414aa4c7
Hope that info is more usefull
Awaiting further orders :D