dummynet issues

[Kevin Smith]

Hi,

I'm experiencing some dummynet issues after upgrading from 7-STABLE to
8.0-RELEASE.
My /var/log/messages is full of these logs:

Nov 29 15:34:18 stone kernel: dummynet: OUCH! pipe should have been idle!
Nov 29 15:34:49 stone last message repeated 409 times
Nov 29 15:36:49 stone last message repeated 1595 times
Nov 29 15:46:50 stone last message repeated 8162 times
Nov 29 15:56:51 stone last message repeated 7099 times
Nov 29 16:06:52 stone last message repeated 4771 times
Nov 29 16:16:53 stone last message repeated 3859 times
Nov 29 16:26:54 stone last message repeated 3493 times
Nov 29 16:36:55 stone last message repeated 5874 times

Also I noticed that traffic shaping is not working any longer , i.e.:
actually outgoing pipes do not limit bandwidth at all.
Until 8 Release upgrading the same configuration was working perfectly.

This is my uname -a

FreeBSD stone.it 8.0-RELEASE FreeBSD 8.0-RELEASE #5: Sat Nov 28 20:22:30
CET 2009 ke...@stone.it:/usr/obj/usr/src/sys/STONE i386

Attached my dmesg.boot and my kernel configuration.

Is anybody experiencing same issues?
Thank you,
regards,

--
Kevin

[Kevin Smith]

Alex Almeida wrote:
> Hi,
>
> The same happened with me, just by setting:
> net.inet.ip.fw.one_pass: 0
>
> And stopped the message, however I was using version 6.4.
>
> I hope that helps you,
>
> Hugs
>
> Alex Almeida
>
>
>
>
> Kevin Smith escreveu:

>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> freebs...@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
>> To unsubscribe, send any mail to "freebsd-ipfw...@freebsd.org"
>
Hi,

I've already net.inet.ip.fw.one_pass set to 0, and also setting it to 1,
even if this is not what I need, does not fix.
Thank you anyway,
regards,

--
Kevin

[Alex Almeida]

Hi,

The same happened with me, just by setting:
net.inet.ip.fw.one_pass: 0

And stopped the message, however I was using version 6.4.

I hope that helps you,

Hugs

Alex Almeida

Kevin Smith escreveu:

[Mailing LIst Member]

I know this may be a rediculous question, given the audience, however, I
will inquire anyhow.

"Have you verified that your kernel has installed cleanly?"

I am still on 7.2-Stable, on my production servers, however, when I have
attempted hasty upgrades, and reinstalled the kernel after compiling it
on the new system, it required a second iteration of cleaning, and
recompiling/installing for the kernel to recognize the net options that
are being referenced.

Hence, when I initially compiled and installed the kernel, I had to
repeat the process a second time, to see all firewalling activated
correctly.(pipes, and other rules)

Of course, as previously indicated, this may be just a new bug for the
newer system, however, I would try that first. Also, make sure you port
the new version of your rules/config file to the 8-Release branch. I
have had trouble going between 6 and 7 with some of these rules not
being recognized but the option compiled.

I can't remember what that was, for a good example, however, just a few
things to investigate.

I apologize if these options have already been investigated.

Respectfully,

Martes

[Kevin Smith]

Hi,

thank you for your answer, but everything seems quite fine with
8-RELEASE except for this issue.
I tried to recompile without SMP or PREEMPTION options and also to apply
oleg@ 's patch at
http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/netinet/ipfw/ip_dummynet.c?rev=1.5.2.2;content-type=text%2Fplain
committed on RELENG_8 with no luck.
Upgrading was fine, I can say that all has been installed cleanly.

[Oleg Bulyzhin]

> _______________________________________________
> freebsd...@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-curre...@freebsd.org"

Please show your pipe/queue configuration commands and your ipfw ruleset.
sysctl net.inet.ip.fw & sysctl net.inet.ip.dummynet output would not hurt too.

--
Oleg.

================================================================
=== Oleg Bulyzhin -- OBUL-RIPN -- OBUL-RIPE -- ol...@rinet.ru ===
================================================================

[Ben Kelly]

On Nov 30, 2009, at 3:12 PM, Oleg Bulyzhin wrote:
> Please show your pipe/queue configuration commands and your ipfw ruleset.
> sysctl net.inet.ip.fw & sysctl net.inet.ip.dummynet output would not hurt too.

I've also run into the problem recently on 9-CURRENT (last synced on 11/13/2009). My configuration looks like:

# Configure traffic shaping.
$fw pipe 10 config bw 950Kbit/s
$fw queue 10 config pipe 10 weight 100
$fw queue 20 config pipe 10 weight 1

# Shape traffic to avoid ACK starvation when our upload is saturated.
$fw add 6100 queue 10 tcp from any to any tcpflags ack iplen 0-80 out via $oif
$fw add 6110 queue 10 udp from any to any iplen 0-80 out via $oif
$fw add 6120 queue 20 tcp from any to any \{ not tcpflags ack or not iplen 0-80 \} out via $oif
$fw add 6130 queue 20 udp from any to any not iplen 0-80 out via $oif

The output of the sysctl elements are:

gate# sysctl net.inet.ip.fw
net.inet.ip.fw.dyn_keepalive: 1
net.inet.ip.fw.dyn_short_lifetime: 5
net.inet.ip.fw.dyn_udp_lifetime: 10
net.inet.ip.fw.dyn_rst_lifetime: 1
net.inet.ip.fw.dyn_fin_lifetime: 1
net.inet.ip.fw.dyn_syn_lifetime: 20
net.inet.ip.fw.dyn_ack_lifetime: 300
net.inet.ip.fw.static_count: 42
net.inet.ip.fw.dyn_max: 4096
net.inet.ip.fw.dyn_count: 232
net.inet.ip.fw.curr_dyn_buckets: 256
net.inet.ip.fw.dyn_buckets: 256
net.inet.ip.fw.default_to_accept: 0
net.inet.ip.fw.tables_max: 128
net.inet.ip.fw.default_rule: 65535
net.inet.ip.fw.verbose_limit: 0
net.inet.ip.fw.verbose: 0
net.inet.ip.fw.one_pass: 0
net.inet.ip.fw.autoinc_step: 100
net.inet.ip.fw.enable: 1
gate# sysctl net.inet.ip.dummynet
net.inet.ip.dummynet.debug: 0
net.inet.ip.dummynet.pipe_byte_limit: 1048576
net.inet.ip.dummynet.pipe_slot_limit: 100
net.inet.ip.dummynet.io_pkt_drop: 1601
net.inet.ip.dummynet.io_pkt_fast: 146359
net.inet.ip.dummynet.io_pkt: 26208842
net.inet.ip.dummynet.io_fast: 0
net.inet.ip.dummynet.tick_lost: 0
net.inet.ip.dummynet.tick_diff: 1352176
net.inet.ip.dummynet.tick_adjustment: 239751
net.inet.ip.dummynet.tick_delta_sum: -494
net.inet.ip.dummynet.tick_delta: 1
net.inet.ip.dummynet.red_max_pkt_size: 1500
net.inet.ip.dummynet.red_avg_pkt_size: 512
net.inet.ip.dummynet.red_lookup_depth: 256
net.inet.ip.dummynet.max_chain_len: 16
net.inet.ip.dummynet.expire: 1
net.inet.ip.dummynet.search_steps: 0
net.inet.ip.dummynet.searches: 0
net.inet.ip.dummynet.extract_heap: 16
net.inet.ip.dummynet.ready_heap: 0
net.inet.ip.dummynet.hash_size: 64

Thanks for the help.

- Ben_______________________________________________

[Kevin Smith]

Hi,

this is my pipe/queue configuration:

/sbin/ipfw pipe 1 config bw 256kbits/s
/sbin/ipfw queue 3 config pipe 1 weight 40 mask all
/sbin/ipfw queue 4 config pipe 1 weight 50 mask all
/sbin/ipfw add queue 3 all from any to any out via tun\? uid asterisk
/sbin/ipfw add queue 3 all from any to any 80 out via tun\?
/sbin/ipfw add queue 3 all from any to any 53 out via tun\?
/sbin/ipfw add queue 3 all from me 4300 to any out via tun\?
/sbin/ipfw add queue 3 all from me 1194 to any out via tun\?
/sbin/ipfw add queue 4 all from any to any out via tun\? tcpflags
\!syn,ack not jail ${MLDONKEYJID:=1}
/sbin/ipfw add queue 4 all from any to any out via tun\? not jail
${MLDONKEYJID:=1}
/sbin/ipfw queue 1 config pipe 1 weight 1 gred 0.8/16/39/1 mask all
/sbin/ipfw queue 2 config pipe 1 weight 2 gred 0.02/3/6/0.06 mask all
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 40
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 41
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 42
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 43
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 44
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 45
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 46
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 47
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 48
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 49
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 50
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 51
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 52
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 53
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 55
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 56
/sbin/ipfw add queue 2 all from any to any out via tun\? iplen 57
/sbin/ipfw add queue 1 all from any to any out via tun\? jail
${MLDONKEYJID:=1}

and these are some system settings:

net.inet.ip.dummynet.debug: 0
net.inet.ip.dummynet.pipe_byte_limit: 1048576
net.inet.ip.dummynet.pipe_slot_limit: 100

net.inet.ip.dummynet.io_pkt_drop: 1316
net.inet.ip.dummynet.io_pkt_fast: 146311
net.inet.ip.dummynet.io_pkt: 3006844
net.inet.ip.dummynet.io_fast: 0
net.inet.ip.dummynet.tick_lost: 0
net.inet.ip.dummynet.tick_diff: 18983852
net.inet.ip.dummynet.tick_adjustment: 17727039
net.inet.ip.dummynet.tick_delta_sum: 453
net.inet.ip.dummynet.tick_delta: 1000

net.inet.ip.dummynet.red_max_pkt_size: 1500
net.inet.ip.dummynet.red_avg_pkt_size: 512
net.inet.ip.dummynet.red_lookup_depth: 256
net.inet.ip.dummynet.max_chain_len: 16
net.inet.ip.dummynet.expire: 1

net.inet.ip.dummynet.search_steps: 3047766
net.inet.ip.dummynet.searches: 3006844

net.inet.ip.dummynet.extract_heap: 16
net.inet.ip.dummynet.ready_heap: 0
net.inet.ip.dummynet.hash_size: 64

net.inet.ip.fw.dyn_keepalive: 1

net.inet.ip.fw.dyn_short_lifetime: 5
net.inet.ip.fw.dyn_udp_lifetime: 10
net.inet.ip.fw.dyn_rst_lifetime: 1
net.inet.ip.fw.dyn_fin_lifetime: 1
net.inet.ip.fw.dyn_syn_lifetime: 20
net.inet.ip.fw.dyn_ack_lifetime: 300

net.inet.ip.fw.static_count: 68
net.inet.ip.fw.dyn_max: 4096
net.inet.ip.fw.dyn_count: 0
net.inet.ip.fw.curr_dyn_buckets: 256
net.inet.ip.fw.dyn_buckets: 256
net.inet.ip.fw.default_to_accept: 1

net.inet.ip.fw.tables_max: 128
net.inet.ip.fw.default_rule: 65535
net.inet.ip.fw.verbose_limit: 0

net.inet.ip.fw.verbose: 1

net.inet.ip.fw.one_pass: 0
net.inet.ip.fw.autoinc_step: 100
net.inet.ip.fw.enable: 1

Please don't hexitate to ask me for further infos.
Thank you for your help,
regards,

--
Kevin

[Oleg Bulyzhin]

On Mon, Nov 30, 2009 at 03:38:50PM -0500, Ben Kelly wrote:
>
> I've also run into the problem recently on 9-CURRENT (last synced on 11/13/2009). My configuration looks like:
>

My quick attempt to reproduce the issue failed. Perhaps i'm missing something.

How are you measuring connection bandwidth?

[Ben Kelly]

On Nov 30, 2009, at 6:45 PM, Oleg Bulyzhin wrote:

> On Mon, Nov 30, 2009 at 03:38:50PM -0500, Ben Kelly wrote:
>>
>> I've also run into the problem recently on 9-CURRENT (last synced on 11/13/2009). My configuration looks like:
>>
>
> My quick attempt to reproduce the issue failed. Perhaps i'm missing something.
>
> How are you measuring connection bandwidth?

I actually have not measured my bandwidth to validate dummynet. I have simply observed these messages repeating in my log:

dummynet: OUCH! pipe should have been idle!

Under normal conditions I don't really need the dummynet rules to shape traffic for my configuration to work, so it has not been a high priority for me yet. Do you see the log messages?

Thanks.

- Ben_______________________________________________

freebs...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw

To unsubscribe, send any mail to "freebsd-ipfw...@freebsd.org"

[Oleg Bulyzhin]

On Mon, Nov 30, 2009 at 11:58:55PM -0500, Ben Kelly wrote:
>
> I actually have not measured my bandwidth to validate dummynet. I have simply observed these messages repeating in my log:
>
> dummynet: OUCH! pipe should have been idle!
>
> Under normal conditions I don't really need the dummynet rules to shape traffic for my configuration to work, so it has not been a high priority for me yet. Do you see the log messages?
>
> Thanks.
>
> - Ben

It seems i've found the problem. Please test attached patch (it's for R8.0
sources and include r198845). I'm interested in some feedback:
1) does it solve 'OUCH' messages problem?
2) does it solve bandwidth problem (if there was any)?

[Ben Kelly]

On Dec 1, 2009, at 2:33 PM, Kevin Smith wrote:

> Oleg Bulyzhin wrote:
>> On Mon, Nov 30, 2009 at 11:58:55PM -0500, Ben Kelly wrote:
>>> I actually have not measured my bandwidth to validate dummynet. I have simply observed these messages repeating in my log:
>>>
>>> dummynet: OUCH! pipe should have been idle!
>>>
>>> Under normal conditions I don't really need the dummynet rules to shape traffic for my configuration to work, so it has not been a high priority for me yet. Do you see the log messages?
>>>
>>> Thanks.
>>>
>>> - Ben
>>
>> It seems i've found the problem. Please test attached patch (it's for R8.0
>> sources and include r198845). I'm interested in some feedback:
>> 1) does it solve 'OUCH' messages problem?
>> 2) does it solve bandwidth problem (if there was any)?
>>
>>

> The patch fixes the problem: now it seems all ok, no more "OUCH"
> messages and pipe bandwidth limiting works again.
> Thank you very much, Oleg!!
> Best regards,

I just verified that it got rid of the log messages for me as well. I still haven't actually measured the dummynet bandwidth, though.

For reference, I used only this part of the patch against 9-CURRENT since the rest seemed to already be applied:

Index: sys/netinet/ipfw/ip_dummynet.c
===================================================================
--- sys/netinet/ipfw/ip_dummynet.c (revision 252)
+++ sys/netinet/ipfw/ip_dummynet.c (working copy)
@@ -1426,7 +1426,9 @@
q->numbytes += pipe->bandwidth;
}
} else { /* WF2Q. */
- if (pipe->idle_time < curr_time) {
+ if (pipe->idle_time < curr_time &&
+ pipe->scheduler_heap.elements == 0 &&
+ pipe->not_eligible_heap.elements == 0) {
/* Calculate available burst size. */
pipe->numbytes +=
(curr_time - pipe->idle_time - 1) * pipe->bandwidth;

Thanks for the quick fix Oleg!

[Kevin Smith]

Oleg Bulyzhin wrote:
> On Mon, Nov 30, 2009 at 11:58:55PM -0500, Ben Kelly wrote:
>> I actually have not measured my bandwidth to validate dummynet. I have simply observed these messages repeating in my log:
>>
>> dummynet: OUCH! pipe should have been idle!
>>
>> Under normal conditions I don't really need the dummynet rules to shape traffic for my configuration to work, so it has not been a high priority for me yet. Do you see the log messages?
>>
>> Thanks.
>>
>> - Ben
>
> It seems i've found the problem. Please test attached patch (it's for R8.0
> sources and include r198845). I'm interested in some feedback:
> 1) does it solve 'OUCH' messages problem?
> 2) does it solve bandwidth problem (if there was any)?
>
>

The patch fixes the problem: now it seems all ok, no more "OUCH"
messages and pipe bandwidth limiting works again.
Thank you very much, Oleg!!
Best regards,

--
Kevin

[Julian Elischer]

this should be made an errata item for 8.0

[Oleg Bulyzhin]

On Tue, Dec 01, 2009 at 12:48:43PM -0800, Julian Elischer wrote:

> this should be made an errata item for 8.0

I'm not sure about the procedure, should i contact re@ team?