traffic bandwidth limit with dummynet

[bored to death]

hello,

i'm trying to limit my input traffic bandwidth on freebsd. i used
ipfw+dummynet. without limitation, i have almost 1Gbit/s input traffic
on my system. when i try to limit the bandwidth, it works fine on low
to normal limitation number (up to 450 Mbit/s).
but when i limit my traffic to an amount bigger than 500MBit/s, my
input traffic lowers down to average of 430 MBit/s. no matter what
limitations i set(from 500MBit/s to 1000MBit/s) my input traffic rate
is getting stucked at 430MBit/s.

i suspected low hardware performance, so i tried this on 2 different
systems with freebsd-8.0 and the result is exactly the same.

my ipfw rules are:

Code:
ipfw -q flush
ipfw -q add 500 set 1 pipe 1 all from any to any in
ipfw pipe 1 config bw 700Mbit/s
ipfw add 1000 allow all from any to anyso my question is:
1- is ipfw+dummynet traffic limit control, has performance problem that
cannot limit traffic correctly if limitation number is bigger than
500MBit/s?
2- is there any other way (other than ipfw+dummynet) to limit input bandwidth in freebsd?

thank you.

_______________________________________________
freebs...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw...@freebsd.org"

[Luigi Rizzo]

On Thu, Jun 03, 2010 at 09:29:20AM -0700, bored to death wrote:
> hello,
>
> i'm trying to limit my input traffic bandwidth on freebsd. i used
> ipfw+dummynet. without limitation, i have almost 1Gbit/s input traffic
> on my system. when i try to limit the bandwidth, it works fine on low
> to normal limitation number (up to 450 Mbit/s).
> but when i limit my traffic to an amount bigger than 500MBit/s, my
> input traffic lowers down to average of 430 MBit/s. no matter what
> limitations i set(from 500MBit/s to 1000MBit/s) my input traffic rate
> is getting stucked at 430MBit/s.

try to run the kernel with higher value of HZ (in /boot/loader.conf

kern.hz=4000

and see if the situation improves.
I think the symptoms you are seeing depend on the fact that
packets are released every 1/HZ seconds probably causing some
queue overflow at high packet rates and [relatively] low HZ
values.

cheers
luigi

[bored to death]

thank you luigi for your reply, it helped.

i changed the hz parameter to 1000 and then 4000 and then 8000 in my /boot/loader.conf. the result got much better.
i configured my system as a router and i send 1GB traffic rate passing by it and set an 800MBytes bandwidth limit on input traffic with dummynet.
this was the result:
with hz=1 (default) between 200MBytes/s and 300MBytes/s
with hz=1000 between 200MBytes/s and 300MBytes/s
with hz=4000 between 350MBytes/s and 450MBytes/s
with hz=8000 between 250MBytes/s and 550MBytes/s

the maximum traffic rate is got so much better, but 2 problems still remain:
1- the maximum rate is still not high enough.
2- the rate variation range is high (250-550) and it's not a steady enough.

i've also tried setting different "queue" and "burst" values for the pipe. the result is a little better when i set "queue" to a value between 80MBytes and 90MBytes and "burst" to a big number.

any other ideas?

________________________________
From: Luigi Rizzo <ri...@iet.unipi.it>
To: bored to death <bored_to...@yahoo.com>
Cc: freebsd-ipfw <freebs...@freebsd.org>
Sent: Thu, June 3, 2010 9:57:33 PM
Subject: Re: traffic bandwidth limit with dummynet

[Luigi Rizzo]

On Fri, Jun 04, 2010 at 01:19:32AM -0700, bored to death wrote:
> thank you luigi for your reply, it helped.
>
> i changed the hz parameter to 1000 and then 4000 and then 8000 in my /boot/loader.conf. the result got much better.
> i configured my system as a router and i send 1GB traffic rate passing by it and set an 800MBytes bandwidth limit on input traffic with dummynet.
> this was the result:
> with hz=1 (default) between 200MBytes/s and 300MBytes/s
> with hz=1000 between 200MBytes/s and 300MBytes/s
> with hz=4000 between 350MBytes/s and 450MBytes/s
> with hz=8000 between 250MBytes/s and 550MBytes/s
>
> the maximum traffic rate is got so much better, but 2 problems still remain:
> 1- the maximum rate is still not high enough.
> 2- the rate variation range is high (250-550) and it's not a steady enough.
>
> i've also tried setting different "queue" and "burst" values for the pipe. the result is a little better when i set "queue" to a value between 80MBytes and 90MBytes and "burst" to a big number.
>
> any other ideas?
>

HZ=1000 is the default, for the records.
Setting the burst size should have no practical effects,
whereas setting the queue size e.g.
o
ipfw pipe 10 config bw 800Mbit/s queue 200kbytes

should help a lot, but check your configuration with 'ipfw pipe show'
because if you supply an invalid parameter ipfw silently uses
a default or something different.
As an example, you said you used 80-90 Mbytes but the max queue
size is 100 packets or 1023Kbytes and larger values do not produce
the desired effect.

As a rule of thumb, to make sure that drops are not caused
by short queues, you should set the queue size to 1/HZ seconds
worth of data -- at HZ=1000 and 1Gbit/s this means 128Kbytes.
Note that after the dummynet queue, there might be some other
queue that saturates. As an example, when using the box as a router,
packets go in bursts to the output interface, and the burst can
be as large as 1500 packets per tick on a fully saturated Gig-E
(the interface's queue ranges normally between 128 and 1024 slots).
The only fix for this is probably using higher values of HZ.

chers
luigi

[bored to death]

thank you luigi,

your explanation really cleared everything out for me.

i changed my pipe 1 config to:
ipfw pipe 1 config bw 800Mbits/s queue 200K

and set HZ to 4000

and this solved my problem completely.
i checked limitations with various values between 400Mbits/s to more than 1000Mbits/s and it works like a charm.

(the problem was when i set queue to 80MBytes, queue value was actually set to "80 slots")

thanks again luigi.

________________________________
From: Luigi Rizzo <ri...@iet.unipi.it>
To: bored to death <bored_to...@yahoo.com>
Cc: freebsd-ipfw <freebs...@freebsd.org>

Sent: Fri, June 4, 2010 4:27:25 PM

Subject: Re: traffic bandwidth limit with dummynet

On Fri, Jun 04, 2010 at 01:19:32AM -0700, bored to death wrote:

[Luigi Rizzo]

On Fri, Jun 04, 2010 at 05:25:54AM -0700, bored to death wrote:
> thank you luigi,
>
> your explanation really cleared everything out for me.
>
> i changed my pipe 1 config to:
> ipfw pipe 1 config bw 800Mbits/s queue 200K
>
> and set HZ to 4000
>
> and this solved my problem completely.

glad it helped

luigi

[Brandon Gooch]

Thanks for the explanation Luigi, this is bound to help others in
regard to queue configuration.

-Brandon