HFSC / PF limit of 62 / scaling questions
Jason Eggleston
thousands at least) of customers to the Internet using OpenBSD. The best
way I have found so far is HFSC, due to its apparent ability (as much as I
can gather) to overcommit bandwidth with a limit (upperlimit) as opposed to
cbq's general "borrow" idea, and guarantee a minimum throughput.
It became apparent that HFSC is limited to 62 queues per interface, at least
at the top level. Is there any way to change that?
I have included a sample PF file that classifies traffic from customers into
queues, and also a script to generate a config with up to 16K customers.
Unless it's a simple question (for which I'd appreciate a simple answer),
I'd like to have a discussion about it and also offer any coding assistance
if necessary. Algorithmically, nothing hits me as to how this wouldn't be
possible to do in software. It just requires someone interested :)
As it stands, the scaling for the PF rules for classifying the traffic could
be improved by grouping by class C and using anchors, similar to a hash
table with existing functionality in pf, but I didn't put that into the
example below.
I'm essentially trying to use OpenBSD as a Packeteer. I don't subscribe to
te...@openbsd.org, so please CC me on any replies.
problem:
# /home/jason/altqtest.py 63 em0 em1 > pf.conf
# pfctl -f pf.conf
pfctl: DIOCADDALTQ: Cannot allocate memory
# /home/jason/altqtest.py 62 em0 em1 > pf.conf
# pfctl -f pf.conf
#
The pf.conf that fails:
# cat pf.conf
outif = "em0"
inif = "em1"
altq on $outif bandwidth 10Mb hfsc queue { co,\
co1,\
co2,\
co3,\
co4,\
co5,\
co6,\
co7,\
co8,\
co9,\
co10,\
co11,\
co12,\
co13,\
co14,\
co15,\
co16,\
co17,\
co18,\
co19,\
co20,\
co21,\
co22,\
co23,\
co24,\
co25,\
co26,\
co27,\
co28,\
co29,\
co30,\
co31,\
co32,\
co33,\
co34,\
co35,\
co36,\
co37,\
co38,\
co39,\
co40,\
co41,\
co42,\
co43,\
co44,\
co45,\
co46,\
co47,\
co48,\
co49,\
co50,\
co51,\
co52,\
co53,\
co54,\
co55,\
co56,\
co57,\
co58,\
co59,\
co60,\
co61,\
co62,\
co63 }
queue co bandwidth 2Mb hfsc(default)
queue co1 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co2 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co3 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co4 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co5 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co6 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co7 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co8 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co9 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co10 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co11 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co12 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co13 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co14 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co15 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co16 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co17 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co18 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co19 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co20 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co21 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co22 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co23 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co24 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co25 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co26 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co27 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co28 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co29 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co30 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co31 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co32 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co33 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co34 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co35 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co36 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co37 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co38 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co39 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co40 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co41 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co42 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co43 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co44 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co45 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co46 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co47 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co48 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co49 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co50 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co51 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co52 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co53 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co54 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co55 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co56 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co57 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co58 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co59 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co60 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co61 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co62 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue co63 bandwidth 1Kb hfsc(upperlimit 4Mb)
altq on $inif bandwidth 10Mb hfsc queue { ci,\
ci1,\
ci2,\
ci3,\
ci4,\
ci5,\
ci6,\
ci7,\
ci8,\
ci9,\
ci10,\
ci11,\
ci12,\
ci13,\
ci14,\
ci15,\
ci16,\
ci17,\
ci18,\
ci19,\
ci20,\
ci21,\
ci22,\
ci23,\
ci24,\
ci25,\
ci26,\
ci27,\
ci28,\
ci29,\
ci30,\
ci31,\
ci32,\
ci33,\
ci34,\
ci35,\
ci36,\
ci37,\
ci38,\
ci39,\
ci40,\
ci41,\
ci42,\
ci43,\
ci44,\
ci45,\
ci46,\
ci47,\
ci48,\
ci49,\
ci50,\
ci51,\
ci52,\
ci53,\
ci54,\
ci55,\
ci56,\
ci57,\
ci58,\
ci59,\
ci60,\
ci61,\
ci62,\
ci63 }
queue ci bandwidth 2Mb hfsc(default)
queue ci1 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci2 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci3 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci4 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci5 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci6 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci7 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci8 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci9 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci10 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci11 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci12 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci13 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci14 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci15 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci16 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci17 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci18 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci19 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci20 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci21 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci22 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci23 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci24 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci25 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci26 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci27 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci28 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci29 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci30 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci31 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci32 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci33 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci34 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci35 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci36 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci37 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci38 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci39 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci40 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci41 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci42 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci43 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci44 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci45 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci46 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci47 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci48 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci49 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci50 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci51 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci52 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci53 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci54 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci55 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci56 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci57 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci58 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci59 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci60 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci61 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci62 bandwidth 1Kb hfsc(upperlimit 4Mb)
queue ci63 bandwidth 1Kb hfsc(upperlimit 4Mb)
pass out on $outif from 10.0.0.1 to any queue co1
pass out on $inif from any to 10.0.0.1 queue ci1
pass out on $outif from 10.0.0.2 to any queue co2
pass out on $inif from any to 10.0.0.2 queue ci2
pass out on $outif from 10.0.0.3 to any queue co3
pass out on $inif from any to 10.0.0.3 queue ci3
pass out on $outif from 10.0.0.4 to any queue co4
pass out on $inif from any to 10.0.0.4 queue ci4
pass out on $outif from 10.0.0.5 to any queue co5
pass out on $inif from any to 10.0.0.5 queue ci5
pass out on $outif from 10.0.0.6 to any queue co6
pass out on $inif from any to 10.0.0.6 queue ci6
pass out on $outif from 10.0.0.7 to any queue co7
pass out on $inif from any to 10.0.0.7 queue ci7
pass out on $outif from 10.0.0.8 to any queue co8
pass out on $inif from any to 10.0.0.8 queue ci8
pass out on $outif from 10.0.0.9 to any queue co9
pass out on $inif from any to 10.0.0.9 queue ci9
pass out on $outif from 10.0.0.10 to any queue co10
pass out on $inif from any to 10.0.0.10 queue ci10
pass out on $outif from 10.0.0.11 to any queue co11
pass out on $inif from any to 10.0.0.11 queue ci11
pass out on $outif from 10.0.0.12 to any queue co12
pass out on $inif from any to 10.0.0.12 queue ci12
pass out on $outif from 10.0.0.13 to any queue co13
pass out on $inif from any to 10.0.0.13 queue ci13
pass out on $outif from 10.0.0.14 to any queue co14
pass out on $inif from any to 10.0.0.14 queue ci14
pass out on $outif from 10.0.0.15 to any queue co15
pass out on $inif from any to 10.0.0.15 queue ci15
pass out on $outif from 10.0.0.16 to any queue co16
pass out on $inif from any to 10.0.0.16 queue ci16
pass out on $outif from 10.0.0.17 to any queue co17
pass out on $inif from any to 10.0.0.17 queue ci17
pass out on $outif from 10.0.0.18 to any queue co18
pass out on $inif from any to 10.0.0.18 queue ci18
pass out on $outif from 10.0.0.19 to any queue co19
pass out on $inif from any to 10.0.0.19 queue ci19
pass out on $outif from 10.0.0.20 to any queue co20
pass out on $inif from any to 10.0.0.20 queue ci20
pass out on $outif from 10.0.0.21 to any queue co21
pass out on $inif from any to 10.0.0.21 queue ci21
pass out on $outif from 10.0.0.22 to any queue co22
pass out on $inif from any to 10.0.0.22 queue ci22
pass out on $outif from 10.0.0.23 to any queue co23
pass out on $inif from any to 10.0.0.23 queue ci23
pass out on $outif from 10.0.0.24 to any queue co24
pass out on $inif from any to 10.0.0.24 queue ci24
pass out on $outif from 10.0.0.25 to any queue co25
pass out on $inif from any to 10.0.0.25 queue ci25
pass out on $outif from 10.0.0.26 to any queue co26
pass out on $inif from any to 10.0.0.26 queue ci26
pass out on $outif from 10.0.0.27 to any queue co27
pass out on $inif from any to 10.0.0.27 queue ci27
pass out on $outif from 10.0.0.28 to any queue co28
pass out on $inif from any to 10.0.0.28 queue ci28
pass out on $outif from 10.0.0.29 to any queue co29
pass out on $inif from any to 10.0.0.29 queue ci29
pass out on $outif from 10.0.0.30 to any queue co30
pass out on $inif from any to 10.0.0.30 queue ci30
pass out on $outif from 10.0.0.31 to any queue co31
pass out on $inif from any to 10.0.0.31 queue ci31
pass out on $outif from 10.0.0.32 to any queue co32
pass out on $inif from any to 10.0.0.32 queue ci32
pass out on $outif from 10.0.0.33 to any queue co33
pass out on $inif from any to 10.0.0.33 queue ci33
pass out on $outif from 10.0.0.34 to any queue co34
pass out on $inif from any to 10.0.0.34 queue ci34
pass out on $outif from 10.0.0.35 to any queue co35
pass out on $inif from any to 10.0.0.35 queue ci35
pass out on $outif from 10.0.0.36 to any queue co36
pass out on $inif from any to 10.0.0.36 queue ci36
pass out on $outif from 10.0.0.37 to any queue co37
pass out on $inif from any to 10.0.0.37 queue ci37
pass out on $outif from 10.0.0.38 to any queue co38
pass out on $inif from any to 10.0.0.38 queue ci38
pass out on $outif from 10.0.0.39 to any queue co39
pass out on $inif from any to 10.0.0.39 queue ci39
pass out on $outif from 10.0.0.40 to any queue co40
pass out on $inif from any to 10.0.0.40 queue ci40
pass out on $outif from 10.0.0.41 to any queue co41
pass out on $inif from any to 10.0.0.41 queue ci41
pass out on $outif from 10.0.0.42 to any queue co42
pass out on $inif from any to 10.0.0.42 queue ci42
pass out on $outif from 10.0.0.43 to any queue co43
pass out on $inif from any to 10.0.0.43 queue ci43
pass out on $outif from 10.0.0.44 to any queue co44
pass out on $inif from any to 10.0.0.44 queue ci44
pass out on $outif from 10.0.0.45 to any queue co45
pass out on $inif from any to 10.0.0.45 queue ci45
pass out on $outif from 10.0.0.46 to any queue co46
pass out on $inif from any to 10.0.0.46 queue ci46
pass out on $outif from 10.0.0.47 to any queue co47
pass out on $inif from any to 10.0.0.47 queue ci47
pass out on $outif from 10.0.0.48 to any queue co48
pass out on $inif from any to 10.0.0.48 queue ci48
pass out on $outif from 10.0.0.49 to any queue co49
pass out on $inif from any to 10.0.0.49 queue ci49
pass out on $outif from 10.0.0.50 to any queue co50
pass out on $inif from any to 10.0.0.50 queue ci50
pass out on $outif from 10.0.0.51 to any queue co51
pass out on $inif from any to 10.0.0.51 queue ci51
pass out on $outif from 10.0.0.52 to any queue co52
pass out on $inif from any to 10.0.0.52 queue ci52
pass out on $outif from 10.0.0.53 to any queue co53
pass out on $inif from any to 10.0.0.53 queue ci53
pass out on $outif from 10.0.0.54 to any queue co54
pass out on $inif from any to 10.0.0.54 queue ci54
pass out on $outif from 10.0.0.55 to any queue co55
pass out on $inif from any to 10.0.0.55 queue ci55
pass out on $outif from 10.0.0.56 to any queue co56
pass out on $inif from any to 10.0.0.56 queue ci56
pass out on $outif from 10.0.0.57 to any queue co57
pass out on $inif from any to 10.0.0.57 queue ci57
pass out on $outif from 10.0.0.58 to any queue co58
pass out on $inif from any to 10.0.0.58 queue ci58
pass out on $outif from 10.0.0.59 to any queue co59
pass out on $inif from any to 10.0.0.59 queue ci59
pass out on $outif from 10.0.0.60 to any queue co60
pass out on $inif from any to 10.0.0.60 queue ci60
pass out on $outif from 10.0.0.61 to any queue co61
pass out on $inif from any to 10.0.0.61 queue ci61
pass out on $outif from 10.0.0.62 to any queue co62
pass out on $inif from any to 10.0.0.62 queue ci62
pass out on $outif from 10.0.0.63 to any queue co63
pass out on $inif from any to 10.0.0.63 queue ci63
$ dmesg
OpenBSD 3.5-current (GENERIC) #270: Sun Aug 1 16:16:36 MDT 2004
der...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel(R) Xeon(TM) CPU 2.40GHz ("GenuineIntel" 686-class) 2.40 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLU
SH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,CNXT-ID
real mem = 1073324032 (1048168K)
avail mem = 987529216 (964384K)
using 4278 buffers containing 53768192 bytes (52508K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(00) BIOS, date 11/18/03, BIOS32 rev. 0 @ 0xf0010
pcibios0 at bios0: rev. 2.1 @ 0xf0000/0x10000
pcibios0: PCI IRQ Routing Table rev. 1.0 @ 0xf47e0/256 (14 entries)
pcibios0: no compatible PCI ICU found: ICU vendor 0x8086 product 0x2480
pcibios0: Warning, unable to fix up PCI interrupt routing
pcibios0: PCI bus #7 is the last bus
bios0: ROM list: 0xc0000/0x8000 0xc8000/0x3e00 0xcc000/0x1000
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 vendor "Intel", unknown product 0x254c rev
0x01
ppb0 at pci0 dev 2 function 0 "Intel E7500 MCH HI_B vppb 1" rev 0x01
pci1 at ppb0 bus 1
"Intel P64H2 IOxAPIC" rev 0x04 at pci1 dev 28 function 0 not configured
ppb1 at pci1 dev 29 function 0 "Intel 82870P2 P64H2 PCI-PCI" rev 0x04
pci2 at ppb1 bus 2
em0 at pci2 dev 1 function 0 "Intel PRO/1000MT DP (82546EB)" rev 0x01: irq
9, address: 00:e0:81:26:4c:de
em1 at pci2 dev 1 function 1 "Intel PRO/1000MT DP (82546EB)" rev 0x01: irq
9, address: 00:e0:81:26:4c:df
"Intel P64H2 IOxAPIC" rev 0x04 at pci1 dev 30 function 0 not configured
ppb2 at pci1 dev 31 function 0 "Intel 82870P2 P64H2 PCI-PCI" rev 0x04
pci3 at ppb2 bus 3
ppb3 at pci0 dev 3 function 0 "Intel E7500 MCH HI_C vppb 1" rev 0x01
pci4 at ppb3 bus 4
"Intel P64H2 IOxAPIC" rev 0x04 at pci4 dev 28 function 0 not configured
ppb4 at pci4 dev 29 function 0 "Intel 82870P2 P64H2 PCI-PCI" rev 0x04
pci5 at ppb4 bus 5
em2 at pci5 dev 3 function 0 "Intel PRO/1000MF DP (82546EB)" rev 0x03: irq
7, address: 00:04:23:a1:7b:ac
em3 at pci5 dev 3 function 1 "Intel PRO/1000MF DP (82546EB)" rev 0x03: irq
7, address: 00:04:23:a1:7b:ad
"Intel P64H2 IOxAPIC" rev 0x04 at pci4 dev 30 function 0 not configured
ppb5 at pci4 dev 31 function 0 "Intel 82870P2 P64H2 PCI-PCI" rev 0x04
pci6 at ppb5 bus 6
em4 at pci6 dev 1 function 0 "Intel PRO/1000MT DP (82546EB)" rev 0x01: irq
7, address: 00:e0:81:26:4c:dc
em5 at pci6 dev 1 function 1 "Intel PRO/1000MT DP (82546EB)" rev 0x01: irq
7, address: 00:e0:81:26:4c:dd
uhci0 at pci0 dev 29 function 0 "Intel 82801CA/CAM USB" rev 0x02: irq 9
usb0 at uhci0: USB revision 1.0
uhub0 at usb0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
ppb6 at pci0 dev 30 function 0 "Intel 82801BA AGP" rev 0x42
pci7 at ppb6 bus 7
fxp0 at pci7 dev 1 function 0 "Intel 82557" rev 0x10: irq 5, address
00:e0:81:26:50:b4
inphy0 at fxp0 phy 1: i82555 10/100 media interface, rev. 4
vga1 at pci7 dev 2 function 0 "ATI Rage XL" rev 0x27
wsdisplay0 at vga1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pciide0 at pci7 dev 3 function 0 "HighPoint HPT372A IDE" rev 0x02: DMA
pciide0: using irq 11 for native-PCI interrupt
wd0 at pciide0 channel 0 drive 0: <Maxtor 6Y080M0>
wd0: 16-sector PIO, LBA, 78167MB, 160086528 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 6
wd1 at pciide0 channel 1 drive 0: <Maxtor 6Y080M0>
wd1: 16-sector PIO, LBA, 78167MB, 160086528 sectors
wd1(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 6
ichpcib0 at pci0 dev 31 function 0 "Intel 82801CA LPC" rev 0x02
pciide1 at pci0 dev 31 function 1 "Intel 82801CA IDE" rev 0x02: DMA, channel
0 configured to compatibility, channel 1 configured to compatibility
pciide1: channel 0 disabled (no drives)
atapiscsi0 at pciide1 channel 1 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: <TEAC, CD-224E, 1.9A> SCSI0 5/cdrom removable
cd0(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 2
"Intel 82801CA/CAM SMBus" rev 0x02 at pci0 dev 31 function 3 not configured
isa0 at ichpcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pmsi0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pmsi0 mux 0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
sysbeep0 at pcppi0
lm0 at isa0 port 0x290/8: W83627HF
npx0 at isa0 port 0xf0/16: using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
biomask ef45 netmask efe5 ttymask ffe7
pctr: user-level cycle counter enabled
dkcsum: wd0 matched BIOS disk 80
wd1: no disk label
dkcsum: wd1 matched BIOS disk 81
root on wd0a
rootdev=0x0 rrootdev=0x300 rawdev=0x302
-----altqtest.py **********************
#!/usr/local/bin/python
import sys
# number of customers to simulate
numcusts = int(sys.argv[1])
#output interface
outif = sys.argv[2]
#input interface
inif = sys.argv[3]
def altqdef(interface, basename):
print "altq on %s bandwidth 10Mb hfsc queue { %s,\\" % \
(interface, basename)
for n in xrange(1, numcusts):
print "\t%s%d,\\" % (basename, n)
print "\t%s%d }\n" % (basename, n+1)
# define customer queues
print "queue %s bandwidth 2Mb hfsc(default)" % basename
for n in xrange(1, numcusts+1):
print "queue %s%d bandwidth 1Kb hfsc(upperlimit 4Mb)" \
% (basename, n)
print
# enable queueing with references to customer queues
print 'outif = "%s"' % outif
print 'inif = "%s"' % inif
print
altqdef("$outif", "co")
altqdef("$inif", "ci")
# classify customer traffic in one direction
for n in xrange(1, numcusts+1):
c = n >> 8;
d = n % 256;
print "pass out on $outif from 10.0.%d.%d to any queue co%d" \
% (c, d, n)
print "pass out on $inif from any to 10.0.%d.%d queue ci%d" \
% (c, d, n)
Thanks,
-Jason
Jason Eggleston
Do you (or anyone else) know of any plans to implement a scalable algorithm
or any show-stopper theoretical problems with doing it?
Thanks,
-Jason
-----Original Message-----
From: Kenjiro Cho [mailto:k...@iijlab.net]
Sent: Tuesday, August 03, 2004 10:27 AM
To: jeggl...@nextweb.net
Cc: te...@openbsd.org
Subject: Re: HFSC / PF limit of 62 / scaling questions
Jason Eggleston wrote:
> I am looking for a way to rate limit thousands (preferably in the low
> thousands at least) of customers to the Internet using OpenBSD. The best
> way I have found so far is HFSC, due to its apparent ability (as much as I
> can gather) to overcommit bandwidth with a limit (upperlimit) as opposed
to
> cbq's general "borrow" idea, and guarantee a minimum throughput.
>
> It became apparent that HFSC is limited to 62 queues per interface, at
least
> at the top level. Is there any way to change that?
you can increase the max number of classes by modifying the following
line in sys/altq/altq_hfsc.h.
#define HFSC_MAX_CLASSES 64
but HFSC currently uses linear algorithms to access different queues
so that it would work with hundreds of queues with a high-end CPU but
would not scale to thousands of queues.
-Kenjiro
Kenjiro Cho
Jason Eggleston
bandwidth, at least in -current, so that may suit our purposes. Although I
still hope there are plans to make hfsc scalable, as I see the IPFW2 rate
limiting is claimed to be O(log(n)) and hfsc appears to be much better than
cbq.
Thanks,
-Jason
-----Original Message-----
From: Jason Eggleston [mailto:jeggl...@nextweb.net]
Sent: Tuesday, August 03, 2004 11:10 AM
To: 'Kenjiro Cho'
Cc: 'te...@openbsd.org'
Subject: RE: HFSC / PF limit of 62 / scaling questions
Thank you for the information.
Do you (or anyone else) know of any plans to implement a scalable algorithm
or any show-stopper theoretical problems with doing it?
Thanks,
-Jason
-----Original Message-----
From: Kenjiro Cho [mailto:k...@iijlab.net]
Sent: Tuesday, August 03, 2004 10:27 AM
To: jeggl...@nextweb.net
Cc: te...@openbsd.org
Subject: Re: HFSC / PF limit of 62 / scaling questions