AltQ
Kenjiro Cho
CCed to Luigi (author of dummynet) and the altq mailing-list.
There are people concerned about ALTQ's modifying every driver and
prefer dummynet for that reason. But dummynet and ALTQ achieve
different goals.
Let me clarify 2 things:
1. architectural difference between dummynet and ALTQ
2. why ALTQ needs to modify drivers
dummynet does packet shaping whereas ALTQ does packet scheduling.
- packet shaping: transmission with a constant rate
- packet scheduling: dynamic selection of packets to transmit
dummynet:
- implemented in the network (IP) layer
- cleaner implementation (no need to modify drivers)
- uses timers to delay packets
ALTQ:
- implemented at the network interface level
- modifies drivers not to spoil packet scheduling
- is able to make use of interrupts
There are 2 issues to support packet scheduling effectively:
- a packet scheduler need to make use of transmission complete
interrupts to trigger scheduling
if_done hook was once added to struct ifnet in 4.3-reno for
this reason but it was never supported by drivers.
- drivers should not buffer too many packets
unfortunately, many network drivers buffer packets as many as
possible, which creates a long waiting queue after packets are
scheduled.
Ideally, packet scheduling is cleanly implemented if all drivers are
rewritten to meet the above 2 requirements.
ALTQ tries to identify points to fix in drivers.
-Kenjiro
Thierry Deval
On 26-Jan-00 Chris Cappuccio wrote:
> I ported ALTQ to OpenBSD several months ago, based mostly on the code used to
> run ALTQ on NetBSD. I submitted this back to the author, through his work it
> evolved along with several other parts of ALTQ to become version 2.1.
Yes Chris, I remember seeing it on the list someday (on Oct 12th 1999), but
it is not a real port (in the port tree) yet, as it needs kernel rebuild. (No
offense)
And indeed, in AltQ 2.1, they support several cards based on stock 2.6 .
> I asked Kenjiro Cho, author of ALTQ, why not do its work at ether_input() and
> ether_output(), and he said:
>
> "Packet scheduling should be done from a driver interrupts (tx complete
> interrupts) and it is if_start (xx_start in the driver) in the BSD systems."
Kenjiro told us about the interrupts, and also about a if_done hook (wich is
obviously found nowhere, except for ip_rsvp_vif_done in netinet mroute.[ch] )
Then, I would suggest envisaging the integration of AltQ into the base source
(despite Theo's reluctancy to change the network drivers' API), at least the
kernel parts.
For the userland parts, we'll have to see (on the long run) if we can get rid
of an altqd daemon, and try to merge the controls with ipfilter (which, btw, is
not IP centric only :)
Thierry
Chris Cappuccio
run ALTQ on NetBSD. I submitted this back to the author, through his work it
evolved along with several other parts of ALTQ to become version 2.1.
I asked Kenjiro Cho, author of ALTQ, why not do its work at ether_input() and
ether_output(), and he said:
"Packet scheduling should be done from a driver interrupts (tx complete
interrupts) and it is if_start (xx_start in the driver) in the BSD systems."
On Mon, 24 Jan 2000, Thierry Deval wrote:
| OK, my best best would be to base alt-q on bpf, or maybe just under it.
|
| I'll look at all this HUGE networking code, and see were AltQ could fit.
| Any suggestion from you gurus are welcome. And when I come back with something,
| there will certainly be a need for deep cleaning ;-)
|
| CYA
|
| Thierry
|
| On 24-Jan-00 a...@eltex.ru wrote:
| > -----BEGIN PGP SIGNED MESSAGE-----
| >
| > nuqneH,
| >
| > That's the reason i never use AltQ on my machines: i just don't like the
| > way it works and this way creates enormous amount of kernel changes.
| >
| > There is a thing that looks much better, called dummynet, but it is
| > freebsd-only and even worse, requires ipfw to work. There were some
| > discussions about porting it to ipfilter, but nobody actually tried..
| >
| > We have no acceptable traffic shaper for now :(
| >
| > Theo de Raadt <der...@cvs.openbsd.org> said :
| >
| >> > Is there any project of integrating the AltQ development
| >> > (http://www.csl.sony.co.jp/~kjc/software.html) into our source tree ?
| >> >
| >> > I think it might be a usefull addition to the system.
| >> > It may be of great interest to ISPs and small-to-large office managers :-)
| >>
| >> I am concerned that the current code will require changes to every single
| >> network device driver in the source tree.
| >>
| >
| >
| > _ _ _ _ _ _ _
| > {::} {::} {::} CU in Hell _| o |_ | | _|| | / _||_| |_ |_ |_
| > (##) (##) (##) /Arkan#iD |_ o _||_| _||_| / _| | o |_||_||_|
| > [||] [||] [||] Do i believe in Bible? Hell,man,i've seen one!
| >
| > -----BEGIN PGP SIGNATURE-----
| > Version: 2.6.3i
| > Charset: noconv
| >
| > iQCVAwUBOIxo0qH/mIJW9LeBAQFTbAP/e9DX0QJqfTftFuSNjKXaeLn5Tn7ljiXJ
| > ZhTRSFTdxbarMyB8OXI9HQv4K2gN2p5vKUIdGvjvqqtyyitTSDwly7zsPoaCrfh9
| > KdEf48ZpwxAS7W7EyiUnNleFrxREHwONZhXkF/bzmHFCidhXzRmPD5R0R6KHRF2q
| > XpcuX2Qu5L4=
| > =wsJA
| > -----END PGP SIGNATURE-----
|
| ----------------------------------
| E-Mail: Thierry Deval <TDe...@PrimeOBJ.COM>
| Date: 24-Jan-00
| Time: 19:37:32
|
| Prime Objective
| OpenBSD - Linux - Windows NT
| OO Development - Network Systems
| ----------------------------------
|
---
Gates' Law: Every 18 months, the speed of software halves.
Chris Cappuccio
|
| Yes Chris, I remember seeing it on the list someday (on Oct 12th 1999), but
| it is not a real port (in the port tree) yet, as it needs kernel rebuild. (No
| offense)
| And indeed, in AltQ 2.1, they support several cards based on stock 2.6 .
|
I never said that I made it part of /usr/ports
| Then, I would suggest envisaging the integration of AltQ into the base source
| (despite Theo's reluctancy to change the network drivers' API), at least the
| kernel parts.
| For the userland parts, we'll have to see (on the long run) if we can get rid
| of an altqd daemon, and try to merge the controls with ipfilter (which, btw, is
| not IP centric only :)
How is IP Filter not IP centric ?
Altqd deals with a ruleset that is dissimilar to ipfilter. It makes no sense
to integrate it with ipf.
Finally, ALTQ does not change the API of any ethernet drivers. It just adds
code to them.
Thierry Deval
On 01-Feb-00 Chris Cappuccio wrote:
> On Tue, 1 Feb 2000, Thierry Deval wrote:
>
> | Yes, that was my idea. (with the approval of the high players :-)
> |
> | We should also try to extend AltQ beyond the x86 platform. (in
> collaboration
> | with the NetBSD team)
> | Kenjiro, what would be the main difficulties in implementing AltQ on other
> | CPUs ?
> |
>
> Making ALTQ work on other platforms requires that you allocate a device major
> number for altq.
Couldn't we use the same majors across platforms ? Do they diverge that much ?
> Also, if the architecture has a special high resolution
> timer (like the pentium's TSC timer) then support should be added in
> altq/altq_subr.c.
Wow, low-level coding... What archs do have such features (do we consider
variants like 68360 <-> m68k; of course they are not supported) ?
Let's go to the datasheets ...
>
> This is mostly trivial work.. Also I wonder if ALTQ is big endian or 64-bit
> clean...
These will have to be checked, definitely.
Chris Cappuccio
| Yes, that was my idea. (with the approval of the high players :-)
|
| We should also try to extend AltQ beyond the x86 platform. (in collaboration
| with the NetBSD team)
| Kenjiro, what would be the main difficulties in implementing AltQ on other
| CPUs ?
|
Making ALTQ work on other platforms requires that you allocate a device major
number for altq. Also, if the architecture has a special high resolution
timer (like the pentium's TSC timer) then support should be added in
altq/altq_subr.c.
This is mostly trivial work.. Also I wonder if ALTQ is big endian or 64-bit
clean...
---
James WorK
>
> I'll look at all this HUGE networking code, and see were AltQ could fit.
> Any suggestion from you gurus are welcome. And when I come back with something,
> there will certainly be a need for deep cleaning ;-)
I fitted AltQ 2.1-post-release in OpenBSD-current tonight. I didn't
add any interface support beyond the one that is already into AltQ
2.1. Should we merge it to OpenBSD tree? Adding new drivers
support should be kinda easy.
James WorK, j...@lords.com
KeyID: 3072/1024/0x409C9044
Fingerprint: 7984 10FD 0460 4202 BF90 3881 CDE4 D78E 409C 9044
"Behold, I send you forth as sheep in the midst of wolves:
be ye therefore wise as serpents, and harmless as doves." - Matthew 10:16
Thierry Deval
On 01-Feb-00 James WorK wrote:
>> OK, my best best would be to base alt-q on bpf, or maybe just under it.
>>
>> I'll look at all this HUGE networking code, and see were AltQ could fit.
>> Any suggestion from you gurus are welcome. And when I come back with
>> something,
>> there will certainly be a need for deep cleaning ;-)
>
> I fitted AltQ 2.1-post-release in OpenBSD-current tonight. I didn't
> add any interface support beyond the one that is already into AltQ
> 2.1. Should we merge it to OpenBSD tree? Adding new drivers
> support should be kinda easy.
>
Yes, that was my idea. (with the approval of the high players :-)
We should also try to extend AltQ beyond the x86 platform. (in collaboration
with the NetBSD team)
Kenjiro, what would be the main difficulties in implementing AltQ on other
CPUs ?
Maybe you could submit the diffs (huge as they are, don't pollute the list,
just send to whom are interested -> at least me ;-)
Regards,
Thierry
Chris Cappuccio
| Couldn't we use the same majors across platforms ? Do they diverge that much ?
|
if you look at, say, /usr/src/sys/arch/sparc/sparc/conf.c, major numbers
up to 123 are allocated. Under /usr/src/sys/arch/i386/i386/conf.c, it is up
to 63.
Major numbers not portable (which is why there's a separate /dev/MAKEDEV for
each architecture) simply because they are for devices and some devices may
exist on some platforms and not others.
| > Also, if the architecture has a special high resolution
| > timer (like the pentium's TSC timer) then support should be added in
| > altq/altq_subr.c.
|
| Wow, low-level coding... What archs do have such features (do we consider
| variants like 68360 <-> m68k; of course they are not supported) ?
| Let's go to the datasheets ...
|
Well, it will use microtime if you don't do this. Unless there is code to
support the alternative timer feature (such as, the i386 arch has code to
support TSC) then it needs to be done (could be taken from netbsd if they
have it and we dont for instance)
The RTMX real-time code includes a high resolution kernel timer (according to
www.rtmx.com).. When the RTMX real-time code is merged into current, this may
be an alternative.
The point of all this is for the developers to accept the commit of ALTQ into
the tree. This will not happen until ALTQ is modified to support all
architectures, and either a vast majority of network drivers are modified to
work with ALTQ or ALTQ is redesigned to work in a different manner (Which I
don't think will happen at all ;)
So, the real question is, does anyone want to modify network drivers?? The
modifications became more complex between altq-2.0 and altq-2.1 and are not
documented in the altq how-to-modify-drivers.txt. (I don't want to do this
if someone else does ;)
Of course if people using other architectures are interested in ALTQ then
please get it together!!
-chris
ito...@iijlab.net
>> I fitted AltQ 2.1-post-release in OpenBSD-current tonight. I didn't
>> add any interface support beyond the one that is already into AltQ
>> 2.1. Should we merge it to OpenBSD tree? Adding new drivers
>> support should be kinda easy.
>Yes, that was my idea. (with the approval of the high players :-)
>We should also try to extend AltQ beyond the x86 platform. (in collaboration
>with the NetBSD team)
>Kenjiro, what would be the main difficulties in implementing AltQ on other
>CPUs ?
>Maybe you could submit the diffs (huge as they are, don't pollute the list,
>just send to whom are interested -> at least me ;-)
just checking: the latest ALTQ is in KAME tree (www.kame.net/anoncvs
available) so be sure to use that. if you mean ALTQ in KAME tree
by "2.1-post-release", that's just fine.
IIRC there was one 64bit issue - int-to-pointer typecast somewhere
in libaltq.
itojun
Kenjiro Cho
Chris Cappuccio wrote:
> On Tue, 1 Feb 2000, Thierry Deval wrote:
>
> |
> | We should also try to extend AltQ beyond the x86 platform. (in collaboration
> | with the NetBSD team)
> | Kenjiro, what would be the main difficulties in implementing AltQ on other
> | CPUs ?
> |
>
> timer (like the pentium's TSC timer) then support should be added in
> altq/altq_subr.c.
Right, that's my understanding too.
> This is mostly trivial work.. Also I wonder if ALTQ is big endian or 64-bit
> clean...
I'm planning to play with alpha, and possibly sparc. I already got
machines for it but haven't had time to do the port.
Also, if someone knows how to read a high resolution timer on these
architecures (I believe they have one), please let me know.
-Kenjiro