Bug#890343: systemd: Debian could use the fq_codel network queuing algorithm by default

[Laurent Bonnaud]

Package: systemd
Version: 237-2
Severity: normal


Dear Maintainer,

by default Debian uses the pfifo_fast network queuing algorithm:

# tc -s qdisc show
[...]
qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 413728102 bytes 475015 pkt (dropped 0, overlimits 0 requeues 785)
backlog 0b 0p requeues 785

The systemd source package contains this file:

./systemd-237/sysctl.d/50-default.conf:net.core.default_qdisc = fq_codel

whose purpose is to set the default queuing algorithm to fq_codel. According to the NEWS file, this is a better alternative:

* The default sysctl.d/ snippets will now set:

net.core.default_qdisc = fq_codel

This selects Fair Queuing Controlled Delay as the default
queuing discipline for network interfaces. fq_codel helps
fight the network bufferbloat problem. It is believed to be
a good default with no tuning required for most workloads.
Downstream distributions may override this choice. On 10Gbit
servers that do not do forwarding, "fq" may perform better.
Systems without a good clocksource should use "pfifo_fast".

However the 50-default.conf file is not in the Debian binary package.
Is this intentional or an omission?
Could it be possible to enable fq_codel by default?


-- Package-specific info:

-- System Information:
Debian Release: buster/sid
APT prefers unstable-debug
APT policy: (500, 'unstable-debug'), (500, 'unstable'), (1, 'experimental-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 4.14.0-3-rt-amd64 (SMP w/2 CPU cores; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages systemd depends on:
ii adduser 3.117
ii libacl1 2.2.52-3+b1
ii libapparmor1 2.12-2
ii libaudit1 1:2.8.2-1
ii libblkid1 2.31.1-0.1
ii libc6 2.27-0experimental0
ii libcap2 1:2.25-1.2
ii libcryptsetup12 2:2.0.1-1
ii libgcrypt20 1.8.2-1
ii libgpg-error0 1.27-6
ii libidn11 1.33-2.1
ii libip4tc0 1.6.2-1
ii libkmod2 25-1
ii liblz4-1 0.0~r131-2+b1
ii liblzma5 5.2.2-1.3
ii libmount1 2.31.1-0.1
ii libpam0g 1.1.8-3.7
ii libseccomp2 2.3.1-2.1
ii libselinux1 2.7-2+b1
ii libsystemd0 237-2
ii mount 2.31.1-0.1
ii procps 2:3.3.12-4
ii util-linux 2.31.1-0.1

Versions of packages systemd recommends:
ii dbus 1.13.0-1
ii libpam-systemd 237-2

Versions of packages systemd suggests:
ii policykit-1 0.113-6
pn systemd-container <none>

Versions of packages systemd is related to:
pn dracut <none>
ii initramfs-tools 0.130
ii udev 237-2

--
Laurent.

[Felipe Sateler]

Changelog has this by Marco d'Itri:

> * Do not install sysctl.d/50-default.conf because the systemd package
> should not change kernel policies, at least until it will become
> the only supported init system.

And it makes sense: why would some users get this but others don't?

--

Saludos,
Felipe Sateler

[Laurent Bonnaud]

On 2/13/18 6:29 PM, Felipe Sateler wrote:

> Changelog has this by Marco d'Itri:
>
>> * Do not install sysctl.d/50-default.conf because the systemd package
>> should not change kernel policies, at least until it will become
>> the only supported init system.

Thanks for pointing this out! I did not find it with "grep codel".

> And it makes sense: why would some users get this but others don't?

Indeed. I will therefore reassign this bug to the kernel...

Thanks again,

--
Laurent.

[Chentao Credungtao]

> And it makes sense: why would some users get this but others don't?

It does indeed make sense.

But in the case I install Debian with systemd (which is the default), would you recommend to manually change the default to fq_codel (like here : https://serverfault.com/questions/951735/deleting-root-pfifo-fast-tc-on-linux) ?

Given that fq_codel is supposed to be ?? better ?? than pfifo_fast, that would make sense too, no ?

[Matt Taggart]

#890343 was originally opened against systemd asking to install the
upstream systemd sysctl.d/50-default.conf file that sets:

net.core.default_qdisc = fq_codel

As explained in #950701 (and the systemd debian changelog) the debian
systemd maintainers felt that systemd in debian should not be changing
kernel policies (and I agree).
So #890343 was reassigned to linux to consider changing the default.

fq_codel is better in every way than pfifo_fast and I am unaware of any
reason why it would not be a better default. (but don't trust me, ask
the kernel networking experts)

Can we change it?

Related thoughts:
* ubuntu issue, looks like maybe they already switched by default due to
the systemd change?
https://bugs.launchpad.net/ubuntu/+source/procps/+bug/1773157
* another ubuntu issue that might help
https://bugs.launchpad.net/ubuntu/+bug/940541
* are there other related defaults that should change too? like tcp_ecn,
syncookies, BBR, etc
* what are upstream kernel defaults and why?

--
Matt Taggart
tag...@debian.org

[Noah Meyerhans]

On Thu, Apr 23, 2020 at 03:34:06PM -0700, Matt Taggart wrote:
> fq_codel is better in every way than pfifo_fast and I am unaware of any
> reason why it would not be a better default. (but don't trust me, ask the
> kernel networking experts)

Isn't CAKE supposed to be even better than fq_codel, including better
handling of both large numbers of flows (e.g. busy routers) and small
systems with limited resources.

https://www.bufferbloat.net/projects/codel/wiki/Cake/

If we consider a change (which I think we should), is there a reason we
wouldn't go with CAKE?

noah

[Matt Taggart]

In net/sched/Kconfig, the options for NET_SCH_DEFAULT are:
fq, codel, fq_codel, sfq, pfifo_fast

Also Documentation/admin-guide/sysctl/net.rst explains:

default_qdisc
-------------

The default queuing discipline to use for network devices. This allows
overriding the default of pfifo_fast with an alternative. Since the

default queuing discipline is created without additional parameters so
is best suited to queuing disciplines that work well without
configuration like stochastic fair queue (sfq), CoDel (codel) or fair
queue CoDel (fq_codel). Don't use queuing disciplines like
Hierarchical Token Bucket or Deficit Round Robin which require setting
up classes and bandwidths. Note that physical multiqueue interfaces
still use mq as root qdisc, which in turn uses this default for its
leaves. Virtual devices (like e.g. lo or veth) ignore this setting and
instead default to noqueue.

CAKE, while better for AQM, requires interface specific parameters and
so won't work for a default (and also is generally used in conjunction
with fq_codel)

--
Matt Taggart
tag...@debian.org

[Dave Taht]

cake and its default parameters is designed to run just fine as a
default qdisc at whatevr the line rate is in linux. I do it all the
time. I find the per host/per flow fq useful, the sane diffserv
markings helpful, the gso splitting best for low latency apps like
videoconferencing, and the stats often enlightening, and I do wish
more folk ran it by default.

however, cake is far more cpu intensive (and slightly more memory
intensive) than fq_codel is and given the wide range of hardware
debian runs on, I cannot support
the idea of it being the default qdisc. I could put up a benchmark or
three as to the difference in cpu on the raspberry pi 3 and 4 to
illustrate this point.

Certainly pfifo_fast must die! Up until last week I thought fq_codel
WAS the default in debian (in addition to *wrt,ubuntu/fedora/rhel8,
arch and so many others) but hd been fooled by linode supplying their
own kernel for everything.

I'd of course love to see it built by default (along with sch_fq and
fq_codel), and more people just reach for the lovely one liner of

tc qdisc add eth0 root cake bandwidth XXX

whenever they need to shape something, AND use it at line rate
(without the bandwidth param) where they can afford it... but I think
the decision tree for a linux default at this point would be

fq_codel (for generic workloads)
sch fq (for tcp serving workloads)
cake (if you can afford it)
bfifo
pfifo_fast

I had a long and (apology, all, oft testy!) debate on the benefits of
fq_codel vs pfifo_fast vs sch fq over here, with the most relevant
comment here:

https://github.com/systemd/systemd/issues/9725#issuecomment-413369212

--
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

[Matt Taggart]

Regarding the default qdisc setting, here's an interesting article in
Phoronix about fq_pie

Linux 5.9 To Allow Defaulting To FQ-PIE Queuing Discipline For Fighting
Bufferbloat
https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.9-FQ-PIE-QDISC

I think this only means it's possible for it to be set as the default in
5.9, I don't know what the upstream kernel will ship with as a default.

The Debian default needs to be updated, fq_codel would be a good choice,
maybe fq_pie is a good choice starting in 5.9 as well.

Any comments on this or my previous questions from April? What needs to
happen for this to change?

Thanks,

--
Matt Taggart
tag...@debian.org

[Noah Meyerhans]

On Thu, Apr 23, 2020 at 03:34:06PM -0700, Matt Taggart wrote:

> #890343 was originally opened against systemd asking to install the upstream
> systemd sysctl.d/50-default.conf file that sets:
>
> net.core.default_qdisc = fq_codel
>
> As explained in #950701 (and the systemd debian changelog) the debian
> systemd maintainers felt that systemd in debian should not be changing
> kernel policies (and I agree).
> So #890343 was reassigned to linux to consider changing the default.
>
> fq_codel is better in every way than pfifo_fast and I am unaware of any
> reason why it would not be a better default. (but don't trust me, ask the
> kernel networking experts)
>
> Can we change it?

I strongly agree that we should make this change for the bullseye
release.

I'm looking into provding a patch to implement the switch to fq_codel by
default, but it appears to require something more than just a kernel
config change. I have tried the following with the 5.10 kernel from the
current sid branch:

CONFIG_NET_SCH_FQ_CODEL=m
CONFIG_DEFAULT_FQ_CODEL=y
CONFIG_DEFAULT_NET_SCH="fq_codel"

Then we don't see any change at all to the qdisc in use:

admin@ip-10-0-0-162:~$ grep -i fq_codel /boot/config-$(uname -r)
CONFIG_NET_SCH_FQ_CODEL=m
CONFIG_DEFAULT_FQ_CODEL=y
CONFIG_DEFAULT_NET_SCH="fq_codel"
admin@ip-10-0-0-162:~$ /sbin/sysctl net.core.default_qdisc
net.core.default_qdisc = pfifo_fast
admin@ip-10-0-0-162:~$ tc qdisc show dev ens5
qdisc mq 0: root
qdisc pfifo_fast 0: parent :2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
admin@ip-10-0-0-162:~$ ip link show dev ens5
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 02:47:e2:7c:be:ff brd ff:ff:ff:ff:ff:ff
altname enp0s5

If we statically link the fq_codel module into the kernel, then we see:

admin@ip-10-0-0-162:~$ grep -i fq_codel /boot/config-$(uname -r)
CONFIG_NET_SCH_FQ_CODEL=y
CONFIG_DEFAULT_FQ_CODEL=y
CONFIG_DEFAULT_NET_SCH="fq_codel"
admin@ip-10-0-0-162:~$ /sbin/sysctl net.core.default_qdisc
net.core.default_qdisc = fq_codel
admin@ip-10-0-0-162:~$ /sbin/tc qdisc show dev ens5
qdisc mq 0: root
qdisc fq_codel 0: parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
qdisc fq_codel 0: parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
admin@ip-10-0-0-162:~$ ip link show dev ens5
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 02:47:e2:7c:be:ff brd ff:ff:ff:ff:ff:ff
altname enp0s5

So in this case, we have fq_codel configured, but not as the root
qdisc for the interface. If we manually set it:

admin@ip-10-0-0-162:~$ sudo /sbin/tc qdisc add root dev ens5 fq_codel

Then we get the following configuration:

admin@ip-10-0-0-162:~$ /sbin/tc qdisc show dev ens5
qdisc fq_codel 8001: root refcnt 3 limit 10240p flows 1024 quantum 9015 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
admin@ip-10-0-0-162:~$ ip link show dev ens5
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 02:47:e2:7c:be:ff brd ff:ff:ff:ff:ff:ff
altname enp0s5

I believe that this is what we want. Is that accurate?

The recent thread at https://www.mail-archive.com/net...@vger.kernel.org/msg380410.html
also seems relevant.

noah

[Ivan Baldo]

    Hello.

    Not an expert nor kernel developer, etc., just simple sysadmin
here, but will answer anyway since nobody did yet.

    I think we want the mq qdisc to distribute the load between cores,
to support very high speed network cards or too slow CPUs.

    Also, if net.core.default_qdisc = fq_codel is used, it also has the
mq qdisc and fq_codel childs for each CPU core, so that's the default
behavior in other distros.

    I guess being under mq it could impact fq_codel's ability somewhat,
but it's better to support all the hardware spectrum by default,
otherwise we should use cake as default since it can handle up to
10Gbit/s without troubles; we don't, because Debian is a generic
operating system so it should work on embedded devices, laptops,
workstations and big iron servers...

    Also, I don't see a problem with statically linking fq_codel if we
are using it as default, since it will probably get loaded anyway...

    Thanks for pushing this forward, doing these tests, etc.!!!

    Bye.



El 7/1/21 a las 21:12, Noah Meyerhans escribió:

--
Ivan Baldo - iba...@adinet.com.uy - http://ibaldo.codigolibre.net/
Freelance C++/PHP programmer and GNU/Linux systems administrator.
The sky is not the limit!

[Vincent Blut]

Hi Noah,

I was about to start a thread on the debian-kernel ML about switching from
pfifo_fast to FQ_Codel when I stumble across this bug report.

My proposal would differ from yours though in that it would not touch the kernel
configuration but would instead consist in patching procps to provide a
configuration file (let's say default_qdisc.conf) to set the value of the
net.core.default_qdisc variable to fq_codel via sysctl.
This would allow to benefit from FQ_Codel without depending on a specific Linux
version.

Opinion?

> noah

Cheers,
Vincent

[Noah Meyerhans]

On Sun, Jan 17, 2021 at 10:29:44PM -0300, Ivan Baldo wrote:
>     I think we want the mq qdisc to distribute the load between cores, to
> support very high speed network cards or too slow CPUs.

Yep, you're right. Though it's not about CPU cores, but about tx queues
on the NIC hardware.

>     Also, if net.core.default_qdisc = fq_codel is used, it also has the mq
> qdisc and fq_codel childs for each CPU core, so that's the default behavior
> in other distros.

Yep, so I think the behavior when we set the default qdisc to fq_codel
in the kernel config has the effect we're looking for, after all.

noah

[Noah Meyerhans]

On Wed, Jan 20, 2021 at 10:22:16PM +0100, Vincent Blut wrote:
> My proposal would differ from yours though in that it would not touch the kernel
> configuration but would instead consist in patching procps to provide a
> configuration file (let's say default_qdisc.conf) to set the value of the
> net.core.default_qdisc variable to fq_codel via sysctl.
> This would allow to benefit from FQ_Codel without depending on a specific Linux
> version.

We could do that. However, in the past (earlier in this bug, even) it's
been pointed out that other packages should not be responsible for
setting kernel policies, so changes like this should be the
responsibility of the kernel packages. That seems like a sensible
position to take.

One possible way for the kernel team to take ownership of this would be
for it to introduce a new "debian-kernel-sysctl" package or something
like that to provide some sysctl.d drop-in files. It could then set
net.core.default_qdisc, and potentially others in various scenarios.
Such a package can be installed indepdendently of whether the user is
running a Debian-provided kernel package.

The other alternative is the one I've proposed, which involves changing
the compile-time defaults in Debian's kernel packages. This obviously
only affects users of those packages. However, I think that's fine;
people who are building their own packages may very well be starting
from Debian's config, in which case they'll still get this change, or
may be constructing their own configuration from scratch, in which case
they're assuming ownership of all the parameters.

noah

[Noah Meyerhans]

Control: tags -1 + patch

A proposed patch is at
https://salsa.debian.org/kernel-team/linux/-/merge_requests/309

[Vincent Blut]

Le 2021-01-20 13:58, Noah Meyerhans a écrit :
> On Wed, Jan 20, 2021 at 10:22:16PM +0100, Vincent Blut wrote:
> > My proposal would differ from yours though in that it would not touch the kernel
> > configuration but would instead consist in patching procps to provide a
> > configuration file (let's say default_qdisc.conf) to set the value of the
> > net.core.default_qdisc variable to fq_codel via sysctl.
> > This would allow to benefit from FQ_Codel without depending on a specific Linux
> > version.
>
> We could do that. However, in the past (earlier in this bug, even) it's
> been pointed out that other packages should not be responsible for
> setting kernel policies, so changes like this should be the
> responsibility of the kernel packages. That seems like a sensible
> position to take.

If this is the position of the kernel team, then fine. But some packages *do*
tweak kernel parameters using the sysctl interface mechanism. So does the kernel
team provides documention about what is acceptable?

[Noah Meyerhans]

On Wed, Jan 20, 2021 at 11:39:16PM +0100, Vincent Blut wrote:
> > We could do that. However, in the past (earlier in this bug, even) it's
> > been pointed out that other packages should not be responsible for
> > setting kernel policies, so changes like this should be the
> > responsibility of the kernel packages. That seems like a sensible
> > position to take.
>
> If this is the position of the kernel team, then fine. But some packages *do*
> tweak kernel parameters using the sysctl interface mechanism. So does the kernel
> team provides documention about what is acceptable?

I think the distinction is that the other packages that tweak sysctl
values don't claim to be doing so on behalf of the kernel team. If the
kernel team is responsible for the values being set, then the settings
should come from a package that the kernel team owns, not some other
package.

AFAIK, there are no guidelines or policy anywhere in Debian about
whether or not a package can provide its own sysctl settings.

noah

[Vincent Blut]

Le 2021-01-20 14:46, Noah Meyerhans a écrit :
> On Wed, Jan 20, 2021 at 11:39:16PM +0100, Vincent Blut wrote:
> > > We could do that. However, in the past (earlier in this bug, even) it's
> > > been pointed out that other packages should not be responsible for
> > > setting kernel policies, so changes like this should be the
> > > responsibility of the kernel packages. That seems like a sensible
> > > position to take.
> >
> > If this is the position of the kernel team, then fine. But some packages *do*
> > tweak kernel parameters using the sysctl interface mechanism. So does the kernel
> > team provides documention about what is acceptable?
>
> I think the distinction is that the other packages that tweak sysctl
> values don't claim to be doing so on behalf of the kernel team. If the
> kernel team is responsible for the values being set, then the settings
> should come from a package that the kernel team owns, not some other
> package.

Makes sense!

> AFAIK, there are no guidelines or policy anywhere in Debian about
> whether or not a package can provide its own sysctl settings.

That would be nice to have a statement on this, though.

> noah

Cheers,
Vincent

[Ben Hutchings]

On Wed, 2021-01-20 at 14:46 -0800, Noah Meyerhans wrote:
> On Wed, Jan 20, 2021 at 11:39:16PM +0100, Vincent Blut wrote:
> > > We could do that.  However, in the past (earlier in this bug,
> > > even) it's
> > > been pointed out that other packages should not be responsible
> > > for
> > > setting kernel policies, so changes like this should be the
> > > responsibility of the kernel packages.  That seems like a
> > > sensible
> > > position to take.
> >
> > If this is the position of the kernel team, then fine. But some
> > packages *do*
> > tweak kernel parameters using the sysctl interface mechanism. So
> > does the kernel
> > team provides documention about what is acceptable?
>
> I think the distinction is that the other packages that tweak sysctl
> values don't claim to be doing so on behalf of the kernel team.  If
> the
> kernel team is responsible for the values being set, then the
> settings
> should come from a package that the kernel team owns, not some other
> package.

Right, maybe in linux-base? Although that might annoy derivatives that
want different defaults.

procps is the wrong place, not just because it's out of our hands, but
because systemd applies sysctl configuration now and procps is
optional.

Ben.

> AFAIK, there are no guidelines or policy anywhere in Debian about
> whether or not a package can provide its own sysctl settings.
>
> noah
>

--
Ben Hutchings
Computers are not intelligent. They only think they are.

[Vincent Blut]

Hi,

Is there a definitive answer from the kernel team about how this should be
implemented? In the meantime, Noah sent [1].

> Ben.

Cheers,
Vincent

[1] https://salsa.debian.org/kernel-team/linux/-/merge_requests/309/diffs

[Noah Meyerhans]

On Fri, Feb 26, 2021 at 06:58:50PM +0100, Vincent Blut wrote:
> > > I think the distinction is that the other packages that tweak sysctl
> > > values don't claim to be doing so on behalf of the kernel team.  If
> > > the
> > > kernel team is responsible for the values being set, then the
> > > settings
> > > should come from a package that the kernel team owns, not some other
> > > package.
> >
> > Right, maybe in linux-base? Although that might annoy derivatives that
> > want different defaults.
> >
> > procps is the wrong place, not just because it's out of our hands, but
> > because systemd applies sysctl configuration now and procps is
> > optional.
>
> Is there a definitive answer from the kernel team about how this should be
> implemented? In the meantime, Noah sent [1].

I've rebased
https://salsa.debian.org/kernel-team/linux/-/merge_requests/309 against
the current 'master' kernel branch on Salsa. Basic test results are
below. It'd be nice if the kernel team could have a look and give
feedback on the approach or recommend an alternative one if this isn't
the one they'd like to pursue.

# before reboot:
admin@ip-10-0-0-136:~$ /sbin/tc qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc mq 0: dev ens5 root
qdisc pfifo_fast 0: dev ens5 parent :2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: dev ens5 parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
admin@ip-10-0-0-136:~$ sudo apt install ./linux-image-5.16.0-rc3-cloud-amd64-unsigned_5.16~rc3-1~exp2_amd64.deb
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'linux-image-5.16.0-rc3-cloud-amd64-unsigned' instead of './linux-image-5.16.0-rc3-cloud-amd64-unsigned_5.16~rc3-1~exp2_amd64.deb'
The following additional packages will be installed:
firmware-linux-free
Suggested packages:
linux-doc-5.16 debian-kernel-handbook grub-pc | grub-efi-amd64 | extlinux
The following NEW packages will be installed:
firmware-linux-free linux-image-5.16.0-rc3-cloud-amd64-unsigned
...
admin@ip-10-0-0-136:~$ sudo reboot
Connection to 18.236.97.48 closed by remote host.
...
admin@ip-10-0-0-136:~$ uname -a
Linux ip-10-0-0-136 5.16.0-rc3-cloud-amd64 #1 SMP PREEMPT Debian 5.16~rc3-1~exp2 (2021-12-01) x86_64 GNU/Linux
admin@ip-10-0-0-136:~$ /sbin/tc qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc mq 0: dev ens5 root
qdisc fq_codel 0: dev ens5 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
qdisc fq_codel 0: dev ens5 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64