Huge number of iptables rules in k3s cluster

[Michael Kelly]

Hi all,

I don't know if this is an awx issue or a k3s issue or some combination of both.

Several months ago I created a couple of single node k3s clusters using https://github.com/kurokobo/awx-on-k3s each with 4 vCPUs.

What I've noticed over time is that some of the iptables rules are being duplicated over and over again.

For example

~# iptables -L -n | sort | uniq -c | sort -k1n | tail -4
# Warning: iptables-legacy tables present, use iptables-legacy to see them
     25 target     prot opt source               destination
  21931 RETURN     udp  --  0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL multiport dports 30000:32767 /* allow LOCAL UDP traffic to node ports - 76UCBPIZNGJNWNUZ */
  21932 RETURN     tcp  --  0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL multiport dports 30000:32767 /* allow LOCAL TCP traffic to node ports - LR7XO7NXDBGQJD2M */
  65790 ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            mark match 0x20000/0x20000 /* rule to explicitly ACCEPT traffic that comply to network policies */

When I run top, I can see 4 iptables consuming most of the CPU resource.

The net effect of this is that some awx jobs fail with timeout errors.

Stopping the k3s service, running k3s-killall.sh and starting the k3s service resolves the issue, for a while.

Any ideas about what's happening here.

[Wei-Yen Tan]

Yes I have encountered this with system resources with k3s?. Do you have the normal iptables installed?  

https://github.com/k3s-io/k3s/issues/3117

Sent from Outlook for iOS

---

From: awx-p...@googlegroups.com <awx-p...@googlegroups.com> on behalf of Michael Kelly <michae...@sohonet.com>
Sent: Thursday, October 19, 2023 6:11:46 AM
To: AWX Project <awx-p...@googlegroups.com>
Subject: [awx-project] Huge number of iptables rules in k3s cluster

 

--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/1a65316d-d14f-4cfc-ae1e-a3c72a909c64n%40googlegroups.com.