Has anyone got K3s running on the cluster hat?

[Ben Heininger]

Hi all,

Has anyone got K3s running on the cluster hat?

I have the RPi4-4gb with the clusterhat v2.5 and 4x Pi-Zero v1.3.

I tried following the instructions from Rancher and Network Chuck to get K3s running on my cluster. 

I have the cluster setup with cBridge and with Chis' help managed to get them all working successfully but I cannot get K3s to work.

I have the server running on the host and K3s-agent running on the Nodes (Zeros) but all i get is:

kubectl get nodes
NAME      STATUS   ROLES                  AGE   VERSION
cbridge   Ready    control-plane,master   8d    v1.23.6+k3s1

No nodes found.

I have double checked that the server and agent are running on each device. 

They all have the node-token from the host.

They all have legacy iptables configured. 

I've uninstalled and reinstalled, ran the killall script and restarted each service on each node and the host.

However I cant get the host to see the nodes (╯°□°）╯︵ ┻━┻

I have been trying now for at least a week and posted for help on the rancher forum but haven't found any fixes.

If someone else has successfully done it, please help me work out why this isnt working

Thank in advance

Ben

[Ben Heininger]

FYI, dont do "sudo apt install docker.io on the master. 

It does something to the networking and I could no longer see the clusters P1-4.

as soon as I sudo apt purge docker.io and rebooted everything returned.

That rules out being able to use docker on the cluster.... :'(

[Chris Burton]

Hi,

FYI, dont do "sudo apt install docker.io on the master. 

It does something to the networking and I could no longer see the clusters P1-4.

as soon as I sudo apt purge docker.io and rebooted everything returned.

That rules out being able to use docker on the cluster.... :'(

It's been a while but I'd assume https://groups.google.com/g/clusterhat/c/_vLhJfPO_eo/m/yJVdfOZ1DwAJ still applies.

Chris. 

[Ben Heininger]

Thank you again Chris you have saved the day.
Question is like you mentioned about understanding why they did that to Docker.

You would think there would be some way to restrict the access to the docker containers?!?!

And considering that it was over  years ago that this started occurring.

Oh well Docker fixed, K3s still broken.

Chipping away at this 1 boulder at a time :/

[Jason Horwitz]

I've been running k3s on my pi4 with 4x Zero2W's semi-successfully using bullseye cnat image.  I found these articles helpful: 

* dashed https://360techexplorer.com/install-k3s-on-raspberry-pi/

* https://medium.com/@prasenjitsarkar_79320/k3s-cluster-on-raspberry-pi-gotchas-14f781e7bf6c

I say semi-successfully only because my nodes (seemingly) randomly become unresponsive after some hours/days and I cannot figure out why - though Ive spotted many similar reports on this thread - and it seems to coincide with write attempts (Ive also seen nodes become unresponsive during a manual apt-get upgrade for example).  

If anyone has solved the unresponsive-node issue I'd love to hear about it - Im at my wits end trying to figure out how to stabilize this k3s cluster so I can actually use it for something...

[Chris Burton]

Hi, 

If anyone has solved the unresponsive-node issue I'd love to hear about it - Im at my wits end trying to figure out how to stabilize this k3s cluster so I can actually use it for something...

If anyone can explain how I can replicate the "unresponsive-node issue" I may be able to look into it but you'd need to explain how to setup docker/k3s/etc. as I have no clue.

Chris.

[Ben Heininger]

Unfortunately I cant help there.
I re-watched Network Chuck's video on YT about kubernetes on Pi Cluster and he briefly mentioned that you have to have 64bit OS on all the Pis.

He of course isnt using the cluster hat but had 8x RPi4s and unfortunately Zero 2 w are unavailable for who knows how long.

I think this might be part of why I cant get it to work as all my Zeros are v1.3 and cant use 64 bit os or I cant get 64bit OS to load on them.

[Enigma Mamba]

I have a K3S cluster running on the Turing Pi Cluster (RP3 compute modules @ 1GB). That stuff barely runs reliable and I can see the 1GB being used heavily on the master and the nodes doing the cluster monitorin g already with no docker productive images active. I can hardly image K3S being able to run on just 512MB per node. Just my $ 0.02.