Re: How to configure Weave Net after kubeadm

[Torsten Bronger]

Hallöchen!

br...@weave.works writes:

> You should definitely not be using --pod-network-cidr with Weave
> Net, as it turns on Kubernetes' own "cloud-provider" IP address
> management which will fight with Weave Net.

Right.

> You're correct: the IPALLOC_RANGE environment variable is provided
> to change the address range used by weave-kube.

And it indeed works. I even don't need the "ip route add
10.96.0.0/12 dev eno33559296" thingy (probably because it is
realised through iptables since Kubernetes 1.2).

> A /24 seems rather small for a pod network - I guess it will work,
> but it doesn't allow for a lot of growth in your cluster.

Yes, but our IT department prefers it this small. We should
complain when it's getting too tight.

Anyway, in case anybody has a similar setup (local and public
interfaces, necessity to set IP ranges), here's how it works with
Kubernetes 1.4.4: ([1] is
http://kubernetes.io/docs/getting-started-guides/kubeadm/)

- Set the current hostname in /etc/hosts to the local IP
(e.g. 192.168.something) on each node and the master.

- Install (not start!) the Kubernetes packages on master and all
nodes according to [1].

- Patch /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on
master and all nodes, so that the DNS server IP is in your
services network, e.g. for 10.67.97.0/24 it may be 10.67.97.10.

- Then start the systemd services on master and all nodes according
to [1]

- Call kubeadm on master according to [1].

- Pass "--api-external-dns-names <hostnames>" to kubeadm.
<hostnames> is comma-separated,
e.g. "kubmaster,kubmaster.example.com".

- Also pass "--service-cidr 10.67.97.0/24" to kubeadm (of course,
with *your* services network).

- Call kubeadm on the nodes according to [1], i.e. use the "join"
command.

- Download
https://raw.githubusercontent.com/weaveworks/weave/master/prog/weave-kube/weave-daemonset.yaml
and add the environment variable IPALLOC_RANGE to the container
"weave" with e.g. the value "172.25.97.0/24". This is the pod
network. Call "kubectl apply -f <filename>" on the result.

It is a little bit tricky but there we go at the bleeding edge.

Tschö,
Torsten.

--
Torsten Bronger

[Torsten Bronger]

Hallöchen!

Torsten Bronger writes:

> [...]

>
> Anyway, in case anybody has a similar setup (local and public
> interfaces, necessity to set IP ranges), here's how it works with
> Kubernetes 1.4.4: ([1] is
> http://kubernetes.io/docs/getting-started-guides/kubeadm/)
>
> - Set the current hostname in /etc/hosts to the local IP
> (e.g. 192.168.something) on each node and the master.

As it turned out, this is not enough because inter-node
communication to pods via services doesn't work yet. You have to
route the service traffic to an interface connecting the nodes. On
nmcli-based systems (RedHat), this is:

nmcli connection modify {connection_name} +ipv4.routes {service_cidr}
systemctl restart network.service