TECTONIC Install failing at Connect Nodes

[ke...@scott-land.net]

Hi, I am trying to install a Techtonic two node cluster but the install is failing at the Tectonic" install page "Connect Nodes". I have managed to copy over my "assets" zip file and have logged into my "controller" and run the following commands in the terminal...

scp assets-show-dev.zip co...@ctl1.example.com:/home/core/ &&
ssh co...@ctl1.example.com 'unzip assets-show-dev.zip \
&& sudo mv assets /opt/bootkube/assets \
&& sudo chmod +x /opt/bootkube/assets/bootkube-start \
&& sudo systemctl start bootkube'

The above commands complete ok, so I ran the following command  and was able to see that some download was happening (I can't remember what it was) then lots of text went up the screen and ended in the following.....

journalctl -u bootkube -f

Feb 14 13:19:54 ctl1.example.com bootkube-start[1908]: [ 5115.763398] bootkube[5]: I0214 13:19:54.359585       5 log.go:19] http: TLS handshake error from 192.168.20.21:48194: read tcp 192.168.20.21:443->192.168.20.21:48194: read: connection reset by peer
Feb 14 13:19:54 ctl1.example.com bootkube-start[1908]: [ 5115.834138] bootkube[5]: I0214 13:19:54.430432       5 log.go:19] http: TLS handshake error from 192.168.20.22:33762: read tcp 192.168.20.21:443->192.168.20.22:33762: read: connection reset by peer
Feb 14 13:19:54 ctl1.example.com bootkube-start[1908]: [ 5115.834836] bootkube[5]: I0214 13:19:54.431156       5 log.go:19] http: TLS handshake error from 192.168.20.22:33764: read tcp 192.168.20.21:443->192.168.20.22:33764: read: connection reset by peer
Feb 14 13:19:54 ctl1.example.com bootkube-start[1908]: [ 5115.844672] bootkube[5]: I0214 13:19:54.440965       5 log.go:19] http: TLS handshake error from 192.168.20.22:33766: read tcp 192.168.20.21:443->192.168.20.22:33766: read: connection reset by peer
Feb 14 13:19:54 ctl1.example.com bootkube-start[1908]: [ 5115.860094] bootkube[5]: I0214 13:19:54.456388       5 log.go:19] http: TLS handshake error from 192.168.20.22:33768: read tcp 192.168.20.21:443->192.168.20.22:33768: read: connection reset by peer
Feb 14 13:19:54 ctl1.example.com bootkube-start[1908]: [ 5116.013105] bootkube[5]: Error: error while checking pod status: timed out waiting for the condition
Feb 14 13:19:54 ctl1.example.com bootkube-start[1908]: [ 5116.013575] bootkube[5]: Error: error while checking pod status: timed out waiting for the condition
Feb 14 13:19:54 ctl1.example.com bootkube-start[1908]: [ 5116.013890] bootkube[5]: error while checking pod status: timed out waiting for the condition
Feb 14 13:19:55 ctl1.example.com systemd[1]: bootkube.service: Main process exited, code=exited, status=1/FAILURE
Feb 14 13:19:55 ctl1.example.com systemd[1]: bootkube.service: Unit entered failed state.
Feb 14 13:19:55 ctl1.example.com systemd[1]: bootkube.service: Failed with result 'exit-code'.


FYI My Setup consists the following 4 servers,

 1. Provisioner (Centos7 + dnsmasq, bootcfg)   192.168.20.10
 2. Controller  192.168.20.21
 3. Node1       192.168.20.22 
 4. Node2       192.168.20.23


Not sure if it's relevant but the Controller and Node 1 and Node 2 as printed the following text to the console.

perf : interrupt took too long (2514 > 2500), lowering kernel.perf_event_mat_sample_rate to 79000
perf : interrupt took too long (2514 > 2500), lowering kernel.perf_event_mat_sample_rate to 63000
perf : interrupt took too long (2514 > 2500), lowering kernel.perf_event_mat_sample_rate to 50000

I am not sure what to do next so any help would be really appreciated.

Thanks
Keith

[ke...@scott-land.net]

I restarted the bootkube service and checked the logs and found the following errors. I am not sure what the problem is so any help would be appreciated.

 5 log.go:19] http: TLS handshake error from 192.168.20.21:42432: read tcp 192.168.20.21:443->192.168.20.21:42432: read: connection reset by peer
Feb 14 16:30:54 ctl1.example.com bootkube-start[4419]: [16576.111333] bootkube[5]: I0214 16:30:54.707612       5 log.go:19] http: TLS handshake error from 192.168.20.23:36640: read tcp 192.168.20.21:443->192.168.20.23:36640: read: connection reset by peer
Feb 14 16:30:54 ctl1.example.com bootkube-start[4419]: [16576.123021] bootkube[5]: I0214 16:30:54.719286       5 log.go:19] http: TLS handshake error from 192.168.20.23:36638: read tcp 192.168.20.21:443->192.168.20.23:36638: read: connection reset by peer
Feb 14 16:30:54 ctl1.example.com bootkube-start[4419]: [16576.188099] bootkube[5]: E0214 16:30:54.784392       5 create.go:35] Error creating assets: error when creating "/assets/manifests/kube-dns-svc.yaml": Service "kube-dns" is invalid: spec.clusterIP: Invalid value: "10.3.0.10": provided IP is already allocated
Feb 14 16:30:54 ctl1.example.com bootkube-start[4419]: [16576.188634] bootkube[5]: Error creating assets: error when creating "/assets/manifests/kube-dns-svc.yaml": Service "kube-dns" is invalid: spec.clusterIP: Invalid value: "10.3.0.10": provided IP is already allocated
Feb 14 16:30:54 ctl1.example.com bootkube-start[4419]: [16576.188986] bootkube[5]: NOTE: Bootkube failed to create some cluster assets. It is important that manifest errors are resolved and resubmitted to the apiserver.
Feb 14 16:30:54 ctl1.example.com bootkube-start[4419]: [16576.189354] bootkube[5]: For example, after resolving issues: kubectl create -f <failed-manifest>
Feb 14 16:30:54 ctl1.example.com bootkube-start[4419]: [16576.351504] bootkube[5]: I0214 16:30:54.947702       5 log.go:19] http: TLS handshake error from 192.168.20.21:42434: read tcp 192.168.20.21:443->192.168.20.21:42434: read: connection reset by peer
Feb 14 16:30:55 ctl1.example.com bootkube-start[4419]: [16576.443579] bootkube[5]: I0214 16:30:55.039871       5 log.go:19] http: TLS handshake error from 192.168.20.22:58678: read tcp 192.168.20.21:443->192.168.20.22:58678: read: connection reset by peer
Feb 14 16:30:55 ctl1.example.com bootkube-start[4419]: [16576.461560] bootkube[5]: I0214 16:30:55.057867       5 log.go:19] http: TLS handshake error from 192.168.20.22:58680: read tcp 192.168.20.21:443->192.168.20.22:58680: read: connection reset by peer

[Rob Szumski]

Looks like the IP range provided for kube-dns is already taken for some reason:

Feb 14 16:30:54 ctl1.example.com bootkube-start[4419]: [16576.188099] bootkube[5]: E0214 16:30:54.784392       5 create.go:35] Error creating assets: error when creating "/assets/manifests/kube-dns-svc.yaml": Service "kube-dns" is invalid: spec.clusterIP: Invalid value: "10.3.0.10": provided IP is already allocated

Since it looks like you are on bare metal, is there anything that would conflict with that?

 - Rob

--
You received this message because you are subscribed to the Google Groups "CoreOS User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to coreos-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Keith]

Hi Rob,

No there are no other machines involved in my setup of Tectonic and these machines are all on a single vlan to isolate then from other network kit, I did restart all the servers in the cluster to see if that might help and it's possibly that thats caused the IP conflict.