CLusterhat and docker - Controller loses connectivity to pi0's once docker installed.

[Andrew McKenzie]

Is someone albe to help me get docker running on the ClusterHAT please?

I have a Pi3 (tried also with a Pi2 and get same issue) as the controller and 2 rpi0's plugged into the clusterHAT.

Everything work fine until I install docker on the controller, done via curl -sSL get.docker.com | sh. As soon as that done and I reboot, the controller can no longer communicate to the 2 pi zero's on the HAT.

I'm clearly missing something but I've no idea what. COuld someone who has managed to get docker running enlighten me please?

The images being used for the pi's are the ClusterHAT ones for the controller and Pi1 and pi2.

Thanks.

[Nos Doughty]

I am experiencing this too.

There are a few of us at work who are playing with pi0 clusters for docker swarm as part of our innovation time. What we've found is that those who did it a few weeks earlier, and installed docker 1.12 then upgraded to 1.13 are fine. Those of us who have done it after 1.13 was released have the same issue. Our theory is that it's a conflict between the docker0 and br0 interfaces. Note that it's not a case of running docker, even installing it but not running it will cause this issue.

I'm going to keep zeroing away on what's going wrong then post an update when I work it out.

[Ted Wood]

Can you please give the output of `iptables -L` on the controller? If I remember right, docker 1.13 adds some firewall rules that may be causing problems.

I'm noticing a similar behavior on my setup where the Pi0s on the hat will not talk to my dhcp server without running `sudo dhclient br0` on the controller manually after booting. Perhaps try that and then turning on a Pi0 to see if that allows communication to resume.

[Chris Burton]

Hi,

On Thursday, 9 February 2017 22:27:31 UTC, Ted Wood wrote:

Can you please give the output of `iptables -L` on the controller? If I remember right, docker 1.13 adds some firewall rules that may be causing problems.

It looks like https://github.com/docker/docker/issues/30338 is the issue you're seeing.

Testing by installing with "curl -sSL get.docker.com | sh", "clusterhat on p1", then running "iptables -P FORWARD ACCEPT" (which changes the default from DROP to ACCEPT) I can then see DHCP requests for P1 and I can contact it from other devices on the network.

It doesn't look like they currently give any clues as to how you should change the default in the config but before trying I'd advise understanding why it was changed (https://github.com/docker/docker/issues/14041 for example).

Chris.

[Nos Doughty]

Cool!

I've be away on training this week so I haven't had a chance to follow up. I'll follow this up during the week ahead.

Cheers

[Niels Dommerholt]

Hi Chris,

Thanks a lot. The "iptables -P FORWARD ACCEPT" fixed it for me. 

Kind regards,

Niels Dommerholt

[Chris Johnson]

I had Docker Swarm running nicely, but now I lose all network connectivity about 10 seconds after I turn on any Pi Zero. The iptables suggestion for me doesn't make any difference. Any more details on this problem?

[Chris Burton]

Hi, 

I had Docker Swarm running nicely, but now I lose all network connectivity about 10 seconds after I turn on any Pi Zero. The iptables suggestion for me doesn't make any difference. Any more details on this problem?

Can you try adding "denyinterfaces eth0 ethpi1 ethpi2 ethpi3 ethpi4" to the end of your /etc/dhcpcd.conf file and let me know if you still have the same problem? It looks like the Controller Pi is requesting multiple IPs from the DHCP server, doesn't release the first one and causes SSH to disconnect when the IP changes.

Chris.

[Nos Doughty]

Thanks everyone, the iptables adjustment fixed it for me.

I did have the dhcp problem also, but it was coming and going, not sure why. I now have my router (also the dhcp server) set to give out the same ips each time, so perhaps that's a fix for the dhcp issue also?

Cheers

[Stuart Hunter]

The iptables fix worked for me too.  I already had my router dish out static IP addresses to specific MAc addresses.

[Ben Heininger]

so I have to run that every time I restart the Controller Pi?

[Chris Burton]

Hi,

so I have to run that every time I restart the Controller Pi?

Basically yes, every time docker changes the default you would need to change it back.

Chris.