bridge setup

[luca paganotti]

Hi all,

I know this may be an old topic, but even if I followed the instructions as reported @ https://clusterhat.com/setup-software I can't see my zeroes via ethernet but only via serial console.

I installed ClusterHAT-2018-03-13-1-NAT image on my controller. which is a RPI3+ as typing:

$ cat /sys/firmware/devicetree/base/model

gives

Raspberry Pi 3 Model B Plus Rev 1.3

Then I setup, as recommended, my four zeroes using:

• ClusterHAT-2018-03-13-lite-1-p1 image
  
• ClusterHAT-2018-03-13-lite-1-p2 image
  
• ClusterHAT-2018-03-13-lite-1-p3 image
  
• ClusterHAT-2018-03-13-lite-1-p4 image
  

I can reach each one of my zeroes with minicom and I get (example only for one of them but I get the same thing for all of them)

pi@p1:~$ sudo ifconfig -a

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

        inet 127.0.0.1  netmask 255.0.0.0

        inet6 ::1  prefixlen 128  scopeid 0x10<host>

        loop  txqueuelen 1  (Local Loopback)

        RX packets 9492  bytes 901740 (880.6 KiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 9492  bytes 901740 (880.6 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet 172.19.181.1  netmask 255.255.255.0  broadcast 172.19.181.255

        inet6 fe80::7ff1:e1d8:3761:d620  prefixlen 64  scopeid 0x20<link>

        ether 00:22:82:ff:ff:01  txqueuelen 1000  (Ethernet)

        RX packets 31839  bytes 934504 (912.6 KiB)

        RX errors 0  dropped 9  overruns 0  frame 0

        TX packets 11458  bytes 491102 (479.5 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500

        ether b8:27:eb:2b:62:07  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 0  bytes 0 (0.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

these results hold also for pi2, pi3 and pi4 where the only differences are (apart phisical addresses) that p2 is not a zerow and usb0 addresses change accordingly to the pi number

On the controller I've setup my wlan at first, I got from docs that I can't bridge it to see the zeroes so I reverted setting up eth0 with a static IP.

I've hacked very little the configuration files, but never been able to see my zeroes via ssh or ping them in 172.19.181.0/24 net from the controller.

My controller gives me this:

$ sudo brctl show

bridge name bridge id STP enabled interfaces

br0 8000.002282fffe01 no ethpi1

ethpi2

ethpi3

ethpi4

$ sudo ifconfig -a

br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet6 fe80::589a:5aff:fe34:81be  prefixlen 64  scopeid 0x20<link>

        ether 00:22:82:ff:fe:01  txqueuelen 1000  (Ethernet)

        RX packets 41785  bytes 1179468 (1.1 MiB)

        RX errors 0  dropped 4  overruns 0  frame 0

        TX packets 29  bytes 3010 (2.9 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet 192.168.1.45  netmask 255.255.255.0  broadcast 192.168.1.255

        inet6 fe80::ba27:ebff:fe41:c951  prefixlen 64  scopeid 0x20<link>

        ether b8:27:eb:41:c9:51  txqueuelen 1000  (Ethernet)

        RX packets 3633  bytes 472358 (461.2 KiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 1185  bytes 163154 (159.3 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ethpi1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet6 fe80::222:82ff:feff:fe01  prefixlen 64  scopeid 0x20<link>

        ether 00:22:82:ff:fe:01  txqueuelen 1000  (Ethernet)

        RX packets 11113  bytes 313624 (306.2 KiB)

        RX errors 1  dropped 0  overruns 0  frame 1

        TX packets 30729  bytes 1301162 (1.2 MiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ethpi2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet6 fe80::222:82ff:feff:fe02  prefixlen 64  scopeid 0x20<link>

        ether 00:22:82:ff:fe:02  txqueuelen 1000  (Ethernet)

        RX packets 9903  bytes 279594 (273.0 KiB)

        RX errors 1  dropped 0  overruns 0  frame 1

        TX packets 31938  bytes 1352062 (1.2 MiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ethpi3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet6 fe80::222:82ff:feff:fe03  prefixlen 64  scopeid 0x20<link>

        ether 00:22:82:ff:fe:03  txqueuelen 1000  (Ethernet)

        RX packets 10707  bytes 302232 (295.1 KiB)

        RX errors 1  dropped 0  overruns 0  frame 1

        TX packets 31135  bytes 1318238 (1.2 MiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ethpi4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet6 fe80::222:82ff:feff:fe04  prefixlen 64  scopeid 0x20<link>

        ether 00:22:82:ff:fe:04  txqueuelen 1000  (Ethernet)

        RX packets 10062  bytes 284018 (277.3 KiB)

        RX errors 1  dropped 0  overruns 0  frame 1

        TX packets 31779  bytes 1345412 (1.2 MiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

        inet 127.0.0.1  netmask 255.0.0.0

        inet6 ::1  prefixlen 128  scopeid 0x10<host>

        loop  txqueuelen 1000  (Local Loopback)

        RX packets 2  bytes 78 (78.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 2  bytes 78 (78.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4098<BROADCAST,MULTICAST>  mtu 1500

        ether b8:27:eb:14:9c:04  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 0  bytes 0 (0.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I would like to be able to update my zeroes and then install docker on all the boxes.

Can someone please give some hints on how to solve my problem?

Thank you by now for any answer.

Luca Paganotti

[Chris Burton]

Hi,

On the controller I've setup my wlan at first, I got from docs that I can't bridge it to see the zeroes so I reverted setting up eth0 with a static IP.

I've hacked very little the configuration files, but never been able to see my zeroes via ssh or ping them in 172.19.181.0/24 net from the controller. 

Which config files did you edit and in what way?

Did you alter /etc/dhcpcd.conf as the only thing I can see wrong is br0 isn't configured correctly as this should have the IP 172.19.181.254 ?

To use the NAT image with WiFi all you should need to edit is /etc/wpa_supplicant/wpa_supplicant.conf (or wpa_supplicant.conf in boot partition) and it should just work.

Chris.

[luca paganotti]

Hi Chris, thanks for answering,

Yes I noticed that my br0 is not setup correctly but I couldn't find a way to do that, so I decided to write to the group.

Here is my /etc/dhcpcd.conf:

# A sample configuration for dhcpcd.

# See dhcpcd.conf(5) for details.

# Allow users of this group to interact with dhcpcd via the control socket.

#controlgroup wheel

# Inform the DHCP server of our hostname for DDNS.

hostname

# Use the hardware address of the interface for the Client ID.

clientid

# or

# Use the same DUID + IAID as set in DHCPv6 for DHCPv4 ClientID as per RFC4361.

# Some non-RFC compliant DHCP servers do not reply with this set.

# In this case, comment out duid and enable clientid above.

#duid

# Persist interface configuration when dhcpcd exits.

persistent

# Rapid commit support.

# Safe to enable by default because it requires the equivalent option set

# on the server to actually work.

option rapid_commit

# A list of options to request from the DHCP server.

option domain_name_servers, domain_name, domain_search, host_name

option classless_static_routes

# Most distributions have NTP support.

option ntp_servers

# Respect the network MTU. This is applied to DHCP routes.

option interface_mtu

# A ServerID is required by RFC2131.

require dhcp_server_identifier

# Generate Stable Private IPv6 Addresses instead of hardware based ones

slaac private

# Example static IP configuration:

interface eth0

static ip_address=192.168.1.45/24

#static ip6_address=fd51:42f8:caae:d92e::ff/64

static routers=192.168.1.1

#static domain_name_servers=192.168.0.1 8.8.8.8 fd51:42f8:caae:d92e::1

static domain_name_servers=192.168.1.1 8.8.8.8

# It is possible to fall back to a static IP if DHCP fails:

# define static profile

#profile static_eth0

#static ip_address=192.168.1.23/24

#static routers=192.168.1.1

#static domain_name_servers=192.168.1.1

# fallback to static profile on eth0

#interface eth0

#fallback static_eth0

# ClusterHAT

denyinterfaces ethpi1 ethpi2 ethpi3 ethpi4

profile clusterhat_fallback_usb0

static ip_address=172.19.181.253/24 #ClusterHAT

static routers=172.19.181.254

static domain_name_servers=8.8.8.8 208.67.222.222

profile clusterhat_fallback_br0

static ip_address=172.19.181.254/24

interface usb0

fallback clusterhat_fallback_usb0

interface br0

fallback clusterhat_fallback_br0

as you can notice the only change in this file is in the "# Example static IP configuration:" section, where I tried to put my LAN address. No other changes to this file. I must say anyway that br0 was not functioning also before any change to this file.

Next you will find my /etc/network/interfaces file

# interfaces(5) file used by ifup(8) and ifdown(8)

# Include files from /etc/network/interfaces.d:

source-directory /etc/network/interfaces.d

allow-hotplug eth0

iface eth0 inet static

        address 192.168.1.45

        netmask 255.255.255.0

        gateway 192.168.1.1

        broadcast 192.168.1.255

auto br0

iface br0 inet manual

        bridge_ports none

        bridge_stp off

        bridge_waitport 0

        bridge_fd 0

# allow-hotplug wlan0

#       address 192.168.1.45

#       netmask 255.255.255.0

#       gateway 192.168.1.1

#    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

here I added a static setup to eth0, commenting my wlan0 setup, no other changes.

Then, here it comes my /etc/wpa_supplicant/wpa_supplicant.conf file:

country=IT

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev

update_config=1

network={

        ssid="<my wifi SSID>"

        key_mgmt="<my wifi password>"

}

where obviously wifi SSID and password are not the ones I wrote :-)

Hope this could be helpfull.

Have a nice day and thanks again ...

[Chris Burton]

Hi,

 

If you check the /var/log/syslog file you should see something similar to this.

May 28 08:12:34 cnat dhcpcd[293]: Not running dhcpcd because /etc/network/interfaces
May 28 08:12:34 cnat dhcpcd[293]: defines some interfaces that will use a
May 28 08:12:34 cnat dhcpcd[293]: DHCP client or static address

With newer Raspbian you should configure the static IP in /etc/dhcpcd.conf so just removing the eth0 config from /etc/network/interfaces and rebooting should fix it as you've already configured the static IP for eth0 in /etc/dhcpcd.conf.

Chris.

[luca paganotti]

Hi,

inspecting syslog logs tells me that I've no such messages in anyone of the syslog files from may 24, no previous syslog log file in my hand. Indeed it doesn't work 

[Chris Burton]

Hi, 

inspecting syslog logs tells me that I've no such messages in anyone of the syslog files from may 24, no previous syslog log file in my hand. Indeed it doesn't work 

And did you remove the config from /etc/network/interfaces? If you don't then it's not going to work.

Chris. 

[luca paganotti]

Hi, yes,

I commented all the iface section in the interfaces file. The result is that my pi cnat box is no more reachable in my network. I think this is related to dhcpcd setup or the overall networking system setup of this linux box. The file you mentioned are exactly as I described, so maybe the hack is elsewhere.

I can work on this starting from saturday, thank you. If you have some other hint it will be highly appreciated.

[luca paganotti]

Ok, it was a five minutes check, I thought dhcpcd enabled in the controller image by default ... That was not the case, now the bridge it's correctly setup and the zeroes are "seeing" the outside world ...

Me stupid ...

sudo systemctl enable --now dhcpcd.service

Thanks for the help.

[Chris Burton]

Hi,

Ok, it was a five minutes check, I thought dhcpcd enabled in the controller image by default ... That was not the case, now the bridge it's correctly setup and the zeroes are "seeing" the outside world ...

Me stupid ...

sudo systemctl enable --now dhcpcd.service

Glad to hear you got it working.

The images have dhcpcd enabled by default so I can only assume some of your other changes caused the issue.

Chris.