Updated (2018-03-13) Cluster HAT images and important information for Pi3+ users

[Chris Burton]

Updated Cluster HAT images (based on Raspbian Stretch with Desktop/Lite 2018-03-13)

Changes

=======

Along with the normal Controller image which use bridging I have created a new NAT image which doesn't bridge the Ethernet on the Controller Pi but uses NAT which can be used for both WiFi and Ethernet sharing.

Details of the new NAT image

============================

Due to what looks like a bug in the Pi3+ Ethernet driver adding the Pi3+ built in Ethernet device (eth0) to a bridge causes a kernel oops (https://github.com/raspberrypi/linux/issues/2437) and boot never gets to a login prompt! (This will also be a problem for those using the Pi3+ as a bridged Access Point)

To work around this problem on the Cluster HAT I've released the Controller NAT (Network Address Translation) Lite and Desktop [still uploading] images for both WiFi and Pi3+ users. We're keeping the standard bridged Controller and P1/P2/P3/P4 images for those not using a Pi3+.

The downside if using the new NAT configuration is you will no longer be able to log into the Pi Zeros directly from the local network (via WiFi or Ethernet). You must first log into the controller and then to Pi Zeros. If you need to access the Pi Zeros directly you could add iptables rules or use ssh port forwarding to forward connections via the Controller to the Pi Zero.

As with all of the other types of image (Controller/P1/P2/P3/P4/etc.) using the "Intermediate" setup you can reconfigure a Cluster HAT 2018-03-14 image to use NAT or the standard bridged network by appending " quiet init=/sbin/reconfig-clusterhat cnat" or " quiet init=/sbin/reconfig-clusterhat c".

Network configuration on the NAT (cnat) image

=============================================

ethpi1/ethpi2/ethpi3/ethpi4/etc. are still automatically bridged with br0 

BUT eth0 on the controller is NOT added to the bridge br0 on boot (this is the step which causes the Oops).

Pi Zeros are assigned the IP 172.19.181.X where X is the P number (P1=172.19.181.1, etc) and the Controller br0 interface is assigned 172.19.181.254.

NAT:When using with Ethernet

============================

Allows eth0 to either be configured with DHCP from the local network as normal or you can add a static IP address for eth0 to "/etc/dhcpcd.conf" see the commented out "interface eth0" section, connections from the Pi Zeros will use nat on the Controller to access the network/internet via the Controllers internet connection.

NAT:When using with WiFi

========================

When using WiFi you need to configure the wireless as normal either by placing a wpa_supplicant.conf in the boot partition or using the command line. Again connections from the Pi Zeros will use nat on the Controller to access the network/internet via the Controllers internet connection.

# Sample wpa_supplicant.conf file, replace GB with your 2 letter country code.

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev

update_config=1

country=GB

network={

    ssid="YourSSID"

    psk="YourPSK"

}

NAT:When using standalone

=========================

When no Ethernet/WiFi is available the Pi Zeros will still have IPs 172.19.181.X (where X is the P number) and 172.19.181.254 on the Controller br0 which allows communication between controller/zero an zero to zero.

Upgrade

=======

Create a backup image of your SD cards (https://www.raspberrypi.org/documentation/linux/filesystem/backup.md).

Then upgrade the Pi Zeros

# Run on each Pi Zero

sudo apt-get update

sudo apt-get dist-upgrade

sudo apt-get -y install subversion

sudo svn --force export https://github.com/burtyb/clusterhat-image/trunk/files /

echo 'iptables-persistent iptables-persistent/autosave_v4 boolean false' | debconf-set-selections

echo 'iptables-persistent iptables-persistent/autosave_v6 boolean false' | debconf-set-selections

APT_LISTCHANGES_FRONTEND=none apt-get -y install iptables-persistent

echo '#net.ipv4.ip_forward=1 # Cluster HAT NAT' >> /etc/sysctl.conf

  cat << EOF >> /etc/iptables/rules.v4

# Generated by iptables-save v1.6.0 on Fri Mar 13 00:00:00 2018

*filter

:INPUT ACCEPT [7:1365]

:FORWARD ACCEPT [0:0]

:OUTPUT ACCEPT [0:0]

-A FORWARD -i br0 ! -o br0 -j ACCEPT

-A FORWARD -o br0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

COMMIT

# Completed on Fri Mar 13 00:00:00 2018

# Generated by iptables-save v1.6.0 on Fri Mar 13 00:00:00 2018

*nat

:PREROUTING ACCEPT [8:1421]

:INPUT ACCEPT [7:1226]

:OUTPUT ACCEPT [0:0]

:POSTROUTING ACCEPT [0:0]

-A POSTROUTING -s 172.19.181.0/24 ! -o br0 -j MASQUERADE

COMMIT

# Completed on Fri Mar 13 00:00:00 2018

EOF

cat << EOF >> /etc/dhcpcd.conf

# ClusterHAT

denyinterfaces eth0 ethpi1 ethpi2 ethpi3 ethpi4

profile clusterhat_fallback_usb0

static ip_address=172.19.181.253/24 #ClusterHAT

static routers=172.19.181.254

static domain_name_servers=8.8.8.8 208.67.222.222

profile clusterhat_fallback_br0

static ip_address=172.19.181.254/24

interface usb0

fallback clusterhat_fallback_usb0

interface br0

fallback clusterhat_fallback_br0

EOF

On P1 append " quiet init=/sbin/reconfig-clusterhat p1" to the end of the cmdline.txt file in /boot (keeping it all on one line) and shutdown (repeat changing "p1" to p2/p3/p4 and then c for the other installations).

Then update the Controller

# Run on the Controller

sudo apt-get update

sudo apt-get dist-upgrade

sudo apt-get -y install subversion

sudo svn --force export https://github.com/burtyb/clusterhat-image/trunk/files /

echo 'iptables-persistent iptables-persistent/autosave_v4 boolean false' | debconf-set-selections

echo 'iptables-persistent iptables-persistent/autosave_v6 boolean false' | debconf-set-selections

APT_LISTCHANGES_FRONTEND=none apt-get -y install iptables-persistent

echo '#net.ipv4.ip_forward=1 # Cluster HAT NAT' >> /etc/sysctl.conf

  cat << EOF >> /etc/iptables/rules.v4

# Generated by iptables-save v1.6.0 on Fri Mar 13 00:00:00 2018

*filter

:INPUT ACCEPT [7:1365]

:FORWARD ACCEPT [0:0]

:OUTPUT ACCEPT [0:0]

-A FORWARD -i br0 ! -o br0 -j ACCEPT

-A FORWARD -o br0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

COMMIT

# Completed on Fri Mar 13 00:00:00 2018

# Generated by iptables-save v1.6.0 on Fri Mar 13 00:00:00 2018

*nat

:PREROUTING ACCEPT [8:1421]

:INPUT ACCEPT [7:1226]

:OUTPUT ACCEPT [0:0]

:POSTROUTING ACCEPT [0:0]

-A POSTROUTING -s 172.19.181.0/24 ! -o br0 -j MASQUERADE

COMMIT

# Completed on Fri Mar 13 00:00:00 2018

EOF

cat << EOF >> /etc/dhcpcd.conf

# ClusterHAT

denyinterfaces eth0 ethpi1 ethpi2 ethpi3 ethpi4

profile clusterhat_fallback_usb0

static ip_address=172.19.181.253/24 #ClusterHAT

static routers=172.19.181.254

static domain_name_servers=8.8.8.8 208.67.222.222

profile clusterhat_fallback_br0

static ip_address=172.19.181.254/24

interface usb0

fallback clusterhat_fallback_usb0

interface br0

fallback clusterhat_fallback_br0

EOF

Append " quiet init=/sbin/reconfig-clusterhat c" to the end of the cmdline.txt file in /boot (keeping it all on one line) and reboot or if you want to switch to the new NAT configuration append " quiet init=/sbin/reconfig-clusterhat cnat" instead.

Please remember the "c" configuration will not work on a Raspberry Pi B 3+, the "cnat" image will work on any (A+/B+/2/3/3+).

======

Build script and files will be updated shortly on https://github.com/burtyb/clusterhat-image/ .

If you have problems with the images or suggestions please open a ticket at https://secure.8086.net/billing/submitticket.php?step=2&deptid=1 or reply here.

Chris.

[Jerome Ibanes]

Some of the isos are 404; thank you for looking into that.

[Chris Burton]

Hi,

On Monday, 19 March 2018 23:50:04 UTC, Jerome Ibanes wrote:

Some of the isos are 404; thank you for looking into that.

Sorry about that, I'd somehow uploaded the "img" and not "zip" for one of them and had the capitalisation "nat" not "NAT" for the other - should be fixed now.

Chris.

[Andrew Shaw Care]

It seems like that issue has been closed... Is it now possible to create new images for the Raspberry Pi B 3+ using bridging?

[Chris Burton]

Hi,

It seems like that issue has been closed... Is it now possible to create new images for the Raspberry Pi B 3+ using bridging?

Sadly I don't class the Pi3+ as stable yet as there are other issues I'm waiting for such as https://github.com/raspberrypi/linux/issues/2449 and https://github.com/raspberrypi/firmware/issues/968 (at the moment I'm unable to build the Cluster HAT images using a Pi3+ as the network performance is so poor).

If you want to give it a try then just boot the "cnat" image, update and append " quiet init=/sbin/reconfig-clusterhat c" to /boot/cmdline.txt (making sure it's still one line) and reboot. If it doesn't work then you can switch back by appending " quiet init=/sbin/reconfig-clusterhat cnat" by editing the cmdline.txt file in the boot (FAT32) filesystem.

Chris.

[Patrick Nooijen]

i'm using the new images with my pi3 and clusterhat v2. I can not get i to work with my network. Out of the box is gets the 172 range ip. How can i set dhcp so the controller and zero get a dhcp address from my router?

Op maandag 19 maart 2018 13:58:27 UTC+1 schreef Chris Burton:

[Patrick Nooijen]

Editing the dhcpcd file i hot the ip address in, but i can not ping my router?

[Chris Burton]

Hi, 

Editing the dhcpcd file i hot the ip address in, but i can not ping my router?

If you want both controller and Pi Zeros to obtain IP addresses using DHCP on the local network then you'll need to use the "controller" image (not CNAT).

With the network connection on the Ethernet (eth0) port on the controller and you shouldn't need to edit anything.

Does the image you're using on the controller have a hostname of "controller" ?

What did you edit in dhcpcd.conf?

What does "ifconfig -a;brctl show;ip route" look like?

Chris. 

[Patrick Nooijen]

I use the normal controller image. Without edit i dont get a dhcp address. After editing i can Access the internet but no lan ip's are accessable?

[Patrick Nooijen]

Juist replaced my sd card and that works out of the box. Strange.... but thanks!

[Mitchell Harding]

Hi is this compatible with usbboot as well?

[Chris Burton]

Hi,

Hi is this compatible with usbboot as well?

The upgrade instructions here aren't for the usboot image.

But the normal problem with the usbboot image after an update is if the kernel changes you'll need to rebuild the initramfs otherwise the Pi Zeros won't be able to boot - see  https://groups.google.com/d/msg/clusterhat/XoyQ_x6s9L4/Cq9oP6bCAwAJ (those commands need to be ran as root so "sudo -i" should get you that from the pi user).

Chris.

[jcl...@umbc.edu]

Hi Chris,

When I try to run "echo 'iptables-persistent iptables-persistent/autosave_v4 boolean false' | debconf-set-selections", I get the following errors:

"

debconf: DbDriver "passwords" warning: could not open /var/cache/debconf/passwords.dat: Permission denied

debconf: DbDriver "config" could not write /var/cache/debconf/config.dat-new: Permission denied

"

In my /var/cache/debconf folder, I have:

config.dat , confit.dat-old , passwords.dat , templates.dat , templates.dat-old

However, my passwords.dat file is completely empty.

I have ran numerous updates and upgrades but have no been able to find a solution online whatsoever.

-Jake

[jcl...@umbc.edu]

I also tried changing permissions on the files so that I could read/write to the respective files, however I get an error saying that config can not be found.

[Chris Burton]

Hi, 

When I try to run "echo 'iptables-persistent iptables-persistent/autosave_v4 boolean false' | debconf-set-selections", I get the following errors:

Since you're replied to a "usbboot" image reply I'm not sure if you're using that image but if you are don't follow any of the instructions here.

If you're using the image in the title of the email then you most likely need to switch to the root user (on both controller and pi zeros) before running the commands shown after "sudo svn".

You can do this with "sudo -i" to make life easier and then at the end of the upgrade steps you can just "reboot". 

I'd also advise putting the permissions back on any files you changed to try and fix the permissions error.

Chris.

[jcl...@umbc.edu]

Ah I see my issue here. Do you have instructions for setting up a 3B+ cluster hat with Pi Zero Wireless?

[Chris Burton]

 Hi,

Ah I see my issue here. Do you have instructions for setting up a 3B+ cluster hat with Pi Zero Wireless?

https://clusterhat.com/setup-software works with Pi Zero or Pi Zero W.

If you want to do something non-standard then please start a new thread explaining what you're trying to do.

Chris.

[dave r]

Hi Chris - awesome job on the clusterhat - thank you.

I was wondering if the updates at your bug report:

https://github.com/raspberrypi/linux/issues/2437

...mean that this issue is now fixed... and if so...

...wondering if this means that we can now use the non-NAT based images for the RPi B 3+ ?

Thanks.

Dave.

[Chris Burton]

Hi, 

...wondering if this means that we can now use the non-NAT based images for the RPi B 3+ ?

I (quietly) released updated images a few days ago (https://clusterhat.com/setup-software) so for new installs it should already work OK.

I haven't put upgrade notes on here yet as I need to test upgrading the clusterhat "files" side of things a bit more - I'll do that as soon as I can.

But (after backing up) you should be able to upgrade Raspbian to get the new kernel on the previous image and get non-NAT (bridged) network working on a Pi 3b+.

sudo apt-get update
sudo apt-get dist-upgrade

# and these if you're running "with desktop"
sudo apt-get install qpdfview
sudo apt-get purge xpdf
sudo apt-get install rp-prefapps
sudo apt-get install piwiz

Then if you was running NAT you can switch to bridged network by appending " quiet init=/sbin/reconfig-clusterhat c" to the /boot/cmdline.txt on the controller and reboot (remember this will reset any changes to the hostname/etc. as normal).

Chris.

[David Glaude]

Hi Chris,

Did you see improvement on the network side, both Ethernet and USB gadget with the PiZero?
Do we practically benefit from the + of RPi B 3+?

Also, do you have plan to update the usbboot version, it seems 3+ and usbboot is a great combo (especially if you do PoE too like in the picture you shared).

Regards.

David Glaude

[Chris Burton]

Hi, 

Did you see improvement on the network side, both Ethernet and USB gadget with the PiZero?

The network side of the controller to the outside world is obviously faster if you have a gigabit network but the speed between Pi Zero and Controller (or network via the controller) hasn't changed from my testing.

 

Do we practically benefit from the + of RPi B 3+?

You'd probably benefit from better thermal management with a HAT on top, other than that it depends if you'd benefit from any of the new features (faster CPU/network/etc).

Chris.