Rpi4 + radxa zero

ClusterHAT

unread,

Sep 18, 2023, 2:30:51 PM9/18/23

to ClusterHAT

Hi! I have the next setup:
Rpi4 as controller with NAT image
ClusterHat v2.5
2x RADXA Zero
2x USB-C to MicroUsb adapter

For now, I've managed to connect one Zero to the controller using this dhcpcd.conf file:
```
# ClusterCTRL

reboot 15

denyinterfaces ethpi* ethupi* ethupi*.10 brint usb0.10

profile clusterctrl_fallback_usb0

static ip_address=172.19.181.2/24 #ClusterCTRL

static routers=172.19.181.253

static domain_name_servers=8.8.8.8 208.67.222.222

profile clusterctrl_fallback_br0

static ip_address=172.19.181.253/24

interface usb0

fallback clusterctrl_fallback_usb0

interface br0

fallback clusterctrl_fallback_br0
```
However, when I plug in the second one, it's connected through the usb1 interface, and an invalid IP is being assigned.

Can someone help me configure the nodes? Thank you!"

Chris Burton

unread,

Sep 22, 2023, 2:30:37 PM9/22/23

to ClusterHAT

Hi,

Please understand I have no idea what software you're using on the RADXA Zero and have no idea what changes you have or haven't made.

static ip_address=172.19.181.2/24 #ClusterCTRL

Did you change this to a different IP on the second Zero?

However, when I plug in the second one, it's connected through the usb1 interface, and an invalid IP is being assigned.

Can someone help me configure the nodes? Thank you!"

It might be due to the IP conflict if you haven't used a different IP on the second Zero and it's falling back to a 169.254.x.x IP.

Otherwise what do you mean by invalid IP?

Is the first Zero accessible on 172.19.181.2 IP?

If you boot the second Zero with the first Zero powered off does it work as expected?

Chris.

Alejandro Perez Favieres

unread,

Sep 30, 2023, 10:06:11 AM9/30/23

to ClusterHAT

Exactly, Chris. It only works with one node at a time. I'm using a static IP address of 172.19.181.1/24 for node 1 and a static IP address of 172.19.181.2/24 for node two. When I only plug in one of each node, everything works as expected. The problem appears when I power on the other node; the master sees two interfaces (usb0 and usb1), but only the node attached to the usb0 interface works.

I'm using a fresh intallation os Ubuntu Server 20.04

Chris Burton

unread,

Oct 1, 2023, 10:23:39 AM10/1/23

to ClusterHAT

Hi,

Exactly, Chris. It only works with one node at a time. I'm using a static IP address of 172.19.181.1/24 for node 1 and a static IP address of 172.19.181.2/24 for node two. When I only plug in one of each node, everything works as expected. The problem appears when I power on the other node; the master sees two interfaces (usb0 and usb1), but only the node attached to the usb0 interface works.

I'm using a fresh intallation os Ubuntu Server 20.04

I don't know how Ubuntu Server 20.04 works to give you OTG or if you've made changes to do it but is it giving different MAC addresses on both sides of whichever type of Gadget Ethernet device you're using? Running "ip -a" on both node and controller should show you the MAC on the line with "link/ether" for the device.

Chris.

Alejandro Perez Favieres

unread,

Oct 1, 2023, 1:45:05 PM10/1/23

to ClusterHAT

Thank you Chris!

I used this service to convert the devices to USBGadget: https://github.com/radxa-pkg/radxa-otgutils. I'm looking for the MAC address, but it seems to be generated randomly on each device.

These are the logs in the dmesg after plugging in the devices:

[ 305.721576] usb 1-1.1.3: new full-speed USB device number 23 using xhci_hcd

[ 305.823506] usb 1-1.1.3: not running at top speed; connect to a high speed hub

[ 305.829754] usb 1-1.1.3: New USB device found, idVendor=1d6b, idProduct=0104, bcdDevice= 1.00

[ 305.829769] usb 1-1.1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3

[ 305.829775] usb 1-1.1.3: Product: OTG Utils

[ 305.829781] usb 1-1.1.3: Manufacturer: Radxa

[ 305.829786] usb 1-1.1.3: SerialNumber: 0123456789ABCDEF

[ 305.841746] cdc_ether 1-1.1.3:1.0 usb0: register 'cdc_ether' at usb-0000:01:00.0-1.1.3, CDC Ethernet Device, 3a:fb:47:d3:45:06

[ 2275.562555] usb 1-1.1.4: new full-speed USB device number 24 using xhci_hcd

[ 2275.664468] usb 1-1.1.4: not running at top speed; connect to a high speed hub

[ 2275.672716] usb 1-1.1.4: New USB device found, idVendor=1d6b, idProduct=0104, bcdDevice= 1.00

[ 2275.672730] usb 1-1.1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3

[ 2275.672736] usb 1-1.1.4: Product: OTG Utils

[ 2275.672741] usb 1-1.1.4: Manufacturer: Radxa

[ 2275.672746] usb 1-1.1.4: SerialNumber: 0123456789ABCDEF

[ 2275.684706] cdc_ether 1-1.1.4:1.0 usb1: register 'cdc_ether' at usb-0000:01:00.0-1.1.4, CDC Ethernet Device, 56:91:05:4a:06:48

And the content in the ifcongif -a:

usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 172.19.181.254 netmask 255.255.255.0 broadcast 172.19.181.255

inet6 fe80::9fbe:9a85:6046:a538 prefixlen 64 scopeid 0x20<link>

ether 3a:fb:47:d3:45:06 txqueuelen 1000 (Ethernet)

RX packets 580 bytes 79415 (77.5 KiB)

RX errors 1 dropped 0 overruns 0 frame 0

TX packets 914 bytes 86172 (84.1 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

usb1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 172.19.181.254 netmask 255.255.255.0 broadcast 172.19.181.255

inet6 fe80::3554:dae:9766:f700 prefixlen 64 scopeid 0x20<link>

ether 56:91:05:4a:06:48 txqueuelen 1000 (Ethernet)

RX packets 37 bytes 4239 (4.1 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 32 bytes 4504 (4.3 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

The only suspicious thing I have noticed is that both devices are using the same serial number.

Chris Burton

unread,

Oct 1, 2023, 3:47:39 PM10/1/23

to ClusterHAT

Hi,

The only suspicious thing I have noticed is that both devices are using the same serial number.

If you want to do it the same way as the ClusterCTRL pX images do then you shouldn't have an IP on the usbX interfaces (and they shouldn't be the same IP if you don't). Remove the IPs from usb0 and usb1 on the controller side and then add them into the bridge br0 (which should have the IP).

brctl addif br0 usb0

brctl addif br0 usb1

Using the ClusterCTRL images on a node (Pi Zero) it would normally set a MAC address (on both controller and node side of the USB Gadget device) with the last 2 hex digits being the node number in hex it gets from /etc/defaults/clusterctrl (see https://github.com/burtyb/clusterhat-image/blob/master/files/usr/sbin/composite-clusterctrl ). If you can get your Zeros to have a MAC "00:22:82:ff:fe:XX on the controller side and 00:22:82:ff:ff:XX on the node itself then it should auto setup using the config explained below. If not then you would need to replicate what these are doing to get it automatically added into the bridge.

The usb0 device is then renamed to ethpiX based on this MAC address (see https://github.com/burtyb/clusterhat-image/blob/master/files/etc/udev/rules.d/90-clusterctrl.rules ).

Then once the usbX device has the correct ethpiX name it's automatically added to the bridge (see https://github.com/burtyb/clusterhat-image/blob/master/files/usr/share/clusterctrl/interfaces.cnat ).

Chris.

Alejandro Perez Favieres

unread,

Oct 2, 2023, 1:58:52 AM10/2/23

to ClusterHAT

Yeah, that was it! Finally, I have both nodes working. Thanks for everything!

Alejandro Perez Favieres

unread,

Oct 11, 2023, 6:20:16 AM10/11/23

to ClusterHAT

Hi Chris Burton.

As I mentioned in the last post, if I manually add the usb0 interface to the bridge, it works fine.

Now, I'm trying to add it to the bridge automatically. To achieve this, I have modified the MAC and configuration of the USB gadget to be similar to the 'clusterctl' config.

Currently, I have the 'ethpip1' interface, but I can't obtain an IP.

ethpip1: flags=4098<BROADCAST,MULTICAST> mtu 1500

ether 00:22:82:ff:fe:01 txqueuelen 1000 (Ethernet)

RX packets 0 bytes 0 (0.0 B)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 0 bytes 0 (0.0 B)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

dmegs logs:

[ 67.493066] usb 1-1.1.4: New USB device found, idVendor=3171, idProduct=0020, bcdDevice= 1.00

[ 67.493097] usb 1-1.1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3

[ 67.493104] usb 1-1.1.4: Product: ClusterCTRL

[ 67.493110] usb 1-1.1.4: Manufacturer: 8086 Consultancy

[ 67.493116] usb 1-1.1.4: SerialNumber: p1

[ 67.568233] cdc_ether 1-1.1.4:1.0 eth1: register 'cdc_ether' at usb-0000:01:00.0-1.1.4, CDC Ethernet Device, 00:22:82:ff:fe:01

[ 67.568419] usbcore: registered new interface driver cdc_ether

[ 67.686107] cdc_ether 1-1.1.4:1.0 ethpip1: renamed from eth1

Thank you! All your support it's appreciate.

Chris Burton

unread,

Oct 17, 2023, 9:52:04 AM10/17/23

to ClusterHAT

Hi,

Currently, I have the 'ethpip1' interface, but I can't obtain an IP.

ethpip1: flags=4098<BROADCAST,MULTICAST> mtu 1500

ether 00:22:82:ff:fe:01 txqueuelen 1000 (Ethernet)

RX packets 0 bytes 0 (0.0 B)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 0 bytes 0 (0.0 B)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

This is expected, ethpi1 (the controller side) of the USB gadget shouldn't have an IP address, it's normally bridged to "br0" which has the IP address.

Do you have a "br0" interface and does that have an IP? What does "brctl show" say?

Chris.

Alejandro Perez Favieres

unread,

Oct 17, 2023, 11:09:43 AM10/17/23

to ClusterHAT

After a fresh restart, getting the ethpip1 interface up:

bridge name bridge id STP enabled interfaces
br0 8000.d83add1be5aa no
brint 8000.000000000000 no
cni0 8000.3e96c376b7f8 no veth3a6c8f88
veth6564a1c2
veth7667f613
veth91ba20dd
vetha00a6aa2
vetha147c52b
vethb2b3367e
vethd6339949
vethf2073a75
vethfafa6d9c

If I add manually the ethpip1 to the bridge, the br0 interfaces show the ethpip1, but I can't do ping to 172.19.181.1

PD: cni0 is used by the Kubernetes cluster.

Chris Burton

unread,

Oct 20, 2023, 3:08:59 AM10/20/23

to ClusterHAT

Hi,

After a fresh restart, getting the ethpip1 interface up:

bridge name bridge id STP enabled interfaces
br0 8000.d83add1be5aa no
brint 8000.000000000000 no
cni0 8000.3e96c376b7f8 no veth3a6c8f88
veth6564a1c2
veth7667f613
veth91ba20dd
vetha00a6aa2
vetha147c52b
vethb2b3367e
vethd6339949
vethf2073a75
vethfafa6d9c

If I add manually the ethpip1 to the bridge, the br0 interfaces show the ethpip1, but I can't do ping to 172.19.181.1

Do you have an IP in the same range on br0 (the ClusterCTRL CNAT image uses 172.19.181.254/24 on br0)?

On the controller if you have 172.19.181.254 on br0 and any ethpiX in the br0 bridge, on the Px you have 172.19.181.1/2/etc on usb0 then you should be able to ping 172.19.181.1/2/etc. from the controller.

Chris.

Alejandro Perez Favieres

unread,

Oct 20, 2023, 3:42:55 AM10/20/23

to ClusterHAT

I'm not sure because when I exec the ifconfig -a command don't show the ip:

br0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether d8:3a:dd:1b:e5:aa txqueuelen 1000 (Ethernet)

RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0

TX packets 8 bytes 1016 (1016.0 B)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

But I can do ping to the 172.19.180.254 and I see the config of the bridge. So I guess It's wright configured.

#
# Interface definitions for Cluster CTRL (bridged controller)
#

auto brint br0

# Interfaces bridged to brint (internal network)
# - usbboot - Pi node "usb0.10" (VLAN 10 for NFSROOT)
# - SD boot - unused

iface brint inet static
bridge_ports none
address 172.19.180.254
netmask 255.255.255.0
bridge_stp off
bridge_waitport 0
bridge_fd 0

Chris Burton

unread,

Oct 26, 2023, 3:40:03 AM10/26/23

to ClusterHAT

Hi,

I'm not sure because when I exec the ifconfig -a command don't show the ip:

Using "ip a" might give a better idea of all IPs on the interfaces, if it replies to ping it must be there somewhere on the network :).

Chris.

Reply all

Reply to author

Forward