Pi4b Clusterhat 2.4 (bridge) image does not communicate via USB
139 views
Skip to first unread message
Tony Brack
May 9, 2021, 11:57:11 PM5/9/21
to ClusterHAT
Greetings All,
I have used numerous up to date cnat or cbridge images to try to set up my Pi4b with a ClusterHAT 2.4. I am seeing MAC addresses on the Pi4b as well as serial devices, but neither is able to communicate with the Pi ZEROs. I have also tried to attach a Pi ZERO (P20) image with a USB connection, also to no avail. 32 and 64 bit images show the same behavior. I have seen extremely intermittent success. One of the Pis is a Pi ZERO-W, so I should be able to post its view of the connection.
Currently the Pi4b and Pi ZEROs have:
- 2020-08-20-8-ClusterCTRL-arm64-lite-CBRIDGE
- 2020-12-02-1-ClusterCTRL-armhf-lite-p1 to p5
I have applied the latest apt update to all of these images.
I have created a script that reports most of the current state:
Static hostname: Pi4b-64-cluster
Icon name: computer
Machine ID: b659d7240f29459aab330cb98fbf6d05
Boot ID: 6c0d87d483324813ac0b916d2e3235c6
Operating System: Debian GNU/Linux 10 (buster)
Kernel: Linux 5.10.17-v8+
Architecture: arm64
-- testing clusterhat --
clusterhat:1
clusterctrl:False
maxpi:4
throttled:0x0
hat_version:2.4
hat_version_major:2
hat_version_minor:4
hat_size:4
hat_uuid:de91a4ce-ac7f-11e9-a2a3-2a2ae2dbcce4
hat_vendor:8086 Consultancy
hat_product_id:0x0004
hat_alert:0
hat_hub:1
hat_wp:1
hat_led:1
hat_wplink:0
hat_xra1200p:True
p1:1
p2:1
p3:1
p4:1
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.120 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::79dc:886f:660d:fa2d prefixlen 64 scopeid 0x20<link>
ether dc:a6:32:ad:52:49 txqueuelen 1000 (Ethernet)
RX packets 486550 bytes 61897779 (59.0 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 189856 bytes 43163844 (41.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
brint: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.19.180.254 netmask 255.255.255.0 broadcast 172.19.180.255
inet6 fe80::6057:baff:fe79:157c prefixlen 64 scopeid 0x20<link>
ether 62:57:ba:79:15:7c txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 52 bytes 5909 (5.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether dc:a6:32:ad:52:49 txqueuelen 1000 (Ethernet)
RX packets 486432 bytes 70446842 (67.1 MiB)
RX errors 0 dropped 89 overruns 0 frame 0
TX packets 291836 bytes 47540932 (45.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ethpi1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::222:82ff:feff:fe01 prefixlen 64 scopeid 0x20<link>
ether 00:22:82:ff:fe:01 txqueuelen 1000 (Ethernet)
RX packets 180 bytes 38348 (37.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1 bytes 134 (134.0 B)
TX errors 310880 dropped 0 overruns 0 carrier 0 collisions 0
ethpi2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::222:82ff:feff:fe02 prefixlen 64 scopeid 0x20<link>
ether 00:22:82:ff:fe:02 txqueuelen 1000 (Ethernet)
RX packets 25232 bytes 740753 (723.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1 bytes 134 (134.0 B)
TX errors 311112 dropped 0 overruns 0 carrier 0 collisions 0
ethpi3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::222:82ff:feff:fe03 prefixlen 64 scopeid 0x20<link>
ether 00:22:82:ff:fe:03 txqueuelen 1000 (Ethernet)
RX packets 25045 bytes 704174 (687.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 7 bytes 1420 (1.3 KiB)
TX errors 310996 dropped 0 overruns 0 carrier 0 collisions 0
ethpi4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::222:82ff:feff:fe04 prefixlen 64 scopeid 0x20<link>
ether 00:22:82:ff:fe:04 txqueuelen 1000 (Ethernet)
RX packets 25043 bytes 704118 (687.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 65 bytes 15411 (15.0 KiB)
TX errors 277704 dropped 0 overruns 0 carrier 0 collisions 0
ethpi20: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::222:82ff:feff:fe14 prefixlen 64 scopeid 0x20<link>
ether 00:22:82:ff:fe:14 txqueuelen 1000 (Ethernet)
RX packets 25066 bytes 712878 (696.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 34 bytes 8347 (8.1 KiB)
TX errors 310822 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 1562 bytes 2806072 (2.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1562 bytes 2806072 (2.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.15 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 2001:1970:55e2:6500:6b0d:9a41:6f11:7e14 prefixlen 64 scopeid 0x0<global>
inet6 2001:1970:55e2:6500::72ae prefixlen 128 scopeid 0x0<global>
inet6 fe80::ec5c:6172:550a:44b0 prefixlen 64 scopeid 0x20<link>
ether 6e:aa:e8:7f:6d:57 txqueuelen 1000 (Ethernet)
RX packets 155447 bytes 160697508 (153.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 9447 bytes 1372562 (1.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default gw.home.brack.c 0.0.0.0 UG 213 0 0 br0
default 192.168.0.1 0.0.0.0 UG 307 0 0 wlan0
172.19.180.0 0.0.0.0 255.255.255.0 U 0 0 0 brint
192.168.0.0 0.0.0.0 255.255.255.0 U 307 0 0 wlan0
192.168.1.0 0.0.0.0 255.255.255.0 U 213 0 0 br0
PING 192.168.1.120 (192.168.1.120) 56(84) bytes of data.
64 bytes from 192.168.1.120: icmp_seq=1 ttl=64 time=0.104 ms
--- 192.168.1.120 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.104/0.104/0.104/0.000 ms
PING 192.168.1.121 (192.168.1.121) 56(84) bytes of data.
From 192.168.1.120 icmp_seq=1 Destination Host Unreachable
--- 192.168.1.121 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
PING 192.168.1.122 (192.168.1.122) 56(84) bytes of data.
From 192.168.1.120 icmp_seq=1 Destination Host Unreachable
--- 192.168.1.122 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
PING 192.168.1.123 (192.168.1.123) 56(84) bytes of data.
From 192.168.1.120 icmp_seq=1 Destination Host Unreachable
--- 192.168.1.123 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
PING 192.168.1.124 (192.168.1.124) 56(84) bytes of data.
From 192.168.1.120 icmp_seq=1 Destination Host Unreachable
--- 192.168.1.124 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
PING 192.168.1.125 (192.168.1.125) 56(84) bytes of data.
From 192.168.1.120 icmp_seq=1 Destination Host Unreachable
--- 192.168.1.125 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
? (192.168.0.1) at 02:00:00:00:00:04 [ether] on wlan0
gw.home.brack.ca (192.168.1.1) at 00:0c:29:80:d2:68 [ether] on br0
p1.home.brack.ca (192.168.1.121) at <incomplete> on br0
p2.home.brack.ca (192.168.1.122) at <incomplete> on br0
p3.home.brack.ca (192.168.1.123) at <incomplete> on br0
p4.home.brack.ca (192.168.1.124) at <incomplete> on br0
p20.home.brack.ca (192.168.1.125) at <incomplete> on br0
CNCSBRACKAD3C.home.brack.ca (192.168.1.13) at 18:66:da:40:aa:f8 [ether] on br0
thunderbolt.home.brack.ca (192.168.1.167) at 54:bf:64:2f:5d:16 [ether] on br0
-- listing serial ports --
crw-rw---- 1 root gpio 254, 0 May 9 12:05 /dev/gpiochip0
crw-rw---- 1 root gpio 254, 1 May 9 12:05 /dev/gpiochip1
crw-rw---- 1 root gpio 246, 0 May 9 12:05 /dev/gpiomem
crw-rw---- 1 root video 238, 0 May 9 12:05 /dev/rpivid-h264mem
crw-rw---- 1 root video 240, 0 May 9 12:05 /dev/rpivid-hevcmem
crw-rw---- 1 root video 239, 0 May 9 12:05 /dev/rpivid-intcmem
crw-rw---- 1 root video 237, 0 May 9 12:05 /dev/rpivid-vp9mem
lrwxrwxrwx 1 root root 7 May 9 12:05 /dev/ttypi1 -> ttyACM0
lrwxrwxrwx 1 root root 7 May 9 12:05 /dev/ttypi1a -> ttyACM1
lrwxrwxrwx 1 root root 7 May 9 12:05 /dev/ttypi2 -> ttyACM2
lrwxrwxrwx 1 root root 7 May 9 12:06 /dev/ttypi20 -> ttyACM8
lrwxrwxrwx 1 root root 7 May 9 12:06 /dev/ttypi20a -> ttyACM9
lrwxrwxrwx 1 root root 7 May 9 12:05 /dev/ttypi2a -> ttyACM3
lrwxrwxrwx 1 root root 7 May 9 12:05 /dev/ttypi3 -> ttyACM4
lrwxrwxrwx 1 root root 7 May 9 12:05 /dev/ttypi3a -> ttyACM5
lrwxrwxrwx 1 root root 7 May 9 12:05 /dev/ttypi4 -> ttyACM6
lrwxrwxrwx 1 root root 7 May 9 12:05 /dev/ttypi4a -> ttyACM7
I am about to test with a standard DW2 ethernet and serial 'gadget' Pis and will post the results of that testing as well.
Tony
Tony Brack
May 10, 2021, 2:04:08 AM5/10/21
to ClusterHAT
I took the same Pi ZERO that I used as P20 (updated P5 image) and plugged it in to another machine running another cbridge image without a ClusterHAT and was able to communicate with both the serial interface and the network interface. The working host is a Pi4b running 32 bit:
Thanks
pi@flirc-4b:~ $ hostnamectl
Static hostname: flirc-4b
Icon name: computer
Machine ID: 062cd7dd815f4306b6b44abe2752e446
Boot ID: 48819624fc074c1696968d498e9bdd77
Operating System: Raspbian GNU/Linux 10 (buster)
Kernel: Linux 5.10.17-v7l+
Architecture: arm
pi@flirc-4b:~ $ ping -c 1 p20
PING p20.home.brack.ca (192.168.1.125) 56(84) bytes of data.
64 bytes from p20.home.brack.ca (192.168.1.125): icmp_seq=1 ttl=64 time=0.540 ms
--- p20.home.brack.ca ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.540/0.540/0.540/0.000 ms
pi@flirc-4b:~ $ arp -a | sort -k2
? (192.168.0.1) at 02:00:00:00:00:04 [ether] on wlan0
p20.home.brack.ca (192.168.1.125) at 00:22:82:ff:ff:14 [ether] on br0
laptop (192.168.1.167) at 54:bf:64:2f:5d:16 [ether] on br0
gw.home.brack.ca (192.168.1.1) at 00:0c:29:80:d2:68 [ether] on br0
pi@flirc-4b:~ $ route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default gw.home.brack.c 0.0.0.0 UG 205 0 0 br0
default 192.168.0.1 0.0.0.0 UG 303 0 0 wlan0
172.19.180.0 0.0.0.0 255.255.255.0 U 0 0 0 brint
192.168.0.0 0.0.0.0 255.255.255.0 U 303 0 0 wlan0
192.168.1.0 0.0.0.0 255.255.255.0 U 205 0 0 br0
Tony Brack
May 10, 2021, 2:20:39 AM5/10/21
to ClusterHAT
From the failing Pi4b (with a ClusterHAT) - reconnecting p20
pi@Pi4b-64-cluster:~ $ ifconfig ethpi20
ethpi20: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::222:82ff:feff:fe14 prefixlen 64 scopeid 0x20<link>
ether 00:22:82:ff:fe:14 txqueuelen 1000 (Ethernet)
RX packets 538 bytes 26910 (26.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 17 bytes 2516 (2.4 KiB)
TX errors 4988 dropped 0 overruns 0 carrier 0 collisions 0
pi@Pi4b-64-cluster:~ $ arp p20
Address HWtype HWaddress Flags Mask Iface
p20.home.brack.ca (incomplete) br0
pi@Pi4b-64-cluster:~ $ ping -c 1 p20
PING p20.home.brack.ca (192.168.1.125) 56(84) bytes of data.
From Pi4b-64-cluster.home.brack.ca (192.168.1.120) icmp_seq=1 Destination Host Unreachable
--- p20.home.brack.ca ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
-- dmesg output --
[38862.733439] EXT4-fs error (device mmcblk0p2): htree_dirblock_to_tree:1030: inode #41948: block 11294: comm apt-get: bad entry in directory: rec_len % 4 != 0 - offset=0, inode=4294967295, rec_len=65535, name_len=255, size=4096
[46646.658111] usb 1-1.2: USB disconnect, device number 14
[46646.661046] rndis_host 1-1.2:1.0 ethpi20: unregister 'rndis_host' usb-0000:01:00.0-1.2, RNDIS device
[46646.691297] br0: port 6(ethpi20) entered disabled state
[46646.716088] device ethpi20 left promiscuous mode
[46646.716106] br0: port 6(ethpi20) entered disabled state
[50420.624449] usb 1-1.2: new full-speed USB device number 15 using xhci_hcd
[50433.188669] usb 1-1.2: new high-speed USB device number 16 using xhci_hcd
[50433.290118] usb 1-1.2: New USB device found, idVendor=3171, idProduct=0020, bcdDevice= 1.00
[50433.290137] usb 1-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[50433.290150] usb 1-1.2: Product: ClusterCTRL
[50433.290162] usb 1-1.2: Manufacturer: 8086 Consultancy
[50433.290174] usb 1-1.2: SerialNumber: 20
[50433.308170] rndis_host 1-1.2:1.0 eth1: register 'rndis_host' at usb-0000:01:00.0-1.2, RNDIS device, 00:22:82:ff:fe:14
[50433.315441] cdc_acm 1-1.2:1.2: ttyACM8: USB ACM device
[50433.318496] cdc_acm 1-1.2:1.4: ttyACM9: USB ACM device
[50433.397261] rndis_host 1-1.2:1.0 ethpi20: renamed from eth1
[50433.595102] br0: port 6(ethpi20) entered blocking state
[50433.595115] br0: port 6(ethpi20) entered disabled state
[50433.595444] device ethpi20 entered promiscuous mode
[50433.617777] br0: port 6(ethpi20) entered blocking state
[50433.617790] br0: port 6(ethpi20) entered forwarding state
[50452.053278] cdc_acm 1-1.2:1.2: failed to set dtr/rts
[50457.173346] cdc_acm 1-1.2:1.2: failed to set dtr/rts
[50470.997640] cdc_acm 1-1.2:1.4: failed to set dtr/rts
[50479.445795] cdc_acm 1-1.2:1.2: failed to set dtr/rts
[50490.965950] cdc_acm 1-1.2:1.4: failed to set dtr/rts
Tony Brack
May 10, 2021, 5:53:09 AM5/10/21
to ClusterHAT
I think I have found something: The interface on the Pi ZERO (P1) shows up as:
usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.19.181.1 netmask 255.255.255.0 broadcast 172.19.181.255
inet6 fe80::33f0:28a0:23fb:ac27 prefixlen 64 scopeid 0x20<link>
ether 00:22:82:ff:ff:01 txqueuelen 1000 (Ethernet)
RX packets 116 bytes 15114 (14.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 535 bytes 101198 (98.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
The same interface on the other side of the USB on the Pi4b shows up as:
ethpi1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::222:82ff:feff:fe01 prefixlen 64 scopeid 0x20<link>
ether 00:22:82:ff:fe:01 txqueuelen 1000 (Ethernet)
RX packets 401 bytes 56352 (55.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1 bytes 134 (134.0 B)
TX errors 429490 dropped 0 overruns 0 carrier 0 collisions 0
pi@Pi4b-64-cluster:/boot $ ping -c 1 p1
PING p1.home.brack.ca (192.168.1.121) 56(84) bytes of data.
From Pi4b-64-cluster.home.brack.ca (192.168.1.120) icmp_seq=1 Destination Host Unreachable
--- p1.home.brack.ca ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
Tony Brack
May 10, 2021, 10:39:54 PM5/10/21
to ClusterHAT
Attached is a shell script that reports (hopefully) all the debug data in the troubleshooting guide as well as a copy of the most recent report from my configuration. I am going to retry with ClusterCTRL-2020-02-13-lite-1-CBRIDGE as the Pi4b image.
Chris Burton
May 11, 2021, 3:00:16 AM5/11/21
to ClusterHAT
Hi,
It looks like you have an IP on wlan0 and a non-standard fallback/DHCP IP on br0 so are you connecting both or have you changed the fallback IP in /etc/dhcpcd.conf on the controller?
If you've connected both then I'd try using one or the other not both and if you use WiFi then you'll need to use the CNAT image.
If you've set the br0 fallback IP on the controller you'll need to alter it on the Pi Zeros too.
If br0 is getting the IP via DHCP on Ethernet with CBRIDGE but the Pi Zero usb0 isn't getting an IP then you can alter the fallback IP in /etc/dhcpcd.conf to the correct IP over serial.
You should be able to log into the Pi Zero P1 over USB Gadget serial from the controller with "screen /dev/ttypi1", press enter and wait a few seconds for the login prompt (press CTRL-a, then press k and y to exit screen).
The MAC addresses should be different on both sides of the usb0/ethpiX USB Gadget Ethernet interface.
Chris.
Tony Brack
May 11, 2021, 10:51:41 PM5/11/21
to ClusterHAT
Hi Chris,
I have 2 routers. wlan0 is connected to my ISP router (192.168.0.1/24) and gets its address from a DHCP static mapping, and br0 is getting a similar IP address from my external gateway that bypasses my ISP router (192.169.1.1/24). I have not touched anything either here or on my 2 working clusters.
Thanks for getting back.
I am connecting to 2 networks and have not altered the configuration in any way, so I will try to answer you as best I can. I have tried multiple "vanilla" cbridge images with the same result, so I have no doubt that I am missing something stupid and obvious. I provided the script so that it can be updated to capture all the important stuff for the next guy! This is not my only cluster, and the rest work. (I think they're running CNAT but not sure)
I will try to follow you:
> It looks like you have an IP on wlan0 and a non-standard fallback/DHCP IP on br0 so are you connecting both or
> have you changed the fallback IP in /etc/dhcpcd.conf on the controller?
I have 2 routers. wlan0 is connected to my ISP router (192.168.0.1/24) and gets its address from a DHCP static mapping, and br0 is getting a similar IP address from my external gateway that bypasses my ISP router (192.169.1.1/24). I have not touched anything either here or on my 2 working clusters.
> If you've connected both then I'd try using one or the other not both and if you use WiFi then you'll need to use the CNAT image.
I have done this with and without WiFi enabled, ... no difference, but I will try again later and advise. CNAT has the same (similar) problem with all kernels tested. Yes: I have tried multiple.
> If you've set the br0 fallback IP on the controller you'll need to alter it on the Pi Zeros too.
No, I have not explicitly set anything on BR0. It is running DHCP (defaulted) and communicates with the outside world properly. I am running headless with ssh and cockpit as well. That is how I am generating these reports.
> If br0 is getting the IP address via DHCP on Ethernet with CBRIDGE but the Pi Zero usb0 isn't getting an IP then you
> can alter the fallback IP in /etc/dhcpcd.conf to the correct IP over serial.
No, the serial ports are not working either. The 5th Pi (mapped to P20) works properly when I connect to my other cluster, or a system that is just running the CBRIDGE image without a hat (used for testing), or even my laptop for that matter, but not when connected on this machine. I am going to try a fresh uSD on a different spare Pi I have kicking around. Again, all of these are getting statically mapped addresses via DHCP/MAC. I took painstaking caution to ensure we also do not have a collision. Again, P20 which works properly everywhere else is the proof. It is a P5 image and is NOT going through the HAT.
*** interesting ***
I checked P1 (4th Pi is a Pi ZERO W, ... Why'd you do that? LOL) and logged in over the WiFi.
It is NOT *successfully* getting its address from the DHCP server and falling back ... ?
(incidentally: it still did not work properly from a CNAT either)
>>> I am going to try this MicroSD on another Pi ZERO W off my laptop.
usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.19.181.1 netmask 255.255.255.0 broadcast 172.19.181.255
inet6 fe80::33f0:28a0:23fb:ac27 prefixlen 64 scopeid 0x20<link>
ether 00:22:82:ff:ff:01 txqueuelen 1000 (Ethernet)
RX packets 116 bytes 15114 (14.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 535 bytes 101198 (98.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
The same interface on the other side of the USB on the Pi4b shows up as:
ethpi1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::222:82ff:feff:fe01 prefixlen 64 scopeid 0x20<link>
ether 00:22:82:ff:fe:01 txqueuelen 1000 (Ethernet)
RX packets 401 bytes 56352 (55.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1 bytes 134 (134.0 B)
TX errors 429490 dropped 0 overruns 0 carrier 0 collisions 0
pi@Pi4b-64-cluster:/boot $ ping -c 1 p1
PING p1.home.brack.ca (192.168.1.121) 56(84) bytes of data.
From Pi4b-64-cluster.home.brack.ca (192.168.1.120) icmp_seq=1 Destination Host Unreachable
--- p1.home.brack.ca ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
*** interesting - This tells me the bridge isn't bridging ... ***
I am attaching dhcpd.conf from the CBRIDGE (Pi4b-64-cluster). It seems to be mostly vanilla and has not been edited.
> You should be able to log into the Pi Zero P1 over USB Gadget serial from the controller
> with "screen /dev/ttypi1", press enter and wait a few seconds for the login prompt (press CTRL-a,
> then press k and y to exit screen).
screen (anything) just sits unresponsive, but the devices ARE created
No, none of these work, ... not even P20 which should be the easiest one. If I unplug P20 and plug it in anywhere else I can communicate via the serial gadget, but not in this CBRIDGE with either of these:
lrwxrwxrwx 1 root root 7 Feb 13 2020 /dev/ttypi20 -> ttyACM0
lrwxrwxrwx 1 root root 7 Feb 13 2020 /dev/ttypi20a -> ttyACM1
You will note that the correct IP addresses were registered in the DNS server on the router (pfSense) and that arp shows the correct MAC addresses mapping to the expected IP addresses:
pi@Pi4b-64-cluster:~ $ arp -a | sort -k2
gw.home.brack.ca (192.168.1.1) at 00:0c:29:80:d2:68 [ether] on br0
p1.home.brack.ca (192.168.1.121) at 00:22:82:ff:ff:01 [ether] on br0
p2.home.brack.ca (192.168.1.122) at 00:22:82:ff:ff:02 [ether] on br0
p3.home.brack.ca (192.168.1.123) at 00:22:82:ff:ff:03 [ether] on br0
p4.home.brack.ca (192.168.1.124) at 00:22:82:ff:ff:04 [ether] on br0
p20.home.brack.ca (192.168.1.125) at 00:22:82:ff:ff:14 [ether] on br0
In most of my report script outputs I get the correct IP addresses and no mention of the MAC address in arp:
gw.home.brack.ca (192.168.1.1) at 00:0c:29:80:d2:68 [ether] on br0
p1.home.brack.ca (192.168.1.121) at <incomplete> on br0
p2.home.brack.ca (192.168.1.122) at <incomplete> on br0
p3.home.brack.ca (192.168.1.123) at <incomplete> on br0
p4.home.brack.ca (192.168.1.124) at 00:22:82:ff:ff:04 [ether] on br0
p20.home.brack.ca (192.168.1.125) at <incomplete> on br0
--- p1.home.brack.ca ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 106ms
pi@Pi4b-64-cluster:~ $ ping p2
PING p2.home.brack.ca (192.168.1.122) 56(84) bytes of data.
--- p2.home.brack.ca ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 84ms
^C
pi@Pi4b-64-cluster:~ $ ping p3
PING p3.home.brack.ca (192.168.1.123) 56(84) bytes of data.
^C
--- p3.home.brack.ca ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 198ms
pi@Pi4b-64-cluster:~ $ ping p4
PING p4.home.brack.ca (192.168.1.124) 56(84) bytes of data.
^C
--- p4.home.brack.ca ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 129ms
pi@Pi4b-64-cluster:~ $ ping p20
PING p20.home.brack.ca (192.168.1.125) 56(84) bytes of data.
^C
--- p20.home.brack.ca ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 92ms
The network is working: (silly, but to be thorough)
pi@Pi4b-64-cluster:~ $ ping flirc-4b
PING flirc-4b.home.brack.ca (192.168.1.85) 56(84) bytes of data.
64 bytes from flirc-4b.home.brack.ca (192.168.1.85): icmp_seq=1 ttl=64 time=0.586 ms
64 bytes from flirc-4b.home.brack.ca (192.168.1.85): icmp_seq=2 ttl=64 time=0.216 ms
64 bytes from flirc-4b.home.brack.ca (192.168.1.85): icmp_seq=3 ttl=64 time=0.209 ms
64 bytes from flirc-4b.home.brack.ca (192.168.1.85): icmp_seq=4 ttl=64 time=0.217 ms
64 bytes from flirc-4b.home.brack.ca (192.168.1.85): icmp_seq=5 ttl=64 time=0.232 ms
^C
--- flirc-4b.home.brack.ca ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 100ms
rtt min/avg/max/mdev = 0.209/0.292/0.586/0.147 ms
> The MAC addresses should be different on both sides of the usb0/ethpiX
> USB Gadget Ethernet interface.
They are. 00:22:82:ff:fe:01 versus 00:22:82:ff:ff:01
Thanks again,
Tony
Tony Brack
May 14, 2021, 8:57:45 AM5/14/21
to ClusterHAT
Hi All,
I finally gave up and did 2 things:
1. swapped out the physical Pi
2. switched to CNAT
The cluster is now up and running, but I have to go through either the Pi ZERO W or the CNAT host to access any of the other Pis. This was not the intent. This configuration still does not work properly in a bridged configuration, including the serial ports.
Tony
Reply all
Reply to author
Forward
0 new messages