RDMA network Unreachable

135 views
Skip to first unread message

bin tang

unread,
Jul 31, 2021, 5:42:43 AM7/31/21
to cloudlab-users
Hi, 
I'm using the RDMA environment, and before, I only needed to configure the ib0 IP address on both nodes. Now, this doesn't work.

I did it like this.

cp1.
sudo modprobe ib_ipoib
sudo ifconfig ib0 192.168.0.211/24

cp2.
sudo modprobe ib_ipoib
sudo ifconfig ib0 192.168.0.212/24

on cp2.
ping 192.168.0.211

result.
PING 192.168.0.211 (192.168.0.211) 56(84) bytes of data.
From 192.168.0.212 icmp_seq=1 Destination Host Unreachable
From 192.168.0.212 icmp_seq=2 Destination Host Unreachable
From 192.168.0.212 icmp_seq=3 Destination Host Unreachable
From 192.168.0.212 icmp_seq=3 Destination Host Unreachable

How should I configure the RDMA environment now? Is it updated?

bin tang

unread,
Jul 31, 2021, 7:10:42 AM7/31/21
to cloudlab-users
sudo hca_self_test.ofed

---- Performing Adapter Device Self Test ----
Number of CAs Detected ................. 1
PCI Device Check ....................... PASS
Kernel Arch ............................ x86_64
Host Driver Version .................... MLNX_OFED_LINUX-4.1-1.0.2.0 (OFED-4.1-1.0.2): 4.4.0-140-generic
Host Driver RPM Check .................. PASS
Firmware on CA #0 VPI .................. v2.36.5000
Host Driver Initialization ............. PASS
Number of CA Ports Active .............. 0
Port State of Port #1 on CA #0 (VPI)..... DOWN (InfiniBand)
Error Counter Check on CA #0 (VPI)...... PASS
Kernel Syslog Check .................... PASS
Node GUID on CA #0 (VPI) ............... 00:02:c9:03:00:16:a6:c0
------------------ DONE ---------------------

ibstat

CA 'mlx4_0'
CA type: MT4099
Number of ports: 1
Firmware version: 2.36.5000
Hardware version: 1
Node GUID: 0x0002c9030016a6c0
System image GUID: 0x0002c9030016a6c3
Port 1:
State: Armed
Physical state: LinkUp
Rate: 56
Base lid: 23
LMC: 0
SM lid: 16
Capability mask: 0x02514868
Port GUID: 0x0002c9030016a6c1
Link layer: InfiniBand
Reply all
Reply to author
Forward
0 new messages