[URGENT HELP NEEDED] Infiniband SRP boot from SAN.

[Willard Griselda]

Hi all, I really need your help!

I am new to Infiniband, SRP and ESOS. I have an ESOS installed server with Mellanox ConnectX-3 FCBT installed SAN, I have same model NIC installed on a Dell R720 diskless server, I want to install Debian 12 for the R720 on SRP shared block storage and boot from there.

On ESOS, I created dev_disk, created initiator. On Dell R720, I boot from a Debian 12 live system, installed packages of `rdma*`, `srptools`, `ibutils`, `lsscsi` and `mstflint` inside of it. After I done that, I executed `lsscsi` command, the SRP shared(from ESOS) block storage was automatically detected as `/dev/sdb`. I finished installation process(GPT+Grub+BIOS instead of UEFI, because I heard ConnectX-3 doesn't support UEFI boot) on `/dev/sdb`.

But after I reboot the machine and removed the USB stick it still won't boot. When I was in Live system I see the R720 server connected with the SRP target on ESOS tui, but I don't see that connection after I reboot the server and remove the USB drive.

What should I do? Where should I start to troubleshoot?

Version info:

Mellanox CX-3 FCBT: Firmware v2.42.5000 and Flexboot v3.4.752

Dell R720: BIOS version 2.9.0.

ESOS version: I installed with this zip file: https://download.esos-project.com/packages/master/esos-master_547868d_dgvszq.zip

I have been stucking on this for 2 weeks, please help me! Please let me know for any information you need!

Thanks in advance

[Marc Smith]

I'm not familiar with boot-from-SAN via SRP (IB), but if this is
supported, it must be supported by your adapter (Mellanox IB HCA). I'd
look at the option ROM settings (eg, during BIOS POST time) for the
Mellanox adapter and see what settings are in there. It may not be
supported at all, but if I were you I'd look around in that option ROM
menu and perhaps look at the documentation for ConnectX-3 IB adapters.
Then if you can at least get to the point where the OS loads, you may
need some tweaking in Debian to get it to mount the file systems, but
perhaps that is already there if you installed it that way (eg, if
you're not using IB switch, you need to run OpenSM, IB Subnet
Manager).

--Marc


>
> --
> You received this message because you are subscribed to the Google Groups "esos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to esos-users+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/esos-users/f35865e5-050b-47cd-a9d7-9e4fa3e32415n%40googlegroups.com.