Mellanox ConnectX-xx with Infiniband available in cloudlab hardware?

[sm...@illinois.edu]

Hello,

I was using xl170 machines with ConnectX-4 nics. However, I learnt that these nics do not support infiniband are are supporting only ethernet. Are there mellanox connectx - 4/5 hardware which support infiniband available on cloudlab?

It looks like the MT27700 supports infiniband but the MT27710 does not..

sarthakm@node-0:~/RDMA/release/examples$ lspci | grep Mellanox 03:00.0 

.

Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] 03:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] 07:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] 07:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx].

The reason I say this is because a query on the hardware should return `LINK_TYPE_P1` which allows us to choose ethernet / infiniband, however a "sudo mlxconfig -d /dev/mst/mt4117_pciconf0.1 q" command does not have that option.

Some more reference: https://access.redhat.com/articles/3082811

Thanks

Sarthak

[Mike Hibler]

No. Only the Apt cluster (connectX-3) and first gen Clemson machines
(Qlogic) are wired for Infiniband.

> --
> You received this message because you are subscribed to the Google Groups
> "cloudlab-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to cloudlab-user...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/
> cloudlab-users/c2557abc-b0eb-467f-be52-3ac55f49287an%40googlegroups.com.

[Leigh Stoller]

Just a warning; Do not run your own subnet manager, that will cause chaos
for all users and get your account frozen.

Thanks
Leigh