nfs over rmda

[bob flynn]

anyone have experience of nfs over rmda with isilon

see its possible, i.e. from https://infohub.delltechnologies.com/l/powerscale-onefs-best-practices/nfs-considerations

OneFS 9.2 and later include Remote Direct Memory Access support for applications and clients with NFS over RDMA, and allows substantially higher throughput performance, especially for single connection and read intensive workloads - while also reducing both cluster and client CPU utilization. OneFS 9.2 and later supports NFSv3 over RDMA by leveraging the ROCEv2 network protocol (also known as Routable RoCE or RRoCE). New OneFS CLI and WebUI configuration options have been added, including global enablement, and IP pool configuration, filtering and verification of RoCEv2 capable network interfaces.

NFS over RDMA is also available on all PowerScale and Isilon Gen6 nodes which contain Mellanox ConnectX network adapters on the front end with either 25, 40, or 100 Gig Ethernet connectivity. The ‘isi network interfaces list’ CLI command can be used to easily identify which of a cluster’s NICs support .

Curious of its use 'in the wild', as investigating HPC options with isilon as backend.

-Bob

[Coolgoose]

  i Would only consider for testing or poc  this seems to be a very special requirement. for most of the workloads regular NFS is a good enough. what is ur work load like?  how much iops / thruput is needed. 

[bob flynn]

data feed to GPU's for ML

--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-gr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isilon-user-group/69c906cb-3b2c-4e03-9d8e-32d7f5973bd3n%40googlegroups.com.

[Yama Nawabi]

I have run some Blackmagic resolve on a Linux client with NFS over RDMA, definitely needed to use the mlnx_tune scripts on my workstation. With RDMA I'm not totally in the know on how to measure throughput since the standard NIC counters don't show throughput.

Makes an old system work really well, since the context switching (copying memory from user to kernel mode) and interrupts, which were the bottlenecks before, are somewhat out of the picture and really uncap performance.

Didn't have any issues with stability, but I didn't do it at HPC scale.

-Yama