[Lustre-community] recommendation lustre o2ib fabric manager ?

[Chris Hunter]

We have a cluster with QDR infiniband. Lustre MDS & OSS servers are
using mellanox HCA while clients use qlogic HCAs. I believe the switches
are qlogic QDR 12000 series. We are using ofed distributed with RHEL 6.2.

We currently use the qlogic IB fabric manager (subnet manager). We have
heard qlogic_fm is good for MPI but bad for lustre. We have the option
of running ofed opensm. Does anyone have any experience/opinions about
using qlogic fabric manager versus opensm ?

thanks,
chris hunter
yale hpc group
_______________________________________________
Lustre-community mailing list
Lustre-c...@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-community

[Stearman, Marc]

The primary difference where Lustre is concerned is that Mellanox does RDMA in hardware, and QLogic did RDMA in software. Doesn't really matter which subnet manager you run.

-Marc

----
D. Marc Stearman
Lustre Operations Lead
stea...@llnl.gov
925.423.9670

[Chris Hunter]

Thanks for the response.

Qlogic & Mellanox claim to do a better job with routing on large/complex
networks than opensm. They use buzz words such as "Dispersive Routing"
or "Adaptive Routing". Is there any substance to these statements ?

regards,

chris hunter
yale hpc group