Infiniband and RDMA in containers

[rama...@gmail.com]

Is there a way to create containers with Infiniband and RDMA support to run MPI applications on stock CoreOS kernel?

After starting the node I can see module ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008), but Infiniband stack doesn't work in privileged container.

[Brandon Philips]

What doesn't work in a priviledged container?

On Thu, Nov 17, 2016 at 7:00 AM <rama...@gmail.com> wrote:

Is there a way to create containers with Infiniband and RDMA support to run MPI applications on stock CoreOS kernel?

After starting the node I can see module ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008), but Infiniband stack doesn't work in privileged container.

--
You received this message because you are subscribed to the Google Groups "CoreOS User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to coreos-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[rama...@gmail.com]

The problem solved by loading existing infiniband-related modules (ib_uverbs, ib_umad, rdma_cm, rdma_ucm). After this Infiniband stack can work in containers.

понедельник, 21 ноября 2016 г., 23:13:54 UTC+3 пользователь Brandon Philips написал:

[Brandon Philips]

Yes, you can add these to your /etc/modules file to have them load at boot time automatically.

[邱志林]

Are you using container with docker? I found that most of the solution is by setting containers to using host network stack, like running with option --net=host. It that how you make it works? Or Could you please show us how you make it ?

在 2016年12月4日星期日 UTC+8下午3:09:34，rama...@gmail.com写道：

[Paul Morgan]

On Mon, Jul 24, 2017 at 5:57 AM, 邱志林 <gek...@163.com> wrote:

Are you using container with docker? I found that most of the solution is by setting containers to using host network stack, like running with option --net=host. It that how you make it works? Or Could you please show us how you make it ?

​Take a look at:

* https://github.com/jumanjihouse/docker-infiniband#run-an-image

* https://github.com/jumanjihouse/docker-opensm#run-opensm

hth,

-paul​