Module ib_uverbs not found in directory /lib/modules/4.4.0-72-generic

[me]

Hi, 

I'm trying to test the disni library. As for now I don't have an RDMA Hardware, I installed Softiwarp. Everything went fine with the installation except that when I run the check ibv_devices, 

I got this error: Failed to get IB devices list: Function not implemented; the cause is that ib_uverbs module failed to load with this: modprobe: FATAL: Module ib_uverbs not found in directory /lib/modules/4.4.0-72-generic, as well as the related user space module. Is this related to the support of the kernel version , how can I recover from this error ?

[patrick stuedi]

Yes, softiwarp needs a kernel with RDMA support, that is with the rdma core and hw modules.

If you have a kernel that got built including RDMA support (which I think is default) then the ib_uverbs modules and others should show up at kernel/drivers/infiniband/core, for instance this is how it looks for me:

$ ls -al /lib/modules/3.13.0-77-generic/kernel/drivers/infiniband/core/

total 652

drwxr-xr-x 2 root root   4096 Feb  6  2016 .

drwxr-xr-x 5 root root   4096 Feb  6  2016 ..

-rw-r--r-- 1 root root  21004 Jan 20  2016 ib_addr.ko

-rw-r--r-- 1 root root  79796 Jan 20  2016 ib_cm.ko

-rw-r--r-- 1 root root 137412 Jan 20  2016 ib_core.ko

-rw-r--r-- 1 root root  71004 Jan 20  2016 ib_mad.ko

-rw-r--r-- 1 root root  52108 Jan 20  2016 ib_sa.ko

-rw-r--r-- 1 root root  32100 Jan 20  2016 ib_ucm.ko

-rw-r--r-- 1 root root  32244 Jan 20  2016 ib_umad.ko

-rw-r--r-- 1 root root  80524 Jan 20  2016 ib_uverbs.ko

-rw-r--r-- 1 root root  23476 Jan 20  2016 iw_cm.ko

-rw-r--r-- 1 root root  74860 Jan 20  2016 rdma_cm.ko

-rw-r--r-- 1 root root  35812 Jan 20  2016 rdma_ucm.ko

If, instead of using the built-in kernen support for RDMA, you're installing the Mellanox MOFED, then your RDMA modules might be in the updates directory, e.g., /lib/modules/3.13.0-77-generic/updates/...

In your case apparently they are at neither those locations. So you might need to compile a kernel with RDMA support, or install the Mellanox OFED package. That's my understanding of things. If my colleagues (also on the mailing list) are familiar with this particular issue they may suggest a faster fix...

--
You received this message because you are subscribed to the Google Groups "zrlio-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zrlio-users+unsubscribe@googlegroups.com.
To post to this group, send email to zrlio...@googlegroups.com.
Visit this group at https://groups.google.com/group/zrlio-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/zrlio-users/631127c9-3ec3-4845-b138-1645df89f1ee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ines Messadi]

Thank you for you answer.

In my case, the folder looks like this : 

$ ls

ib_addr.ko  ib_cm.ko  ib_core.ko  ib_mad.ko  ib_sa.ko  iw_cm.ko  rdma_cm.ko

I think I should try a different kernel with RDMA support.

[Ines Messadi]

I thought everything was fine after installingMellanox OFED package but I'm mistaken I got this error when loading siw.ko

sudo insmod ./siw.ko

insmod: ERROR: could not insert module ./siw.ko: Invalid parameters

$ dmesg | tail

[   44.033359] siw: disagrees about version of symbol ib_unregister_device

[   44.033366] siw: Unknown symbol ib_unregister_device (err -22)

[   44.033372] siw: disagrees about version of symbol ib_register_device

[   44.033373] siw: Unknown symbol ib_register_device (err -22)

[   44.033384] siw: disagrees about version of symbol ib_dispatch_event

[   44.033385] siw: Unknown symbol ib_dispatch_event (err -22)

[   44.033416] siw: disagrees about version of symbol ib_dealloc_device

[   44.033418] siw: Unknown symbol ib_dealloc_device (err -22)

[   44.033421] siw: disagrees about version of symbol ib_alloc_device

[   44.033422] siw: Unknown symbol ib_alloc_device (err -22)

On Sun, Apr 30, 2017 at 9:35 PM, Ines Messadi <ines.m...@gmail.com> wrote:

I finally installed the the Mellanox OFED package, found the modules in /lib/modules/3.13.0-77-generic/updates/ and everything went fine.

Thank you again.

[Bernard Metzler]

Hmmm, if you built siw within the environment of the given kernel, then it's symbols will not fit with MOFED symbols of the now installed core RDMA stack. I suggest not to use MOFED (you would have to rebuild siw against MOFED), but use your distro's plain RDMA support. From what you wrote earlier - there seem to be RDMA support available, but user level RDMA was missing. So, un-install MOFED (sorry), and add user level RDMA support to your installation. If you have user level RDMA support added, just modprobe rdma_ucm. Now insert siw again, and it should be ok.
It will depend on your distro how you get user level rdma installed. You may contact me directly for help.

Best regards,
Bernard.

Ines Messadi ---05/01/2017 12:08:03 AM---I thought everything was fine after installingMellanox OFED package but I'm mistaken I got this erro

To unsubscribe from this group and stop receiving emails from it, send an email to zrlio-users...@googlegroups.com.

To post to this group, send email to

zrlio...@googlegroups.com.
Visit this group at https://groups.google.com/group/zrlio-users.

To view this discussion on the web visit

https://groups.google.com/d/msgid/zrlio-users/631127c9-3ec3-4845-b138-1645df89f1ee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "zrlio-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to zrlio-users...@googlegroups.com.

To post to this group, send email to zrlio...@googlegroups.com.
Visit this group at https://groups.google.com/group/zrlio-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/zrlio-users/CALDU1HSo%3Dgw0iUYrWh8-JxySkc-pzG6RSj_3Xeob9R3OibG3wA%40mail.gmail.com.

[liang...@gmail.com]

Hi! I'm facing the same problem -- my distro which is the default 16.10 ubuntu on EC2 does not come with rdma support. I am wondering how I should proceed enabling RDMA support in the kernel to use SoftiWarp?

Many thanks!

To unsubscribe from this group and stop receiving emails from it, send an email to zrlio-users+unsubscribe@googlegroups.com.

To post to this group, send email to

zrlio...@googlegroups.com.
Visit this group at https://groups.google.com/group/zrlio-users.

To view this discussion on the web visit

https://groups.google.com/d/msgid/zrlio-users/631127c9-3ec3-4845-b138-1645df89f1ee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "zrlio-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to zrlio-users+unsubscribe@googlegroups.com.