beegfs-client module build fails for latest RHEL 9 kernel?

433 views
Skip to first unread message

Martin Errenst

unread,
Jun 7, 2023, 8:03:07 AM6/7/23
to beegfs-user
Hello everyone,

we are using the beegfs-client 7.3.3 on a couple Alma Linux 9 machines.
The latest kernel update from a couple of days/weeks ago seems to break the beegfs-client kernel module build:

# uname -a
Linux wn1901 5.14.0-284.11.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 9 05:49:00 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux

and the first observable errors from the build log:

  CC [M] /var/lib/dkms/beegfs/7.3.3/build/build/../source/app/config/Config.o
/var/lib/dkms/beegfs/7.3.3/build/build/../source/filesystem/FhgfsOpsFile.c:156:5: error: ‘struct address_space_operations’ has no member named ‘readpages’; did you mean ‘readpage’?
  156 |    .readpages      = FhgfsOpsPages_readpages,
      |     ^~~~~~~~~
      |     readpage
/var/lib/dkms/beegfs/7.3.3/build/build/../source/filesystem/FhgfsOpsFile.c:156:22: error: initialization of ‘int (*)(struct address_space *, struct writeback_control *)’ from incompatible pointer type ‘int (*)(struct file *, struct address_space *, struct list_head *, unsigned int)’ [-Werror=incompatible-pointer-types]
  156 |    .readpages      = FhgfsOpsPages_readpages,
      |                      ^~~~~~~~~~~~~~~~~~~~~~~

[...]

Did anyone run into the same issue and maybe knows a way around?

I am also curious if there are any best practices when weighing off recommended kernel updates against waiting for updates to projects that build against the kernel (e.g. beegfs-client).

Cheers
   Martin Errenst

John Hearns

unread,
Jun 7, 2023, 8:20:51 AM6/7/23
to fhgfs...@googlegroups.com
I saw a similar error when upgrading a Redhat 8.x system to 7.3.3
The best course is to consult Thinkparq support - they immediately identified the header file which was causing the problem.
Of course, this assumes a support agreement isin place...

--
You received this message because you are subscribed to the Google Groups "beegfs-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fhgfs-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fhgfs-user/e03836a8-06fb-48c3-bb85-8f6777098fc2n%40googlegroups.com.

jeanfranco...@gmail.com

unread,
Jun 8, 2023, 9:19:24 PM6/8/23
to beegfs-user
Hello,

Same problem here, with AlmaLinux 9.2:

[root@beegfs01 sock]# uname -a
Linux beegfs01.jeffcourteau.int 5.14.0-284.11.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 9 05:49:00 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux

I get the exact same errors. Full startup log here:
Jun  8 21:08:45 beegfs01 systemd[1]: Starting Start BeeGFS Client...
Jun  8 21:08:45 beegfs01 beegfs-client[54316]: Starting BeeGFS Client:
Jun  8 21:08:45 beegfs01 beegfs-client[54316]: - Loading BeeGFS modules
Jun  8 21:08:45 beegfs01 beegfs-client[54316]: - BeeGFS module autobuild
Jun  8 21:08:49 beegfs01 beegfs-client[55025]: Building beegfs client module
Jun  8 21:09:11 beegfs01 beegfs-client[55239]: feature detection gives: -DKERNEL_HAS_SCHED_SIG_H -DKERNEL_HAS_LINUX_STDARG_H -DKERNEL_HAS_STATX -DKERNEL_HAS_KREF_READ -DKERNEL_HAS_FILE_DENTRY -DKERNEL_HAS_SUPER_SETUP_BDI_NAME -DKERNEL_HAS_KERNEL_READ -DKERNEL_HAS_SKWQ_HAS_SLEEPER -DKERNEL_WAKE_UP_SYNC_KEY_HAS_3_ARGUMENTS -DKERNEL_HAS_IOV_ITER_KVEC_NO_TYPE_FLAG_IN_DIRECTION -DKERNEL_HAS_PROC_OPS -DKERNEL_HAS_SOCKPTR_T -DKERNEL_HAS_SOCK_SETSOCKOPT_SOCKPTR_T_PARAM -DKERNEL_HAS_TIME64 -DKERNEL_HAS_KTIME_GET_TS64 -DKERNEL_HAS_KTIME_GET_REAL_TS64 -DKERNEL_HAS_KTIME_GET_COARSE_REAL_TS64 -DKERNEL_HAS_SETATTR_PREPARE -DKERNEL_HAS_POSIX_GET_ACL -DKERNEL_HAS_SET_ACL -DKERNEL_HAS_IDMAPPED_MOUNTS -DKERNEL_HAS_XATTR_HANDLERS_INODE_ARG
Jun  8 21:09:16 beegfs01 beegfs-client[55556]: /opt/beegfs/src/client/client_module_7/build/../source/common/net/sock/NetworkInterfaceCard.c: In function ‘__NIC_filterInterfacesForRDMA’:
Jun  8 21:09:16 beegfs01 beegfs-client[55556]: /opt/beegfs/src/client/client_module_7/build/../source/common/net/sock/NetworkInterfaceCard.c:309:47: error: ‘IBVSocket’ has no member named ‘cm_id’
Jun  8 21:09:16 beegfs01 beegfs-client[55556]:  309 |          nicAddrCopy->ibdev = rdmaSock.ibvsock.cm_id->device;
Jun  8 21:09:16 beegfs01 beegfs-client[55556]:      |                                               ^
Jun  8 21:09:16 beegfs01 beegfs-client[55239]: make[3]: *** [scripts/Makefile.build:321: /opt/beegfs/src/client/client_module_7/build/../source/common/net/sock/NetworkInterfaceCard.o] Error 1
Jun  8 21:09:16 beegfs01 beegfs-client[55239]: make[3]: *** Waiting for unfinished jobs....
Jun  8 21:09:17 beegfs01 beegfs-client[55026]: make[2]: *** [Makefile:1923: /opt/beegfs/src/client/client_module_7/build/../source] Error 2
Jun  8 21:09:17 beegfs01 beegfs-client[54798]: make[1]: *** [Makefile:174: module] Error 2
Jun  8 21:09:17 beegfs01 beegfs-client[54328]: make: *** [AutoRebuild.mk:34: auto_rebuild] Error 2
Jun  8 21:09:17 beegfs01 systemd[1]: beegfs-client.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jun  8 21:09:17 beegfs01 systemd[1]: beegfs-client.service: Failed with result 'exit-code'.
Jun  8 21:09:17 beegfs01 systemd[1]: Failed to start Start BeeGFS Client.
Jun  8 21:09:17 beegfs01 systemd[1]: beegfs-client.service: Consumed 49.075s CPU time.


Here are my build args from /etc/beegfs/beegfs-client-automount.conf. I disable RDMA support, I have the most basic virtual machines setup in KVM:
buildArgs=-j8 BEEGFS_NO_RDMA=1


If I let RDMA support enabled, the errors change (Remove BEEGFS_NO_RDMA arg), but still, compile doesn't succeed:
Jun  8 21:13:26 beegfs01 systemd[1]: Starting Start BeeGFS Client...
Jun  8 21:13:26 beegfs01 beegfs-client[55797]: Starting BeeGFS Client:
Jun  8 21:13:26 beegfs01 beegfs-client[55797]: - Loading BeeGFS modules
Jun  8 21:13:26 beegfs01 beegfs-client[55797]: - BeeGFS module autobuild
Jun  8 21:13:29 beegfs01 beegfs-client[56509]: Building beegfs client module
Jun  8 21:13:53 beegfs01 beegfs-client[56723]: feature detection gives: -DKERNEL_HAS_SCHED_SIG_H -DKERNEL_HAS_LINUX_STDARG_H -DKERNEL_HAS_STATX -DKERNEL_HAS_KREF_READ -DKERNEL_HAS_FILE_DENTRY -DKERNEL_HAS_SUPER_SETUP_BDI_NAME -DKERNEL_HAS_KERNEL_READ -DKERNEL_HAS_SKWQ_HAS_SLEEPER -DKERNEL_WAKE_UP_SYNC_KEY_HAS_3_ARGUMENTS -DKERNEL_HAS_IOV_ITER_KVEC_NO_TYPE_FLAG_IN_DIRECTION -DKERNEL_HAS_PROC_OPS -DKERNEL_HAS_SOCKPTR_T -DKERNEL_HAS_SOCK_SETSOCKOPT_SOCKPTR_T_PARAM -DKERNEL_HAS_TIME64 -DKERNEL_HAS_KTIME_GET_TS64 -DKERNEL_HAS_KTIME_GET_REAL_TS64 -DKERNEL_HAS_KTIME_GET_COARSE_REAL_TS64 -DKERNEL_HAS_SETATTR_PREPARE -DKERNEL_HAS_POSIX_GET_ACL -DKERNEL_HAS_SET_ACL -DKERNEL_HAS_IDMAPPED_MOUNTS -DKERNEL_HAS_XATTR_HANDLERS_INODE_ARG
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:156:5: error: ‘struct address_space_operations’ has no member named ‘readpages’; did you mean ‘readpage’?
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:  156 |    .readpages      = FhgfsOpsPages_readpages,
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:      |     ^~~~~~~~~
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:      |     readpage
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:156:22: error: initialization of ‘int (*)(struct address_space *, struct writeback_control *)’ from incompatible pointer type ‘int (*)(struct file *, struct address_space *, struct list_head *, unsigned int)’ [-Werror=incompatible-pointer-types]
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:  156 |    .readpages      = FhgfsOpsPages_readpages,
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:      |                      ^~~~~~~~~~~~~~~~~~~~~~~
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:156:22: note: (near initialization for ‘fhgfs_address_ops.writepages’)
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:158:22: warning: initialized field overwritten [-Woverride-init]
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:  158 |    .writepages     = FhgfsOpsPages_writepages,
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:      |                      ^~~~~~~~~~~~~~~~~~~~~~~~
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:158:22: note: (near initialization for ‘fhgfs_address_ops.writepages’)
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:159:5: error: ‘struct address_space_operations’ has no member named ‘set_page_dirty’
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:  159 |    .set_page_dirty = __set_page_dirty_nobuffers,
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:      |     ^~~~~~~~~~~~~~
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:159:22: error: initialization of ‘bool (*)(struct address_space *, struct folio *)’ {aka ‘_Bool (*)(struct address_space *, struct folio *)’} from incompatible pointer type ‘int (*)(struct page *)’ [-Werror=incompatible-pointer-types]
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:  159 |    .set_page_dirty = __set_page_dirty_nobuffers,
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:      |                      ^~~~~~~~~~~~~~~~~~~~~~~~~~
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:159:22: note: (near initialization for ‘fhgfs_address_ops.dirty_folio’)
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:171:5: error: ‘struct address_space_operations’ has no member named ‘readpages’; did you mean ‘readpage’?
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:  171 |    .readpages      = FhgfsOpsPages_readpages,
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:      |     ^~~~~~~~~
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:      |     readpage
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:171:22: error: initialization of ‘int (*)(struct address_space *, struct writeback_control *)’ from incompatible pointer type ‘int (*)(struct file *, struct address_space *, struct list_head *, unsigned int)’ [-Werror=incompatible-pointer-types]
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:  171 |    .readpages      = FhgfsOpsPages_readpages,
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:      |                      ^~~~~~~~~~~~~~~~~~~~~~~
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:171:22: note: (near initialization for ‘fhgfs_address_pagecache_ops.writepages’)
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:173:22: warning: initialized field overwritten [-Woverride-init]
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:  173 |    .writepages     = FhgfsOpsPages_writepages,
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:      |                      ^~~~~~~~~~~~~~~~~~~~~~~~
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:173:22: note: (near initialization for ‘fhgfs_address_pagecache_ops.writepages’)
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:174:5: error: ‘struct address_space_operations’ has no member named ‘set_page_dirty’
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:  174 |    .set_page_dirty = __set_page_dirty_nobuffers,
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:      |     ^~~~~~~~~~~~~~
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:174:22: error: initialization of ‘bool (*)(struct address_space *, struct folio *)’ {aka ‘_Bool (*)(struct address_space *, struct folio *)’} from incompatible pointer type ‘int (*)(struct page *)’ [-Werror=incompatible-pointer-types]
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:  174 |    .set_page_dirty = __set_page_dirty_nobuffers,
Jun  8 21:14:36 beegfs01 beegfs-client[58854]:      |                      ^~~~~~~~~~~~~~~~~~~~~~~~~~
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.c:174:22: note: (near initialization for ‘fhgfs_address_pagecache_ops.dirty_folio’)
Jun  8 21:14:36 beegfs01 beegfs-client[58851]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsPages.c: In function ‘_FhgfsOpsPages_readpages’:
Jun  8 21:14:36 beegfs01 beegfs-client[58851]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsPages.c:1007:16: error: implicit declaration of function ‘read_cache_pages’; did you mean ‘read_cache_page’? [-Werror=implicit-function-declaration]
Jun  8 21:14:36 beegfs01 beegfs-client[58851]: 1007 |       retVal = read_cache_pages(mapping, pageList, FhgfsOpsPages_readPageCallBack, &pageData);
Jun  8 21:14:36 beegfs01 beegfs-client[58851]:      |                ^~~~~~~~~~~~~~~~
Jun  8 21:14:36 beegfs01 beegfs-client[58851]:      |                read_cache_page
Jun  8 21:14:36 beegfs01 beegfs-client[58854]: cc1: some warnings being treated as errors
Jun  8 21:14:37 beegfs01 beegfs-client[56723]: make[3]: *** [scripts/Makefile.build:321: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFile.o] Error 1
Jun  8 21:14:37 beegfs01 beegfs-client[56723]: make[3]: *** Waiting for unfinished jobs....
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFileNative.c: In function ‘beegfs_readpages’:
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFileNative.c:1808:10: error: implicit declaration of function ‘read_cache_pages’; did you mean ‘read_cache_page’? [-Werror=implicit-function-declaration]
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: 1808 |    err = read_cache_pages(mapping, pages, beegfs_readpages_add_page, context);
Jun  8 21:14:37 beegfs01 beegfs-client[58883]:      |          ^~~~~~~~~~~~~~~~
Jun  8 21:14:37 beegfs01 beegfs-client[58883]:      |          read_cache_page
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFileNative.c: At top level:
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFileNative.c:2149:5: error: ‘const struct address_space_operations’ has no member named ‘set_page_dirty’
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: 2149 |    .set_page_dirty = beegfs_set_page_dirty,
Jun  8 21:14:37 beegfs01 beegfs-client[58883]:      |     ^~~~~~~~~~~~~~
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFileNative.c:2149:22: error: initialization of ‘void (*)(struct page *)’ from incompatible pointer type ‘int (*)(struct page *)’ [-Werror=incompatible-pointer-types]
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: 2149 |    .set_page_dirty = beegfs_set_page_dirty,
Jun  8 21:14:37 beegfs01 beegfs-client[58883]:      |                      ^~~~~~~~~~~~~~~~~~~~~
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFileNative.c:2149:22: note: (near initialization for ‘fhgfs_addrspace_native_ops.freepage’)
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFileNative.c:2153:5: error: ‘const struct address_space_operations’ has no member named ‘readpages’; did you mean ‘readpage’?
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: 2153 |    .readpages = beegfs_readpages,
Jun  8 21:14:37 beegfs01 beegfs-client[58883]:      |     ^~~~~~~~~
Jun  8 21:14:37 beegfs01 beegfs-client[58883]:      |     readpage
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFileNative.c:2153:17: error: initialization of ‘int (*)(struct address_space *, struct page *, struct page *, enum migrate_mode)’ from incompatible pointer type ‘int (*)(struct file *, struct address_space *, struct list_head *, unsigned int)’ [-Werror=incompatible-pointer-types]
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: 2153 |    .readpages = beegfs_readpages,
Jun  8 21:14:37 beegfs01 beegfs-client[58883]:      |                 ^~~~~~~~~~~~~~~~
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFileNative.c:2153:17: note: (near initialization for ‘fhgfs_addrspace_native_ops.migratepage’)
Jun  8 21:14:37 beegfs01 beegfs-client[58883]: cc1: some warnings being treated as errors
Jun  8 21:14:37 beegfs01 beegfs-client[56723]: make[3]: *** [scripts/Makefile.build:321: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsFileNative.o] Error 1
Jun  8 21:14:37 beegfs01 beegfs-client[58851]: cc1: some warnings being treated as errors
Jun  8 21:14:37 beegfs01 beegfs-client[56723]: make[3]: *** [scripts/Makefile.build:321: /opt/beegfs/src/client/client_module_7/build/../source/filesystem/FhgfsOpsPages.o] Error 1
Jun  8 21:14:38 beegfs01 beegfs-client[56510]: make[2]: *** [Makefile:1923: /opt/beegfs/src/client/client_module_7/build/../source] Error 2
Jun  8 21:14:38 beegfs01 beegfs-client[56282]: make[1]: *** [Makefile:174: module] Error 2
Jun  8 21:14:38 beegfs01 beegfs-client[55809]: make: *** [AutoRebuild.mk:34: auto_rebuild] Error 2
Jun  8 21:14:38 beegfs01 systemd[1]: beegfs-client.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jun  8 21:14:38 beegfs01 systemd[1]: beegfs-client.service: Failed with result 'exit-code'.
Jun  8 21:14:38 beegfs01 systemd[1]: Failed to start Start BeeGFS Client.
Jun  8 21:14:38 beegfs01 systemd[1]: beegfs-client.service: Consumed 3min 23.545s CPU time.


Not that I wouldn't buy support for the product, but since it doesn't work out of the box, I will not buy support for a product that won't work on the most up-to-date-supposed-to-be-supported-OS...


Any help and/or software update would be greatly appreciated.

Thanks,

J-F Courteau

Philipp Falk

unread,
Jun 9, 2023, 12:12:58 PM6/9/23
to fhgfs...@googlegroups.com
Hey J-F,

we invest a lot of time and resources to keep our kernel module compatible
with all major distributions, but supporting us by buying a support
contract is always appreciated and helps us allocate more resources to be
able to support new kernel and OS versions even faster ;D.

The kernel had a pretty substantial API change in page handling starting
from version 5.18 that mandated some client module modifications on our
end. Some of those changes seem to have made their way into RedHat 9.2,
which hadn't even been released when we released version 7.3.3.

It is always a good idea to check our release notes ([1] for 7.3.3) before
upgrading operating systems. We have a detailed list of supported operating
systems and kernels we run build tests for. If the OS version (including
minor) is not included in that list, especially for RedHat based
distributions, chances are high that there will be issues with the build.

All other major distributions keep their kernels pretty stable and
therefore aren't affected as much and give us more time to prepare, which
is why we have some code that is currently in testing to support kernels up
to version 6.1 (Debian 12). If all goes well, that code will be released
with our next maintenance releases, very shortly after the Debian 12
release, and also add support for RHEL/Rocky/Alma 9.2.

Best regards and have a good weekend
- Philipp

[1] https://doc.beegfs.io/7.3.3/release_notes.html

* jeanfranco...@gmail.com <jeanfranco...@gmail.com> [230609 03:19]:
> Hello,
>
> *Same problem here, with AlmaLinux 9.2:*
>
> [root@beegfs01 sock]# uname -a
> Linux beegfs01.jeffcourteau.int 5.14.0-284.11.1.el9_2.x86_64 #1 SMP
> PREEMPT_DYNAMIC Tue May 9 05:49:00 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux
>
> *I get the exact same errors. Full startup log here:*
> *Here are my build args from /etc/beegfs/beegfs-client-automount.conf. I
> disable RDMA support, I have the most basic virtual machines setup in KVM:*
> buildArgs=-j8 BEEGFS_NO_RDMA=1
>
>
> *If I let RDMA support enabled, the errors change (Remove BEEGFS_NO_RDMA
> arg), but still, compile doesn't succeed:*
> > CC [M] */var/lib/dkms/beegfs/7.3.3/build/build/*../source/app/config/Config.o
> >
> > */var/lib/dkms/beegfs/7.3.3/build/build/*../source/filesystem/FhgfsOpsFile.c:156:5:
> > error: ‘struct address_space_operations’ has no member named ‘readpages’;
> > did you mean ‘readpage’?
> > 156 | .readpages = FhgfsOpsPages_readpages,
> > | ^~~~~~~~~
> > | readpage
> > */var/lib/dkms/beegfs/7.3.3/build/build/*../source/filesystem/FhgfsOpsFile.c:156:22:
> > error: initialization of ‘int (*)(struct address_space *, struct
> > writeback_control *)’ from incompatible pointer type ‘int (*)(struct file
> > *, struct address_space *, struct list_head *, unsigned int)’
> > [-Werror=incompatible-pointer-types]
> > 156 | .readpages = FhgfsOpsPages_readpages,
> > | ^~~~~~~~~~~~~~~~~~~~~~~
> >
> > [...]
> >
> > Did anyone run into the same issue and maybe knows a way around?
> >
> > I am also curious if there are any best practices when weighing off
> > recommended kernel updates against waiting for updates to projects that
> > build against the kernel (e.g. beegfs-client).
> >
> > Cheers
> > Martin Errenst
> >
>
> --
> You received this message because you are subscribed to the Google Groups "beegfs-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fhgfs-user+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/fhgfs-user/2191421c-5e8a-4a03-acc6-168273c42d05n%40googlegroups.com.


--
Philipp Falk | Head of Engineering | m: philip...@thinkparq.com
ThinkParQ GmbH | Trippstadter Strasse 113 | 67663 Kaiserslautern | Germany
CEO: Frank Herold | COB: Dr. Franz-Josef Pfreundt | Registered: Amtsgericht Kaiserslautern HRB 31565 I VAT-ID-No.:DE 292001792

Confidentiality Notice: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential or proprietary information.

Raphael

unread,
Jun 15, 2023, 9:21:34 AM6/15/23
to beegfs-user
Hello,
I have the same problem on Alma 9.0. (Which is supported according to the release notes)

# uname -a
Linux sub 5.14.0-284.11.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 9 05:49:00 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/os-release
NAME="AlmaLinux"
VERSION="9.0 (Emerald Puma)"
ID="almalinux"
ID_LIKE="rhel centos fedora"
VERSION_ID="9.0"
PLATFORM_ID="platform:el9"
PRETTY_NAME="AlmaLinux 9.0 (Emerald Puma)"
....

Philipp Falk

unread,
Jun 15, 2023, 10:01:06 AM6/15/23
to fhgfs...@googlegroups.com
Hello Raphael,

your operating system version seems to be AlmaLinux 9.0, but it looks like
your kernel was upgraded at some point.

* Raphael <r264...@gmail.com> [230615 15:21]:
> # uname -a
> Linux sub 5.14.0-284.11.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 9
> 05:49:00 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux

As you can see, the version string here says el9_2. I can not find an Alma
9.0 mirror right now, but the kernel version for that OS was probably
5.14.0-<something>.el9.x86_64. For now, you will have to downgrade your
kernel to make the BeeGFS client module work.

Best regards
- Philipp
> To view this discussion on the web visit https://groups.google.com/d/msgid/fhgfs-user/ff05bd7c-07f3-4986-847c-32ed9b9acee6n%40googlegroups.com.

Raphael

unread,
Jun 20, 2023, 6:34:03 AM6/20/23
to beegfs-user

So, about the kernel versions.

5.14.0-284.11.1.el9_2 is actually the kernel used by Alma 9.0 and Rocky 9.1 right now. (as strange as that is)

Old Alma images can be found here: https://repo.almalinux.org/vault/9.0/isos/x86_64/
If you install it, you will have a version ending in el9_0. But after running dnf update-minimal you end up on 5.14.0-284.11.1.el9_2.

This is not a peculiarity of Alma, as Rocky 9.1 does the same thing (images can be found here https://dl.rockylinux.org/vault/rocky/9.1/isos/x86_64/)

So, as weird as RedHat may be about their kernels, we are talking about the official repos for Alma 9.0 and Rocky 9.1.
I do understand that it's difficult to clean up after changes like this. But if Alma 9.0 is listed as fully supported I do not expect the first update to break everything.
I think there should at least be an up-to-date listing of supported kernel versions?
That way users could at least look this up beforehand.

Best,
  Raphael
Message has been deleted
Message has been deleted

Robert Henning

unread,
Mar 11, 2024, 4:59:49 AMMar 11
to fhgfs...@googlegroups.com
Hi,

My temporary fix for this was to just go back to a previous kernel. See the steps below that I used. I'm hoping that this will be resolved in the near future. 

dnf install kernel-5.14.0-284.30.1.el9_2.x86_64 kernel-devel-5.14.0-284.30.1.el9_2.x86_64

[root@id14b6 ~]# grubby --info=ALL | grep title
title="Red Hat Enterprise Linux (5.14.0-362.18.1.el9_3.x86_64) 9.3 (Plow)"
title="Red Hat Enterprise Linux (5.14.0-362.8.1.el9_3.x86_64) 9.3 (Plow)"
title="Red Hat Enterprise Linux (5.14.0-284.30.1.el9_2.x86_64) 9.2 (Plow)"
title="Red Hat Enterprise Linux (0-rescue-2c095a0d483c4c9a9634f1f07a6ff096) 9.2 (Plow)"

grub2-set-default 2 (This is the third one in the list. Numbering starts at 0)

reboot

Then rebuild client again.

Regards,

Rob

On Fri, Mar 8, 2024 at 6:23 PM Filippo Savazzi <filo.s...@gmail.com> wrote:
Hello, I'll jump in this 9 month olds conversation: did somebody manage to solve this issue?

I'm having a similar problem here: I'm doing some test on virtual machine before upgrading our cluster in production; this VM runs on RHEL 9.3 and I'm trying to install BeeGFS 7.4.2.
No success whatsoever, I was wondering whether someone found a way to circumvent (or even fix, why not?!) these kernel issues.

Here's some details about the host:

uname -r
5.14.0-362.8.1.el9_3.x86_64

cat /etc/os-release
NAME="Red Hat Enterprise Linux"
VERSION="9.3 (Plow)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="9.3"
PLATFORM_ID="platform:el9"
PRETTY_NAME="Red Hat Enterprise Linux 9.3 (Plow)"

Here instead is an extract of the error (this is after I tried to fix it based on errors from previous trials, hence installing dwarves, symlinking btf/vmlinux, etc..):

beegfs-client[10322]: Starting BeeGFS Client:
beegfs-client[10322]: - Loading BeeGFS modules
beegfs-client[10322]: - BeeGFS module autobuild
beegfs-client[11066]: Building beegfs client module
beegfs-client[11290]: feature detection gives: -DKERNEL_HAS_SCHED_SIG_H -DKERNEL_HAS_LINUX_STDARG_H -DKERNEL_HAS_STATX -DKERNEL_HAS_K>
beegfs-client[13800]: feature detection gives: -DKERNEL_HAS_SCHED_SIG_H -DKERNEL_HAS_LINUX_STDARG_H -DKERNEL_HAS_STATX -DKERNEL_HAS_K>
beegfs-client[14043]: /bin/sh: line 1: ./tools/bpf/resolve_btfids/resolve_btfids: No such file or directory
beegfs-client[14027]: make[4]: *** [scripts/Makefile.modfinal:61: /opt/beegfs/src/client/client_module_7/build/../source/beegfs.ko] E>
beegfs-client[14027]: make[4]: *** Deleting file '/opt/beegfs/src/client/client_module_7/build/../source/beegfs.ko'
beegfs-client[13800]: make[3]: *** [scripts/Makefile.modpost:140: __modpost] Error 2
beegfs-client[11067]: make[2]: *** [Makefile:1842: modules] Error 2
beegfs-client[10828]: make[1]: *** [Makefile:177: module] Error 2
beegfs-client[10336]: make: *** [AutoRebuild.mk:34: auto_rebuild] Error 2

systemd[1]: beegfs-client.service: Main process exited, code=exited, status=2/INVALIDARGUMENT

Best,
Filippo

Quentin Le Burel

unread,
Mar 15, 2024, 11:31:04 AMMar 15
to fhgfs...@googlegroups.com
Hi I just tried with rocky 9.3 and kernel version 5.14.0-362.18.1.el9_3.0.1.x86_64, worked fine out of the box.

[root@rocky93 ~]# modinfo beegfs
filename:       /lib/modules/5.14.0-362.18.1.el9_3.0.1.x86_64/updates/fs/beegfs_autobuild/beegfs.ko
version:        7.4.2
alias:          fs-beegfs
author:         Fraunhofer ITWM, CC-HPC
description:    BeeGFS parallel file system client (http://www.beegfs.com)
license:        GPL v2
rhelversion:    9.3
srcversion:     D799114F137FBECE6AA46C8
depends:        ib_core,rdma_cm
retpoline:      Y
name:           beegfs
vermagic:       5.14.0-362.18.1.el9_3.0.1.x86_64 SMP preempt mod_unload modversions
[root@rocky93 ~]#

[root@rocky93 ~]# locate resolve_btfids
/usr/src/kernels/5.14.0-362.18.1.el9_3.0.1.x86_64/tools/bpf/resolve_btfids
/usr/src/kernels/5.14.0-362.18.1.el9_3.0.1.x86_64/tools/bpf/resolve_btfids/Build
/usr/src/kernels/5.14.0-362.18.1.el9_3.0.1.x86_64/tools/bpf/resolve_btfids/Makefile
/usr/src/kernels/5.14.0-362.18.1.el9_3.0.1.x86_64/tools/bpf/resolve_btfids/main.c
[root@rocky93 ~]# rpm -qf /usr/src/kernels/5.14.0-362.18.1.el9_3.0.1.x86_64/tools/bpf/resolve_btfids
kernel-devel-5.14.0-362.18.1.el9_3.0.1.x86_64

Can you make sure to do yum update kernel* and try again ?

Regards
Quentin







Reply all
Reply to author
Forward
0 new messages