Ext4 nvdimm fsdax ftruncate + RDMA memory registration

[Sébastien Valat]

Hi,

I want to use the nvdimm for its large available space and use it as ram in my program.

It does:

1. A first allocation step
   1. open a file on an nvdimm ext4 fsdax mount point
   2. ftruncate it a first time to get a first chunk
   3. mmap the first part of the file
   4. register the memory to libfabric for RDMA operations
2. A second allocation step to get more memory
   1. ftruncate again the file to make it grow (double the size)
      ----------------- DEADLOCK ---------------------
      ISSUE: here the process block on ftuncate() which does not return.
      DMESG REPORT: [860756.289775] EXT4-fs error (device pmem0) in ext4_setattr:5081: error 512
      -----------------------------------------------------
      
   2. mmap it
   3. register the new mmap part

I grow the file with ftruncate to avoid getting too many file descriptor and not creating tons of files. Any idea ? (I have fallback solutions, but this one seams to fall on a bug).

I run under centos 7.7, kernel 3.10.0-1062, mofed 5.1. Maybe this is already fixed on new kernels.

Cheers,

Sébastien.