Question about <libpmem.h> (also can see https://github.com/pmem/pmdk/issues/5131)

[Wentao Huang]

Error for pmem_map_file, return with NULL, with errno 12, Cannot allocate memory.

Hi guys, I have a problem when running programs on Optane DCPMM, please help check the following code:

#include <errno.h>

#include <iostream>

#include <libpmem.h>

using namespace std;

#define TEST_LIMIT 100000

typedef struct Mystruct

{

double array[16384];

} Mystruct __attribute__((packed));

int main()

{

string pmem_path = "/optane/toy/";

size_t mapped_len = 0;

int is_pmem = 0;

for (int i = 0; i < TEST_LIMIT; i ++) {

string path = pmem_path + "___" + to_string(i);

Mystruct * addr = (Mystruct *) pmem_map_file(path.c_str(), sizeof(Mystruct), 

PMEM_FILE_CREATE|PMEM_FILE_EXCL, 0666, &mapped_len, &is_pmem);

// pmem_unmap(addr, sizeof(Mystruct));

if (addr == NULL) {

cout << i << "th iteration, Error number: " << errno << endl;

perror("");

exit(1);

}

}

return 0;

}

The error is when the loop iterates at the 65478th iteration, the pmem_file_map function returns NULL. The mapped_len and is_pmem for this iteration keep value at 0. The error number is 12, which means cannot allocate memory or out of memory. However, I check the space left on the persistent memory, there is still plenty of space available (still remains 300GB). I am not sure why it reports cannot allocate memory.

When I remove the comment on line 27 ( pmem_unmap(addr, sizeof(Mystruct)); ), the program can finish running successfully. I guess there is something wrong here for function pmem_map_file. I don't know why it behaves this way. Many thanks for the help if you guys have any ideas for this question.

[Jan K]

There's a limit on number of memory mappings a program can use (just
as there is a limit on number of open files). That's a system limit
imposed on mmap and has nothing to do with pmem.
I might be wrong, but I think you've just hit this limit. It's 65530
on my box, and I've never changed it, so that's probably the default.
Try to lower/rise vm.max_map_count (for instance by running 'sysctl
vm.max_map_count=n') and see if the loop ENOMEMs on a different
iteration.

Regards,
Jan

> --
> You received this message because you are subscribed to the Google Groups
> "pmem" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pmem+uns...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pmem/47f27a93-4187-4172-9d43-c8feb0d9edb9n%40googlegroups.com.
>

[Wentao Huang]

Hi Jan,

Thank you for sharing your opinion. You are right about this, I guess the main issue is that I exceeded that mapping limit. I  have rise the value of vm.max_map_count and it does successfully finish the program running. It's more about the system calling of mmap, we should be careful if we want to create a large number of mappings.

Best,

Wentao