vmmalloc

28 views

Skip to first unread message

Adrian Jackson

unread,

Mar 28, 2019, 8:16:46 AM3/28/19

to pmem

Hi,

We're interested in using vmmalloc from parallel (MPI) programs, but is there anyway to combine the dax file systems from two sockets to give a single vmmalloc pool? i.e. can I use vmmalloc on top of a linux device mapper setup (i.e. https://pmem.io/2018/05/15/using_persistent_memory_devices_with_the_linux_device_mapper.html) or is that not sensible?

We don't actually need a single memory pool, as each process running on a node will have it's own vmmalloc pool, but if we're using two dax filesystems (one for each socket) it'll require a bit of scripting footwork to ensure MPI processes use the correct filesystem end point for their specific VMMALLOC_POOL_DIR.

cheers

adrianj

Piotr Balcer

unread,

Apr 1, 2019, 4:24:16 AM4/1/19

to pmem

Hi,

libvmem and libvmmalloc are no longer being developed, instead we recommend using libmemkind. It uses newer, less modified, version of jemalloc and it being actively developed and supported.

But to answer your question: lvm will work just fine. But, keep in mind that the allocator will have no information about the underlying NUMA nodes, and will just allocate linearly from the provided memory. For that reason, I recommend carefully thinking about whether to use linear or stripped volumes.