Creating PMDK pmemobj pools on existing regions of memory

[Daniel Waddington]

Hi,

Can I confirm that there is no way to create a pmemobj pool on an existing mapped contiguous region of memory? Also, I assume that for pmemobj pools take over the whole of the dax file - i.e. can't operate in a partition.

What would it take to support this?

We have a push-down storage system (MCAS - https://github.com/IBM/mcas/). I'd like to have the option to use PMDK in the active data objects.

Daniel

[ppbb...@gmail.com]

Hi Daniel,

We are slowly working towards that goal. The problem is that there's no clean abstraction layer that would allow us to do so, and so we need to create one. The release of libpmem2 was the first step towards that, and we are currently working on libpmemset (https://github.com/pmem/pmdk/issues/4914), which will create a clean separation between the library core logic and the file and map handling. Once this is implemented, we intend to move libpmemobj and the rest of our higher level libraries to this new abstraction layer - this will be a massive improvement in terms of maintainability of PMDK's codebase, but will also enable new use cases - like the ability to create pmemobj pools on custom "sets" that are built on top of arbitrary memory pages.

As for your second question - are you asking about device dax? If so, then the answer is yes, you cannot create a PMEMobjpool on a part of the device. If you are asking about regular files, then I'm not sure I follow. But yes - libpmemobj does use the entire file for its heap. There's no way to "partition" a file. But you can always create multiple files :)

The MCAS projects looks cool! Thanks for sending this link.

Piotr

[Daniel Waddington]

Great! libpmemset seems to be what I am after. I guess you will have pmemset_source_from_xxx where xxx could be an existing region of memory.

Daniel

[ppbb...@gmail.com]

I'm happy to hear that.

You will be able to accomplish what you want by composing libpmem2 [0] together with the proposed API for libpmemset [1]. You will be then be able to use the resulting set as a base for the PMEMobjpool.

This is not yet possible with the existing pmemset APIs, but it will be in the final version.

[0] - https://github.com/pmem/pmdk/blob/master/src/include/libpmem2.h

[1] - https://github.com/pmem/pmdk/blob/master/src/include/libpmemset.h