Persistent allocator question

[Krishna Harathi]

In our usage of libpmemobj (NVML 1.2), we are seeing 100% overhead in allocations.

For instance,  an allocation request for 131200 bytes (typical) yields an actual allocated size of 262080 bytes.

This may be expected as per design.

Is there any compile/build/config options that we can use to improve the utilization?

[Piotr Balcer]

Hi Krishna,
I'm afraid that this works "as designed" :) The culprit here is the trade-off to
optimize for average case. That exact number you've given is the threshold at
which the allocator stops using runs (used for small/medium allocations) and
starts using the global heap and chunks, which are 256 kilobytes in size.
The bigger that number is, the contention on the heap is lower, but the probable
fragmentation is higher. I do understand that this is not ideal for applications
that exlusively allocate big objects (that happen to be not aligned to any of
the allocation classes).

This all can be changed by simply adjusting few #define's in the alloc_class.c
file, but I've been working on a more systemic solution to this exact problem
that will retain the average case overhead and fragmentation, while smoothing
out the transition sizes between using runs and chunks.
Here's the relevant pull request: https://github.com/pmem/nvml/pull/1629
You can try the code from this branch and see if it helps for your problem.

The overall, average, overhead per object I'm aiming for is ~10%.

Hope that answers your question.

Piotrek

[Krishna Harathi]

Hi Piotrek,

Excellent, thanks for your help. I will definitely try your changes.

Regards,

Krishna Harathi