libpmemobj min allocation size

[Lu]

Hi,

It seems the minimum allocation size of libpmemobj is 128 bytes, maybe determined by the MIN_RUN_SIZE configuration. I'm curious why choosing this size as the minimum allocation size instead of some smaller values i.e. 16? If my program allocates a lot of small objects wouldn't this cause much internal fragmentation?

Regards,

Lu

[Piotr Balcer]

Hi Lu,
This is mostly because up until very recently the header size of an allocation was 64 bytes in size. We've shrunk that down to 16 bytes, so now it's more
realistic to actually create small allocation classes (< 64), but doing that without breaking the on-media layout would be a non trivial effort.
Having said that, I've attached a very simple patch (for current master) that reduces the smallest allocation size to 16 bytes of usable space.
I do not recommend using it beyond testing, and this patch breaks backward compatibility of libpmemobj.

If you feel this is something we should support, please feel free to create a feature request in our github issues repository - https://github.com/pmem/issues/issues.

Piotr

[Lu Zhang]

Hi Piotr,

Thanks for the reply and patch! I also like to reproduce the fragmentation and performance graphs you made here: http://pmem.io/2017/03/29/progress-report-q1-17.html

Are the data generated by some open source utilities in the NVML repo?

Regards,

Lu

--
You received this message because you are subscribed to the Google Groups "pmem" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pmem+unsubscribe@googlegroups.com.
To post to this group, send email to pm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pmem/856a88ea-bced-4e1c-9c41-27d9244e5438%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Piotr Balcer]

Happy to help ;)

Yes, most of the data comes from pmembench, a simple framework we've made.
You can find it in our repo (src/benchmarks). Take a look at one of the cfg files to see how to run it.

The data for the fragmentation chart however was generated by a small tool I've made that amounted to:

PMEMoid oid;
size_t usable_size;
for (size_t n = 100; n < 4 megabytes; n += 100) {
  pmemobj_alloc(pop, &oid, n, ...);
  usable_size = pmemobj_usable_size(oid);
  printf("%lu, %lu", n, usable_size);

  pmemobj_free(&oid);
}

(pseudocode)

Piotr

W dniu piątek, 7 lipca 2017 20:14:28 UTC+2 użytkownik Lu Zhang napisał:

Hi Piotr,

Thanks for the reply and patch! I also like to reproduce the fragmentation and performance graphs you made here: http://pmem.io/2017/03/29/progress-report-q1-17.html

Are the data generated by some open source utilities in the NVML repo?

Regards,

Lu

On Fri, Jul 7, 2017 at 12:39 AM, Piotr Balcer <ppbb...@gmail.com> wrote:

Hi Lu,
This is mostly because up until very recently the header size of an allocation was 64 bytes in size. We've shrunk that down to 16 bytes, so now it's more
realistic to actually create small allocation classes (< 64), but doing that without breaking the on-media layout would be a non trivial effort.
Having said that, I've attached a very simple patch (for current master) that reduces the smallest allocation size to 16 bytes of usable space.
I do not recommend using it beyond testing, and this patch breaks backward compatibility of libpmemobj.

If you feel this is something we should support, please feel free to create a feature request in our github issues repository - https://github.com/pmem/issues/issues.

Piotr

W dniu piątek, 7 lipca 2017 00:34:22 UTC+2 użytkownik Lu napisał:

Hi,

It seems the minimum allocation size of libpmemobj is 128 bytes, maybe determined by the MIN_RUN_SIZE configuration. I'm curious why choosing this size as the minimum allocation size instead of some smaller values i.e. 16? If my program allocates a lot of small objects wouldn't this cause much internal fragmentation?

Regards,

Lu

--
You received this message because you are subscribed to the Google Groups "pmem" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pmem+uns...@googlegroups.com.

To post to this group, send email to pm...@googlegroups.com.