tcmalloc vs glibc 2.26

[mska...@redhat.com]

Recent glibc improved malloc implementation.

NEWS for version 2.26
=====================

Major new features:

* A per-thread cache has been added to malloc. Access to the cache requires
  no locks and therefore significantly accelerates the fast path to allocate
  and free small amounts of memory. Refilling an empty cache requires
locking
  the underlying arena. Performance measurements show significant gains in a
  wide variety of user workloads. Workloads were captured using a special
  instrumented malloc and analyzed with a malloc simulator. Contributed by
  DJ Delorie with the help of Florian Weimer, and Carlos O'Donell. 

(https://sourceware.org/ml/libc-alpha/2017-08/msg00010.html)

As I understand mongodb suggest using libtcmalloc allocator. Right?

How do you measure allocator performance? Any chance to switch to glibc allocator?

Thanks for answers,
Marek

[Andrew Morrow]

Hi Marek -

I think it is unlikely that we would change the default allocator anytime soon. We know tcmalloc well, and we surface instrumentation data that it provides in our always-on diagnostic data. Even if we were to find that the glibc allocator offered superior performance over tcmalloc, we would first need to understand what allocator instrumentation glibc provides and integrate with it in a way that did not degrade our diagnostic capabilities before we would considering switching.

Definitely something we will keep an eye on, of course, as it would allow us to shed a dependency.

Thanks,

Andrew

--
You received this message because you are subscribed to the Google Groups "mongodb-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mongo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-dev/25e9f1cb-6250-45c5-b152-3d232203b4d0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[MARK CALLAGHAN]

Andrew - I don't disagree with you, change is expensive. Regardless I look forward to a better glibc malloc. In the past it provided slightly worse throughput and much worse fragmentation compared to tcmalloc and jemalloc in my tests. From what I read the throughput problems might have been fixed but I have yet to learn of improvements to avoiding fragmentation.
* http://smalldatum.blogspot.com/2017/11/concurrent-large-allocations-glibc.html
* http://smalldatum.blogspot.com/2015/10/myrocks-versus-allocators-glibc.html
* http://smalldatum.blogspot.com/2014/12/malloc-and-mongodb-performance.html

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-dev/CAHX05qFg-eouBu-h%3DB3iUXKwmW3T0i1%2Bh6b-zEs_0QFYwTj6SA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Mark Callaghan
mdca...@gmail.com