jemalloc Vs libc on Linux
112 views
Skip to first unread message
Sandeep R
Feb 22, 2017, 3:32:53 AM2/22/17
to Redis DB
Hi,
I have been expermenting to understand the memory requirements of Redis, especially
with respect to the usage of jemalloc Vs libc on Linux. From the following link, I
understand that jemalloc was introduced to reduce the fragmentation
While I understand that the above link was written for Redis 2.4.x, I was not sure
if it was still relevant (though I see that Redis uses jemalloc on Linux, by default).
For experiment, I created a Redis cache size of 1GB (maxmemory 1gb), populated the
cache (strings) and collected "INFO memory" at various intervals. I observed that
the fragmenation ratio for libc varies between 1.00 to 1.03. However, for jemalloc,
I observed that it ranges from 1.15 to 1.23!
I also tried removing some random keys, and observed the fragmentation ratio. After
removing the keys too, I observed that libc had better value (1.07) as compared to
jemalloc (1.22).
Apart from the metrics provided by Redis info memory, I also looked at process statistics
at /proc/pid/status, and observed that the memory utilization of Redis is much higher in
case of jemalloc when compared to libc (see below for actual values)!
I am wondering if the fragmentation is still an issue, or am I missing something in
my tests?
Some details below about the fields I have been looking at:
---------------------------------------------------------------------------------------------
$ uname -a
Linux xxxx.localdomain 4.1.12-61.1.25.el7uek.x86_64 #2 SMP Wed Jan 11 19:01:23 PST 2017 x86_64 x86_64 x86_64 GNU/Linux
$ ldd --version
ldd (GNU libc) 2.17
---------------------------------------------------------------------------------------------
$ redis-cli info memory # On Full Cache with libc
# Memory
used_memory:1073680400
used_memory_human:1023.94M
used_memory_rss:1087504384
used_memory_rss_human:1.01G
used_memory_peak:1073697368
used_memory_peak_human:1023.96M
total_system_memory:540628930560
total_system_memory_human:503.50G
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:1073741824
maxmemory_human:1.00G
maxmemory_policy:noeviction
mem_fragmentation_ratio:1.01
mem_allocator:libc
---------------------------------------------------------------------------------------------
$ redis-cli info memory # On Full Cache with jemalloc
# Memory
used_memory:1073716392
used_memory_human:1023.98M
used_memory_rss:1424330752
used_memory_rss_human:1.33G
used_memory_peak:1073732592
used_memory_peak_human:1023.99M
total_system_memory:540628930560
total_system_memory_human:503.50G
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:1073741824
maxmemory_human:1.00G
maxmemory_policy:noeviction
mem_fragmentation_ratio:1.33
mem_allocator:jemalloc-4.0.3
---------------------------------------------------------------------------------------------
$ cat /proc/<pid>/status | grep Vm # With Libc
VmPeak: 1196336 kB
VmSize: 1196336 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 1062016 kB
VmRSS: 1062016 kB
VmData: 1080480 kB
VmStk: 136 kB
VmExe: 860 kB
VmLib: 3004 kB
VmPTE: 2132 kB
VmPMD: 16 kB
VmSwap: 0 kB
---------------------------------------------------------------------------------------------
$ cat /proc/<pid>/status | grep Vm # With jemalloc (Peak usage more than twice of maxmemory!!)
VmPeak: 2115300 kB
VmSize: 2115300 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 1391624 kB
VmRSS: 1390948 kB
VmData: 1999160 kB
VmStk: 136 kB
VmExe: 1144 kB
VmLib: 3004 kB
VmPTE: 3936 kB
VmPMD: 20 kB
VmSwap: 0 kB
---------------------------------------------------------------------------------------------
Best Regards,
Sandeep
Reply all
Reply to author
Forward
0 new messages