Java Memory Allocation FlameGraph

[jake maloney]

Hi!

I recently wrote a tool to help generate memory allocation profiles of programs. It uses an agent which samples memory allocations and outputs data to produce flamegraphs. It has been a useful tool for me to understand the memory allocation patterns of applications I work with and thought it could be of some use to you guys. I think its also fitting that I share it with this group since it was posts from here that gave me the original idea. 

https://github.com/jmaloney10/allocation-instrumenter/tree/master 

The tool is largely based on the work of https://github.com/google/allocation-instrumenter so thanks to them and all the work they did! I wrote a brief blog post about it too but I don't want to seem like I am plugging the post for reads or anything so unless asked for I'll hold off on posting the link. My main goal is to share the tool! I would also appreciate any feedback about improvements, ideas, etc.!

[Stefano Doni]

Hi,

How would your approach differ from Brendan Gregg one outlined here:

http://www.brendangregg.com/FlameGraphs/memoryflamegraphs.html

AFAICT, profiling with perf would not interfere with the way the JVM manage memory (the caveats you cite) and would also show code paths that led to actual physical memory allocation by tracing page faults.

Thanks for sharing!

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Wojciech Kudla]

I hope to get corrected if I'm wrong here, but you can't trace allocation on the JVM heap like this (by looking at page faults) because it has nothing to do with malloc()/calloc() (essentially sbrk() or mmap(). It's just bump-the-pointer which doesn't necessarily mean it comes with a page fault. The heap may even get pre-touched or you might be running with page size big enough to avoid faults.
Instrumenting is probably the only reliable way to trace allocations on java heap.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.

[jake maloney]

Perhaps I should have clarified. The tool measures heap allocations which, to the best of my knowledge, are done the way Wojciech describes and can't be measured from listening to syscalls because the system has no knowledge of the internal JVM heap management. It could be interesting to couple the two methods to provide a way of monitoring off heap allocation as well!

Side note: I think you need to be using -XX:+AlwaysPreTouch for the JVM to touch all pages on startup.

On Wednesday, December 14, 2016 at 1:49:28 AM UTC-6, Wojciech Kudla wrote:

I hope to get corrected if I'm wrong here, but you can't trace allocation on the JVM heap like this (by looking at page faults) because it has nothing to do with malloc()/calloc() (essentially sbrk() or mmap(). It's just bump-the-pointer which doesn't necessarily mean it comes with a page fault. The heap may even get pre-touched or you might be running with page size big enough to avoid faults.
Instrumenting is probably the only reliable way to trace allocations on java heap.

On Wed, 14 Dec 2016, 07:33 Stefano Doni, <stefan...@moviri.com> wrote:

Hi,

How would your approach differ from Brendan Gregg one outlined here:

http://www.brendangregg.com/FlameGraphs/memoryflamegraphs.html

AFAICT, profiling with perf would not interfere with the way the JVM manage memory (the caveats you cite) and would also show code paths that led to actual physical memory allocation by tracing page faults.

Thanks for sharing!

On 13 Dec 2016 18:28, "jake maloney" <jake.m...@gmail.com> wrote:

Hi!

I recently wrote a tool to help generate memory allocation profiles of programs. It uses an agent which samples memory allocations and outputs data to produce flamegraphs. It has been a useful tool for me to understand the memory allocation patterns of applications I work with and thought it could be of some use to you guys. I think its also fitting that I share it with this group since it was posts from here that gave me the original idea. 

https://github.com/jmaloney10/allocation-instrumenter/tree/master 

The tool is largely based on the work of https://github.com/google/allocation-instrumenter so thanks to them and all the work they did! I wrote a brief blog post about it too but I don't want to seem like I am plugging the post for reads or anything so unless asked for I'll hold off on posting the link. My main goal is to share the tool! I would also appreciate any feedback about improvements, ideas, etc.!

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsub...@googlegroups.com.

[Stefano Doni]

Ok,

I was not aware of the way the JVM actually allocates memory, I can now see the value of your approach.

Many thanks!

On 14 Dec 2016 20:12, "jake maloney" <jake.m...@gmail.com> wrote:

Perhaps I should have clarified. The tool measures heap allocations which, to the best of my knowledge, are done the way Wojciech describes and can't be measured from listening to syscalls because the system has no knowledge of the internal JVM heap management. It could be interesting to couple the two methods to provide a way of monitoring off heap allocation as well!

Side note: I think you need to be using -XX:+AlwaysPreTouch for the JVM to touch all pages on startup.

On Wednesday, December 14, 2016 at 1:49:28 AM UTC-6, Wojciech Kudla wrote:

I hope to get corrected if I'm wrong here, but you can't trace allocation on the JVM heap like this (by looking at page faults) because it has nothing to do with malloc()/calloc() (essentially sbrk() or mmap(). It's just bump-the-pointer which doesn't necessarily mean it comes with a page fault. The heap may even get pre-touched or you might be running with page size big enough to avoid faults.
Instrumenting is probably the only reliable way to trace allocations on java heap.

On Wed, 14 Dec 2016, 07:33 Stefano Doni, <stefan...@moviri.com> wrote:

Hi,

How would your approach differ from Brendan Gregg one outlined here:

http://www.brendangregg.com/FlameGraphs/memoryflamegraphs.html

AFAICT, profiling with perf would not interfere with the way the JVM manage memory (the caveats you cite) and would also show code paths that led to actual physical memory allocation by tracing page faults.

Thanks for sharing!

On 13 Dec 2016 18:28, "jake maloney" <jake.m...@gmail.com> wrote:

Hi!

I recently wrote a tool to help generate memory allocation profiles of programs. It uses an agent which samples memory allocations and outputs data to produce flamegraphs. It has been a useful tool for me to understand the memory allocation patterns of applications I work with and thought it could be of some use to you guys. I think its also fitting that I share it with this group since it was posts from here that gave me the original idea. 

https://github.com/jmaloney10/allocation-instrumenter/tree/master 

The tool is largely based on the work of https://github.com/google/allocation-instrumenter so thanks to them and all the work they did! I wrote a brief blog post about it too but I don't want to seem like I am plugging the post for reads or anything so unless asked for I'll hold off on posting the link. My main goal is to share the tool! I would also appreciate any feedback about improvements, ideas, etc.!

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsubscribe...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsubscribe...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.