[ANN] clj-async-profiler — embeddable profiler with flame graphs, based on Java's async-profiler

[Alexander Yakushev]

I've just released a wrapper around https://github.com/jvm-profiling-tools/async-profiler that allows controlling the profiler directly from the REPL of the program you want to profile. The JAR file ships the profiling agent and the flamegraph generation script from https://github.com/brendangregg/FlameGraph, so that you can profile your code and instantly generate flame graphs from it.

Introductory blog post: http://clojure-goes-fast.com/blog/profiling-tool-async-profiler/

Repository: https://github.com/clojure-goes-fast/clj-async-profiler

Hope you'll like it!

[Alex Miller]

Totally awesome - nice work!

[Luke Burton]

Like it?? I had to *physically prevent* myself from dropping what I was doing to hack on this, already having a full plate at work … I've wanted something like this for such a long time! 

My first reaction is to create a pedestal interceptor that allows you to remotely trigger a flamegraph. I first saw this in rack-mini-profiler though I'm sure others have done it too.

My second reaction is to wonder how difficult it would be to aggregate samples across a cluster of machines running the same code, similar to riemann-jvm-profiler.

My third reaction is to wonder how close you can get to having an "always on" flamegraph. I want a scrubber so I can reintroduce the time dimension. Imagine having a report of some perf issues and being able to say "let's go see what the flamegraph looked like at that time" - that'd be some Star Trek level observability right there.

Great work!

Luke.

[Alexander Yakushev]

Alex and Luke, thank you for your kind words!

Regarding your questions, Luke. #3. Authors of async-profiler promise that it's quite low-overhead. Of course, some experimentation and benchmarking is needed, but I think it is possible to have the profiler always on. Some work would be required too: async-profiler doesn't flush intermediate data, so an "always on" profiler would probably have to restart repeatedly in predefined timespans, say every 1 minute.

Re: #2. That would be actually quite simple. The format of folded stacktraces before they hit the flamegraph is is the following:

<stacktrace> <number-of-occurences>

For example:

clojure.core$main.invoke;my_app$_main;my_app$init-config; 1

clojure.core$main.invoke;my_app$_main;my_app$do_hard_work;my_app$burn_cpu; 254

You can mash such files from multiple machines together, adding the numbers where stacks match.

Best regards,

Alex