Sept 10 Meeting Notes

[Alexander Maidak]

New attendee's:

The Deere IBM account Team decided to Pay the LUG a visit, Welcome!
Kaz Koehring
David Lloyd
Doug Allen
John Henley

Next Month is TV-Time-Linux anything involving digital video content and linux:
  * Transcoding
  * Video storage
  * Video streaming
  * Digital content acquisition automation

Chris Cooper presents on Perf
-----------------------------

There are counters and tracepoints inside the kernel which the perf tool can read/aggreaget on demand

You install perf with:
 debian: apt-get install linux-tools (a terrible name)
 redhat variants: yum install perf

 You'll want your debug symbols to make reading the perf output easier. symbol resolution with binary objects

There are ~2000 built in events
 - CPU COUNTERS
 - Kernel Tracepoints
 - kprobes
 - uprobes

You can see all the counters with "perf list"

Linux Perf Events Sources JPG: http://www.brendangregg.com/perf_events/perf_events_map.png

Events are only instrumented went attached to.

What questions can you answer with perf?
 - Whats using the cpu?
 - What is causing all these context switches?
 - What is using all my disk IO?
 - Where to start optimizing (not just for Devs)?

CPU Profiling: Why not use Top?

"sudo perf record -a -g -F 99 --call-graph dwarf -- sleep 30"
 - "-a" Records all CPU events
 - "-g" record graphs (stack traces)
 - "-F" 99 samples per second
 - Use dwarf for stack traces (as most things omit frame pointers)

dwarf is a way to get stack traces without having frame pointers on
Chris shared an example where htop show 10% usage on CPU but load of 1
read the data with "perf report" - not useful output
You can visual things into a frame graph using brenden greggs tool

sudo perf record -a -g -F 99 -e context-switches -- sleep 30
 -> could visualize with a flamegraph as well
In the era of specter/meltdown context switches are more expensive

Chris drilled into a flamegraph for our education
Chris used one of Brendan's tools to export the perf data into a interactive SVG image which made it a lot easier to interpret.
Chris starts from the bottom when reading a flame graph from the bottom looks for "shelves"
A lot of this comes form Brendan Gregg
https://github.com/brendangregg/perf-tools has lots of cool tools
Any 4.0 > kernel works great
https://github.com/iovisor/bcc#tools
Overheads are getting better with back ends like EBPF
Chris recommends reading Brendan Greg's Linux Perf Stuff

Great presentation Chris!

[William Bean]

Thanks for sending out notes, Alex!  And thanks for giving the presentation again, Chris; glad others were able to see this time.

Already excited for next month.

--

---
You received this message because you are subscribed to the Google Groups "QCLUG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qclug-list+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qclug-list/CAN1cgW8f5nAW-DKmgJ%3DVnbSLccb30GJMHRgtF-saDF92S5Lg_w%40mail.gmail.com.