Flamegraphs

Flamegraphs

Purpose

Guide agents through the pipeline from profiler data to SVG flamegraph, and teach interpretation of flamegraphs to drive concrete optimisation decisions.

Triggers

• "How do I generate a flamegraph from perf data?"
• "How do I read a flamegraph?"
• "The flamegraph shows a wide frame — what does that mean?"
• "How do I generate a flamegraph from Callgrind?"
• "I want to compare two flamegraphs (before/after)"

Workflow

1. Install FlameGraph tools

git clone https://github.com/brendangregg/FlameGraph
# No install needed; scripts are in the repo
export PATH=$PATH:/path/to/FlameGraph

2. perf → flamegraph (most common path)

# Step 1: record
perf record -F 999 -g -o perf.data ./prog

# Step 2: generate script output
perf script -i perf.data > out.perf

# Step 3: collapse stacks
stackcollapse-perf.pl out.perf > out.folded

# Step 4: generate SVG
flamegraph.pl out.folded > flamegraph.svg

# Step 5: view
xdg-open flamegraph.svg     # Linux
open flamegraph.svg          # macOS

One-liner:

perf record -F 999 -g ./prog && perf script | stackcollapse-perf.pl | flamegraph.pl > fg.svg

3. Differential flamegraph (before/after)

# Collect two profiles
perf record -g -o before.data ./prog_old
perf record -g -o after.data ./prog_new

# Collapse
perf script -i before.data | stackcollapse-perf.pl > before.folded
perf script -i after.data  | stackcollapse-perf.pl > after.folded

# Diff (red = regressed, blue = improved)
difffolded.pl before.folded after.folded | flamegraph.pl > diff.svg

4. Callgrind → flamegraph

valgrind --tool=callgrind --callgrind-out-file=cg.out ./prog
stackcollapse-callgrind.pl cg.out | flamegraph.pl > fg.svg

5. Other profiler inputs

# Go pprof
go tool pprof -raw -output=prof.txt prog
stackcollapse-go.pl prof.txt | flamegraph.pl > fg.svg

# DTrace
dtrace -x ustackframes=100 -n 'profile-99 /execname=="prog"/ { @[ustack()] = count(); }' \
  -o out.stacks sleep 10
stackcollapse.pl out.stacks | flamegraph.pl > fg.svg

# Java (async-profiler)
async-profiler -d 30 -f out.collapsed PID
flamegraph.pl out.collapsed > fg.svg

6. Reading flamegraphs

A flamegraph is a call-stack visualisation:

• X axis: time on CPU (not time sequence) — wider = more time
• Y axis: call stack depth — taller = deeper call chain
• Color: random (no significance) — unless using differential mode

What to look for:

Identifying the actionable hotspot:

1. Find the widest top frame (a frame with no or narrow children above it) 2. That is where CPU time is actually spent 3. Trace down to understand what called it and why

Differential flamegraph:

• Red frames: more time in new profile (regression)
• Blue frames: less time in new profile (improvement)
• Frames only in one profile appear solid colored

7. flamegraph.pl options

flamegraph.pl --title "My App" \
              --subtitle "Release build, workload X" \
              --width 1600 \
              --height 16 \
              --minwidth 0.5 \
              --colors java \
              out.folded > fg.svg

References

For tool installation, stackcollapse scripts, and palette options, see references/tools.md.

Related skills

• Use skills/profilers/linux-perf to collect perf data
• Use skills/profilers/valgrind to collect Callgrind data
• Use skills/compilers/clang for LLVM PGO from sampling profiles

Flamegraph Tools Reference

Source: <https://github.com/brendangregg/FlameGraph> Source: <https://www.brendangregg.com/flamegraphs.html>

Table of Contents

1. FlameGraph scripts 2. stackcollapse scripts by profiler 3. flamegraph.pl options 4. Differential flamegraphs 5. Alternative flamegraph tools 6. Reading patterns quick reference

---

FlameGraph scripts

All from: git clone https://github.com/brendangregg/FlameGraph

---

stackcollapse scripts by profiler

Linux perf

perf record -F 999 -g -o perf.data ./prog
perf script -i perf.data > out.perf
stackcollapse-perf.pl out.perf > out.folded
flamegraph.pl out.folded > fg.svg

Valgrind Callgrind

valgrind --tool=callgrind --callgrind-out-file=cg.out ./prog
stackcollapse-callgrind.pl cg.out > out.folded
flamegraph.pl out.folded > fg.svg

DTrace (macOS / FreeBSD / Solaris)

# CPU sampling
sudo dtrace -x ustackframes=100 \
  -n 'profile-997 /execname=="prog"/ { @[ustack()] = count(); }' \
  -o out.stacks \
  sleep 30

stackcollapse.pl out.stacks > out.folded
flamegraph.pl out.folded > fg.svg

# Kernel + user combined
sudo dtrace -x stackframes=100 \
  -n 'profile-997 { @[stack(),ustack()] = count(); }' \
  -o out.stacks sleep 30

Go pprof

go tool pprof -raw -output=cpu.pprof ./prog
stackcollapse-go.pl cpu.pprof > out.folded
flamegraph.pl out.folded > fg.svg

Java (async-profiler)

# async-profiler: https://github.com/async-profiler/async-profiler
asprof -d 30 -f out.collapsed -t <PID>
# -f out.collapsed writes pre-collapsed format
flamegraph.pl out.collapsed > fg.svg

# Or use jstack periodically
for i in $(seq 1 100); do jstack <PID> >> jstacks.txt; sleep 0.1; done
stackcollapse-jstack.pl jstacks.txt > out.folded
flamegraph.pl out.folded > fg.svg

Rust (cargo-flamegraph)

cargo install flamegraph
# Runs perf internally and generates flamegraph.svg
cargo flamegraph --bin mybin

---

flamegraph.pl options

flamegraph.pl [options] folded_input > output.svg

Color palettes

---

Differential flamegraphs

Compare two profiles: red = more time (regression), blue = less time (improvement).

# Collect two profiles
perf record -g -o before.data ./prog_old
perf script -i before.data | stackcollapse-perf.pl > before.folded

perf record -g -o after.data ./prog_new
perf script -i after.data  | stackcollapse-perf.pl > after.folded

# Generate diff (positive = regression in 'after')
difffolded.pl before.folded after.folded | flamegraph.pl > diff.svg

# Invert: positive = improvement in 'after'
difffolded.pl -n after.folded before.folded | flamegraph.pl --negate > diff_inv.svg

Reading the diff:

• Saturated red: function took significantly more time in the new profile
• Saturated blue: function took significantly less time
• Pale colors: small change
• Gray/white: no change
• Frame only in one profile: appears fully colored (red or blue)

---

Alternative flamegraph tools

Speedscope (browser-based, interactive)

# Install
npm install -g speedscope

# Use
perf script | speedscope -
# Opens in browser with left-heavy / right-heavy / sandwich views

Firefox Profiler

# Supports perf data via import at profiler.firefox.com
perf script -F +pid > profile.perf
# Upload to https://profiler.firefox.com

inferno (Rust implementation, fast)

cargo install inferno
perf script | inferno-collapse-perf | inferno-flamegraph > fg.svg

pprof (Go, supports flamegraphs)

go tool pprof -http=:8080 profile.pb.gz
# Flamegraph view available in the web UI

---

Reading patterns quick reference

The actionable hotspot is always the widest frame that has no (or very narrow) children above it. That is where CPU time is actually consumed.