Profile Guided Optimization for Rust Binaries Step by Step

Every optimizing compiler guesses. When it compiles an if, it has to decide which side is the likely one, so it can lay out the hot path as straight-line code and push the cold path out of the way. When it sees a call, it has to decide whether inlining is worth the code growth. It makes those guesses from static heuristics, and the heuristics are often right and sometimes badly wrong. Profile-guided optimization, PGO, replaces the guesses with measurements: you run an instrumented build on a representative workload, record which branches were taken and which functions were hot, and recompile with that data.

This guide runs the whole PGO workflow on a real program, shows the speedup it produced, and spends equal time on what can make PGO useless or harmful, because the technique is only as good as the workload you train it on. Everything was measured on Rust 1.99 (edition 2024) on an Apple silicon Mac, with the LLVM tools installed through rustup component add llvm-tools. The numbers are real; yours will differ in size and direction depends on your program.

When PGO is worth considering

PGO changes code layout and inlining decisions, so it helps programs whose speed depends on those: interpreters and virtual machines, parsers, compilers, database engines, anything with a large match in a hot loop and a skewed distribution of cases. It does little for code that is already a tight numeric kernel, bound by memory bandwidth, or dominated by system calls and waiting. The Rust compiler itself is built with PGO, which is one of the reasons it got measurably faster over recent years, and that is the genre of program that benefits.

Before trying it, make sure cheaper levers are already pulled. Turn on lto = "fat" and codegen-units = 1, check opt-level, and consider -C target-cpu=native if you control the deployment hardware. PGO is a final step, worth perhaps 5 to 30 percent on the right program, and it adds build complexity that you will carry for the life of the project.

A program that PGO can help

The test program is a small stack-machine interpreter. It runs a loop that dispatches on an Op enum with ten variants through one match, which is the classic interpreter shape. The branch behavior depends entirely on the bytecode it runs, so the compiler cannot know statically which Op is common.

use std::hint::black_box;
use std::time::Instant;

#[derive(Clone, Copy, Debug)]
enum Op {
    Push(i64),
    Add,
    Mul,
    Dup,
    Swap,
    Drop,
    JumpIfZero(usize),
    Jump(usize),
    Dec,
    Halt,
}

/// A tiny stack machine: the kind of interpreter loop whose branch pattern depends on
/// the program it runs, which is where profile-guided optimization helps.
fn run(code: &[Op], iterations: i64) -> i64 {
    let mut stack: Vec<i64> = Vec::with_capacity(64);
    let mut pc = 0;
    let mut acc = 0i64;
    stack.push(iterations);
    loop {
        match code[pc] {
            Op::Push(v) => stack.push(v),
            Op::Add => {
                let b = stack.pop().unwrap();
                let a = stack.pop().unwrap();
                stack.push(a.wrapping_add(b));
            }
            Op::Mul => {
                let b = stack.pop().unwrap();
                let a = stack.pop().unwrap();
                stack.push(a.wrapping_mul(b));
            }
            Op::Dup => {
                let a = *stack.last().unwrap();
                stack.push(a);
            }
            Op::Swap => {
                let n = stack.len();
                stack.swap(n - 1, n - 2);
            }
            Op::Drop => {
                acc = acc.wrapping_add(stack.pop().unwrap());
            }
            Op::JumpIfZero(t) => {
                if *stack.last().unwrap() == 0 {
                    pc = t;
                    continue;
                }
            }
            Op::Jump(t) => {
                pc = t;
                continue;
            }
            Op::Dec => {
                let a = stack.pop().unwrap();
                stack.push(a - 1);
            }
            Op::Halt => return acc,
        }
        pc += 1;
    }
}

fn main() {
    // counter on the stack; each loop pass does a little arithmetic and counts down
    let code = [
        Op::JumpIfZero(11), // 0: exit when the counter is zero
        Op::Dup,            // 1
        Op::Push(3),        // 2
        Op::Mul,            // 3
        Op::Push(7),        // 4
        Op::Add,            // 5
        Op::Drop,           // 6: fold into the accumulator
        Op::Dec,            // 7
        Op::Jump(0),        // 8
        Op::Halt,           // 9 (unreachable padding)
        Op::Halt,           // 10
        Op::Halt,           // 11
    ];

    // Program B exercises a different mix: the counter lives under a second value, so it
    // needs Swap, and the loop body is Add-heavy instead of Mul-heavy.
    let code_b = [
        Op::JumpIfZero(10), // 0
        Op::Dup,            // 1
        Op::Push(2),        // 2
        Op::Swap,           // 3
        Op::Add,            // 4
        Op::Push(4),        // 5
        Op::Add,            // 6
        Op::Drop,           // 7
        Op::Dec,            // 8
        Op::Jump(0),        // 9
        Op::Halt,           // 10
    ];
    let program: &[Op] = if std::env::var("PROG").as_deref() == Ok("b") { &code_b } else { &code };

    let iterations = black_box(std::env::var("N").ok().and_then(|s| s.parse().ok()).unwrap_or(40_000_000));
    let mut best = f64::MAX;
    let mut result = 0;
    for _ in 0..5 {
        let start = Instant::now();
        result = run(program, iterations);
        best = best.min(start.elapsed().as_secs_f64());
    }
    println!("result {result}, best of 5: {:.0} ms", best * 1000.0);
}

There are two bytecode programs. Program A, the default, counts down from the value on the stack and on each pass multiplies, adds, folds the result into an accumulator and jumps back; only five of the ten operations ever execute, and JumpIfZero almost never takes its branch. Program B, selected with PROG=b, exercises a different mix, including Swap, which A never uses. The first sections use program A alone. Program B exists for the experiment on mismatched training later in the guide. A compiler that knew this would lay out the five hot arms as a tight sequence and treat the rest as rare. The black_box and the N environment variable prevent the compiler from precomputing the answer and let us choose a small workload for training. The program reports the best of five runs, which removes most scheduling noise.

Step one: the baseline

cargo build --release
./target/release/pgolab

Run four times in alternation with the profile-built binary below, the baseline best-of-five time was stable at 667 to 671 ms:

base: result 2400000340000000, best of 5: 671 ms
base: result 2400000340000000, best of 5: 671 ms
base: result 2400000340000000, best of 5: 667 ms
base: result 2400000340000000, best of 5: 670 ms

The first runs of this program, before the measurement method was settled, ranged from 690 to 1,009 ms. That spread is typical of a laptop with thermal throttling and background load, and it is the reason the comparison above uses the best of five, repeated four times, with the two binaries interleaved so drift affects both equally. A PGO result measured with a single run of each is not evidence.

Step two: build an instrumented binary

The first PGO build adds counters to every branch and function. The flag is -C profile-generate, which takes the directory where the raw profile data will be written. Use a separate target directory so the instrumented build does not pollute your normal one:

RUSTFLAGS="-Cprofile-generate=/tmp/pgo-data" \
  CARGO_TARGET_DIR=target-gen cargo build --release

The instrumented binary is a little larger (494,672 bytes against 472,160) and runs a little slower, because every counted branch performs an increment. For this program the slowdown was small, 36 ms against 33 ms on a 2-million-iteration run, but instrumented builds of branchier code can be several times slower, so do not time them.

Step three: run the workload that matters

This is the step where PGO succeeds or fails. Run the instrumented binary on a workload that looks like production:

N=2000000 ./target-gen/release/pgolab
ls /tmp/pgo-data

result 6000017000000, best of 5: 36 ms
default_11598746608350676856_0.profraw

The run produced one .profraw file, a binary record of how many times each instrumented branch went each way. The training run used 2 million iterations, 5 percent of the size of the benchmark, because PGO needs the shape of the behavior, not its volume: the ratio of taken to not-taken branches is the same at any size. For a server, the equivalent is to replay a recorded sample of real traffic, or run a load test with a realistic mix of requests, for a few minutes.

Step four: merge the raw data

Raw profiles must be merged and converted into an indexed format by llvm-profdata. It ships with the llvm-tools rustup component, in the toolchain's own directory, so its version matches the compiler's LLVM:

rustup component add llvm-tools
$(rustc --print sysroot)/lib/rustlib/aarch64-apple-darwin/bin/llvm-profdata \
  merge -o /tmp/pgo-merged.profdata /tmp/pgo-data

Replace the target triple with your host's. The merged file here was 2,320 bytes. If you run several training workloads, put all their .profraw files in the directory and merge combines them, weighting each by how many times it ran. That is how you cover several kinds of traffic in one profile. Using the version of llvm-profdata from the system's LLVM instead of Rust's bundled one is a common source of "unsupported profile version" errors.

Step five: rebuild with the profile

RUSTFLAGS="-Cprofile-use=/tmp/pgo-merged.profdata" \
  CARGO_TARGET_DIR=target-pgo cargo build --release

The compiler reads the merged data and uses it to choose branch weights, order basic blocks, decide inlining, and place hot functions together. Now compare the two release binaries, alternating runs:

base: result 2400000340000000, best of 5: 671 ms
pgo:  result 2400000340000000, best of 5: 483 ms
base: result 2400000340000000, best of 5: 671 ms
pgo:  result 2400000340000000, best of 5: 484 ms
base: result 2400000340000000, best of 5: 667 ms
pgo:  result 2400000340000000, best of 5: 484 ms
base: result 2400000340000000, best of 5: 670 ms
pgo:  result 2400000340000000, best of 5: 482 ms

The profile-guided binary ran in about 483 ms against 670 ms for the baseline: 28 percent less time, or a 1.39 times speedup, and the result value is identical in every run, which is the check that PGO changed speed and not behavior. The binary size barely moved (471,936 bytes against 472,160), so the gain did not come from inlining more code. It came from knowing which match arms are hot and which branches are almost never taken, and arranging the machine code accordingly.

Does LTO already give you this?

A fair objection is that the free settings, lto = "fat" and codegen-units = 1, might capture the same gain. They address a different problem. Link-time optimization lets the compiler inline across crate boundaries and see the whole program; it does nothing to tell it which branch is likely. The experiment is quick: build the baseline with those two settings, then build a profile-guided binary with the same settings, using a profile generated from an instrumented build that also used them.

program A      baseline  661 ms    LTO + 1 codegen unit  661 ms    LTO + PGO  387 ms
program B      baseline  743 ms    LTO + 1 codegen unit  741 ms    LTO + PGO  477 ms

For this program the free settings changed nothing, 661 ms against 661 ms, because it is a single crate with no cross-crate calls to inline. Profile guidance took the same code from 661 to 387 ms on program A, a 41 percent reduction, and from 743 to 477 ms on program B. The two techniques are complementary in a real application: LTO removes the boundaries between your crates and your dependencies, and PGO then tells the optimizer where inside the merged program the time goes. Apply both, in that order, and measure each step separately so you know which one paid for itself.

Where the gain comes from

It is worth knowing why the speedup happens, because it tells you which programs will see it. Look at the interpreter's match. Without a profile, the compiler sees ten arms of unknown likelihood and typically emits a jump table or a chain of comparisons in source order. With the profile, it learns that five arms execute and JumpIfZero almost never takes its jump, so it orders blocks so that the hot sequence falls through without taken branches, moves the cold arms out of the hot instruction cache lines, and inlines the small stack helpers into the arms that use them. The CPU's branch predictor and instruction fetch both prefer that layout. None of this changes what the program computes, which is why the result value is identical in every run above.

That mechanism has a practical consequence. Programs dominated by one big dispatch, a bytecode interpreter, a state machine, a protocol decoder, a SQL executor, gain the most. A program that mostly calls into libraries or the operating system gains little, since its time is spent in code the profile does not cover, and a program already limited by memory latency gains little because layout cannot fix a cache miss. A quick predictor: if a flamegraph shows your time concentrated in a few dozen branchy functions of your own, PGO is worth the experiment. If it shows time spread across memcpy, allocator calls and system calls, it is not.

What can make PGO useless or harmful

The speedup above is the best case: a clean training workload that matches the benchmark exactly. Real projects find the failure modes quickly.

A training workload that does not match production

The profile encodes the behavior of the training run. If production does something different, the optimizer has tuned for the wrong thing, and the paths that production uses get less attention than they would with no profile. The risk is real, and the experiment to quantify it is cheap, so run it. The program has two workloads, A and B. Four binaries were built: the baseline, one trained on A, one trained on B, and one trained on both with the profiles merged. Each was benchmarked on each workload, best of three alternating runs:

The result is not what the warning predicts. The binary trained on the wrong workload was not worse than the baseline on either benchmark; the A-trained binary was 21 percent faster than the baseline even on B, which it never saw. And the B-trained binary beat the A-trained one on workload A itself, 388 ms against 479 ms. The explanation is that B's profile covers more of the instruction set, including arms that A never executes, and that gives the optimizer better information about the shared dispatch loop than A's narrower profile did. The merged profile matched the best single one.

Two honest conclusions follow. This interpreter is forgiving, because both workloads share one dispatch loop whose hot structure is similar, so a mismatch costs little. A program whose workloads exercise disjoint code would show a harder penalty. And the usual advice to train on a broad mix is supported: the merged profile was never worse than either single one. Do this test on your own program with two realistic workloads before trusting a profile, since the only safe statement is the measurement.

A stale profile

Profiles are tied to the exact source and compiler version. Changing the code after recording a profile makes some functions mismatch: the compiler discards the data for functions whose control flow changed and falls back to the heuristics. A small edit costs you only the changed functions, but a profile that predates a large refactor is mostly dead weight. Regenerate the profile in the same pipeline that builds the release, from the same commit. A profile checked into the repository and refreshed once a year is a common way to get a PGO build that does nothing.

The build becomes three builds

PGO builds the program twice and runs it once. In CI that costs build time and a place to run a representative workload, which must be fast and deterministic enough to put in a pipeline. If the workload needs a database or a network, your release job now depends on one. The pragmatic design is a script that builds the instrumented binary, runs a canned workload against it, merges, and builds the final one, run only for tagged releases, with ordinary builds left unoptimized by PGO.

Reproducibility and debugging

A PGO build depends on a data file as well as the source, so two builds of one commit with different profiles produce different binaries. If you promise reproducible builds, the profile must be archived with the release and its hash recorded. And an optimized layout makes stack traces and profilers harder to read, since functions are split into hot and cold pieces. Keep an unoptimized build for debugging.

Tools that automate the workflow

Doing the steps by hand teaches what they are, and for regular use there is cargo-pgo, a Cargo subcommand that wraps the same flags: cargo pgo build for the instrumented binary, running it, then cargo pgo optimize for the final build, with the right llvm-profdata located for you. This guide ran the manual flags and did not run cargo-pgo, so check its documentation for the current syntax. A related technique, BOLT, applies the same idea after linking: it reorders the machine code of a finished binary using a profile collected by Linux perf. It is Linux-only, and it composes with PGO for further gains on large binaries, at the price of a more fragile pipeline.

A decision procedure

Decide in this order. First, profile the program (see profiling Rust programs from flamegraph to a fix) and confirm it is CPU-bound in branchy code and not waiting on I/O. Second, apply the free settings: lto, codegen-units = 1, maybe target-cpu. Third, try PGO on a branch with a training workload you can defend, and measure with alternating, repeated runs as above. Fourth, keep it only if the gain beats your noise by a clear margin, which for this program was 28 percent against a noise floor under 1 percent. Fifth, automate it in the release pipeline and archive the profile with the artifact. If at step three the speedup is a few percent, drop it: the maintenance cost of the pipeline outweighs the gain.

When not to use PGO

Skip it for libraries, because the profile belongs to the final binary and not to a crate that others link. Skip it when you cannot construct a representative workload, since a misleading profile is worse than none. Skip it for programs dominated by I/O wait, where the CPU is not the bottleneck and nothing in the layout matters. And skip it while the code is changing quickly, because each change makes the profile stale and the maintenance burden outweighs the reward. For the order of the cheaper optimizations that come before it, read Cargo release profile settings.