Benchmarking in Rust With Criterion and Divan Compared

Measuring a Rust function takes more than a timer around a loop: the optimizer deletes unused work, the first run is cold, and one sample means nothing. Two crates handle the details. Everything compiles on Rust 1.90 with edition 2024, criterion 0.5 and divan 0.1. Timings are not shown, because they would only describe one machine.

The function

pub fn sum_even_squares(v: &[u64]) -> u64 {
    v.iter().filter(|x| **x % 2 == 0).map(|x| x * x).sum()
}

Criterion

[dev-dependencies]
criterion = { version = "0.5", features = ["html_reports"] }

[[bench]]
name = "sum"
harness = false

use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
use demo::sum_even_squares;

fn bench(c: &mut Criterion) {
    let mut group = c.benchmark_group("sum_even_squares");
    for size in [100usize, 10_000, 1_000_000] {
        let data: Vec<u64> = (0..size as u64).collect();
        group.throughput(Throughput::Elements(size as u64));
        group.bench_with_input(BenchmarkId::from_parameter(size), &data, |b, data| {
            b.iter(|| sum_even_squares(black_box(data)))
        });
    }
    group.finish();
}

criterion_group!(benches, bench);
criterion_main!(benches);

cargo bench

Criterion warms up, takes many samples, reports a time interval and compares to the previous run: change: [-1.2% +0.3% +1.9%] with "No change in performance detected" or "Performance has regressed". HTML reports land in target/criterion/. Name a baseline with -- --save-baseline main and compare later with --baseline main.

Divan

[dev-dependencies]
divan = "0.1"

[[bench]]
name = "sum"
harness = false

use demo::sum_even_squares;

fn main() {
    divan::main();
}

#[divan::bench(args = [100, 10_000, 1_000_000])]
fn sum(bencher: divan::Bencher, size: usize) {
    let data: Vec<u64> = (0..size as u64).collect();
    bencher.bench(|| sum_even_squares(divan::black_box(&data)));
}

The benchmark is a function with an attribute, and args runs it once per value. Divan prints a table with fastest, slowest, median and mean. Allocation counting is built in with a global allocator wrapper:

#[global_allocator]
static ALLOC: divan::AllocProfiler = divan::AllocProfiler::system();

Then each benchmark also reports allocation counts and bytes, which Criterion does not.

Comparison

Pitfalls shared by both

• **Dead code elimination:** use black_box on the input and let the closure return the result.
• **Setup inside the timed loop:** build data outside b.iter, or use batched setup (iter_batched, bencher.with_inputs).
• **Noise:** close other programs, pin CPU frequency if you can, and compare relative differences from runs on one machine.
• **Unrealistic data:** a sorted array benchmarks differently from real data.
• **CI:** shared runners vary too much for pass/fail thresholds; use them to record trends.

The traps are covered at length in benchmarking pitfalls and microbenchmarks that lie. For measuring a whole program, use hyperfine (benchmarking CLIs), and to see where time goes, a flamegraph.