Microbenchmarks That Lie and How to Write Honest Rust Ones
Microbenchmarks mislead when the optimizer deletes work, inputs are unrealistic or caches are warm. Learn the usual lies and how to write benchmarks you can trust.

Microbenchmarks measure small pieces of code in isolation, and they are easy to get wrong in ways that produce confident, wrong numbers. These are the usual ones. Everything here applies to Rust 1.90; the code illustrates mistakes and no timing results are claimed.
Lie 1: the work disappears
use std::time::Instant;
fn square_sum(n: u64) -> u64 {
(0..n).map(|x| x * x).sum()
}
fn main() {
let start = Instant::now();
for _ in 0..1000 {
square_sum(10_000);
}
println!("{:?}", start.elapsed());
}The results of the calls are discarded, the function is pure, and in a release build the optimizer can remove the loop entirely. The output is an absurdly small number. The fix is black_box:
use std::hint::black_box;
use std::time::Instant;
fn square_sum(n: u64) -> u64 {
(0..n).map(|x| x * x).sum()
}
fn main() {
let start = Instant::now();
for _ in 0..1000 {
black_box(square_sum(black_box(10_000)));
}
println!("{:?}", start.elapsed());
}black_box is an identity function the compiler is asked to treat as opaque. Wrapping the argument stops constant folding; wrapping the result stops removal. It is a best-effort hint, so check suspicious results in assembly. Benchmark frameworks do this for you (Criterion and Divan).
Lie 2: data that is too nice
A benchmark that sorts 100 sorted integers measures the best case of the algorithm. Real inputs are larger, messier and sometimes adversarial. Use sizes that match production (including ones larger than the cache), and several shapes: random, sorted, reversed, many duplicates. The sort comparison in stable versus unstable sorting shows why input shape changes the winner.
Lie 3: warm caches and trained branches
Running the same code on the same data a million times keeps everything in cache and teaches the branch predictor the exact pattern. A function that is fast in the loop may be slow when called once between other work. If your code runs cold in production, benchmark it cold: rotate through many different inputs so each call sees data not recently used. See cache friendly code.
Lie 4: setup is inside the timer
Allocating the input, building a Vec or opening a file inside the measured region measures the setup. Create data once outside, or use batched setup in the framework (iter_batched in Criterion, with_inputs in Divan), so only the operation is timed.
Lie 5: noise
Frequency scaling, thermal throttling, other processes and virtual machines move timings by more than the differences you are chasing. Repeat runs, report the spread (not only the mean), close other programs, and compare variants in the same run, interleaved if possible. Do not compare numbers from different machines or days.
Lie 6: allocator and memory effects
A benchmark that allocates may behave differently under the system allocator and under jemalloc, and a freshly started process has a different heap than a long-running one. Test with the allocator you ship, and watch allocation counts as well as time (allocations).
Checklist
| Check | Question |
|---|---|
| Output used | is every result passed to black_box or consumed? |
| Input hidden | could any argument be a compile-time constant? |
| Realistic size | is the data bigger than L2 cache like production? |
| Varied shapes | more than one input distribution? |
| Setup separate | is only the target timed? |
| Repeated and compared | same machine, several runs, spread reported? |
| Confirmed | does a larger benchmark agree? |
When a microbenchmark and the profile disagree, trust the profile of the real workload (profiling with a flamegraph), and read the other traps in benchmarking pitfalls.


