Reducing Allocations in Rust and Finding the Hot Ones

Allocation is rarely the slowest thing a program does, and it is the thing most often hiding in the middle of a hot loop. Each one is a few dozen nanoseconds plus a free, plus a cache miss when the memory is cold. A million allocations a second is not remarkable, and ten million is a profile that is mostly malloc. This post shows how to count them and what to change. It follows the profiling guide.

Rust 1.90, edition 2024.

Count them exactly

Wrap the global allocator with a counter:

use std::alloc::{GlobalAlloc, Layout, System};
use std::sync::atomic::{AtomicUsize, Ordering};

struct Counting;
static ALLOCS: AtomicUsize = AtomicUsize::new(0);

unsafe impl GlobalAlloc for Counting {
    unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
        ALLOCS.fetch_add(1, Ordering::Relaxed);
        // SAFETY: forwarding to the system allocator with the caller's layout.
        unsafe { System.alloc(layout) }
    }
    unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
        // SAFETY: ptr and layout come from a matching `alloc` call.
        unsafe { System.dealloc(ptr, layout) }
    }
}

#[global_allocator]
static A: Counting = Counting;

fn allocations<R>(f: impl FnOnce() -> R) -> (R, usize) {
    let before = ALLOCS.load(Ordering::Relaxed);
    let r = f();
    (r, ALLOCS.load(Ordering::Relaxed) - before)
}

Wrap a function in allocations(|| ...) to see its count. Do it in a test or a small binary, not production. Crates such as dhat give per call-site totals; see the memory measurement posts for the heap profilers.

The usual sources

fn slow(words: &[&str]) -> Vec<String> {
    let mut out = Vec::new();
    for w in words {
        let line = format!("{}!", w);         // 1 allocation per word
        out.push(line.to_uppercase());        // 1 more per word
    }
    out
}

For N words that is about 2N allocations plus the Vec regrowth. Rewrite:

fn faster(words: &[&str]) -> Vec<String> {
    let mut out = Vec::with_capacity(words.len());   // one allocation
    for w in words {
        let mut s = w.to_uppercase();                // one per word, unavoidable here
        s.push('!');
        out.push(s);
    }
    out
}

Allocation count drops from roughly 2N + log N to N + 1. If the caller only needs to write the result somewhere, passing a &mut String or a Write target removes the per word String entirely.

Reuse

let mut buf = String::new();
for item in items {
    buf.clear();
    use std::fmt::Write;
    write!(buf, "{}:{}", item.id, item.name).unwrap();
    sink(&buf);
}

After the first few iterations, buf has the capacity it needs and no further allocation happens. See reusing buffers.

Borrow

Storing &str slices of the input instead of String copies avoids the allocation and the copy. The cost is a lifetime parameter; for parsers that return slices of their input it is worth it. Cow<str> handles the sometimes-copy case.

Small values

Short strings and vectors that stay small fit inline with smallvec or compact_str, trading a larger stack object for no heap use; see small string and small vec optimizations.

Measure the effect

A counting allocator says allocations dropped; a benchmark says whether time did. Check both. Time rarely falls in proportion. The Rust Performance Book has a dedicated chapter on heap allocations.