Data Parallelism in Rust With Rayon and Parallel Iterators

Changing iter() to par_iter() took a prime count over 2,000,000 numbers from 183 ms to 68 ms, a 2.7x speedup for a one-word edit. Changing the same word on a sum of doubled integers made the program slower, from 0.33 ms to 0.52 ms. Rayon makes parallelism easy to write and just as easy to apply where it does not pay, and the difference between those two cases is the thing to learn. Everything here compiles on Rust 1.99 with edition 2024 and rayon 1.12, and the timings are release builds on a 10-core Apple silicon laptop.

The one-word change

Add rayon = "1.12" to Cargo.toml, import the prelude, and replace iter with par_iter. The prelude adds the parallel methods to slices, vectors, ranges and maps.

[package]
name = "rayon-demo"
version = "0.1.0"
edition = "2024"

[dependencies]
rayon = "1.12"

use rayon::prelude::*;
use std::time::Instant;

fn is_prime(n: u64) -> bool {
    n > 1 && (2..).take_while(|d| d * d <= n).all(|d| n % d != 0)
}

fn main() {
    let numbers: Vec<u64> = (1..=2_000_000).collect();

    let start = Instant::now();
    let serial = numbers.iter().filter(|&&n| is_prime(n)).count();
    println!("iter():     {serial} primes in {:>8.1?}", start.elapsed());

    let start = Instant::now();
    let parallel = numbers.par_iter().filter(|&&n| is_prime(n)).count();
    println!("par_iter(): {parallel} primes in {:>8.1?}", start.elapsed());

    // cheap per-item work: parallelism should not help
    let start = Instant::now();
    let s1: u64 = numbers.iter().map(|n| n * 2).sum();
    let t1 = start.elapsed();
    let start = Instant::now();
    let s2: u64 = numbers.par_iter().map(|n| n * 2).sum();
    let t2 = start.elapsed();
    assert_eq!(s1, s2);
    println!("sum of doubles: iter {t1:>8.1?}, par_iter {t2:>8.1?}");

    println!("threads in the global pool: {}", rayon::current_num_threads());
}

iter():     148933 primes in  182.8ms
par_iter(): 148933 primes in   67.5ms
sum of doubles: iter  325.3µs, par_iter  521.2µs
threads in the global pool: 10

Both answers match, 148,933 primes, so the parallel version computed the same thing. The speedup of 2.7x on 10 threads is modest, and that has a cause: the cost of is_prime grows with n, so the last chunks are far more expensive than the first, and some of this laptop's cores are slower efficiency cores. Rayon's work stealing balances the uneven chunks far better than a fixed split would (compare the hand-rolled threads in spawning threads), and it still does not give ten times.

The second comparison is the warning. Doubling a number takes about a nanosecond, so splitting two million of them across threads costs more in scheduling than it saves. As a rule of thumb, an item should cost at least a microsecond of work, or the whole job should take at least a few milliseconds serially, before a parallel iterator is worth trying.

How it works in one paragraph

Rayon keeps a global pool of worker threads, by default one per logical core (RAYON_NUM_THREADS overrides it). A parallel iterator splits its input in half recursively, puts one half on the current worker's queue and processes the other. An idle worker steals the oldest queued half from a busy one. Splitting stops when the pieces are small enough, and the partial results are combined on the way back up. Nothing is spawned per item, which is why it is so much cheaper than creating a thread each.

What the closure may not do

The closure runs on several threads at once, so it must be Fn and Send + Sync, which rules out mutating a captured variable. The natural attempt at a parallel sum is the first one most people write.

// does not compile
use rayon::prelude::*;

fn main() {
    let mut total = 0u64;
    (1..=100u64).into_par_iter().for_each(|n| {
        total += n;
    });
    println!("{total}");
}

error[E0594]: cannot assign to `total`, as it is a captured variable in a `Fn` closure
 --> src/main.rs:7:9
  |
5 |       let mut total = 0u64;
  |           --------- `total` declared here, outside the closure
6 |       (1..=100u64).into_par_iter().for_each(|n| {
  |                                    -        --- in this closure
  |  __________________________________|
  | |
7 | |         total += n;
  | |         ^^^^^^^^^^ cannot assign
8 | |     });
  | |______- expects `Fn` instead of `FnMut`

This is the compiler preventing a data race. A Mutex around total would compile and would also make every thread queue for the lock, which defeats the point. The right fix is to express the computation as a reduction, so that each thread sums its own piece and Rayon adds the pieces together.

use rayon::prelude::*;

fn main() {
    let total: u64 = (1..=100u64).into_par_iter().sum();
    println!("sum: {total}");

    let (min, max) = [8, 3, 9, 1, 7]
        .par_iter()
        .map(|&n| (n, n))
        .reduce(|| (i32::MAX, i32::MIN), |a, b| (a.0.min(b.0), a.1.max(b.1)));
    println!("min {min}, max {max}");

    let words = ["rust", "is", "fast", "and", "safe"];
    let lengths: Vec<usize> = words.par_iter().map(|w| w.len()).collect();
    println!("{lengths:?}");
}

sum: 5050
min 1, max 9
[4, 2, 4, 3, 4]

sum is a ready-made reduction. reduce takes an identity closure and a combining closure, and the combine step must be associative, because Rayon decides how to group the pieces. Subtraction would give a different answer each run; addition, min and max are safe. And collect into a Vec preserves input order, so the lengths line up with the words even though they were computed on different threads.

Per-thread scratch space

Some jobs need a mutable buffer, such as a String to format into or a parser state, and allocating one per item is wasteful. map_init gives each worker its own value, created once per piece of work and reused across items.

use rayon::prelude::*;
use std::fmt::Write;

fn main() {
    let ids: Vec<u32> = (1..=6).collect();
    let labels: Vec<String> = ids
        .par_iter()
        .map_init(String::new, |buf, id| {
            buf.clear();
            write!(buf, "item-{id:03}").unwrap();
            buf.clone()
        })
        .collect();
    println!("{labels:?}");
}

["item-001", "item-002", "item-003", "item-004", "item-005", "item-006"]

The buffer is not shared between threads, so no lock is involved, and results still come out in input order. for_each_init is the same idea for side effects.

Chunks when items are tiny

If each item is cheap but the total is large, give Rayon bigger units instead of abandoning it. par_chunks(n) hands each task a slice of n items, so the scheduling overhead is paid once per chunk, and the work inside a chunk is a plain serial loop the compiler can vectorize. A chunk size in the thousands is a reasonable first guess, and with_min_len on any parallel iterator sets a floor on how finely it will split. Tune it with the same benchmark you used to decide on parallelism in the first place.

Controlling the pool

The global pool is shared by every Rayon call in the process, so a library that uses it competes with your own code. For an isolated job, or a limit lower than the core count, build a pool and run the work inside it with install.

use rayon::prelude::*;
use rayon::ThreadPoolBuilder;

fn main() {
    let pool = ThreadPoolBuilder::new()
        .num_threads(2)
        .thread_name(|i| format!("cruncher-{i}"))
        .build()
        .expect("build pool");

    let (threads, total) = pool.install(|| {
        let total: u64 = (1..=1_000u64).into_par_iter().map(|n| n * n).sum();
        (rayon::current_num_threads(), total)
    });
    println!("ran on {threads} threads, total {total}");
}

ran on 2 threads, total 333833500

Inside install, current_num_threads reports the pool you built and not the global one. This is the tool for the case where a web service would otherwise let a parallel job take every core and starve request handling.

Sorting, and the cost of the machinery

Rayon also parallelizes in-place operations, and par_sort_unstable is the one that most often pays. To see how the benefit scales we sorted 5,000,000 pseudo-random u64 values with the standard sort and then with Rayon pools of 1, 2, 4 and 8 threads, always starting from a fresh unsorted copy.

use rayon::prelude::*;
use rayon::ThreadPoolBuilder;
use std::time::Instant;

fn main() {
    let original: Vec<u64> = (0..5_000_000u64)
        .map(|i| i.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407) >> 20)
        .collect();

    let mut serial = original.clone();
    let start = Instant::now();
    serial.sort_unstable();
    println!("sort_unstable:           {:>8.1?}", start.elapsed());

    for threads in [1, 2, 4, 8] {
        let pool = ThreadPoolBuilder::new().num_threads(threads).build().unwrap();
        let mut copy = original.clone();
        let start = Instant::now();
        pool.install(|| copy.par_sort_unstable());
        println!("par_sort_unstable, {threads} threads: {:>8.1?}", start.elapsed());
        assert_eq!(copy, serial);
    }
}

sort_unstable:             83.0ms
par_sort_unstable, 1 threads:  131.2ms
par_sort_unstable, 2 threads:   69.5ms
par_sort_unstable, 4 threads:   36.8ms
par_sort_unstable, 8 threads:   23.9ms

Two results stand out. On one thread the parallel sort is 58% slower than the serial one (131 ms against 83 ms), because it is a different algorithm that does extra work to be splittable. Parallelism only pays once you have enough threads to cover that overhead, here at 2 threads and beyond. And the scaling is close to linear, halving the time each time the threads double, up to a 3.5x gain over the serial sort at 8. We first ran this on data that was already sorted in reverse, and the one-thread time was 6 ms: the standard sort recognizes that pattern and finishes almost instantly. A benchmark that feeds an algorithm its best-case input measures nothing, so use data shaped like yours.

When parallel iterators are the wrong tool

• Tiny or cheap work, as the sum of doubles showed. Measure first, and build in release mode: a debug build exaggerates the closure's cost and hides the overhead (release versus debug speed, benchmarking pitfalls).
• Waiting, not computing. Rayon workers are for CPU work; a closure that blocks on a network call occupies a worker and the pool stalls. Use async tasks for I/O.
• Inside an async runtime. A long parallel job run directly from an async task blocks that worker. Start it from spawn_blocking and await the result (blocking the Tokio runtime).
• Order-dependent logic. A running total or a state machine over the sequence cannot be split without changing its meaning.