Bounded Channels and Backpressure in Tokio Done Explained

A bounded channel is the cheapest backpressure mechanism in async Rust: when the consumer falls behind, the buffer fills, and the producer's send().await stops until there is room. The number you pass to mpsc::channel is therefore not a performance knob to be set to a big value and forgotten. It decides how much work can pile up in memory, how long a message can wait, how fast a burst is absorbed and what the system does when it is overloaded. This post sizes a channel from first principles, measures throughput against capacity on one machine, and shows the three overload policies (wait, shed, drop old) with code. Everything compiled on Rust 1.99, edition 2024, with tokio 1.

What the capacity buys

A channel's buffer smooths over differences in speed between producer and consumer. A capacity of one means the producer and consumer hand off messages in lockstep: every send must wait for the previous message to be taken. A larger buffer lets each side run ahead of the other for a while, which reduces the number of times a task has to park and wake. Past a point the extra room buys nothing and costs memory and latency, because a message at the back of a long queue waits for everything before it.

The measurement shows the shape. The benchmark sends two million u64 values from one task to another through a Tokio mpsc channel on an Apple M1 Max in a release build, best of three runs, with a consumer that only sums them:

Going from 1 to 32 slots made the channel 3.5 times faster, from 5.2 to 18 million messages a second, and everything past 32 was flat or slightly worse: at 4,096 slots the throughput fell by a quarter, plausibly because the larger buffer no longer fits in the cache of one core. The curve is the general rule. A capacity of tens is enough to absorb scheduling jitter between tasks, and hundreds or thousands are for absorbing bursts of work, which is a different job with a different sizing method.

Sizing for a burst: Little's law

If messages arrive at λ per second on average and each spends W seconds in the system, then on average L = λ × W messages are in the system. For a channel feeding a worker that needs 20 ms per job at 400 jobs per second arriving, the worker is saturated (400 × 0.02 = 8 workers' worth of time), so you need eight workers or the queue grows without bound. The buffer cannot fix a rate mismatch; it can only absorb a temporary excess. Size it for the burst you want to survive without making a producer wait:

fn burst_capacity(arrival_per_sec: f64, drain_per_sec: f64, burst_secs: f64) -> usize {
    let excess = (arrival_per_sec - drain_per_sec).max(0.0);
    (excess * burst_secs).ceil() as usize
}

fn main() {
    // A worker pool drains 500 jobs/s. Traffic spikes to 800/s for 2 seconds.
    println!("{}", burst_capacity(800.0, 500.0, 2.0));
    // A faster pool (900/s) never builds a queue from the same spike.
    println!("{}", burst_capacity(800.0, 900.0, 2.0));
    // A 10 s spike at 600/s against 500/s drain.
    println!("{}", burst_capacity(600.0, 500.0, 10.0));
}

600
0
1000

A 2-second spike at 800 jobs per second against a 500-per-second drain leaves 600 jobs queued at the peak, so a capacity of 600 absorbs it with no waiting. The third case needs 1,000 slots for a 10-second event, and the next question is whether you want a producer to wait that long or to be told "overloaded" at once; the channel's size is also a statement of how stale a message may get. A job that waits 2 seconds in the queue and has a client with a 1-second timeout is processed for nobody.

Policy 1: wait (the default)

send().await makes the producer wait, which propagates the pressure upstream. For a pipeline of tasks, each stage slows to the speed of the slowest one, and memory stays bounded. This is right for batch pipelines and internal queues where producers can afford to wait. For a request handler it means the client's request hangs, which is rarely what you want; combine it with a timeout.

Policy 2: shed load

When latency matters more than completeness, refuse the work. try_send returns TrySendError::Full immediately, and the handler turns that into an error response. A service that fails fast at its limit stays responsive for the requests it does accept:

use tokio::sync::mpsc::{self, error::TrySendError};
use tokio::time::{sleep, Duration};

#[derive(Debug)]
enum Accepted {
    Queued(u32),
    Rejected(u32),
}

fn submit(tx: &mpsc::Sender<u32>, job: u32) -> Accepted {
    match tx.try_send(job) {
        Ok(()) => Accepted::Queued(job),
        Err(TrySendError::Full(job)) => Accepted::Rejected(job),
        Err(TrySendError::Closed(job)) => Accepted::Rejected(job),
    }
}

#[tokio::main(flavor = "current_thread", start_paused = true)]
async fn main() {
    let (tx, mut rx) = mpsc::channel::<u32>(3);

    let results: Vec<_> = (1..=6).map(|job| submit(&tx, job)).collect();
    println!("{results:?}");
    println!("queue depth: {}", tx.max_capacity() - tx.capacity());

    let mut handled = Vec::new();
    while let Ok(Some(job)) = tokio::time::timeout(Duration::from_millis(10), rx.recv()).await {
        sleep(Duration::from_millis(50)).await;
        handled.push(job);
    }
    println!("handled {handled:?}");
}

[Queued(1), Queued(2), Queued(3), Rejected(4), Rejected(5), Rejected(6)]
queue depth: 3
handled [1, 2, 3]

Six jobs arrive at once into a three-slot queue, three are accepted and three are rejected with their payload returned so the caller can report or retry them. In an HTTP server that rejection is a 503 with a Retry-After, and the queue depth (max_capacity() - capacity()) is the number to export as a metric and alert on. Shedding is covered further in graceful degradation and designing for backpressure.

Policy 3: reserve first, then do the work

A subtle waste: a producer builds an expensive message (a database read, a render), then waits to send it, and may find the channel closed. reserve() splits the send into two steps: first claim a slot, which is where the waiting happens, then fill it with a value you compute after you know it will be accepted:

use tokio::sync::mpsc;
use tokio::time::{sleep, Duration};

async fn expensive_render(id: u32) -> String {
    sleep(Duration::from_millis(30)).await;
    format!("page {id}")
}

#[tokio::main(flavor = "current_thread", start_paused = true)]
async fn main() {
    let (tx, mut rx) = mpsc::channel::<String>(1);

    let consumer = tokio::spawn(async move {
        let mut got = Vec::new();
        while let Some(page) = rx.recv().await {
            sleep(Duration::from_millis(100)).await;
            got.push(page);
        }
        got
    });

    for id in 1..=3 {
        let permit = tx.reserve().await.unwrap();
        let page = expensive_render(id).await;
        permit.send(page);
    }
    drop(tx);

    println!("{:?}", consumer.await.unwrap());
}

["page 1", "page 2", "page 3"]

The permit guarantees that the later send cannot fail or wait. If the process is shutting down, the producer discovers it at reserve before spending 30 ms rendering. A dropped Permit releases the slot, so an error between the two steps does not leak capacity. It is also cancel safe in a select!, as the data-loss pitfalls in cancellation safety would suggest you want.

Choosing a number

1. Start with a small capacity, 32 to 128, for a task-to-task channel: it takes full advantage of the throughput curve above.
2. For a queue between a front end and a worker pool, size it for the burst you intend to absorb with the formula above, then cap it by the staleness you can accept (capacity divided by drain rate gives the worst wait).
3. Export the depth. A queue that is sometimes full is working; one that is always full means the consumer is too slow; one that is always empty means the capacity is irrelevant.
4. Decide the overload policy in the same sentence as the size: wait, shed, or drop the oldest (which needs a different structure, such as a watch or a ring buffer, since mpsc cannot evict).

What a bigger buffer hides

An oversized channel does not remove overload, it delays the symptom. The service looks healthy while the queue fills, latency rises for every request behind the burst, and the failure arrives later and all at once, usually as an out-of-memory kill. Small buffers make the problem visible early. The channel types and their loss behaviour are laid out in the four Tokio channels, and the runtime-level view of the same problem is in blocking the Tokio runtime.