Blocking the Tokio Runtime and How to Find the Culprit

The most common performance bug in async Rust is not a slow algorithm. It is a synchronous call inside an async fn that holds a worker thread for milliseconds or seconds while dozens of other tasks wait for that thread. Nothing crashes, nothing logs, and the symptom is a latency spike nobody can reproduce. This post shows the failure with a measurement, lists the usual culprits, explains how to find the one in your code (a heartbeat probe, the runtime's own metrics, tokio-console), and covers the three fixes: spawn_blocking, a dedicated thread, and the async version of the call. Everything compiled on Rust 1.99, edition 2024, with tokio 1.

Why one blocked call hurts everyone

Async tasks are cooperative. A worker thread runs one task until it awaits something that is not ready, then moves to the next. A call that blocks the thread (a sleep, a lock wait, a file read, a CPU loop) never reaches an await, so the worker cannot run anything else until it returns. With the default of one worker per core, blocking all of them stops the runtime; blocking one reduces capacity by that fraction, and any task whose wakeup lands on that worker's queue waits. You can watch it with a task that does nothing but tick, and record the longest gap between ticks:

use std::time::Duration;
use tokio::time::{sleep, Instant};

async fn heartbeat() -> Duration {
    let mut worst = Duration::ZERO;
    let mut last = Instant::now();
    for _ in 0..30 {
        sleep(Duration::from_millis(10)).await;
        let now = Instant::now();
        worst = worst.max(now - last - Duration::from_millis(10));
        last = now;
    }
    worst
}

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let probe = tokio::spawn(heartbeat());

    tokio::task::yield_now().await;
    std::thread::sleep(Duration::from_millis(250));

    let worst = probe.await.unwrap();
    println!("worst delay over 200 ms: {}", worst > Duration::from_millis(200));
}

worst delay over 200 ms: true

The probe asked to be woken every 10 ms and was kept waiting for a quarter of a second, because the main task called std::thread::sleep, which blocks the only thread. On a multi-thread runtime other workers would have picked the probe up and hidden the problem until every worker was busy, which is the production pattern: fine in testing with one request, terrible at the concurrency level where all the workers are blocked at once. A heartbeat like this one is a useful alarm to run in a service; export the worst gap as a metric, and a value over a few milliseconds says something is blocking.

The usual suspects

The password-hash case deserves a number, because it is the one that catches people who did everything else right. A bcrypt or Argon2 hash at a sensible cost takes tens to hundreds of milliseconds of pure CPU. Call it directly in a login handler and every login ties up a worker for that long: on an 8-core box, eight concurrent logins freeze the server's async side for the duration.

Fix 1: spawn_blocking

tokio::task::spawn_blocking moves a closure onto the runtime's separate blocking pool and returns a handle you await. The worker stays free:

use std::time::Duration;
use tokio::time::{sleep, Instant};

fn expensive_hash(input: &str) -> u64 {
    let mut h = 0xcbf29ce484222325u64;
    for _ in 0..30_000_000 {
        for b in input.bytes() {
            h ^= b as u64;
            h = h.wrapping_mul(0x100000001b3);
        }
    }
    h
}

async fn heartbeat() -> Duration {
    let mut worst = Duration::ZERO;
    let mut last = Instant::now();
    for _ in 0..40 {
        sleep(Duration::from_millis(10)).await;
        let now = Instant::now();
        worst = worst.max(now - last - Duration::from_millis(10));
        last = now;
    }
    worst
}

#[tokio::main(flavor = "multi_thread", worker_threads = 2)]
async fn main() {
    let probe = tokio::spawn(heartbeat());

    let hash = tokio::task::spawn_blocking(|| expensive_hash("hunter2")).await.unwrap();
    let worst = probe.await.unwrap();

    println!("hash computed: {}", hash != 0);
    println!("heartbeat stayed under 20 ms: {}", worst < Duration::from_millis(20));
}

hash computed: true
heartbeat stayed under 20 ms: true

The hash took about 1.1 seconds in a debug build (the whole program ran in 1.4 seconds), all of it on a blocking-pool thread, while the heartbeat kept its schedule and its worst delay stayed under 20 ms. The closure must be Send + 'static, so inputs are moved in, and the output comes back through the handle. If the work can panic, the handle returns a JoinError, as with any task. The pool has a size limit (512 by default), so heavy CPU work in bulk should go through a semaphore or rayon; the trade is in spawn_blocking versus rayon.

Fix 2: block_in_place

On a multi-thread runtime, tokio::task::block_in_place(|| ...) runs the closure on the current worker, after handing that worker's queue to another thread. It avoids the closure's 'static requirement, so it can borrow local data, and costs one thread handoff. It panics on a current-thread runtime. Use it for blocking work that needs to borrow, and reach for spawn_blocking otherwise, since the latter works on both flavours.

Fix 3: a dedicated thread or the async version

For long-lived blocking work, such as reading from a serial port or polling a legacy SDK, a plain std::thread with a channel to the async side is clearer than occupying a pool slot forever. And when an async version of the library exists (tokio-postgres for a blocking Postgres client, reqwest without the blocking feature), switch to it: the best fix is for the call not to block.

Finding the culprit in a running service

You rarely know which call is blocking. Three instruments find it:

1. The heartbeat above, as a metric, to know that it is happening and when.
2. Runtime metrics. RuntimeMetrics::worker_total_busy_duration and the poll-time histogram (behind the tokio_unstable flag) show workers busy for long stretches per poll. A task whose single poll takes 50 ms is a blocker.
3. tokio-console. It lists tasks with their poll times and flags those that rarely yield; the setup is in finding stuck tasks with tokio-console. A task showing "busy" with a huge poll time is the one.

For a quick local test, run the suspect code on a current_thread runtime with a heartbeat. A blocker shows up immediately because there is no second worker to hide it.

What to do about CPU work that is genuinely large

If the work is not just a hash but the core of your service (image transcoding, a big parse), the architecture question is whether Tokio should be involved at all. A pipeline where async tasks do the I/O and hand batches to a thread pool sized to the cores, receiving results through a channel, keeps both sides at full utilisation without either starving the other. Sizing that pool is a matter of cores and queue depth, covered in tuning Tokio worker threads. The error you get from reaching for the wrong remedy, a block_on inside a task, is catalogued in cannot start a runtime from within a runtime.

A rule of thumb for what counts as blocking

Tokio's guidance is that a task should not run for more than roughly 10 to 100 microseconds between awaits. That is a budget for a poll, not a deadline for a request. A handler that does two milliseconds of JSON parsing is over it a hundred times, which is fine at a hundred requests a second and a problem at ten thousand. Treat the rule as a prompt to measure rather than a prohibition: time a representative poll with Instant, and move the work only if the numbers say it matters.