Tokio Runtime Explained Workers Tasks and the Scheduler

A Tokio runtime is three machines in one box: a pool of worker threads that run async tasks, a driver that turns operating system events into wakeups, and a second pool of threads for blocking calls. Understanding which of those your code is running on explains almost every performance and hang question in async Rust, from "why did my whole server stall" to "why do I have 512 threads". This post inspects each piece with running code, explains the work-stealing scheduler in plain terms, and shows the knobs that matter. Everything compiled on Rust 1.99, edition 2024, with tokio 1.

The three parts

The defaults come from the macro: #[tokio::main] builds a multi-thread runtime with all drivers enabled and as many workers as std::thread::available_parallelism() reports. You can see all three parts at work in one small program:

use std::thread;
use tokio::runtime::Builder;

fn name() -> String {
    thread::current().name().unwrap_or("?").to_string()
}

fn main() {
    let rt = Builder::new_multi_thread()
        .worker_threads(2)
        .thread_name("rt-worker")
        .enable_all()
        .build()
        .unwrap();

    println!("workers: {}", rt.metrics().num_workers());

    rt.block_on(async {
        println!("block_on body runs on: {}", name());

        let on_worker = tokio::spawn(async { name() }).await.unwrap();
        println!("spawned task runs on: {on_worker}");

        let on_blocking = tokio::task::spawn_blocking(name).await.unwrap();
        println!("spawn_blocking runs on: {on_blocking}");
    });
}

workers: 2
block_on body runs on: main
spawned task runs on: rt-worker
spawn_blocking runs on: rt-worker

Two kinds of thread appear, plus the caller. The future passed to block_on runs on the calling thread (here main), not on a worker, which is a detail that matters when you compare behaviour with spawned tasks. The spawned task runs on a worker named by thread_name. The blocking closure runs on a thread from the blocking pool, and the output shows the same name, because thread_name applies to both pools. The name does not tell you which pool a thread belongs to, so give the builder a name that makes logs readable and use the task context, not the thread name, to tell the two apart.

How the multi-thread scheduler works

Each worker owns a local run queue of tasks. A new task spawned from inside a worker goes onto that worker's queue, which is fast and keeps related work on one core. A task spawned from outside the runtime goes onto a shared injection queue. A worker with an empty queue does not sleep immediately: it first checks the injection queue, then steals half the tasks from another worker's queue. The result is that a burst of work on one worker spreads across all of them without a central lock that every task contends on.

Two refinements matter in practice. A worker runs one task at a time until that task awaits something that is not ready, and Tokio gives each task a budget of 128 operations before forcing it to yield, so one busy resource cannot starve its neighbours. And one task per worker, the most recently woken, is held in a "LIFO slot" and run next, which improves cache locality for request/response ping-pong, at the price that tasks in the slot can run slightly out of order. None of this is visible to your code, but it explains why task start order is not spawn order and why you must never depend on it.

Seeing the scheduler share work

Metrics let you watch it. This program spawns 64 short tasks from outside the runtime and then asks how many workers ended up doing something. The assertion is deliberately loose, because the exact split depends on timing:

use std::collections::HashSet;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use tokio::runtime::Builder;

fn main() {
    let rt = Builder::new_multi_thread().worker_threads(4).thread_name("w").enable_all().build().unwrap();
    let seen = Arc::new(Mutex::new(HashSet::new()));

    rt.block_on(async {
        let mut handles = Vec::new();
        for _ in 0..64 {
            let seen = seen.clone();
            handles.push(tokio::spawn(async move {
                std::thread::sleep(Duration::from_millis(5));
                seen.lock().unwrap().insert(std::thread::current().id());
            }));
        }
        for h in handles {
            h.await.unwrap();
        }
    });

    let n = seen.lock().unwrap().len();
    println!("tasks ran on more than one worker: {}", n > 1);
    println!("worker count: {}", rt.metrics().num_workers());
}

tasks ran on more than one worker: true
worker count: 4

The tasks use std::thread::sleep on purpose, to hold a worker for 5 ms each. That is exactly the mistake the next post warns about, used here to make the stealing visible, because blocked workers leave their queues full for others to take. With 64 tasks and 4 workers the work spread. Do not do this in a real service.

The current-thread flavor

A current-thread runtime has no worker pool: everything runs on the thread that calls block_on. It is the right choice for tests (deterministic ordering, works with start_paused), for small tools, and for code that holds non-Send data, since tasks never move between threads. The cost is that it can use only one core and that any blocking call stalls everything.

use tokio::runtime::Builder;

fn main() {
    let rt = Builder::new_current_thread().enable_all().build().unwrap();
    println!("workers: {}", rt.metrics().num_workers());

    let order = rt.block_on(async {
        let mut log = Vec::new();
        let a = tokio::spawn(async { "a" });
        let b = tokio::spawn(async { "b" });
        log.push(b.await.unwrap());
        log.push(a.await.unwrap());
        log
    });
    println!("{order:?}");
}

workers: 1
["b", "a"]

One "worker" reports the calling thread. Because there is only one thread, you can reason about interleavings, and a test that passes here but fails on the multi-thread flavor has found a real race. The manual-versus-macro choice of runtime is covered in tokio::main versus building a runtime, and a single-thread set with non-Send tasks in LocalSet.

The blocking pool, and its limits

Work that cannot be made async (a CPU-heavy parse, a synchronous library, most file system calls) belongs on the blocking pool, via spawn_blocking. The pool starts empty, adds a thread when all are busy, and retires threads idle for ten seconds, up to max_blocking_threads (512 by default). The ceiling is generous because blocking calls are expected to sit on threads for a long time. It is also a trap: a service that sends 10,000 concurrent spawn_blocking calls will create 512 threads and queue the rest, and each thread holds a stack, so memory grows. A semaphore in front of it, covered in limiting concurrency with a semaphore, keeps that bounded. For CPU-bound work, rayon with a channel back is the better fit, a trade-off examined in spawn_blocking versus rayon.

Settings worth knowing

Sizing the worker count properly has its own post, tuning Tokio worker threads. The short version: the default is right for mostly-async services, and it is the amount of blocking in your code, not the worker count, that usually decides throughput.

What the runtime does not do

It does not preempt. A task that never awaits holds its worker, which is why a CPU loop in a handler stalls requests that share the thread. It does not make a task run in parallel with itself; a single task is sequential, and parallelism comes from having several tasks. It does not give you ordering between tasks. And it does not protect you from a panic: a panic inside a spawned task is caught at the task boundary and delivered as a JoinError when you await the handle, but a panic in the future given to block_on propagates to the caller. If you need to see what the scheduler is doing on a live system, the metrics API used above and the tokio-console tool in finding stuck tasks with tokio-console are the two instruments.