Tokio Tasks Never Finish and Five Reasons It Happens

A task that never finishes produces the most frustrating kind of Rust bug: no panic, no error, no log line, and a request that hangs until a client gives up. Every case reduces to a short list of causes. The task is waiting for something that cannot happen (a channel whose sender is still alive but silent, a lock held by a task that is waiting for this one, a waker nobody will call), or it is not waiting at all but monopolising its thread (a loop with no await), or something else is starving it. This post reproduces the five causes with a timeout so each hang is detected and printed instead of freezing the terminal, shows the tell for each, and ends with the tools for finding the stuck task in a running service. Everything compiled on Rust 1.99, edition 2024, with tokio 1.

First, confirm that it is stuck rather than slow

Wrap the suspect in tokio::time::timeout and log what you were waiting for when it expires. This turns "hangs" into a specific line, and it is the cheapest instrument there is. All the examples below use the paused clock, so a 10-second timeout costs no real time:

use std::future::Future;
use tokio::time::{timeout, Duration};

async fn report<F: Future>(label: &str, fut: F) {
    match timeout(Duration::from_secs(10), fut).await {
        Ok(_) => println!("{label}: finished"),
        Err(_) => println!("{label}: STUCK after 10 s"),
    }
}

#[tokio::main(flavor = "current_thread", start_paused = true)]
async fn main() {
    report("quick job", tokio::time::sleep(Duration::from_millis(50))).await;
    report("never-ready", std::future::pending::<()>()).await;
}

quick job: finished
never-ready: STUCK after 10 s

That helper is the skeleton for the rest of the post. std::future::pending() is a future that is never ready; every cause below is a more subtle way to build one.

Cause 1: a channel that never closes

The loop while let Some(job) = rx.recv().await ends only when every sender is dropped. One forgotten clone keeps it waiting forever, and the usual forgotten clone is the one still held by the code that created the channel:

use tokio::sync::mpsc;
use tokio::time::{timeout, Duration};

async fn worker(mut rx: mpsc::Receiver<u32>) -> u32 {
    let mut total = 0;
    while let Some(n) = rx.recv().await {
        total += n;
    }
    total
}

#[tokio::main(flavor = "current_thread", start_paused = true)]
async fn main() {
    let (tx, rx) = mpsc::channel(8);
    let task = tokio::spawn(worker(rx));

    for n in 1..=3 {
        tx.send(n).await.unwrap();
    }

    let mut handle = task;
    match timeout(Duration::from_secs(5), &mut handle).await {
        Ok(total) => println!("worker finished with {}", total.unwrap()),
        Err(_) => println!("worker still running: the main task still holds a Sender"),
    }

    drop(tx);
    println!("after drop(tx): total = {}", handle.await.unwrap());
}

worker still running: the main task still holds a Sender
after drop(tx): total = 6

The worker received all three values and then waited for a fourth, because tx was alive in main. Dropping the sender ended the loop and returned the total. The fix is structural: move the sender into the producer so it is dropped when the producer finishes, and drop any spare clones explicitly before waiting. A task that is the only holder of a sender and waits on the receiver of the same channel will deadlock itself.

Cause 2: a lock cycle

Two tasks that take two async mutexes in opposite orders can each hold one and wait for the other, forever. A timeout reveals it; nothing else will:

use std::sync::Arc;
use tokio::sync::Mutex;
use tokio::time::{sleep, timeout, Duration};

#[tokio::main(flavor = "current_thread", start_paused = true)]
async fn main() {
    let a = Arc::new(Mutex::new(0u32));
    let b = Arc::new(Mutex::new(0u32));

    let (a1, b1) = (a.clone(), b.clone());
    let t1 = tokio::spawn(async move {
        let _ga = a1.lock().await;
        sleep(Duration::from_millis(10)).await;
        let _gb = b1.lock().await;
    });

    let (a2, b2) = (a.clone(), b.clone());
    let t2 = tokio::spawn(async move {
        let _gb = b2.lock().await;
        sleep(Duration::from_millis(10)).await;
        let _ga = a2.lock().await;
    });

    let both = timeout(Duration::from_secs(5), async { let _ = tokio::join!(t1, t2); }).await;
    println!("both tasks finished: {}", both.is_ok());
}

both tasks finished: false

Task 1 holds A and waits for B; task 2 holds B and waits for A. The sleep makes the interleaving certain here; in production it depends on timing, which is why it appears once a week. The fix is a global lock order (always A before B), a single lock around both values, or an actor that owns both, as in sharing state between tasks. The std-mutex version of this bug freezes a worker thread instead of a task and is worse, as in mutex guards across await.

Cause 3: a waker nobody calls

A hand-written future that returns Pending without arranging for a wakeup is polled once and then forgotten. This is the bug of every first-time Future implementation:

use std::future::Future;
use std::pin::Pin;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::task::{Context, Poll};
use tokio::time::{timeout, Duration};

struct Flag(Arc<AtomicBool>);

impl Future for Flag {
    type Output = &'static str;

    fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<Self::Output> {
        if self.0.load(Ordering::SeqCst) {
            Poll::Ready("flag was set")
        } else {
            Poll::Pending
        }
    }
}

#[tokio::main(flavor = "current_thread", start_paused = true)]
async fn main() {
    let flag = Arc::new(AtomicBool::new(false));
    let waiter = tokio::spawn(Flag(flag.clone()));

    tokio::time::sleep(Duration::from_millis(10)).await;
    flag.store(true, Ordering::SeqCst);

    match timeout(Duration::from_secs(5), waiter).await {
        Ok(v) => println!("{}", v.unwrap()),
        Err(_) => println!("waiter never woke up even though the flag is set"),
    }
}

waiter never woke up even though the flag is set

The flag was set, the condition was true, and the task did not notice because nothing told the runtime to poll it again. The future must store cx.waker().clone() and call wake() when the flag changes, or use a primitive that does this for you (a Notify, a watch channel, a oneshot). The machinery is in futures as state machines; the console flags this exact situation as a "lost waker" (tokio-console).

Cause 4: an await-free loop

A task that never awaits never yields. On a single-thread runtime it prevents everything else from running, including the timeout that would detect it, so this cause cannot be demonstrated with a timeout on the same runtime and has to be recognised by its symptom: the whole process is unresponsive and one core is pinned at 100 percent. The fix is an .await in the loop (tokio::task::yield_now().await at minimum) or moving the work to a blocking thread, as in blocking the Tokio runtime. A related trap: a select! loop whose branches are all always ready can starve the others; Tokio's cooperative budget limits it, and biased; removes the fairness that would otherwise save you.

Cause 5: waiting on a task that waits on you

Awaiting a JoinHandle inside the task it belongs to, a request-response over a channel where the responder is the same loop that is awaiting the response, or a block_on inside an async context, all create a wait cycle. The cycle may be between two tasks and a channel, so it rarely has a lock in it to point at. Draw the "who is waiting for whom" graph on paper; the cycle is usually visible in a dozen nodes.

A note on what a timeout does not fix

Adding a timeout around a stuck await turns the hang into an error, and that is a diagnosis, not a repair. The cause remains, and the timeout hides its cost: the stuck task is dropped when the timeout fires, which cancels it at an arbitrary await with all the consequences described in cancellation safety, and a lock cycle still leaves one task holding a mutex until the drop. Use the timeout to find the problem and to keep a service from piling up requests, then remove the cause. A timeout that fires regularly in production is a bug report written by the program itself.

Finding the stuck task in a live service

1. Add a timeout and a label at the boundaries (request handlers, job runners), so a hang becomes a counter and a log line with the operation name.
2. Attach tokio-console in a staging environment: sort by total time, read the warnings, and the stuck task shows as idle with a lost-waker or zero wakes.
3. Take a task dump (task dumps and async backtraces) on a production process: each task's await stack shows what it is parked on.
4. Check the shape of the stack of OS threads with a debugger: if all workers are in a lock wait or a syscall, it is the blocking case; if they are all idle, the tasks are parked and the cause is one of 1 to 3.

The practical rule behind all five: every await should have an answer to "what wakes me up, and is it certain to happen?" The ones that do not are timeouts waiting to be added. For hangs that start at shutdown specifically, see graceful shutdown.