Async Rust and Tokio From Futures to a Production Service
A guide to async Rust and Tokio: futures, the runtime, spawning, blocking, channels and the mistakes that make a service stop answering.
Async Rust works differently from async in JavaScript or C#, and the difference explains nearly every bug in a first Tokio service. A future does not run until something polls it. There is no scheduler in the language, so you choose a runtime. And a task that does not hand control back stops every other task scheduled on its thread. This guide builds the model from the ground up, then walks through the five problems that decide whether a service stays up: borrowing across spawned tasks, blocking calls, unbounded queues, missing deadlines and cancellation. Each one has a program, its real output, and the fix.
Examples ran on Rust 1.99, edition 2024, with tokio 1.54. Add it with tokio = { version = "1", features = ["full"] }. Timings are from an Apple M1 Max and vary by a few milliseconds from run to run.
A future is a value that does nothing yet
Calling an async fn does not run its body. It returns a value of an anonymous type that implements Future, and the body runs only when something polls that value to completion. The first program shows three consequences of that sentence in one run:
use std::time::{Duration, Instant};
use tokio::time::sleep;
async fn fetch(id: u32) -> u32 {
sleep(Duration::from_millis(100)).await;
id * 2
}
#[tokio::main]
async fn main() {
let start = Instant::now();
let (a, b) = tokio::join!(fetch(1), fetch(2));
println!("joined: {a} {b} in {} ms", start.elapsed().as_millis());
let start = Instant::now();
let a = fetch(1).await;
let b = fetch(2).await;
println!("sequential: {a} {b} in {} ms", start.elapsed().as_millis());
let start = Instant::now();
let _forgotten = fetch(3);
println!("created but never awaited: {} ms", start.elapsed().as_millis());
}It prints:
joined: 2 4 in 102 ms
sequential: 2 4 in 203 ms
created but never awaited: 0 msStart at fetch. It sleeps for 100 ms using Tokio's timer, not std::thread::sleep, then returns id * 2. The difference between the two sleeps is the whole reason async exists: Tokio's sleep returns a future that registers a timer and gives the thread back, while the standard library's blocks it. #[tokio::main] is a macro that builds a multi-threaded runtime and runs the async fn main on it.
The first measurement uses tokio::join!, which polls both futures from the same task, one after the other, as each becomes ready. Both sleeps run concurrently, so the total is about 100 ms (102 here). The second measurement awaits the futures one after another, and the total is the sum, 203 ms. Concurrency in async Rust is something you ask for with join!, select! or spawn. Writing two .awaits in a row means sequential execution, exactly as it looks.
The last measurement is the one that surprises programmers coming from other languages. let _forgotten = fetch(3); took 0 ms because no sleep started. The future was created and dropped without a single poll. The compiler knows this is usually a mistake, and warns about the more common form of it:
use std::time::Duration;
use tokio::time::sleep;
async fn save(id: u32) {
sleep(Duration::from_millis(10)).await;
println!("saved {id}");
}
#[tokio::main]
async fn main() {
save(1);
save(2).await;
}warning: unused implementer of `Future` that must be used
--> src/main.rs:11:5
|
11 | save(1);
| ^^^^^^^
|
= note: futures do nothing unless you `.await` or poll them
= note: `#[warn(unused_must_use)]` (part of `#[warn(unused)]`) on by default
saved 2Only saved 2 prints. The call save(1) produced a future that was thrown away, so id 1 was never saved. This is a warning and not an error, so a build with noisy output will hide it. Treat unused_must_use as an error in CI. The post unused future must be used goes into the fixes.
The model in four sentences
An async fn compiles into a state machine that implements the Future trait, which has one method, poll. A runtime calls poll. If the future cannot make progress, it returns Poll::Pending after storing the Waker from its context in whatever it is waiting for. When that thing becomes ready (the timer fires, the socket has bytes) it calls the waker, and the runtime polls the future again. Between awaits the body runs as ordinary code, and nothing else can run on that worker thread until the task returns Pending or finishes.
The last sentence is the one that matters for the rest of this guide. Scheduling in Tokio is cooperative. The runtime cannot interrupt a task. It can only wait for the task to yield at an .await. Every problem below comes from a task that does something other than yield on its way through. If you want to see the machinery without a runtime hiding it, writing a future by hand builds the pieces, and futures as state machines shows what the compiler generates. For the runtime side, the Tokio runtime explained covers worker threads and the task queue.
Spawning, and the mistake of borrowing
tokio::spawn hands a future to the runtime and returns a JoinHandle immediately. The spawned task runs independently, perhaps on another thread, and may outlive the function that created it. The signature says as much: the future must be Send + 'static. The 'static part is what trips people up, because it forbids borrowing from the caller's stack. Here is the natural first attempt at greeting each name concurrently:
use tokio::task::JoinSet;
#[tokio::main]
async fn main() {
let names = vec![String::from("ada"), String::from("grace")];
let mut set = JoinSet::new();
for name in &names {
set.spawn(async move {
println!("hello {name}");
});
}
while set.join_next().await.is_some() {}
}It does not compile:
error[E0597]: `names` does not live long enough
--> src/main.rs:7:17
|
5 | let names = vec![String::from("ada"), String::from("grace")];
| ----- binding `names` declared here
6 | let mut set = JoinSet::new();
7 | for name in &names {
| ^^^^^^ borrowed value does not live long enough
8 | / set.spawn(async move {
9 | | println!("hello {name}");
10 | | });
| |__________- argument requires that `names` is borrowed for `'static`
...
13 | }
| - `names` dropped here while still borrowed
|
note: requirement that the value outlives `'static` introduced here
--> ~/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.54.1/src/task/join_set.rs:145:19
|
145 | F: Send + 'static,
| ^^^^^^^
For more information about this error, try `rustc --explain E0597`.The loop borrows each element of names, and async move moves that borrow into the task. The compiler cannot prove the task finishes before names is dropped at the end of main, since a spawned task may run for as long as the runtime lives. The last note points at the bound in Tokio's source, F: Send + 'static, which is the actual requirement. (Here the program waits for every task on the last line, but the type system has no way to know that.) E0597 has a general treatment; the async-specific version of this error is covered in why spawn needs a static future.
The solutions all give the task ownership of what it uses. For read-only data shared among many tasks, wrap it in an Arc and clone the pointer into each task. For data each task needs alone, move or clone the value in. This is the corrected program:
use std::sync::Arc;
use tokio::task::JoinSet;
#[tokio::main]
async fn main() {
let names = Arc::new(vec![String::from("ada"), String::from("grace")]);
let mut set = JoinSet::new();
for i in 0..names.len() {
let names = Arc::clone(&names);
set.spawn(async move {
format!("hello {}", names[i])
});
}
let mut out = Vec::new();
while let Some(res) = set.join_next().await {
out.push(res.expect("task did not panic"));
}
out.sort();
println!("{out:?}");
}["hello ada", "hello grace"]Each iteration clones the Arc (a pointer copy and an atomic increment, not a copy of the vector) and moves the clone into its task. The task indexes into the shared vector, so no borrow escapes. The tasks are collected into a JoinSet, which owns the JoinHandles and gives back results as each finishes, in completion order, so the output is sorted before printing to keep it deterministic. A JoinSet also aborts every remaining task when it is dropped, which makes it a safer default than a loose pile of handles. See JoinSet and spawn and join.
The Send half of the bound is the other source of spawn errors. A future is Send only if everything it holds across an .await is. Using std::sync::Mutex is the usual cause:
use std::sync::{Arc, Mutex};
use std::time::Duration;
use tokio::time::sleep;
async fn update(state: Arc<Mutex<u32>>) {
let mut guard = state.lock().unwrap();
*guard += 1;
sleep(Duration::from_millis(1)).await;
}
#[tokio::main]
async fn main() {
let state = Arc::new(Mutex::new(0));
tokio::spawn(update(state.clone())).await.unwrap();
}error: future cannot be sent between threads safely
--> src/main.rs:14:18
|
14 | tokio::spawn(update(state.clone())).await.unwrap();
| ^^^^^^^^^^^^^^^^^^^^^ future returned by `update` is not `Send`
|
= help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `std::sync::MutexGuard<'_, u32>`
note: future is not `Send` as this value is used across an await
--> src/main.rs:8:37
|
6 | let mut guard = state.lock().unwrap();
| --------- has type `std::sync::MutexGuard<'_, u32>` which is not `Send`
7 | *guard += 1;
8 | sleep(Duration::from_millis(1)).await;
| ^^^^^ await occurs here, with `mut guard` maybe used later
note: required by a bound in `tokio::spawn`
--> ~/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.54.1/src/task/spawn.rs:176:21
|
174 | pub fn spawn<F>(future: F) -> JoinHandle<F::Output>
| ----- required by a bound in this function
175 | where
176 | F: Future + Send + 'static,
| ^^^^ required by this bound in `spawn`
error: could not compile `lab` (bin "lab") due to 1 previous errorThis is the better of the two Send errors, because the compiler tells you which value (guard, line 6) and which .await (line 8) are the problem. A standard-library MutexGuard must be released on the thread that took it, so holding one over a suspension point would break if the task resumed elsewhere. Two fixes work. Drop the guard before the await by scoping it in a block: { let mut g = state.lock().unwrap(); *g += 1; }. That is the right fix when the lock protects a quick update and no await happens inside. Or use tokio::sync::Mutex, whose guard can be held across an await, at the price of a slower lock and the risk of deadlocks if you hold it while waiting on something that needs the same lock. Tokio Mutex versus std Mutex works through the decision, and holding a mutex guard across an await shows the deadlock. The related error, a non-Send type such as Rc captured by a spawned task, reports the trait Send is not implemented for Rc<i32> under the same headline, and the future cannot be sent between threads collects the variants. Send and Sync explained covers the traits themselves.
Blocking is the failure that does not look like one
A synchronous sleep, a file read with std::fs, a hash computation that takes 300 ms, a lock held by a thread that is waiting on a database: all of these stop the worker thread running them, and the runtime cannot take it back. The first symptom is not an error. It is a service where every request becomes slow at once and CPU is low. The experiment below makes that visible by running the runtime on one thread, so the damage is total instead of partial:
use std::time::{Duration, Instant};
use tokio::time::sleep;
// one worker thread makes the effect obvious: any blocking call stalls every task
#[tokio::main(flavor = "current_thread")]
async fn main() {
let start = Instant::now();
let ticker = tokio::spawn(async move {
let mut last = Instant::now();
let mut worst = Duration::ZERO;
for _ in 0..10 {
sleep(Duration::from_millis(10)).await;
worst = worst.max(last.elapsed());
last = Instant::now();
}
println!("ticker finished at {} ms, worst gap {} ms", start.elapsed().as_millis(), worst.as_millis());
});
sleep(Duration::from_millis(5)).await;
let mode = std::env::args().nth(1).unwrap_or_default();
match mode.as_str() {
"block" => std::thread::sleep(Duration::from_millis(300)),
"spawn_blocking" => {
tokio::task::spawn_blocking(|| std::thread::sleep(Duration::from_millis(300)))
.await
.expect("blocking task did not panic");
}
_ => sleep(Duration::from_millis(300)).await,
}
ticker.await.expect("ticker did not panic");
}The program starts a ticker task that sleeps for 10 ms ten times and records the longest gap between ticks. Meanwhile the main task does 300 ms of "work", in one of three ways chosen on the command line. Running it three times gives:
cargo run --release
cargo run --release -- block
cargo run --release -- spawn_blockingticker finished at 117 ms, worst gap 12 ms
ticker finished at 429 ms, worst gap 311 ms
ticker finished at 120 ms, worst gap 14 msWith the default, the main task awaits a Tokio sleep(300ms). It yields, so the ticker runs freely: ten ticks finish at 117 ms with a worst gap of 12 ms (a 10 ms sleep plus scheduling jitter). With block, the main task calls std::thread::sleep(300ms) and holds the only worker. The ticker is ready to run and cannot. It finishes at 429 ms, and the worst gap is 311 ms, which is the blocking call plus the normal tick. With spawn_blocking, the same sleep moves to a separate pool of threads that Tokio keeps for exactly this purpose, and the ticker is back to 120 ms.
The multi-thread runtime hides this until it does not. With #[tokio::main] you have one worker per CPU core, so a single blocking call delays only the tasks queued behind it on that worker, and the others may steal them. Eight concurrent blocking calls on an eight-core machine are the same as the experiment above. Tokio's guidance is that a task should not run for more than roughly 10 to 100 microseconds between awaits, and anything that cannot meet that belongs on the blocking pool. Use tokio::task::spawn_blocking for blocking I/O and sync-only libraries. For CPU-heavy work that keeps all cores busy, a dedicated pool is better so as not to starve the blocking pool; the choice is in spawn_blocking versus Rayon. The full diagnosis, including how to spot it in a running service, is in blocking the Tokio runtime, and tokio-console shows which task holds a worker.
The related trap is a loop with no await inside it. A task that does loop { compute() } over in-memory data never yields, even though nothing in it blocks. If you cannot move the work off the runtime, call tokio::task::yield_now().await every so often.
Queues need a limit
Tasks talk through channels, and a channel with no bound is a memory leak with a delay. If a producer is faster than its consumer, an unbounded queue grows until the process is killed. A bounded mpsc::channel turns the same situation into backpressure: send is an async call that does not complete while the buffer is full, so the producer slows down to the consumer's pace. The program below uses a buffer of two and a consumer that takes 100 ms per message:
use std::time::Duration;
use tokio::sync::mpsc;
use tokio::time::{Instant, sleep};
#[tokio::main]
async fn main() {
let (tx, mut rx) = mpsc::channel::<u32>(2);
let start = Instant::now();
let producer = tokio::spawn(async move {
for i in 1..=6 {
tx.send(i).await.expect("receiver is alive");
println!("{:>4} ms sent {i}", start.elapsed().as_millis());
}
});
sleep(Duration::from_millis(100)).await;
while let Some(i) = rx.recv().await {
println!("{:>4} ms received {i}", start.elapsed().as_millis());
sleep(Duration::from_millis(100)).await;
}
producer.await.expect("producer did not panic");
} 0 ms sent 1
0 ms sent 2
101 ms received 1
101 ms sent 3
204 ms received 2
204 ms sent 4
305 ms received 3
306 ms sent 5
408 ms received 4
408 ms sent 6
510 ms received 5
612 ms received 6Read the timestamps. The producer fills the two slots of the buffer instantly, then send(3) waits. The consumer starts at 100 ms, takes message 1, and the moment a slot frees up the producer's send of 3 completes (101 ms). From then on, each send happens within a millisecond of a receive. The producer ran six sends over 408 ms instead of finishing in 0 ms, and the queue never held more than two items. With an unbounded channel, all six sends would have printed at 0 ms, and the queue would have held the entire backlog. For six messages that does not matter. For a million per second from a socket, it is the difference between a stable service and an out-of-memory kill. The tradeoff is that a bounded channel can deadlock when two tasks send to each other with full buffers, and that choosing the number takes thought: it should cover short bursts, not the entire expected backlog. Backpressure with bounded channels covers sizing, and channels compared compares mpsc, oneshot, broadcast and watch. The Semaphore in limiting concurrency is the tool when the thing to bound is the number of tasks in flight, not messages in a queue.
Deadlines, and what a timeout actually does
Every call that leaves the process needs a deadline, because networks fail by going quiet. tokio::time::timeout wraps a future and gives up after a duration. What it does on timeout is the subtle part:
use std::time::Duration;
use tokio::time::{sleep, timeout};
struct Guard(&'static str);
impl Drop for Guard {
fn drop(&mut self) {
println!("dropped {}", self.0);
}
}
async fn slow_job() -> &'static str {
let _g = Guard("slow_job state");
println!("job started");
sleep(Duration::from_secs(5)).await;
println!("job finished");
"done"
}
#[tokio::main]
async fn main() {
match timeout(Duration::from_millis(50), slow_job()).await {
Ok(v) => println!("ok {v}"),
Err(e) => println!("timed out: {e}"),
}
}job started
dropped slow_job state
timed out: deadline has elapsedThe job started, then 50 ms later the timeout fired and returned Err(Elapsed). The important line is the second one. When timeout gives up, it drops the future, and dropping a future runs the destructors of everything that future held at its current .await. That is why Guard printed "dropped" and "job finished" never appeared. The abandoned work is cancelled, with nothing running it any further. This is how cancellation works in all of async Rust: a future is cancelled by dropping it, and it stops at the next await point it would have reached, which is the one it is parked on.
That has a consequence worth designing for. Anything the job did before the await is done, and anything after it is not, so a future that writes to a database and then updates a counter can be cancelled between the two. Code that must be all-or-nothing needs either a transaction or a design where the first half is safe to leave. The same applies to select!, which drops the branches that lose; cancellation safety in select lists which Tokio operations are safe to drop mid-flight. Note also the converse: a timeout around a spawn_blocking call does not stop the blocking thread, because there is no await inside it to cancel at. The thread runs to the end and its result is discarded. Timeouts in Tokio and cancelling tasks take both further, and graceful shutdown is the same idea applied to the whole process.
A checklist for a service that stays up
The five problems above add up to a short list that I would apply to any Tokio service before it takes traffic.
- Every external call has a
timeout, and a decision about what the caller sees when it fires. - Every channel is bounded, with a size that was chosen and not copied.
- Nothing synchronous and slow runs on the async workers: file system calls, password hashing, large JSON parses, compression.
- Shared state is behind
Arc, with a std mutex for brief updates and a Tokio mutex only when a lock must cross an await. - The process has a shutdown path that stops accepting work, waits for tasks and then exits.
- Worker thread count is measured and not assumed; see tuning Tokio worker threads, and runtime metrics for the numbers to read.
A task that appears to hang has its own diagnosis path in debugging hung async Rust and a task that never finishes, and a full worked server is built in a chat server on Tokio.
When you do not need any of this
Async solves one problem: many tasks waiting on I/O at the same time without one thread per task. A program that makes a handful of blocking calls, a command-line tool, a batch job that processes files, and a CPU-bound pipeline are all simpler and often faster with plain threads or no concurrency. Threads cost about one stack each (megabytes of reserved address space, kilobytes actually touched), and a typical operating system will run thousands of them without trouble. At tens of thousands of mostly idle connections the per-task cost of a future, a few hundred bytes to a few kilobytes, makes async the right tool. Below that it is a choice about code style. Async also splits the ecosystem in two: every library you call must be async or be run on the blocking pool, and every function that awaits must itself be async, which spreads through a codebase. For plain threads, see spawning threads and Rayon.
Some topics are deliberately left out: writing your own executor (covered step by step in a tiny executor), no_std async with Embassy, and completion-based runtimes such as io_uring (tokio-uring). The best primary sources are the Tokio tutorial and the async book, which covers the language side.
