Threads, channels, locks, atomics and Tokio, with a bias towards the bug you will hit at 3am.
Split work across a bounded pool of workers and merge the results: a producer, bounded channels for backpressure, N workers, a collector and clean shutdown. A Tokio pipeline with ordering, errors and cancellation handled.
Stop one failing dependency from taking the whole service down. A circuit breaker fails fast after repeated errors and probes for recovery; a bulkhead caps concurrency per dependency. Implement both with Tokio.
Use Tokio runtime and task metrics to see worker busy time, scheduling delay, queue depth and slow polls. Export them, read the numbers that matter and tell blocking code from a genuine lack of capacity.
When a hot cache entry expires, a hundred requests recompute it at once. Coalesce concurrent misses so one loads and the rest wait, with moka get_with, a hand-rolled version and a note on stale-while-revalidate.
Retry and timeout code is full of delays, and sleeping in tests makes them slow and flaky. Use Tokio paused time, a scripted flaky dependency and an injected backoff to test every path in milliseconds.
Make shutdown testable by taking a shutdown future instead of listening for signals inside the server. Assert that in-flight requests finish, new ones are refused and the process exits within a deadline.
A hung async service has no useful thread backtrace. Tokio task dumps capture the await stack of every task. Enable tokio_unstable and taskdump, trigger a dump from a signal or endpoint, and read where each task is parked.
A task that hangs forever has a short list of causes: a channel sender that never drops, an await on a lock held across an await, a blocked worker, a missing timeout, or a lost wake-up.
Futures are lazy: calling an async fn without .await does nothing. The unused_must_use warning is the compiler catching a missing await, a forgotten spawn, or a dropped JoinHandle.
A recursive async fn has an infinitely sized future, so the compiler asks for indirection. Return a boxed future with Box::pin, or use the async-recursion crate, and prefer a loop when you can.
tokio-console shows every task, how long it has been idle or busy, and which ones never wake up. Set it up with console-subscriber and read the columns that expose stuck and blocking tasks.
Pin<Box<dyn Future<Output = T>>> looks like noise until each word has a job. Learn why a future must be pinned, why it is boxed, why it is dyn, and how to write it with less ceremony.
We send one email when a new post is published. No spam.