Fan Out and Fan In Work Pipelines in Rust Services Design
Split work across a bounded pool of workers and merge the results: a producer, bounded channels for backpressure, N workers, a collector and clean shutdown. A Tokio pipeline with ordering, errors and cancellation handled.

Many jobs have the same shape: read a stream of items, do slow work on each, collect what comes out. Download and resize images, call an API for each record, enrich rows from a database. Doing it serially is slow; doing it with a task per item is unbounded. The fan-out fan-in pipeline sits between: a fixed number of workers share the input and feed one collector. This post builds it with Tokio channels and covers the design questions. Versions: Rust 1.90 with edition 2024, Tokio 1.47, futures 0.3.
The pipeline
producer ──► [bounded queue] ──► worker 1 ─┐
├─► worker 2 ─┼─► [bounded queue] ──► collector
└─► worker N ─┘Implementation
use std::sync::Arc;
use tokio::sync::{mpsc, Mutex};
#[derive(Debug)]
pub struct Job {
pub index: usize, // position in the input, to restore order if needed
pub url: String,
}
#[derive(Debug)]
pub struct Outcome {
pub index: usize,
pub result: Result<usize, String>, // bytes fetched, or an error message
}
async fn process(job: &Job) -> Result<usize, String> {
// the slow work: a network call, a decode, a query
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
Ok(job.url.len())
}
pub async fn run_pipeline(urls: Vec<String>, workers: usize) -> Vec<Outcome> {
let (job_tx, job_rx) = mpsc::channel::<Job>(workers * 2); // small buffer: backpressure
let (out_tx, mut out_rx) = mpsc::channel::<Outcome>(workers * 2);
let job_rx = Arc::new(Mutex::new(job_rx)); // workers share one receiver
// fan out: N workers pulling from the same queue
let mut handles = Vec::new();
for _ in 0..workers {
let (rx, tx) = (job_rx.clone(), out_tx.clone());
handles.push(tokio::spawn(async move {
loop {
let job = { rx.lock().await.recv().await }; // hold the lock only while waiting for a job
let Some(job) = job else { break }; // channel closed and drained
let result = process(&job).await;
if tx.send(Outcome { index: job.index, result }).await.is_err() {
break; // collector gone: stop
}
}
}));
}
drop(out_tx); // keep only the workers' clones
// producer
let producer = tokio::spawn(async move {
for (index, url) in urls.into_iter().enumerate() {
if job_tx.send(Job { index, url }).await.is_err() {
break;
}
}
// job_tx dropped here: workers see the end of input
});
// fan in: collect until every worker has dropped its sender
let mut results = Vec::new();
while let Some(o) = out_rx.recv().await {
results.push(o);
}
let _ = producer.await;
for h in handles {
let _ = h.await;
}
results.sort_by_key(|o| o.index); // restore input order
results
}The pieces to notice: the job channel has a small capacity, so if workers are slow the producer blocks on send and memory stays bounded. Workers share the receiver through a mutex held only while waiting for the next job, a common pattern for multi-consumer use of Tokio's single-consumer channel (the async-channel crate offers a native multi-consumer channel). Termination flows from the front: when the producer drops job_tx, the queue drains and each worker's recv returns None; workers end and drop their result senders; the collector's recv returns None when the last sender is gone. The explicit drop(out_tx) matters, since a leftover sender would keep the collector waiting forever (why tasks never finish).
The shorter version with streams
use futures::{stream, StreamExt};
pub async fn run_with_streams(urls: Vec<String>, workers: usize) -> Vec<(usize, Result<usize, String>)> {
stream::iter(urls.into_iter().enumerate())
.map(|(i, url)| async move {
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
(i, Ok::<usize, String>(url.len()))
})
.buffer_unordered(workers) // at most `workers` in flight
.collect()
.await
}
buffer_unordered(n) keeps up to n futures running and yields results as they finish. For a single stage of independent work it is the least code and is usually the right first choice (batching and pipelining). Reach for explicit channels when you have several stages with different concurrency, long-lived workers with state (a connection each), or need to feed work continuously rather than from a finished list.
Design questions to answer up front
- Errors: does one failure stop everything (cancel the rest), or is each item's result recorded and the run continues? Carry
Resultin the outcome, as above, so the collector decides. - Order: results arrive in completion order. Attach the input index and sort at the end, or use
buffered(ordered, but a slow head item delays emission) instead ofbuffer_unordered. - Cancellation: accept a shutdown signal; workers should stop pulling new jobs and finish or abort the current one (cancellation safety, graceful shutdown). A
JoinSetaborts its tasks when dropped. - Concurrency level: set by the scarcest downstream resource: the API's rate limit, the database pool size, file descriptors (semaphores).
- Retries: inside the worker, with backoff, bounded attempts and idempotent work (timeouts and retries).
- Panics: a panic in a worker task ends that worker silently; check the join results and respawn or fail the run.
Fan out in a service
Inside a request handler, a bounded pipeline is a way to enrich many records without flooding a dependency: the handler awaits the collector, and the concurrency limit protects the dependency from a single large request. For work that outlives the request, hand it to a durable queue instead of in-process channels, because a restart loses whatever was in memory (background job queues, SKIP LOCKED queues).
Test it
- All inputs produce exactly one outcome (count and set equality).
- The pipeline terminates when the input is empty and when the input is large (use paused time for slow workers, testing without sleeping).
- A failing item is reported, and others still complete.
- Maximum concurrency never exceeds the worker count (track an atomic high-water mark inside
process).
Used with discipline, channels give you bounded, observable concurrency with few moving parts; the rules are bound every queue, end by dropping senders and decide error and order semantics before writing code (backpressure with bounded channels).


