A system is a set of stages connected by queues: accept requests, parse, process, write. If any stage is slower than the one feeding it, the queue in front of it grows. With unbounded queues it grows until the process is killed. Backpressure is the mechanism that connects the queue's fullness back to the producer. In Rust it is mostly a matter of choosing bounded types. Versions: Rust 1.90, Tokio 1.47, futures 0.3.
use tokio::sync::mpsc;\n\n#[tokio::main]\nasync fn main() {\n let (tx, mut rx) = mpsc::unbounded_channel::<Vec<u8>>();\n\n tokio::spawn(async move {\n loop {\n // producer: no pacing, produces as fast as it can\n if tx.send(vec![0u8; 1024]).is_err() {\n break;\n }\n tokio::task::yield_now().await;\n }\n });\n\n while let Some(item) = rx.recv().await {\n // consumer: slower than the producer\n tokio::time::sleep(std::time::Duration::from_millis(1)).await;\n drop(item);\n }\n}The producer runs at thousands of items per second and the consumer at about a thousand. The difference accumulates in the channel at megabytes per second, with no error until the allocator fails. The program has no way to express \"I am full\".
\nuse tokio::sync::mpsc;\n\n#[tokio::main]\nasync fn main() {\n let (tx, mut rx) = mpsc::channel::<Vec<u8>>(100);\n\n let producer = tokio::spawn(async move {\n for i in 0..10_000 {\n // waits here when 100 items are queued: this IS the backpressure\n if tx.send(vec![i as u8; 1024]).await.is_err() {\n break;\n }\n }\n });\n\n while let Some(item) = rx.recv().await {\n tokio::time::sleep(std::time::Duration::from_millis(1)).await;\n drop(item);\n }\n producer.await.unwrap();\n}When the queue holds 100 items, send().await suspends the producer until the consumer takes one. Memory is bounded by 100 items, and the producer's speed matches the consumer's. The same applies across a pipeline: each stage uses a bounded channel to the next, and pressure propagates upstream to the source (bounded channels in Tokio).
When the queue is full, something has to give:
\n| Policy | API | Use when | |---|---|---| | Wait | send().await | the producer can pause (internal pipelines, file readers) | | Reject newest | try_send returning Full | you can tell the caller to retry (HTTP: 503) | | Drop oldest | ring buffer, broadcast lag | only fresh data matters (metrics, telemetry) | | Drop newest | try_send and ignore | best-effort logging | | Error | try_send and propagate | upstream must know |
use tokio::sync::mpsc::error::TrySendError;\n\nfn enqueue(tx: &mpsc::Sender<Job>, job: Job) -> Result<(), &'static str> {\n match tx.try_send(job) {\n Ok(()) => Ok(()),\n Err(TrySendError::Full(_)) => Err(\"busy\"), // caller returns 503 with Retry-After\n Err(TrySendError::Closed(_)) => Err(\"shutting down\"),\n }\n}In a web handler, waiting on a full queue holds the request open and ties up resources, so reject quickly (try_send) and let the client back off (graceful degradation).
Backpressure also applies to spawning tasks. A loop that tokio::spawns per incoming item has no bound:
use std::sync::Arc;\nuse tokio::sync::Semaphore;\n\nasync fn process_all(items: Vec<u64>) {\n let permits = Arc::new(Semaphore::new(32));\n let mut handles = Vec::new();\n for item in items {\n // waits here when 32 tasks are running\n let permit = Arc::clone(&permits).acquire_owned().await.unwrap();\n handles.push(tokio::spawn(async move {\n do_work(item).await;\n drop(permit);\n }));\n }\n for h in handles {\n let _ = h.await;\n }\n}\n\nasync fn do_work(_n: u64) {}Acquiring the permit before spawning is the key: the loop itself is paced. See semaphores.
\nuse futures::{stream, StreamExt};\n\nasync fn fetch(url: String) -> usize {\n url.len()\n}\n\n#[tokio::main]\nasync fn main() {\n let urls: Vec<String> = (0..1000).map(|i| format!(\"https://example.test/{i}\")).collect();\n let total: usize = stream::iter(urls)\n .map(fetch)\n .buffer_unordered(16) // at most 16 futures in flight\n .fold(0, |acc, n| async move { acc + n })\n .await;\n println!(\"{total}\");\n}buffer_unordered(16) polls the source only when fewer than 16 futures are running, so a lazy source is read at the pace of completion (buffer_unordered, async streams). The stream is pull-based, which gives backpressure by construction: nothing is produced until the consumer asks.
TCP has flow control. If your handler stops reading a socket, the kernel buffer fills, the advertised window shrinks and the sender slows down. This only works if your code actually stops reading: an accept loop that reads requests into an unbounded queue defeats it. Read, process, then read again, with a bound in between. For the accept loop itself, acquire a permit before accept() so the kernel's listen backlog absorbs overflow (async TCP with Tokio). HTTP/2 has per-stream and per-connection windows with the same effect.
tx.capacity() and tx.max_capacity() from channels (Tokio runtime metrics).unbounded_channel and spawn inside loops in code review.Related patterns: fan-out and fan-in pipelines, an in-process event bus and the actor model.
","date_published":"2026-11-03T13:32:00+00:00","tags":["Architecture"]},{"id":"https://rusthacks.dev/library/rust-graceful-degradation","url":"https://rusthacks.dev/library/rust-graceful-degradation","title":"Graceful Degradation in Rust Services Under Heavy Load","summary":"A service that stays up by doing less beats one that tries to do everything and falls over. Decide in advance what to drop and how to say no quickly.","content_html":"Services do not fail at the load they were designed for. They fail at 1.5 times that, when queues grow, latencies stretch, clients time out and retry, and the retries make it worse. A service that degrades gracefully keeps serving a useful share of traffic during the spike and recovers on its own afterward. The techniques are standard, and the work is deciding the policy beforehand. Versions: Rust 1.90, Axum 0.8, Tower 0.5, tower-http 0.6.
Suppose a handler takes 50 ms and the service has capacity for 200 requests per second. At 250 requests per second, the extra 50 queue up each second. The queue grows without bound, so the wait grows, so by the time a request is processed the client has given up and sent a retry, work is done for nobody, and goodput (useful work) falls to near zero while the CPU is at 100 percent. The fix is to stop accepting what you cannot finish.
\nuse axum::{error_handling::HandleErrorLayer, http::StatusCode, routing::get, BoxError, Router};\nuse std::time::Duration;\nuse tower::{limit::ConcurrencyLimitLayer, load_shed::LoadShedLayer, timeout::TimeoutLayer, ServiceBuilder};\n\nasync fn handle_overload(err: BoxError) -> (StatusCode, [(&'static str, &'static str); 1], &'static str) {\n if err.is::<tower::load_shed::error::Overloaded>() {\n (StatusCode::SERVICE_UNAVAILABLE, [(\"retry-after\", \"2\")], \"busy, retry shortly\")\n } else if err.is::<tower::timeout::error::Elapsed>() {\n (StatusCode::GATEWAY_TIMEOUT, [(\"retry-after\", \"5\")], \"request timed out\")\n } else {\n (StatusCode::INTERNAL_SERVER_ERROR, [(\"retry-after\", \"5\")], \"internal error\")\n }\n}\n\npub fn app() -> Router {\n let protect = ServiceBuilder::new()\n .layer(HandleErrorLayer::new(handle_overload))\n .layer(LoadShedLayer::new())\n .layer(ConcurrencyLimitLayer::new(256))\n .layer(TimeoutLayer::new(Duration::from_secs(5)));\n\n Router::new().route(\"/search\", get(search)).layer(protect)\n}\n\nasync fn search() -> &'static str {\n tokio::time::sleep(Duration::from_millis(50)).await;\n \"results\"\n}ConcurrencyLimitLayer allows 256 requests in flight. Without LoadShedLayer, the 257th request waits for a permit, which is a queue; with it, the layer returns Overloaded immediately when the inner service is not ready, which the error handler turns into a 503 with Retry-After. Layers are applied from the outside in order listed: error handling outermost, then shedding, then concurrency, then timeout around the handler. The timeout bounds the time a single slow request holds a permit. Note that the stock concurrency limit protects the whole router. Per-route limits (expensive search versus cheap health check) need separate layers on separate routes.
Every call crossing a boundary needs a deadline: incoming requests, database queries, outgoing HTTP calls. Budget them so inner timeouts are shorter than outer ones, or the outer gives up while the inner keeps working (Axum timeouts and limits, Tokio timeouts). Propagate the remaining deadline if you can, so downstream services stop working on abandoned requests.
\nWhen a dependency is slow, serve something rather than fail:
\nuse std::time::Duration;\nuse tokio::time::timeout;\n\nasync fn recommendations(user_id: u64, cache: &Cache) -> Vec<String> {\n match timeout(Duration::from_millis(200), fetch_recommendations(user_id)).await {\n Ok(Ok(items)) => {\n cache.store(user_id, items.clone());\n items\n }\n _ => cache.get_stale(user_id).unwrap_or_else(popular_items),\n }\n}Ladder of fallbacks: fresh result, stale cached result, generic default, omit the feature. Stale-while-revalidate does this systematically (stale while revalidate). A circuit breaker avoids hammering a failing dependency while it recovers (circuit breakers and bulkheads).
\nNot all requests are equal. Health checks must always answer. Writes that complete a purchase matter more than a recommendations widget. Options:
\nlet critical = Router::new().route(\"/checkout\", post(checkout)).layer(ConcurrencyLimitLayer::new(128));\nlet optional = Router::new().route(\"/recommendations\", get(recs)).layer(ConcurrencyLimitLayer::new(32));\nlet app = Router::new().merge(critical).merge(optional);Each group has its own permit pool, so a flood of recommendation requests cannot starve checkout.
\nUnder pressure, switch off the expensive extras: fancy sorting, personalization, large page sizes, synchronous analytics. A flag toggled automatically (when queue depth or latency crosses a threshold) or by an operator makes it a lever rather than a code change (feature flags). Return smaller pages, drop optional fields, skip enrichment calls. Document these modes and test them.
\nAlert on rising shed counts and on latency, and not on CPU alone.
\nA 503 with Retry-After works only if clients respect it. Retries need backoff, jitter and budgets, or they amplify the overload (async resilience patterns). Document the behavior for your API consumers.
Run a load test above capacity and check the shape of the curve: latency for accepted requests should stay bounded, rejection rate should rise, and recovery after the spike should be quick (load testing). Also kill a dependency during the test, and verify fallbacks engage. Related design: backpressure in pipelines, Axum rate limiting and service performance checklist.
","date_published":"2026-11-03T13:30:00+00:00","tags":["Architecture"]},{"id":"https://rusthacks.dev/library/rust-distributed-rate-limiting","url":"https://rusthacks.dev/library/rust-distributed-rate-limiting","title":"Distributed Rate Limiting for Multi Instance Rust Services","summary":"In-process limiters count per instance, so ten replicas allow ten times the limit. A shared counter in Redis makes the limit global, at the cost of a network hop and a failure policy.","content_html":"A rate limiter in each instance is easy and wrong for a fleet. With a limit of 100 requests per minute and ten replicas behind a load balancer, a client can make about a thousand. If the limit protects a downstream system or enforces a contract, it has to be global. That requires shared state, and the shared state brings atomicity, latency and failure questions. Versions: Rust 1.90, redis 0.32, Axum 0.8, Redis 7.4. The single-process version is in building a rate limiter.
The simplest shared limiter: one counter per key per window.
\nuse redis::{aio::ConnectionManager, AsyncCommands};\n\npub struct Decision {\n pub allowed: bool,\n pub remaining: u32,\n pub retry_after_secs: u64,\n}\n\npub async fn fixed_window(\n redis: &mut ConnectionManager,\n key: &str,\n limit: u32,\n window_secs: u64,\n) -> redis::RedisResult<Decision> {\n let now = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_secs();\n let window = now / window_secs;\n let redis_key = format!(\"rl:{key}:{window}\");\n\n let count: u32 = redis.incr(&redis_key, 1).await?;\n if count == 1 {\n let _: () = redis.expire(&redis_key, window_secs as i64 + 1).await?;\n }\n Ok(Decision {\n allowed: count <= limit,\n remaining: limit.saturating_sub(count),\n retry_after_secs: window_secs - (now % window_secs),\n })\n}INCR is atomic, so concurrent instances cannot lose updates. There is a small gap: if the process dies between INCR and EXPIRE, the key never expires, which is why the expiry is set when count == 1 and why production code uses a script or SET key 0 EX n NX first. The known weakness of fixed windows: a client can send limit requests at the end of one window and limit more at the start of the next, doubling the rate for a moment.
A Lua script runs atomically inside Redis, so reading and updating the bucket cannot interleave with another instance:
\nconst TOKEN_BUCKET: &str = r#\"\nlocal key = KEYS[1]\nlocal capacity = tonumber(ARGV[1])\nlocal rate = tonumber(ARGV[2]) -- tokens per second\nlocal now_ms = tonumber(ARGV[3])\nlocal cost = tonumber(ARGV[4])\n\nlocal data = redis.call('HMGET', key, 'tokens', 'ts')\nlocal tokens = tonumber(data[1])\nlocal ts = tonumber(data[2])\nif tokens == nil then\n tokens = capacity\n ts = now_ms\nend\n\nlocal elapsed = math.max(0, now_ms - ts) / 1000.0\ntokens = math.min(capacity, tokens + elapsed * rate)\n\nlocal allowed = 0\nif tokens >= cost then\n tokens = tokens - cost\n allowed = 1\nend\n\nredis.call('HSET', key, 'tokens', tokens, 'ts', now_ms)\nredis.call('PEXPIRE', key, math.ceil(capacity / rate * 1000) + 1000)\n\nlocal retry_ms = 0\nif allowed == 0 then\n retry_ms = math.ceil((cost - tokens) / rate * 1000)\nend\nreturn { allowed, math.floor(tokens), retry_ms }\n\"#;\n\npub async fn token_bucket(\n redis: &mut ConnectionManager,\n key: &str,\n capacity: u32,\n per_second: f64,\n) -> redis::RedisResult<Decision> {\n let now_ms = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_millis() as u64;\n let (allowed, remaining, retry_ms): (i32, i64, i64) = redis::Script::new(TOKEN_BUCKET)\n .key(format!(\"rl:tb:{key}\"))\n .arg(capacity)\n .arg(per_second)\n .arg(now_ms)\n .arg(1)\n .invoke_async(redis)\n .await?;\n Ok(Decision {\n allowed: allowed == 1,\n remaining: remaining.max(0) as u32,\n retry_after_secs: (retry_ms as u64).div_ceil(1000),\n })\n}The script uses the caller's clock for now_ms. Instances with skewed clocks distort refill, so either keep clocks synchronized (NTP) or use redis.call('TIME') inside the script as the single clock. redis::Script sends the script by hash after the first call. The key expires when the bucket would be full again, so idle clients cost no memory.
| Algorithm | Accuracy | Memory per key | Bursts | Notes | |---|---|---|---|---| | Fixed window | boundary spikes up to 2x | one counter | at window start | simplest | | Sliding window log | exact | one entry per request | no | sorted set; costly for large limits | | Sliding window counter | close | two counters | limited | weighted previous window | | Token bucket | exact rate | two fields | up to capacity | most flexible | | Leaky bucket | smooths output | queue | no | for shaping, not rejecting |
\nFor API limits, token bucket or sliding window counter are the usual choices.
\nPick the identity carefully: authenticated user or API key where available, IP otherwise (behind a proxy, from the trusted forwarded header; see behind a reverse proxy). Combine dimensions with tiers: per_ip:1.2.3.4, per_key:abc, per_route:login. Apply a stricter limit on expensive or abusive routes (login, password reset, search) (rate limiting for abuse). Avoid unbounded key growth from attacker-chosen values; keys expire, which bounds it.
use axum::{extract::{Request, State}, http::{header, HeaderValue, StatusCode}, middleware::Next, response::{IntoResponse, Response}};\n\npub async fn limit(State(app): State<AppState>, req: Request, next: Next) -> Response {\n let key = client_key(&req);\n let mut conn = app.redis.clone();\n\n match token_bucket(&mut conn, &key, 100, 100.0 / 60.0).await {\n Ok(d) if !d.allowed => {\n let mut res = StatusCode::TOO_MANY_REQUESTS.into_response();\n res.headers_mut().insert(header::RETRY_AFTER, HeaderValue::from(d.retry_after_secs.max(1)));\n res\n }\n Ok(d) => {\n let mut res = next.run(req).await;\n res.headers_mut().insert(\"x-ratelimit-remaining\", HeaderValue::from(d.remaining));\n res\n }\n Err(e) => {\n tracing::error!(\"rate limiter unavailable: {e}\");\n if app.fail_open { next.run(req).await } else { StatusCode::SERVICE_UNAVAILABLE.into_response() }\n }\n }\n}Retry-After and RateLimit-* headers let clients pace themselves. Test the whole thing with paused time against a fake clock and, for the Redis part, a real Redis in a container (testing rate limiters).
Redis down, slow or partitioned. Options:
\nLog and count failures, and alert on them.
\nA gateway or CDN can enforce limits before traffic reaches you (Envoy, Nginx, cloud API gateways), which removes the code but ties the policy to infrastructure. Cell-based or sticky routing makes local limiters approximately right: route each client to one instance by key. For very high rates, approximate algorithms with local counters that sync every few hundred milliseconds trade accuracy for scale. See also Axum rate limiting and Redis data structures.
","date_published":"2026-11-03T13:26:00+00:00","tags":["Architecture"]},{"id":"https://rusthacks.dev/library/rust-actor-model-tokio","url":"https://rusthacks.dev/library/rust-actor-model-tokio","title":"The Actor Model in Rust With Tokio Tasks and Channels","summary":"An actor is a task that owns its data and processes messages one at a time. Channels replace locks, and a handle type hides the plumbing.","content_html":"Sharing mutable state between tasks usually means Arc<Mutex<T>>, and that works until a critical section needs an .await, or until the locking discipline spreads across the codebase. The actor model offers another structure: give the state to one task, and let everything else talk to it through messages. Tokio's channels make this natural. Code runs on Rust 1.90, Tokio 1.47.
use tokio::sync::{mpsc, oneshot};\n\nenum Command {\n Deposit { cents: u64 },\n Withdraw { cents: u64, reply: oneshot::Sender<Result<u64, String>> },\n Balance { reply: oneshot::Sender<u64> },\n}\n\nstruct AccountActor {\n rx: mpsc::Receiver<Command>,\n balance: u64,\n}\n\nimpl AccountActor {\n async fn run(mut self) {\n while let Some(cmd) = self.rx.recv().await {\n match cmd {\n Command::Deposit { cents } => self.balance += cents,\n Command::Withdraw { cents, reply } => {\n let result = if cents <= self.balance {\n self.balance -= cents;\n Ok(self.balance)\n } else {\n Err(format!(\"insufficient funds: have {}, need {}\", self.balance, cents))\n };\n let _ = reply.send(result);\n }\n Command::Balance { reply } => {\n let _ = reply.send(self.balance);\n }\n }\n }\n // all handles dropped: the loop ends and the task finishes\n }\n}The state (balance) is a plain field. Messages are processed one at a time, so check-then-update in Withdraw is atomic with no lock. The reply senders send a result back to the caller. Ignoring a failed reply.send is correct: the caller may have given up.
#[derive(Clone)]\npub struct AccountHandle {\n tx: mpsc::Sender<Command>,\n}\n\n#[derive(Debug)]\npub struct ActorGone;\n\nimpl AccountHandle {\n pub fn spawn(initial: u64) -> (Self, tokio::task::JoinHandle<()>) {\n let (tx, rx) = mpsc::channel(64);\n let actor = AccountActor { rx, balance: initial };\n let join = tokio::spawn(actor.run());\n (Self { tx }, join)\n }\n\n pub async fn deposit(&self, cents: u64) -> Result<(), ActorGone> {\n self.tx.send(Command::Deposit { cents }).await.map_err(|_| ActorGone)\n }\n\n pub async fn withdraw(&self, cents: u64) -> Result<Result<u64, String>, ActorGone> {\n let (reply, rx) = oneshot::channel();\n self.tx.send(Command::Withdraw { cents, reply }).await.map_err(|_| ActorGone)?;\n rx.await.map_err(|_| ActorGone)\n }\n\n pub async fn balance(&self) -> Result<u64, ActorGone> {\n let (reply, rx) = oneshot::channel();\n self.tx.send(Command::Balance { reply }).await.map_err(|_| ActorGone)?;\n rx.await.map_err(|_| ActorGone)\n }\n}Callers see ordinary async methods and a cloneable handle they can store in app state and pass to many tasks. The channel is bounded at 64, so if the actor is busy, send().await waits, applying backpressure to callers instead of growing memory (backpressure design, bounded channels). ActorGone distinguishes a dead actor from a business error, which is why withdraw returns a nested Result.
#[tokio::main]\nasync fn main() {\n let (account, join) = AccountHandle::spawn(1_000);\n\n let a = account.clone();\n let t1 = tokio::spawn(async move { a.withdraw(700).await });\n let b = account.clone();\n let t2 = tokio::spawn(async move { b.withdraw(700).await });\n\n let (r1, r2) = (t1.await.unwrap().unwrap(), t2.await.unwrap().unwrap());\n println!(\"{r1:?} {r2:?}\"); // exactly one succeeds\n println!(\"balance {}\", account.balance().await.unwrap());\n\n drop(account); // last handle gone, actor loop ends\n join.await.unwrap();\n}Two concurrent withdrawals of 700 from 1,000 cannot both succeed, since the actor serializes them. The shutdown line shows the lifecycle: when every AccountHandle is dropped, recv() returns None and the task completes.
.await a database call while it holds the state, and other callers just queue. A std::sync::Mutex guard across an await is a bug, and a Tokio mutex held across slow I/O stalls everyone anyway (mutex guard across await).ActorGone.Watch the JoinHandle and restart or fail loudly:
async fn supervise(initial: u64) {\n loop {\n let (handle, join) = AccountHandle::spawn(initial);\n publish_handle(handle); // e.g. replace the one in shared state\n match join.await {\n Ok(()) => break, // clean shutdown\n Err(e) if e.is_panic() => tracing::error!(\"account actor panicked, restarting\"),\n Err(_) => break, // cancelled\n }\n }\n}Restarting loses in-memory state, so an actor that matters should persist state or rebuild it from storage on start. Frameworks such as ractor and kameo provide supervision strategies if you need them at volume.
\n| Need | Tool | |---|---| | Simple shared counter or cache | Arc<Mutex<T>> or atomics | | Read-mostly configuration | ArcSwap or watch channel | | Single owner with async work, serialized commands | actor | | Broadcast notifications | event bus | | Distributed, durable | a message broker |
Related: sharing state in Tokio, channels compared and writing a chat server with broadcast.
","date_published":"2026-11-03T13:20:00+00:00","tags":["Architecture"]},{"id":"https://rusthacks.dev/library/rust-in-process-event-bus","url":"https://rusthacks.dev/library/rust-in-process-event-bus","title":"An In Process Event Bus in Rust With Broadcast Channels","summary":"An event bus lets parts of a program react to things without knowing about each other. Tokio's broadcast channel gives a small, typed one with built-in handling for slow subscribers.","content_html":"As an application grows, one action starts to trigger many reactions: an order is placed, so the confirmation email is queued, the cache is cleared, the audit log is written and the metrics counter increments. Calling all of them from the order code couples it to every consumer. An event bus inverts that: the order code publishes OrderPlaced, and interested modules subscribe. Within one process this takes about forty lines. Versions: Rust 1.90, Tokio 1.47.
use tokio::sync::broadcast;\n\n#[derive(Debug, Clone)]\npub enum Event {\n OrderPlaced { order_id: u64, user_id: u64, total_cents: i64 },\n OrderCancelled { order_id: u64 },\n UserRegistered { user_id: u64 },\n}\n\n#[derive(Clone)]\npub struct EventBus {\n tx: broadcast::Sender<Event>,\n}\n\nimpl EventBus {\n pub fn new(capacity: usize) -> Self {\n let (tx, _) = broadcast::channel(capacity);\n Self { tx }\n }\n\n pub fn publish(&self, event: Event) {\n // Err only means nobody is subscribed\n let _ = self.tx.send(event);\n }\n\n pub fn subscribe(&self) -> broadcast::Receiver<Event> {\n self.tx.subscribe()\n }\n}EventBus is cheap to clone (the sender is reference counted), so each module holds its own handle, typically passed in at construction (dependency injection). Event derives Clone because each subscriber gets its own copy.
A subscriber is a task with a receive loop:
\nuse tokio::sync::broadcast::error::RecvError;\n\npub fn spawn_audit_logger(bus: &EventBus) -> tokio::task::JoinHandle<()> {\n let mut rx = bus.subscribe();\n tokio::spawn(async move {\n loop {\n match rx.recv().await {\n Ok(Event::OrderPlaced { order_id, total_cents, .. }) => {\n tracing::info!(order_id, total_cents, \"order placed\");\n }\n Ok(_) => {}\n Err(RecvError::Lagged(missed)) => {\n tracing::warn!(missed, \"audit logger fell behind and missed events\");\n }\n Err(RecvError::Closed) => break,\n }\n }\n })\n}\n\npub fn spawn_cache_invalidator(bus: &EventBus, cache: moka::future::Cache<u64, String>) {\n let mut rx = bus.subscribe();\n tokio::spawn(async move {\n while let Ok(event) = rx.recv().await {\n if let Event::OrderCancelled { order_id } = event {\n cache.invalidate(&order_id).await;\n }\n }\n });\n}Each subscriber filters for the variants it cares about and ignores the rest. Adding a new reaction means adding a subscriber, and the producer does not change. That is the decoupling.
\npub struct OrderService {\n bus: EventBus,\n}\n\nimpl OrderService {\n pub async fn place(&self, user_id: u64, total_cents: i64) -> u64 {\n let order_id = 1001; // persisted in a real service\n self.bus.publish(Event::OrderPlaced { order_id, user_id, total_cents });\n order_id\n }\n}publish is synchronous and never waits for subscribers, so a slow consumer cannot slow the order path. That is both the strength and the hazard.
The channel holds capacity events. If a subscriber falls behind by more than that, the oldest events are discarded for it and its next recv returns Lagged(n). The bus does not block the producer to protect slow consumers. Consequences:
For that case use an mpsc channel per critical subscriber (one sender per consumer, held by the bus), which applies backpressure, or a durable queue (background jobs, Tokio channels compared).
Within one subscriber, events arrive in publication order. Across subscribers there is no ordering. Delivery is at most once per subscriber: no acknowledgements, no retries. Subscribers added later see only later events; a subscriber that starts after UserRegistered was published never sees it. If a handler panics, only its task dies, so supervise: restart the task or log loudly.
One enum is easy and keeps a single channel. If different events have different audiences and rates, run separate buses (OrderBus, UserBus) so a flood of one type does not push another's events out of a shared buffer. Alternatively use a HashMap<TypeId, broadcast::Sender<...>> for a type-keyed bus; the added complexity is rarely worth it.
#[tokio::test]\nasync fn subscribers_receive_published_events() {\n let bus = EventBus::new(16);\n let mut a = bus.subscribe();\n let mut b = bus.subscribe();\n\n bus.publish(Event::UserRegistered { user_id: 7 });\n\n assert!(matches!(a.recv().await.unwrap(), Event::UserRegistered { user_id: 7 }));\n assert!(matches!(b.recv().await.unwrap(), Event::UserRegistered { user_id: 7 }));\n}\n\n#[tokio::test]\nasync fn slow_subscriber_sees_lag_not_a_hang() {\n let bus = EventBus::new(2);\n let mut rx = bus.subscribe();\n for i in 0..5 {\n bus.publish(Event::OrderCancelled { order_id: i });\n }\n assert!(matches!(rx.recv().await, Err(broadcast::error::RecvError::Lagged(3))));\n assert!(matches!(rx.recv().await.unwrap(), Event::OrderCancelled { order_id: 3 }));\n}The second test pins the lag behavior, which is the contract your subscribers must handle.
\nIf one specific thing must happen after another, call it directly: events hide control flow and make \"what runs when X happens?\" harder to answer. Use the bus for optional side effects and notifications, not for the core of a transaction. When the work must be reliable, record the event in the database in the same transaction and process it asynchronously (outbox pattern). Actors are an alternative structure for owned state (actor model). For cross-process events see Kafka and NATS compared, and for the fan-out design in pipelines fan-out and fan-in.
","date_published":"2026-11-03T13:18:00+00:00","tags":["Architecture"]},{"id":"https://rusthacks.dev/library/rust-test-rate-limiters","url":"https://rusthacks.dev/library/rust-test-rate-limiters","title":"Testing Rate Limiters and Quotas in Rust Web Services","summary":"A limiter is a function of time, so tests should control time. Assert the boundaries: the last allowed request, the first rejected one and the moment capacity returns.","content_html":"Rate limiting code is simple to write and easy to get slightly wrong: off by one at the limit, a window that never resets or one key affecting another. Tests that rely on real time are slow and flaky, so the first rule is to take control of the clock. The examples use Rust 1.90, Tokio 1.47 and Axum 0.8, with the token bucket from building a rate limiter.
\nuse std::time::Duration;\n\n#[tokio::test(start_paused = true)]\nasync fn allows_burst_then_rejects() {\n let mut bucket = TokenBucket::new(3, 1.0); // capacity 3, refill 1 per second\n\n assert!(bucket.try_acquire());\n assert!(bucket.try_acquire());\n assert!(bucket.try_acquire());\n assert!(!bucket.try_acquire(), \"fourth request exceeds the burst\");\n}\n\n#[tokio::test(start_paused = true)]\nasync fn refills_at_the_configured_rate() {\n let mut bucket = TokenBucket::new(2, 1.0);\n assert!(bucket.try_acquire());\n assert!(bucket.try_acquire());\n assert!(!bucket.try_acquire());\n\n tokio::time::advance(Duration::from_millis(999)).await;\n assert!(!bucket.try_acquire(), \"not yet one full second\");\n\n tokio::time::advance(Duration::from_millis(2)).await;\n assert!(bucket.try_acquire(), \"one token has accumulated\");\n assert!(!bucket.try_acquire());\n}\n\n#[tokio::test(start_paused = true)]\nasync fn never_exceeds_capacity_after_a_long_pause() {\n let mut bucket = TokenBucket::new(2, 1.0);\n tokio::time::advance(Duration::from_secs(3600)).await;\n assert!(bucket.try_acquire());\n assert!(bucket.try_acquire());\n assert!(!bucket.try_acquire(), \"tokens are capped at capacity\");\n}start_paused = true freezes Tokio's clock, and advance moves it exactly. The 999 millisecond and 1001 millisecond steps test the boundary around one second. This works because the limiter reads tokio::time::Instant, not std::time::Instant (injected clocks). If your limiter uses its own Clock trait, give tests a fake with a settable now.
A fixed window counter needs boundary tests that a token bucket does not: requests at the end of one window and the start of the next.
\n#[tokio::test(start_paused = true)]\nasync fn fixed_window_resets_at_the_boundary() {\n let limiter = FixedWindow::new(2, Duration::from_secs(60));\n assert!(limiter.check(\"k\").await);\n assert!(limiter.check(\"k\").await);\n assert!(!limiter.check(\"k\").await);\n\n tokio::time::advance(Duration::from_secs(60)).await;\n assert!(limiter.check(\"k\").await, \"new window\");\n}State the known weakness in a test too: a burst straddling two windows can allow double the limit, which is why sliding windows exist (distributed rate limiting).
\n#[tokio::test(start_paused = true)]\nasync fn one_client_cannot_exhaust_anothers_budget() {\n let limiter = KeyedLimiter::new(1, 1.0);\n assert!(limiter.check(\"alice\").await);\n assert!(!limiter.check(\"alice\").await);\n assert!(limiter.check(\"bob\").await, \"bob has his own bucket\");\n}Also test that the map of keys does not grow without bound: after a long idle period, entries should be evicted (limiter.len() returns to zero).
use axum::{body::Body, http::{Request, StatusCode}, routing::get, Router};\nuse tower::ServiceExt;\n\nfn app() -> Router {\n Router::new()\n .route(\"/api\", get(|| async { \"ok\" }))\n .layer(rate_limit_layer(2, Duration::from_secs(60)))\n}\n\n#[tokio::test(start_paused = true)]\nasync fn returns_429_with_retry_after() {\n let app = app();\n for _ in 0..2 {\n let res = app.clone().oneshot(Request::get(\"/api\").body(Body::empty()).unwrap()).await.unwrap();\n assert_eq!(res.status(), StatusCode::OK);\n }\n\n let res = app.clone().oneshot(Request::get(\"/api\").body(Body::empty()).unwrap()).await.unwrap();\n assert_eq!(res.status(), StatusCode::TOO_MANY_REQUESTS);\n let retry: u64 = res.headers()[\"retry-after\"].to_str().unwrap().parse().unwrap();\n assert!((1..=60).contains(&retry));\n\n tokio::time::advance(Duration::from_secs(61)).await;\n let res = app.oneshot(Request::get(\"/api\").body(Body::empty()).unwrap()).await.unwrap();\n assert_eq!(res.status(), StatusCode::OK);\n}Clients rely on Retry-After, so assert its range, and consider RateLimit-Remaining headers if you expose them. Identify the client key from the right source: tests should send distinct X-Forwarded-For values from a trusted proxy address and check they are separate buckets, plus that the header from an untrusted peer is ignored (behind a reverse proxy, Axum rate limiting).
A limiter that checks and decrements non-atomically allows extra requests under load:
\n#[tokio::test(flavor = \"multi_thread\", worker_threads = 4)]\nasync fn concurrent_requests_never_exceed_budget() {\n let limiter = std::sync::Arc::new(KeyedLimiter::new(50, 0.0)); // no refill\n let mut tasks = Vec::new();\n for _ in 0..500 {\n let l = limiter.clone();\n tasks.push(tokio::spawn(async move { l.check(\"k\").await }));\n }\n let mut allowed = 0;\n for t in tasks {\n if t.await.unwrap() {\n allowed += 1;\n }\n }\n assert_eq!(allowed, 50);\n}With a zero refill rate the allowed count must be exactly the capacity. Run it repeatedly; the race, if there is one, shows quickly. For lock-free implementations, model check the core with loom (loom).
\nA Redis-backed limiter needs integration tests against a real Redis (Testcontainers), since Lua script behavior and expiry are the point. Verify key expiry by advancing the server clock or using short windows. Keep the algorithm tests above, which are fast, as the main safety net.
","date_published":"2026-11-03T12:12:00+00:00","tags":["Testing"]},{"id":"https://rusthacks.dev/library/rust-test-concurrency-loom","url":"https://rusthacks.dev/library/rust-test-concurrency-loom","title":"Testing Concurrent Rust Code With Loom Interleavings","summary":"Stress tests find concurrency bugs by luck. Loom explores every legal ordering of your threads and memory operations, so a missing Acquire fails deterministically.","content_html":"Concurrency bugs depend on timing. A test that passes a million times can still hide a race that appears on a different CPU. Loom is a model checker for Rust: it runs your code under every ordering that the memory model allows within a bound. It is how Tokio tests its synchronization primitives. Versions: Rust 1.90, loom 0.7.
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};\nuse std::sync::Arc;\nuse std::thread;\n\n#[test]\nfn publish_data() {\n let data = Arc::new(AtomicUsize::new(0));\n let ready = Arc::new(AtomicBool::new(false));\n\n let (d, r) = (data.clone(), ready.clone());\n let writer = thread::spawn(move || {\n d.store(42, Ordering::Relaxed);\n r.store(true, Ordering::Relaxed); // bug: should be Release\n });\n\n while !ready.load(Ordering::Relaxed) {} // bug: should be Acquire\n assert_eq!(data.load(Ordering::Relaxed), 42);\n writer.join().unwrap();\n}On x86 this nearly always passes, because the hardware orders stores strongly. On ARM or under compiler reordering the reader may see ready before data. A normal test cannot show it.
Add a cfg switch so the crate uses loom's types in tests:
[target.'cfg(loom)'.dependencies]\nloom = \"0.7\"\n\n[lints.rust]\nunexpected_cfgs = { level = \"warn\", check-cfg = [\"cfg(loom)\"] }#[cfg(loom)]\nuse loom::{sync::{atomic::{AtomicBool, AtomicUsize, Ordering}, Arc}, thread};\n#[cfg(not(loom))]\nuse std::{sync::{atomic::{AtomicBool, AtomicUsize, Ordering}, Arc}, thread};Library code then uses these imports instead of std::sync directly. The test:
#[cfg(loom)]\n#[test]\nfn publish_data_loom() {\n loom::model(|| {\n let data = Arc::new(AtomicUsize::new(0));\n let ready = Arc::new(AtomicBool::new(false));\n\n let (d, r) = (data.clone(), ready.clone());\n let writer = thread::spawn(move || {\n d.store(42, Ordering::Relaxed);\n r.store(true, Ordering::Relaxed);\n });\n\n while !ready.load(Ordering::Relaxed) {\n loom::thread::yield_now();\n }\n assert_eq!(data.load(Ordering::Relaxed), 42);\n writer.join().unwrap();\n });\n}RUSTFLAGS=\"--cfg loom\" cargo test --release publish_data_loomthread 'publish_data_loom' panicked at src/lib.rs:31:9:\nassertion `left == right` failed\n left: 0\n right: 42Loom found an execution where ready is true and data is still 0. Change the stores and loads to Release and Acquire:
r.store(true, Ordering::Release);\nwhile !ready.load(Ordering::Acquire) { loom::thread::yield_now(); }and the model passes, having explored all legal orderings. Spin loops need yield_now(), otherwise loom cannot progress the other thread. Run in --release, since the exploration is CPU heavy.
| std | loom | |---|---| | std::sync::atomic::* | loom::sync::atomic::* | | std::sync::{Mutex, RwLock, Condvar} | loom::sync::* | | std::sync::Arc | loom::sync::Arc | | std::thread::spawn | loom::thread::spawn | | std::cell::UnsafeCell | loom::cell::UnsafeCell (detects data races) |
The loom::cell::UnsafeCell variant has with and with_mut methods that panic if two threads would access the data concurrently without synchronization. That check is the strongest reason to use loom with lock-free code (atomics explained).
The number of schedules explodes with each thread and each operation. Practical rules:
\nLOOM_MAX_PREEMPTIONS=3 cargo test (the default checks exhaustively within the bound the library sets).LOOM_LOG=trace to see the failing schedule, and LOOM_CHECKPOINT_FILE to replay it.Reach for loom when you write code with atomics, UnsafeCell, custom locks or lock-free structures, where a bug means a rare crash in production. For ordinary mutex-protected state, stress tests and careful design are enough ([Arc<Mutex<T>>](/library/rust-arc-mutex-pattern)). The companion for undefined behavior in unsafe code is Miri (Miri).
The default answer for a new backend is Postgres, and that is usually right. It is not always right. A growing number of services run on SQLite in production because they fit its shape: one node, modest writes, large reads. The choice depends on a handful of questions that you can answer before writing code. Versions in mind: Postgres 17, SQLite 3.46, SQLx 0.8, Rust 1.90.
\n| | SQLite | Postgres | |---|---|---| | Architecture | library in your process, one file | server process, network protocol | | Writers | one at a time | many concurrent (MVCC) | | Readers | many, in-process, no network | many, over connections | | Access from several machines | not supported on shared file systems | built for it | | Setup | none | install, configure, secure, monitor | | Types | dynamic typing, few types | rich types: arrays, JSONB, ranges, enums | | Extensions | limited (FTS5, JSON, R-Tree) | pgvector, PostGIS, pg_cron and more | | Backups | copy the file, Litestream | pg_dump, base backups, WAL archiving, managed | | HA and failover | external tooling | replicas, managed failover |
\nEverything else follows from these rows.
\n1. How many application instances will write? One instance: SQLite works. Two or more behind a load balancer: Postgres. Zero-downtime deploys with overlapping instances also point to Postgres, since two processes writing the same file during a rollout is a hazard.
\n2. How write-heavy is it? SQLite's single writer sustains thousands of small transactions per second on an SSD, which covers most applications. Hundreds of concurrent long-running write transactions do not fit.
\n3. Do other systems need to query the data? BI tools, workers on other machines and analysts connect to Postgres natively. With SQLite you export or replicate.
\n4. Which features do you need? Full-text search exists in both, with different strengths. Vector search, geospatial queries, row level security and listen/notify are Postgres strengths (pgvector, RLS, LISTEN/NOTIFY).
\n5. Who operates it? A solo developer or small team may prefer no database server. A team with a platform group or a managed Postgres has the opposite situation.
\n6. What is the growth path? If you expect to need multiple nodes within a year, starting on Postgres avoids a migration. If the service will stay small, SQLite avoids a permanent operational cost.
\nThe connection model changes the design. SQLite wants one writer connection plus reader connections (SQLite in production). Postgres wants a pool sized to the server (pool sizing). With SQLx, you pick a different feature and type for the pool:
\n# SQLite\nsqlx = { version = \"0.8\", features = [\"runtime-tokio\", \"sqlite\", \"migrate\"] }\n# Postgres\nsqlx = { version = \"0.8\", features = [\"runtime-tokio\", \"postgres\", \"migrate\"] }SQL dialect differences appear quickly:
\n| Concern | SQLite | Postgres | |---|---|---| | Placeholders | ? or ?1 | $1 | | Auto IDs | INTEGER PRIMARY KEY | bigserial or identity | | Timestamps | text or integer | timestamptz | | Upsert | ON CONFLICT (supported) | ON CONFLICT | | Booleans | integers 0 and 1 | boolean | | ALTER TABLE | limited | extensive | | Type enforcement | flexible unless STRICT | strict |
Use STRICT tables in SQLite to get type checking closer to Postgres.
Export tables as CSV, load with COPY (bulk loading with COPY), convert text timestamps and 0/1 booleans, recreate constraints and indexes, and run the application's tests against Postgres. pgloader automates much of it. Dual-write is rarely necessary at this scale, and a short maintenance window is simpler.
Pick SQLite if: single node, writes under a few hundred per second, simple deployment valued, data under tens of gigabytes, team small.
\nPick Postgres if: more than one app instance, other systems need the data, advanced features, regulatory needs for replication, or a managed service is easier for your organization.
\nWhen unsure and the cost of a wrong guess is low, choose the one you can operate confidently. Both are mature. For related design choices, see SQLx versus Diesel versus SeaORM.
","date_published":"2026-11-03T11:48:00+00:00","tags":["Databases"]},{"id":"https://rusthacks.dev/library/rust-postgres-isolation-levels","url":"https://rusthacks.dev/library/rust-postgres-isolation-levels","title":"Postgres Isolation Levels and Serialization Retries in Rust","summary":"Higher isolation levels prevent anomalies by aborting transactions that conflict. Your code must expect those aborts and retry the whole transaction.","content_html":"Two requests update the same account at the same moment. Whether the result is right depends on isolation, which most application code never thinks about until a balance goes negative. This post explains what the levels in Postgres guarantee and how to write Rust that works with the failures the stronger levels produce. Versions: Postgres 17, SQLx 0.8, Rust 1.90.
\n| Level | Each statement sees | Prevents | Still allows | |---|---|---|---| | READ COMMITTED (default) | rows committed before the statement began | dirty reads | non-repeatable reads, phantoms, write skew | | REPEATABLE READ | a snapshot from the transaction's first statement | the above except write skew | write skew | | SERIALIZABLE | snapshot plus conflict detection | all anomalies | nothing, but may abort |
Postgres has no dirty reads at any level (READ UNCOMMITTED behaves as READ COMMITTED).
Two on-call doctors each check that at least two are on call, then each takes themselves off:
\n-- both transactions\nSELECT count(*) FROM on_call WHERE shift = 'night'; -- sees 2\nUPDATE on_call SET active = false WHERE doctor = :me; -- different rows\nCOMMIT;Under READ COMMITTED and REPEATABLE READ both commit, leaving zero doctors on call, since neither wrote the other's row. Under SERIALIZABLE, Postgres detects the read-write dependency cycle and aborts one transaction:
ERROR: could not serialize access due to read/write dependencies among transactions\nDETAIL: Reason code: Canceled on identification as a pivot, during commit attempt.\nHINT: The transaction might succeed if retried.That is SQLSTATE 40001. The error is the feature: the database refused to let the invariant break.
use sqlx::{PgPool, Postgres, Transaction};\n\nasync fn begin_serializable(pool: &PgPool) -> Result<Transaction<'_, Postgres>, sqlx::Error> {\n let mut tx = pool.begin().await?;\n sqlx::query(\"SET TRANSACTION ISOLATION LEVEL SERIALIZABLE\").execute(&mut *tx).await?;\n Ok(tx)\n}SET TRANSACTION must be the first statement in the transaction. Alternatively set default_transaction_isolation for a role or a connection.
use std::time::Duration;\n\nfn is_retryable(e: &sqlx::Error) -> bool {\n match e {\n sqlx::Error::Database(db) => matches!(db.code().as_deref(), Some(\"40001\") | Some(\"40P01\")),\n _ => false,\n }\n}\n\npub async fn with_retry<T, F, Fut>(pool: &PgPool, mut work: F) -> Result<T, sqlx::Error>\nwhere\n F: FnMut(Transaction<'static, Postgres>) -> Fut,\n Fut: std::future::Future<Output = Result<T, sqlx::Error>>,\n{\n let mut attempt = 0u32;\n loop {\n let tx = pool.begin().await?;\n match work(tx).await {\n Ok(v) => return Ok(v),\n Err(e) if is_retryable(&e) && attempt < 5 => {\n attempt += 1;\n let backoff = Duration::from_millis(10 * 2u64.pow(attempt) + fastrand::u64(0..10));\n tracing::warn!(attempt, \"retrying transaction after conflict\");\n tokio::time::sleep(backoff).await;\n }\n Err(e) => return Err(e),\n }\n }\n}The closure takes ownership of a fresh transaction and is expected to commit it. Because the body can run several times, it must be free of side effects other than database writes. A conflict can appear at any statement or at COMMIT, so the check wraps the whole closure including the commit. Cap attempts and add jitter, otherwise colliding transactions retry in lockstep (async resilience patterns).
Two transactions that lock rows in opposite order wait for each other. Postgres detects this after deadlock_timeout (1 s by default) and aborts one with 40P01. The same retry handles it, and the better fix is a consistent lock order: always update rows sorted by id.
Often you do not need SERIALIZABLE. A targeted lock expresses the intent:
SELECT balance FROM accounts WHERE id = $1 FOR UPDATE;Other transactions touching that row wait. For queues, FOR UPDATE SKIP LOCKED (SKIP LOCKED queues). For counters, a single UPDATE accounts SET balance = balance - $2 WHERE id = $1 AND balance >= $2 is atomic without any extra level. For versioned rows, optimistic checks (optimistic locking).
SERIALIZABLE selectively, for the transactions that guard cross-row invariants, and measure abort rates.REPEATABLE READ READ ONLY DEFERRABLE for a consistent snapshot without aborts.For the transaction API itself see SQLx transactions.
","date_published":"2026-11-03T11:36:00+00:00","tags":["Databases"]},{"id":"https://rusthacks.dev/library/rust-build-disk-usage-cli","url":"https://rusthacks.dev/library/rust-build-disk-usage-cli","title":"Build a Disk Usage Analyzer in Rust Like du and ncdu","summary":"du walks a tree and adds up file sizes. The interesting decisions are hard links counted once, symlinks left alone and how to keep going when a directory is unreadable.","content_html":"A disk is full and you want to know why. du answers, in a way that is accurate and awkward to read, and ncdu answers interactively. Building a small version yourself, rdu, is a good exercise because the problem looks trivial, add up the file sizes, and is not. \"Size\" has two meanings that differ by 10 percent on a tidy directory and by a factor of ten on a directory full of small files. Hard links make the same bytes appear in two places. A symlink to a parent directory loops forever if followed. Parallel scanning makes the answer depend on thread timing unless you design against it. And a directory you cannot read must not stop the report.
This guide builds rdu step by step, finds a real nondeterminism bug in the first parallel version by running it twelve times, fixes it, checks the result against du, and measures the speed. It was written on Rust 1.99 (edition 2024) with rayon 1.12 and clap 4.6.7, on macOS, so file-system details such as block size and APFS behavior reflect that platform. The output blocks are real.
A file has an apparent size, the number of bytes in its contents, and a disk usage, the number of bytes of storage the file system allocated for it. They differ for three reasons. Storage is allocated in blocks, so a 9-byte file costs a whole block. Sparse files have holes that cost nothing. And compressing file systems store less than the logical size. The standard library exposes both through std::fs::Metadata: len() is apparent size, and on Unix the MetadataExt::blocks() method returns the number of 512-byte units allocated.
fn measure(md: &fs::Metadata, how: Measure) -> u64 {\n match how {\n Measure::Apparent => md.len(),\n Measure::Disk => md.blocks() * 512,\n }\n}du reports disk usage by default and du --apparent-size reports the other. The tool should do the same, and name the choice in a flag. The difference is visible on a small directory built to show it: a 5 MB file, a 3 MB file, an 800 KB file, 200 tiny log files of a few bytes each, a 120 KB source file, plus empty directories.
$ rdu /tmp/du_demo\n 9.3 MiB 100.0% #################### .\n...\n206 files, 9.3 MiB\n\n$ rdu --apparent /tmp/du_demo\n 8.5 MiB 100.0% #################### .\n 8.4 MiB 98.6% #################### media\n 117.3 KiB 1.3% src\n 8.0 KiB 0.1% logs\n 64 B 0.0% empty\n206 files, 8.5 MiBThe logs directory holds 200 files of about eight bytes each, so its apparent size is 8.0 KiB (the files plus the directory entries). Its disk usage is far larger, because 200 files each occupy at least one 4 KiB block: the disk-usage mode reports logs at 1.5 MiB, a hundred and ninety times the apparent size. This is exactly the case where the two measures disagree, and the one that explains most \"where did my disk go\" puzzles: a directory with a hundred thousand small files is cheap in bytes and expensive in blocks. Always show the disk usage by default and offer the other as a flag.
The demo tree hides the two cases where apparent and disk size diverge most. A second tree makes them obvious: a 1 GiB sparse file created with truncate -s 1G (all hole, no data), a 1 MB file of random bytes, and a directory of 1,000 files that each contain a single byte.
$ rdu /tmp/du_sparse -d 1\n 4.9 MiB 100.0% #################### .\n 3.9 MiB 80.3% ################ many\n1002 files, 4.9 MiB\n\n$ rdu --apparent /tmp/du_sparse -d 1\n 1.0 GiB 100.0% #################### .\n 32.3 KiB 0.0% many\n1002 files, 1.0 GiBThe same directory is 4.9 MiB by one measure and 1.0 GiB by the other. The sparse file contributes a gigabyte of apparent size and nothing at all to disk usage: stat confirms it, reporting 1,073,741,824 bytes and zero blocks. This is why a virtual machine image or a database preallocated with holes looks enormous in ls -l and costs almost nothing on disk, and why copying such a file naively with a tool that does not preserve holes fills the disk. In the other direction, the 1,000 single-byte files total 32.3 KiB of content, but each occupies a full allocation of 8 blocks of 512 bytes, 4,096 bytes, so the directory costs 3.9 MiB, which is 125 times its apparent size. du -sk agreed with the disk-usage figure, reporting 4,980 KiB for the tree. A tool that reports only apparent size would miss the many-small-files case, and one that reports only disk size would miss the sparse file; offer both, default to the one that answers the question \"what is using my disk\".
The scanner makes a small number of decisions about file types, and each one should be stated because users hit them. The table lists the cases with the behavior of rdu and the reason.
| Entry | What rdu does | Why |
|---|---|---|
| Regular file | Charges its size once | The normal case |
| Directory | Charges its own size plus its contents | A directory occupies blocks for its entries |
| Symlink | Charges the link itself and does not follow it | Following can loop or double-count |
| Hard link, second name | Charges nothing; the first path in sorted order owns the bytes | Bytes exist once |
| Device or socket | Charges the metadata size, usually zero | They hold no data |
| Unreadable directory | Reports an error, continues | One failure should not hide the rest |
Real tools expose some of these as options: du -L follows symlinks, -l counts hard links every time they appear, and -x refuses to cross file systems. Which to choose depends on the question. If you are asking \"how much would I free by deleting this directory?\", hard links to files that live elsewhere do not count, because deleting this name leaves the bytes in place. That is a harder question than it looks: the correct answer for one directory depends on what else links to its files, which a single scan from that directory cannot know. A faithful \"what would deleting free\" mode scans the whole file system, counts links per inode globally and charges a directory only for inodes whose every link lies inside it.
The scan is a recursive function that returns an Entry for each path: its total size, the number of files below it and, for directories, the child entries. It uses symlink_metadata, which does not follow symlinks, because following them would loop on a link that points at its own parent and would double-count anything reachable twice.
use std::collections::HashSet;\nuse std::fs;\nuse std::os::unix::fs::MetadataExt;\nuse std::path::{Path, PathBuf};\nuse std::sync::Mutex;\n\nuse rayon::prelude::*;\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum Measure {\n /// Bytes of content, what `ls -l` shows.\n Apparent,\n /// Bytes allocated on disk, what `du` shows: whole blocks, so small files cost a block each.\n Disk,\n}\n\n#[derive(Debug)]\npub struct Entry {\n pub path: PathBuf,\n pub size: u64,\n pub files: u64,\n pub children: Vec<Entry>,\n}\n\n#[derive(Default)]\npub struct Stats {\n pub errors: Mutex<Vec<String>>,\n /// Files with more than one link. They are not charged during the scan, because threads reach\n /// them in a different order each run. `settle` charges each one to its first path, sorted.\n hard_links: Mutex<Vec<(u64, u64, PathBuf, u64)>>,\n}\n\n/// Charge every hard-linked file exactly once, to the path that sorts first, so the report does\n/// not depend on which thread got there first.\npub fn settle(root: &mut Entry, stats: &Stats) {\n let mut claims = std::mem::take(&mut *stats.hard_links.lock().unwrap());\n claims.sort_by(|a, b| a.2.cmp(&b.2));\n let mut charged = HashSet::new();\n for (dev, ino, path, size) in claims {\n if charged.insert((dev, ino)) {\n add_to_ancestors(root, &path, size);\n }\n }\n}\n\nfn add_to_ancestors(entry: &mut Entry, path: &Path, size: u64) {\n if path.starts_with(&entry.path) {\n entry.size += size;\n for child in &mut entry.children {\n add_to_ancestors(child, path, size);\n }\n entry.children.sort_by(|a, b| b.size.cmp(&a.size));\n }\n}\n\nfn measure(md: &fs::Metadata, how: Measure) -> u64 {\n match how {\n Measure::Apparent => md.len(),\n Measure::Disk => md.blocks() * 512,\n }\n}\n\n/// Total size of `path`, with one `Entry` per directory down to `depth` levels.\npub fn scan(path: &Path, how: Measure, depth: usize, stats: &Stats) -> Entry {\n let md = match fs::symlink_metadata(path) {\n Ok(md) => md,\n Err(e) => {\n stats.errors.lock().unwrap().push(format!(\"{}: {e}\", path.display()));\n return Entry { path: path.into(), size: 0, files: 0, children: vec![] };\n }\n };\n // A symlink is counted as the link itself and never followed, which avoids loops.\n if !md.is_dir() {\n let size = measure(&md, how);\n if md.nlink() > 1 && !md.file_type().is_symlink() {\n stats.hard_links.lock().unwrap().push((md.dev(), md.ino(), path.to_path_buf(), size));\n return Entry { path: path.into(), size: 0, files: 1, children: vec![] };\n }\n return Entry { path: path.into(), size, files: 1, children: vec![] };\n }\n\n let names: Vec<PathBuf> = match fs::read_dir(path) {\n Ok(rd) => rd.filter_map(|e| e.ok()).map(|e| e.path()).collect(),\n Err(e) => {\n stats.errors.lock().unwrap().push(format!(\"{}: {e}\", path.display()));\n vec![]\n }\n };\n let mut kids: Vec<Entry> = names.par_iter().map(|p| scan(p, how, depth.saturating_sub(1), stats)).collect();\n let size = measure(&md, how) + kids.iter().map(|k| k.size).sum::<u64>();\n let files = kids.iter().map(|k| k.files).sum();\n // Keep only directories as children, and only down to the requested depth.\n kids.retain(|k| !k.children.is_empty() || fs::symlink_metadata(&k.path).is_ok_and(|m| m.is_dir()));\n if depth == 0 {\n kids.clear();\n }\n kids.sort_by(|a, b| b.size.cmp(&a.size));\n Entry { path: path.into(), size, files, children: kids }\n}\n\npub fn human(bytes: u64) -> String {\n const UNITS: [&str; 5] = [\"B\", \"KiB\", \"MiB\", \"GiB\", \"TiB\"];\n let mut value = bytes as f64;\n let mut unit = 0;\n while value >= 1024.0 && unit < UNITS.len() - 1 {\n value /= 1024.0;\n unit += 1;\n }\n if unit == 0 { format!(\"{bytes} B\") } else { format!(\"{value:.1} {}\", UNITS[unit]) }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n use std::os::unix::fs::symlink;\n\n fn tree() -> PathBuf {\n static NEXT: std::sync::atomic::AtomicU32 = std::sync::atomic::AtomicU32::new(0);\n let n = NEXT.fetch_add(1, std::sync::atomic::Ordering::Relaxed);\n let root = std::env::temp_dir().join(format!(\"rdu-test-{}-{n}\", std::process::id()));\n let _ = fs::remove_dir_all(&root);\n fs::create_dir_all(root.join(\"a\")).unwrap();\n fs::create_dir_all(root.join(\"b\")).unwrap();\n root\n }\n\n fn run(root: &Path, how: Measure) -> Entry {\n let stats = Stats::default();\n let mut e = scan(root, how, 1, &stats);\n settle(&mut e, &stats);\n e\n }\n\n #[test]\n fn apparent_size_adds_up_file_contents() {\n let root = tree();\n fs::write(root.join(\"a/one\"), vec![0u8; 1000]).unwrap();\n fs::write(root.join(\"b/two\"), vec![0u8; 2500]).unwrap();\n let e = run(&root, Measure::Apparent);\n let dirs = fs::symlink_metadata(&root).unwrap().len() + fs::symlink_metadata(root.join(\"a\")).unwrap().len() + fs::symlink_metadata(root.join(\"b\")).unwrap().len();\n assert_eq!(e.size, 3500 + dirs);\n assert_eq!(e.files, 2);\n fs::remove_dir_all(root).unwrap();\n }\n\n #[test]\n fn a_hard_link_is_charged_once_and_to_the_first_path() {\n let root = tree();\n fs::write(root.join(\"b/big\"), vec![0u8; 100_000]).unwrap();\n fs::hard_link(root.join(\"b/big\"), root.join(\"a/alias\")).unwrap();\n for _ in 0..20 {\n let e = run(&root, Measure::Apparent);\n let a = e.children.iter().find(|c| c.path.ends_with(\"a\")).unwrap();\n let b = e.children.iter().find(|c| c.path.ends_with(\"b\")).unwrap();\n assert!(a.size >= 100_000, \"the first sorted path owns the bytes\");\n assert!(b.size < 100_000);\n }\n fs::remove_dir_all(root).unwrap();\n }\n\n #[test]\n fn a_symlink_loop_terminates_and_is_not_followed() {\n let root = tree();\n symlink(&root, root.join(\"a/loop\")).unwrap();\n let e = run(&root, Measure::Apparent);\n assert_eq!(e.files, 1); // only the link itself\n fs::remove_dir_all(root).unwrap();\n }\n\n #[test]\n fn an_unreadable_directory_is_reported_not_fatal() {\n use std::os::unix::fs::PermissionsExt;\n let root = tree();\n fs::write(root.join(\"a/ok\"), b\"x\").unwrap();\n fs::set_permissions(root.join(\"b\"), fs::Permissions::from_mode(0o000)).unwrap();\n let stats = Stats::default();\n let e = scan(&root, Measure::Apparent, 1, &stats);\n assert_eq!(e.files, 1);\n assert_eq!(stats.errors.lock().unwrap().len(), 1);\n fs::set_permissions(root.join(\"b\"), fs::Permissions::from_mode(0o755)).unwrap();\n fs::remove_dir_all(root).unwrap();\n }\n}Several choices in that listing are the substance of the tool.
\nA symlink is counted as the link itself, a few dozen bytes, and never followed. A directory is read with read_dir, and an entry that fails to read is recorded in a shared error list and treated as empty, so one permission problem cannot abort a whole scan. Failure is information to report at the end, not an exception.
The directory's own size is added to the total of its children, which is what du does: a directory is a file with contents of its own. On this APFS volume empty directories have no blocks, so empty reports 0 B in disk mode and 64 B apparent.
Each directory scans its children in parallel with par_iter from rayon, which hands the children to a thread pool sized to the number of cores. Because scan is a pure recursive function that returns values, the parallelism is a one-word change from iter to par_iter. The only shared state is the Stats object, behind a Mutex, and it is touched only on errors and for hard links, both rare.
The first version handled hard links the obvious way. A file can have several names, two directory entries pointing at one inode, and du charges its bytes once. The scanner kept a HashSet of (device, inode) pairs and counted a file only when it was inserted for the first time. That is correct for the total. Run the tool twelve times on the demo tree, in which media/video.bin (800 KB) has a hard link in logs, and look at how the two directories were charged:
1 7.6 MiB media 1.5 MiB logs\n 11 8.4 MiB media 800.0 KiB logsEleven runs charged the 800 KB file to media, and one charged it to logs. The totals were always equal, 9.3 MiB, but the per-directory numbers changed from one run to the next. The cause is the thread pool. Both directories are scanned concurrently, and whichever thread reached the shared inode first inserted it and paid. This is a classic parallel-programming trap: nothing is wrong with the locking, and the result is still not a function of the input.
du has no such problem because it walks the tree in order, so the file is charged to whichever path comes first in its traversal. In the sample, that is logs: du reported logs at 1,584 KiB and media at 7,816 KiB, the same split as the corrected tool.
The fix makes the choice explicit. Files with more than one link are not charged during the scan; their claims are recorded in Stats. After the scan, settle sorts the claims by path, keeps the first claim for each inode, and adds the bytes to that path and every ancestor directory's entry:
pub fn settle(root: &mut Entry, stats: &Stats) {\n let mut claims = std::mem::take(&mut *stats.hard_links.lock().unwrap());\n claims.sort_by(|a, b| a.2.cmp(&b.2));\n let mut charged = HashSet::new();\n for (dev, ino, path, size) in claims {\n if charged.insert((dev, ino)) {\n add_to_ancestors(root, &path, size);\n }\n }\n}With that change, thirty consecutive runs produced one result, 7.6 MiB media and 1.5 MiB logs, matching du. The principle is worth stating plainly: when the order of work is not deterministic, make the rule that depends on order explicit and deterministic, here \"the path that sorts first\". Do not rely on timing being stable.
The report prints each directory with its size, share of the total and a bar scaled to 5 percent per #. Sorting by size puts the answer at the top:
9.3 MiB 100.0% #################### .\n 7.6 MiB 82.1% ################ media\n 7.6 MiB 82.1% ################ media/photos\n 1.5 MiB 16.6% ### logs\n 120.0 KiB 1.3% src\n 0 B 0.0% empty\n206 files, 9.3 MiBWith -d 2, the tool descends two levels and shows that media/photos holds nearly all of media. The human function formats with binary units, KiB, MiB and GiB, because the file system allocates in powers of two and because KB is ambiguous between 1,000 and 1,024 bytes. The final line gives the file count, which for the sample is 206: 200 logs, three media files, the hard link and the source file, with the symlink counted once.
The program exits with status 1 if any entry could not be read, and prints the first failure to standard error. A scan of a directory with a locked subdirectory reports the readable part and then complains:
\n 4.0 KiB 100.0% #################### .\n 4.0 KiB 100.0% #################### open\n 0 B 0.0% locked\n1 files, 4.0 KiB\nrdu: 1 entries could not be read, for example /tmp/du_perm/locked: Permission denied (os error 13)A report that silently omits what it could not read is worse than none, because the totals look authoritative. The error line and the exit status make the gap visible to a script.
\nA tool like this is only trustworthy once it agrees with the established one on real data. The test tree was a Cargo target directory of 25,350 files. du -sk reported 10,959,208 KiB, which is 10.45 GiB; rdu reported 10.5 GiB (it rounds to one decimal place), and the file count of 25,350 matched find -type f | wc -l. Speed, measured with time:
rdu with the thread pool: 0.10 to 0.11 s (one 0.17 s run while the cache warmed).rdu with iter in place of par_iter: 0.20 s.du -sk: 0.09 to 0.10 s.The parallel scan is twice as fast as the sequential one on this machine, and equal to du within noise. The directory metadata was in the operating system's cache, so these are warm-cache numbers; on a cold disk or a network file system, where each stat waits, parallelism helps more, because threads overlap the waiting. The lesson matches the one in building a fast file finder: walking a tree is dominated by system calls, and the cheap win is overlapping them.
The four unit tests build a real directory tree under the system temp directory and assert the properties that went wrong or could go wrong:
\ntest result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00sOne checks that apparent sizes add up, including the directories' own sizes. One creates a hard link and runs the scan twenty times, asserting each time that the directory sorting first owns the bytes: the regression test for the nondeterminism. One creates a symlink that points at its own parent and asserts the scan terminates and counts only the link. One removes all permissions from a subdirectory and asserts that the scan finishes, reports one error, and counts the readable part. Each fixture cleans up after itself, uses the process id in its name so parallel test runs do not collide, and restores permissions before deleting.
\nThe natural next steps each teach something. A -x flag that stays on one file system compares dev from MetadataExt with the root's and skips directories on other devices, so a scan of / does not wander into network mounts. An --exclude pattern filters paths with the globset crate before descending. A --json output serializes the Entry tree with serde for other tools. An interactive browser built with ratatui (a terminal UI guide covers the loop) lets you move into a directory, sort differently and press a key to delete. And on Linux, reading getdents directly with the rustix crate avoids allocating a path for every entry and makes the scan notably faster on huge trees.
For a one-off, du -sh * | sort -h is faster to type than any program, and ncdu has a polished interface and years of edge-case fixes. Write your own when you need the numbers inside another tool, as a library function in a deployment script or a service that watches a cache directory, or when you need a policy du lacks: charging by owner, grouping by extension, ignoring a build directory. The structure here, a pure recursive function with a small shared error log, a deterministic rule for the shared inode, and a test for each sharp edge, is the one to copy.
As written, rdu is one binary with the scanner in a module. The first request after it works is \"can I call this from another program\", and the structure to adopt before that arrives is a library crate with a thin binary: move scan.rs into src/lib.rs, keep the command line in src/main.rs, and let the binary depend on the library like any other crate. The tests move with the scanner, and a second consumer, such as a monitoring service that alerts when a cache directory passes a size, gets scan, settle and Measure without the argument parser.
Two API choices pay off then. Return the error list as part of the result instead of collecting it in a shared Stats that callers must create; a function that returns (Entry, Vec<String>) is harder to misuse than one that needs a mutable accumulator and a separate call to settle. And hide the two-step scan then settle behind one function, scan_tree, so no caller can forget the second step and get the racy answer the first version produced. A bug you fixed once should be impossible to reintroduce by calling the API wrongly, which is a design rule worth applying to every library in this series, and the one that makes the earlier regression test, twenty runs asserting one answer, a safety net and not the only defense.
A fuzzy finder is a small program that changed how people use a terminal. You pipe it a list, a thousand file names, your command history, a hundred branches, type a few letters from anywhere in the item you want, and it narrows the list live and hands back your choice. fzf made this a habit. Building one is a good Rust project because it splits cleanly into two halves that need different skills: an engine that ranks items against a query, and a terminal interface that draws results and reacts to keys. The engine is pure and testable. The interface is where the platform surprises live, and this guide hits one of them on macOS.
The finder here, rfz, uses nucleo-matcher 0.3.1 for scoring and crossterm 0.29.0 for the terminal. It was built and tested on Rust 1.99 (edition 2024) on macOS. The interface was exercised end to end through a pseudo-terminal, not by hand, and every output below is real.
A fuzzy match asks whether the characters of the query appear in the candidate in order, not necessarily next to each other. The query smr matches src/main.rs because s, m and r all occur, in that order. That is a subsequence match, and by itself it is too loose to be useful: it matches nearly everything long enough. The value is in the ranking, which scores a match higher when it looks like what a person meant. Consecutive characters score more than scattered ones. A match at the start of a word, after a /, _ or -, or at a capital letter inside camelCase, scores more than one in the middle of a word. A match in the file name scores more than one in a directory. Shorter candidates win ties.
You do not write that scoring. nucleo-matcher implements it, the same family of algorithm that powers the Helix editor, and exposes it through Pattern and Matcher. What you write is everything around it, and that is enough work to fill an evening.
The engine takes a list of strings and a query and returns ranked hits, each carrying the index of the item, its score and the positions of the matched characters, which the interface needs for highlighting.
\n//! The engine of a fuzzy finder: matching, ranking and the state a UI drives.\n\nuse crossterm::event::{KeyCode, KeyEvent, KeyModifiers};\nuse nucleo_matcher::pattern::{CaseMatching, Normalization, Pattern};\nuse nucleo_matcher::{Config, Matcher, Utf32Str};\n\n/// One ranked result: the index of the item, its score and the matched character positions.\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Hit {\n pub index: usize,\n pub score: u32,\n pub positions: Vec<u32>,\n}\n\npub struct Finder {\n matcher: Matcher,\n items: Vec<String>,\n}\n\nimpl Finder {\n pub fn new(items: Vec<String>) -> Self {\n // `match_paths` gives bonuses for matches after `/` and at the start of a file name.\n Finder { matcher: Matcher::new(Config::DEFAULT.match_paths()), items }\n }\n\n pub fn len(&self) -> usize {\n self.items.len()\n }\n\n pub fn item(&self, index: usize) -> &str {\n &self.items[index]\n }\n\n /// Rank every item against the query, best first. An empty query matches everything in order.\n pub fn search(&mut self, query: &str, limit: usize) -> Vec<Hit> {\n if query.is_empty() {\n return (0..self.items.len().min(limit)).map(|index| Hit { index, score: 0, positions: vec![] }).collect();\n }\n let pattern = Pattern::parse(query, CaseMatching::Smart, Normalization::Smart);\n let mut buf = Vec::new();\n let mut hits = Vec::new();\n for (index, item) in self.items.iter().enumerate() {\n let haystack = Utf32Str::new(item, &mut buf);\n let mut positions = Vec::new();\n if let Some(score) = pattern.indices(haystack, &mut self.matcher, &mut positions) {\n positions.sort_unstable();\n positions.dedup();\n hits.push(Hit { index, score, positions });\n }\n }\n // Higher score first; ties keep the original order, which makes results stable.\n hits.sort_by(|a, b| b.score.cmp(&a.score).then(a.index.cmp(&b.index)));\n hits.truncate(limit);\n hits\n }\n}\n\nimpl Finder {\n /// Rank only the items named in `candidates`. When a query grows by one character, every\n /// match must already be among the previous hits, so the search can start from those.\n pub fn search_among(&mut self, query: &str, candidates: &[usize], limit: usize) -> Vec<Hit> {\n let pattern = Pattern::parse(query, CaseMatching::Smart, Normalization::Smart);\n let mut buf = Vec::new();\n let mut hits = Vec::new();\n for &index in candidates {\n let haystack = Utf32Str::new(&self.items[index], &mut buf);\n let mut positions = Vec::new();\n if let Some(score) = pattern.indices(haystack, &mut self.matcher, &mut positions) {\n positions.sort_unstable();\n positions.dedup();\n hits.push(Hit { index, score, positions });\n }\n }\n hits.sort_by(|a, b| b.score.cmp(&a.score).then(a.index.cmp(&b.index)));\n hits.truncate(limit);\n hits\n }\n\n /// Same result as `search`, using every core: split the items, rank each chunk with its own\n /// matcher, and merge the best.\n pub fn search_parallel(&self, query: &str, limit: usize, threads: usize) -> Vec<Hit> {\n let pattern = Pattern::parse(query, CaseMatching::Smart, Normalization::Smart);\n let chunk = self.items.len().div_ceil(threads.max(1));\n let mut all: Vec<Hit> = std::thread::scope(|scope| {\n let handles: Vec<_> = self\n .items\n .chunks(chunk.max(1))\n .enumerate()\n .map(|(n, slice)| {\n let pattern = &pattern;\n scope.spawn(move || {\n let mut matcher = Matcher::new(Config::DEFAULT.match_paths());\n let mut buf = Vec::new();\n let mut local = Vec::new();\n for (offset, item) in slice.iter().enumerate() {\n let mut positions = Vec::new();\n if let Some(score) = pattern.indices(Utf32Str::new(item, &mut buf), &mut matcher, &mut positions) {\n positions.sort_unstable();\n positions.dedup();\n local.push(Hit { index: n * chunk + offset, score, positions });\n }\n }\n local.sort_by(|a, b| b.score.cmp(&a.score).then(a.index.cmp(&b.index)));\n local.truncate(limit);\n local\n })\n })\n .collect();\n handles.into_iter().flat_map(|h| h.join().unwrap()).collect()\n });\n all.sort_by(|a, b| b.score.cmp(&a.score).then(a.index.cmp(&b.index)));\n all.truncate(limit);\n all\n }\n}\n\n/// What the UI loop should do after a key press.\n#[derive(Debug, PartialEq, Eq)]\npub enum Action {\n None,\n Quit,\n Accept(usize),\n}\n\n/// The state behind the screen: a query, the ranked results and a selected row.\npub struct App {\n pub finder: Finder,\n pub query: String,\n pub hits: Vec<Hit>,\n pub selected: usize,\n limit: usize,\n}\n\nimpl App {\n pub fn new(items: Vec<String>, limit: usize) -> Self {\n let mut app = App { finder: Finder::new(items), query: String::new(), hits: vec![], selected: 0, limit };\n app.refresh();\n app\n }\n\n fn refresh(&mut self) {\n self.hits = self.finder.search(&self.query, self.limit);\n self.selected = self.selected.min(self.hits.len().saturating_sub(1));\n }\n\n pub fn on_key(&mut self, key: KeyEvent) -> Action {\n match (key.code, key.modifiers) {\n (KeyCode::Esc, _) | (KeyCode::Char('c'), KeyModifiers::CONTROL) => Action::Quit,\n (KeyCode::Enter, _) => match self.hits.get(self.selected) {\n Some(hit) => Action::Accept(hit.index),\n None => Action::None,\n },\n (KeyCode::Up, _) | (KeyCode::Char('p'), KeyModifiers::CONTROL) => {\n self.selected = self.selected.saturating_sub(1);\n Action::None\n }\n (KeyCode::Down, _) | (KeyCode::Char('n'), KeyModifiers::CONTROL) => {\n if self.selected + 1 < self.hits.len() {\n self.selected += 1;\n }\n Action::None\n }\n (KeyCode::Backspace, _) => {\n self.query.pop();\n self.selected = 0;\n self.refresh();\n Action::None\n }\n (KeyCode::Char(c), m) if !m.contains(KeyModifiers::CONTROL) => {\n self.query.push(c);\n self.selected = 0;\n self.refresh();\n Action::None\n }\n _ => Action::None,\n }\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n fn paths() -> Vec<String> {\n [\"src/main.rs\", \"src/matcher/mod.rs\", \"src/ui/mod.rs\", \"README.md\", \"tests/main_test.rs\", \"docs/Main.md\", \"Cargo.toml\"]\n .iter()\n .map(|s| s.to_string())\n .collect()\n }\n\n fn key(c: KeyCode) -> KeyEvent {\n KeyEvent::new(c, KeyModifiers::NONE)\n }\n\n #[test]\n fn an_empty_query_lists_everything_in_order() {\n let mut f = Finder::new(paths());\n let hits = f.search(\"\", 100);\n assert_eq!(hits.len(), 7);\n assert_eq!(hits[0].index, 0);\n }\n\n #[test]\n fn subsequence_matches_rank_the_file_name_hit_first() {\n let mut f = Finder::new(paths());\n let hits = f.search(\"main\", 100);\n let top = f.item(hits[0].index);\n assert_eq!(top, \"src/main.rs\");\n assert!(hits.iter().any(|h| f.item(h.index) == \"tests/main_test.rs\"));\n }\n\n #[test]\n fn letters_must_appear_in_order_but_need_not_touch() {\n let mut f = Finder::new(paths());\n let found: Vec<_> = f.search(\"smr\", 10).iter().map(|h| f.item(h.index).to_string()).collect();\n assert!(found.contains(&\"src/main.rs\".to_string()));\n assert!(f.search(\"nms\", 10).iter().all(|h| f.item(h.index) != \"src/main.rs\"));\n }\n\n #[test]\n fn smart_case_is_insensitive_until_you_type_a_capital() {\n let mut f = Finder::new(paths());\n assert!(f.search(\"main\", 10).iter().any(|h| f.item(h.index) == \"docs/Main.md\"));\n let upper: Vec<_> = f.search(\"Main\", 10).iter().map(|h| f.item(h.index).to_string()).collect();\n assert_eq!(upper, [\"docs/Main.md\"]);\n }\n\n #[test]\n fn positions_point_at_the_matched_characters() {\n let mut f = Finder::new(vec![\"src/main.rs\".into()]);\n let hit = &f.search(\"mn\", 1)[0];\n let chars: Vec<char> = \"src/main.rs\".chars().collect();\n let matched: String = hit.positions.iter().map(|&p| chars[p as usize]).collect();\n assert_eq!(matched, \"mn\");\n }\n\n #[test]\n fn keys_edit_the_query_move_the_selection_and_accept() {\n let mut app = App::new(paths(), 50);\n for c in \"main\".chars() {\n assert_eq!(app.on_key(key(KeyCode::Char(c))), Action::None);\n }\n assert_eq!(app.query, \"main\");\n app.on_key(key(KeyCode::Down));\n assert_eq!(app.selected, 1);\n app.on_key(key(KeyCode::Up));\n app.on_key(key(KeyCode::Up)); // already at the top: stays there\n assert_eq!(app.selected, 0);\n let first = app.hits[0].index;\n assert_eq!(app.on_key(key(KeyCode::Enter)), Action::Accept(first));\n app.on_key(key(KeyCode::Backspace));\n assert_eq!(app.query, \"mai\");\n assert_eq!(app.on_key(key(KeyCode::Esc)), Action::Quit);\n assert_eq!(app.on_key(KeyEvent::new(KeyCode::Char('c'), KeyModifiers::CONTROL)), Action::Quit);\n }\n\n #[test]\n fn narrowing_and_parallel_search_agree_with_the_full_search() {\n let items: Vec<String> = (0..2000).map(|i| format!(\"dir{}/file{}_{}.rs\", i % 17, i, i * 7)).collect();\n let mut f = Finder::new(items);\n let full = f.search(\"fil1\", 50);\n let before = f.search(\"fil\", usize::MAX);\n let candidates: Vec<usize> = before.iter().map(|h| h.index).collect();\n assert_eq!(f.search_among(\"fil1\", &candidates, 50), full);\n assert_eq!(f.search_parallel(\"fil1\", 50, 4), full);\n }\n\n #[test]\n fn accepting_with_no_results_does_nothing() {\n let mut app = App::new(paths(), 50);\n for c in \"zzzzzz\".chars() {\n app.on_key(key(KeyCode::Char(c)));\n }\n assert!(app.hits.is_empty());\n assert_eq!(app.on_key(key(KeyCode::Enter)), Action::None);\n }\n}Several choices in Finder::search are worth noticing. The matcher is configured with Config::DEFAULT.match_paths(), which adds bonuses for matches after a path separator, so main prefers src/main.rs over src/domain/model.rs. Pattern::parse with CaseMatching::Smart implements the convention users expect from fzf and ripgrep: the match ignores case until the query contains a capital letter, and then it becomes case-sensitive. An empty query returns everything in the original order, so the screen is useful before the first keystroke. Results sort by score, highest first, with the original index as a tiebreaker, which makes the output stable: two items with the same score always appear in the order they were supplied, so the list does not shuffle as you type. And the search returns at most limit hits, because a screen shows 40 lines, not 250,000.
Run on a small list of file names, the engine ranks like this:
\nquery \"main\"\n 109 src/[m][a][i][n].rs\n 109 tests/[m][a][i][n]_test.rs\n 109 docs/[M][a][i][n].md\n 76 src/do[m][a][i][n]/model.rs\nquery \"smr\"\n 72 [s]rc/[m]ain.[r]s\n 70 [s]rc/ui/[m]od.[r]s\n 65 [s]rc/[m]atcher/mod.[r]s\n 64 [s]rc/domain/[m]odel.[r]s\nquery \"Main\"\n 109 docs/[M][a][i][n].md\nquery \"mod\"\n 84 src/matcher/[m][o][d].rs\n 84 src/ui/[m][o][d].rs\n 84 src/domain/[m][o][d]el.rs\nquery \"cargo\"\n 134 [C][a][r][g][o].tomlThe brackets mark the characters that matched, which is the positions field. Read the results the way a user would. For main, the three files whose names are main score 109 each and tie, and src/domain/model.rs, where the letters of main appear inside the word domain, scores 76. The scattered query smr finds src/main.rs first, with matches at the start of src, the start of main and the start of the extension. The capital in Main switches off case-insensitivity, so only docs/Main.md qualifies. For mod, three files score the same 84, and the stable sort keeps them in input order. The cargo query puts the exact word first at 134.
Because the engine is a plain struct with no terminal in it, its behavior is easy to specify. Eight tests cover it. Here are the two that expose the most:
\n#[test]\nfn letters_must_appear_in_order_but_need_not_touch() {\n let mut f = Finder::new(paths());\n let found: Vec<_> = f.search(\"smr\", 10).iter().map(|h| f.item(h.index).to_string()).collect();\n assert!(found.contains(&\"src/main.rs\".to_string()));\n assert!(f.search(\"nms\", 10).iter().all(|h| f.item(h.index) != \"src/main.rs\"));\n}\n\n#[test]\nfn positions_point_at_the_matched_characters() {\n let mut f = Finder::new(vec![\"src/main.rs\".into()]);\n let hit = &f.search(\"mn\", 1)[0];\n let chars: Vec<char> = \"src/main.rs\".chars().collect();\n let matched: String = hit.positions.iter().map(|&p| chars[p as usize]).collect();\n assert_eq!(matched, \"mn\");\n}test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02sThe first test originally used the out-of-order query rms and failed, and it is a good story about fuzzy matching. The test claimed rms should not match src/main.rs, since m comes before r in the word main. But the subsequence is r from src, then m from main, then s from rs, which is in order. The matcher was right and the test was wrong. A query that truly cannot match is nms: the only m in the path comes before the only n, so no m follows an n. When you write tests for a fuzzy matcher, check by hand that your negative examples are genuinely impossible.
The interface separates what happens from how it looks. App holds the query, the ranked hits and the selected row, and on_key takes a key event and returns an Action: nothing, quit, or accept a particular item.
pub fn on_key(&mut self, key: KeyEvent) -> Action {\n match (key.code, key.modifiers) {\n (KeyCode::Esc, _) | (KeyCode::Char('c'), KeyModifiers::CONTROL) => Action::Quit,\n (KeyCode::Enter, _) => match self.hits.get(self.selected) {\n Some(hit) => Action::Accept(hit.index),\n None => Action::None,\n },\n (KeyCode::Up, _) | (KeyCode::Char('p'), KeyModifiers::CONTROL) => { /* move up */ Action::None }\n (KeyCode::Backspace, _) => { self.query.pop(); self.selected = 0; self.refresh(); Action::None }\n (KeyCode::Char(c), m) if !m.contains(KeyModifiers::CONTROL) => {\n self.query.push(c);\n self.selected = 0;\n self.refresh();\n Action::None\n }\n _ => Action::None,\n }\n}Because KeyEvent is a plain value you can construct in a test, the whole interaction is testable with no terminal. The test keys_edit_the_query_move_the_selection_and_accept types main, presses Down, Up, Up (which stops at the top), Enter, Backspace, Escape and Ctrl-C, asserting the state after each. Another test checks that pressing Enter with no results does nothing instead of accepting a non-existent item. The state machine is the place where most bugs in an interactive tool live, and having it free of I/O means you find them in milliseconds.
Two details of the design are worth stating. The selection resets to the top whenever the query changes, because the old position is meaningless in a new list, and refresh clamps it into range when the hits shrink. And Control-modified letters are not typed into the query: the guard !m.contains(KeyModifiers::CONTROL) is what keeps Ctrl-N and Ctrl-P as navigation instead of inserting an n or a p.
The interface is thin. It switches the terminal to raw mode and the alternate screen, runs a loop that draws and then waits for a key, and restores the terminal on exit. One design choice makes it usable in a shell. The program draws on standard error and prints the chosen item on standard output, so a shell can capture the selection while the interface still appears on screen:
\nuse std::io::{self, BufRead, IsTerminal, Write};\nuse std::process::ExitCode;\nuse std::time::Duration;\n\nuse crossterm::cursor::{Hide, MoveTo, Show};\nuse crossterm::event::{self, Event, KeyEventKind};\nuse crossterm::style::{Attribute, Print, SetAttribute};\nuse crossterm::terminal::{self, Clear, ClearType, EnterAlternateScreen, LeaveAlternateScreen};\nuse crossterm::{execute, queue};\nuse rfz::{Action, App, Hit};\n\n/// Draw the prompt and the visible results. The screen is written to stderr so that the chosen\n/// item can go to stdout and be captured by a shell: `vim \"$(find . -type f | rfz)\"`.\nfn draw(out: &mut impl Write, app: &App, rows: u16) -> io::Result<()> {\n queue!(out, MoveTo(0, 0), Clear(ClearType::All), Print(format!(\"> {}\", app.query)))?;\n for (row, hit) in app.hits.iter().take(rows.saturating_sub(2) as usize).enumerate() {\n queue!(out, MoveTo(0, row as u16 + 1))?;\n queue!(out, Print(if row == app.selected { \"> \" } else { \" \" }))?;\n print_highlighted(out, app.finder.item(hit.index), hit)?;\n }\n out.flush()\n}\n\nfn print_highlighted(out: &mut impl Write, text: &str, hit: &Hit) -> io::Result<()> {\n for (i, ch) in text.chars().enumerate() {\n if hit.positions.contains(&(i as u32)) {\n queue!(out, SetAttribute(Attribute::Bold), Print(ch), SetAttribute(Attribute::Reset))?;\n } else {\n queue!(out, Print(ch))?;\n }\n }\n Ok(())\n}\n\nfn run(items: Vec<String>) -> io::Result<Option<String>> {\n let mut app = App::new(items, 200);\n let mut screen = io::stderr();\n terminal::enable_raw_mode()?;\n execute!(screen, EnterAlternateScreen, Hide)?;\n let result = (|| -> io::Result<Option<String>> {\n loop {\n let (_, rows) = terminal::size()?;\n draw(&mut screen, &app, rows)?;\n if !event::poll(Duration::from_millis(250))? {\n continue;\n }\n if let Event::Key(key) = event::read()? {\n if key.kind != KeyEventKind::Press {\n continue;\n }\n match app.on_key(key) {\n Action::Quit => return Ok(None),\n Action::Accept(i) => return Ok(Some(app.finder.item(i).to_string())),\n Action::None => {}\n }\n }\n }\n })();\n // Restore the terminal on every path, including errors.\n execute!(screen, Show, LeaveAlternateScreen)?;\n terminal::disable_raw_mode()?;\n result\n}\n\nfn main() -> ExitCode {\n if io::stdin().is_terminal() {\n eprintln!(\"rfz: pipe a list of candidates into it, for example: find . -type f | rfz\");\n return ExitCode::from(2);\n }\n let items: Vec<String> = io::stdin().lock().lines().map_while(Result::ok).filter(|l| !l.is_empty()).collect();\n match run(items) {\n Ok(Some(choice)) => {\n println!(\"{choice}\");\n ExitCode::SUCCESS\n }\n Ok(None) => ExitCode::from(130),\n Err(e) => {\n eprintln!(\"rfz: {e}\");\n ExitCode::from(2)\n }\n }\n}That is the same trick fzf uses, and it makes this idiom work: vim \"$(find . -type f | rfz)\". Standard input carries the candidates, standard output carries the result, and the terminal itself, opened separately, carries the interaction. The exit status distinguishes the cases: 0 for a choice, 130 for a cancel, matching the shell convention for an interrupt, and 2 for errors. The restore call runs after the inner loop returns whether it succeeded or failed, so a failure never leaves the user's terminal in raw mode; a terminal left in raw mode looks like a shell that has stopped echoing your typing.
The first build of this program compiled, passed its tests, and failed the first time it ran as a real finder, which is the only time the problem can show. Run under a pseudo-terminal with candidates piped into it, it printed:
\nrfz: Failed to initialize input readerThe cause is a platform detail. A finder reads its candidates from standard input, so standard input is a pipe. Then it must read keystrokes from somewhere else, and the only place left is the controlling terminal, /dev/tty. On Linux, crossterm reads keys from the terminal device when stdin is not a terminal. On macOS it uses a mechanism that cannot poll /dev/tty, and initializing the event reader fails. The fix is a crate feature that opts into a different reader that opens /dev/tty directly:
[dependencies]\ncrossterm = { version = \"0.29\", features = [\"use-dev-tty\", \"libc\"] }\nnucleo-matcher = \"0.3\"\nanyhow = \"1\"The lesson generalizes: a terminal interface that also consumes piped input needs a keyboard source that is independent of stdin. Always test the real combination, producer | tool, in a real terminal, because a unit test of the state machine will not catch it.
A tool that reads a terminal can be tested end to end by giving it a pseudo-terminal and playing the user. Python's pty module does it in a few lines: fork with a pty as the controlling terminal, run a shell command that pipes four candidate paths into rfz and echoes the result, send the keystrokes b, e, t, the Down arrow and Enter, and read what comes out. The run printed the screen redraws as escape sequences, eleven of them as the keys arrived, and then the shell's echo of the choice:
11 screens drawn\nchose: docs/better.mdThe candidates were src/alpha.rs, src/beta.rs, docs/better.md and README.md. Typing bet leaves two matches, src/beta.rs and docs/better.md; Down moved the selection to the second; Enter returned it. The whole chain, key decoding, query editing, matching, selection and the output on stdout captured by $(...), worked. Keep a script like this in the repository; it is the only test that exercises the terminal setup and teardown.
An interactive finder has about 50 milliseconds before the delay is noticeable. The benchmark used 253,500 paths, built by repeating a real listing of 25,350 files from a Cargo target directory ten times with different prefixes, and ran each query once from scratch in a release build on an Apple silicon laptop:
253500 paths (25350 unique shapes)\n m: 20.8ms\n main: 37.7ms\n src/lib: 23.1ms\n serdejson: 19.2ms\n xyzzy: 10.9msEvery query finished well under 50 ms on a quarter of a million paths with one thread, which is fast enough that the simple design, re-scoring everything on every keystroke, is the right one to ship first. The slowest query, main, matched the most items and so did the most scoring and sorting. A query that matches nothing is the fastest, but notice the last row: xyzzy does match something in this data, because the list contains hash-named build artifacts with long random file names, and a five-letter subsequence appears by chance in a long enough string. That is the practical weakness of fuzzy matching on long, random-looking strings, and it is why the ranking matters more than the match.
The search is linear in the number of candidates. Two standard improvements exist, and the measurements show that only one of them helps here, which is why you measure before adding either.
\nThe first is narrowing. When a new query extends the previous one by a character, every match must already be among the previous hits, so the search can start from those. search_among ranks only the indices you give it. The test narrowing_and_parallel_search_agree_with_the_full_search checks that it returns exactly what a full search returns. Typing serde one key at a time over the 253,500 paths gave these timings, with the full scan and the narrowed search measured for every prefix:
query matches full scan narrowed\ns 241440 25.8ms 27.2ms\nse 214210 120.0ms 91.4ms\nser 167100 92.2ms 93.2ms\nserd 76030 55.2ms 63.1ms\nserde 48130 46.5ms 40.8ms\nserde_ 14010 27.7ms 14.7msNarrowing barely helped. The reason is in the second column: paths are long strings, and a short query is a subsequence of almost all of them, so the letter s alone matches 241,440 of 253,500 paths. Narrowing from 241,000 to 214,000 candidates saves nothing worth having. It starts to pay only when the query becomes selective, as in the last row, where it halves the time. The takeaway is not that narrowing is useless, it is that the benefit depends on how selective the first characters are in your data. For short strings such as branch names or commands, short queries are selective and narrowing works well. For long file paths it does not.
The second improvement is parallelism, and it helps unconditionally because the work is independent per item. search_parallel splits the items into chunks, gives each thread its own Matcher (the matcher holds scratch space and cannot be shared), ranks its chunk, and merges the best from each. The five-keystroke typing sequence on the full list took:
typing s-e-r-d-e over 253500 paths\n rescan everything each key: 317 to 327 ms\n narrow from previous hits: 315 to 328 ms\n 8 threads, rescan each key: 62 to 77 msThose are totals for five keystrokes, so the parallel version spent about 13 ms per key against about 64 ms for the single-threaded scan, a speedup of roughly five on this machine. This is the improvement to reach for first. The numbers also show where the ceiling is: the merge and the final sort of all matching items are single-threaded, and with 240,000 matches that sort is a real cost. Keeping only the top limit hits per thread, as the code does, with a bounded heap instead of a full sort, is the next optimization.
A third change matters more for feel than for raw speed: do the search off the interface thread. If the search takes 60 ms, the screen should still accept keystrokes during it and drop results that were computed for a query the user has since changed. The nucleo crate packages exactly this, with a worker pool, a stream of items that can arrive while searching, and snapshots the interface reads. Move to it when your lists are large or slow to produce. For lists under 100,000 items, the single-threaded engine in this guide is already fast enough to feel instant.
Each of these teaches a distinct part of the problem. A preview pane that shows the first lines of the selected file. Multi-select with Tab, returning several lines. A --query flag to start with text already typed. Highlighting in the drawn result rows, which the code already does with bold text on the positions. Mouse support. Shell key bindings, so Ctrl-R searches history and Ctrl-T inserts a path. And a mode that reads NUL-separated items for paths containing newlines. For the other tools in this family, see a fast file finder with parallel walking, which produces the lists you pipe in, and terminal user interfaces with Ratatui, which replaces the hand-written drawing with a layout system when the screen grows beyond a list.
A finder earns its keep through the shell bindings around it. The simplest is a function that picks a file and opens it: edit() { local f; f=\"$(find . -type f -not -path './.git/*' | rfz)\" && ${EDITOR:-vi} \"$f\"; }. The && is doing real work: when the user cancels with Escape the program exits with status 130, the capture is empty, and the editor is never started. For history search, feed it fc -ln 1 and insert the choice into the command line with a key binding in your shell's line editor. For branches, git branch --format='%(refname:short)' | rfz | xargs git checkout is a one-line branch switcher. Each of these works because of the three decisions made earlier: candidates on stdin, the chosen item alone on stdout, and the interface and diagnostics on standard error, so none of the plumbing sees the screen.
Test the bindings the way you test the program: with a pseudo-terminal and a known candidate list, asserting on the text the shell function receives. A binding that works on your machine and breaks under a different shell or without a controlling terminal, such as inside a cron job or a CI container where no /dev/tty exists, should fail with a clear message and a non-zero status rather than hang waiting for keys that will never come.
Converting Markdown to HTML looks like a solved problem, and it is: call one function from a library and you are done. That is also how the interesting parts get missed. A real converter has to decide what to do with raw HTML in the input, which is a security question. It has to give headings stable anchors, rewrite links between pages, and build a table of contents, which are design questions. And it has to behave as a command line tool, reading standard input, writing to a file, and failing with a clear message. This guide builds mdhtml, a Markdown to HTML converter on top of pulldown-cmark, and spends most of its time on the decisions around the one-line conversion.
The tool compiles and runs on Rust 1.99 (edition 2024) with pulldown-cmark 0.13 and clap 4.6.7. All output blocks below are what the programs printed.
pulldown-cmark is a pull parser. It does not build a tree. It walks the document and hands you a sequence of events: start of a heading, some text, end of a heading, start of a paragraph. A tiny document makes the shape clear. This program prints the events for a heading, a paragraph with emphasis and a link, and a list:
use pulldown_cmark::{Options, Parser};\n\nfn main() {\n let markdown = \"# Title\\n\\nA *brief* [link](a.md).\\n\\n- one\\n\";\n for event in Parser::new_ext(markdown, Options::ENABLE_TABLES) {\n println!(\"{event:?}\");\n }\n}Start(Heading { level: H1, id: None, classes: [], attrs: [] })\nText(Borrowed(\"Title\"))\nEnd(Heading(H1))\nStart(Paragraph)\nText(Borrowed(\"A \"))\nStart(Emphasis)\nText(Borrowed(\"brief\"))\nEnd(Emphasis)\nText(Borrowed(\" \"))\nStart(Link { link_type: Inline, dest_url: Borrowed(\"a.md\"), title: Borrowed(\"\"), id: Borrowed(\"\") })\nText(Borrowed(\"link\"))\nEnd(Link)\nText(Borrowed(\".\"))\nEnd(Paragraph)\nStart(List(None))\nStart(Item)\nText(Borrowed(\"one\"))\nEnd(Item)\nEnd(List(false))Every event is a plain enum value, and the text is Borrowed: the parser points into your input instead of copying it. Two consequences matter. The event stream is an ordinary Rust iterator, so you can map, filter and filter_map it, which is how every customization in this guide works. And because nothing is built in memory, converting a megabyte takes a few milliseconds.
To get HTML you pass the events to html::push_html. The simplest converter is three lines:
let parser = Parser::new_ext(markdown, Options::ENABLE_TABLES | Options::ENABLE_FOOTNOTES);\nlet mut html = String::new();\npulldown_cmark::html::push_html(&mut html, parser);Options turns on extensions beyond core CommonMark: tables, footnotes, strikethrough, task lists and smart punctuation. They are off by default so that the parser matches the standard unless you ask for more.
Markdown allows raw HTML, and the parser passes it straight through. That is correct for a trusted author writing a blog and dangerous for anything else. Convert this document, which has a script tag, to see what comes out:
\n<h2 id=\"features\">Features</h2>\n...\n<script>alert('hi')</script>\n<p>Footnotes work too.<sup class=\"footnote-reference\"><a href=\"#1\">1</a></sup></p>The <script> element appears in the output exactly as written. If this text came from a user, a comment, a wiki edit or a pasted README, you have just published a cross-site scripting vulnerability. The converter must offer a safe mode, and the event stream makes one easy to build: raw HTML arrives as Event::Html for blocks and Event::InlineHtml for fragments, so a filter_map that drops both removes it.
use pulldown_cmark::{html, CowStr, Event, HeadingLevel, Options, Parser, Tag, TagEnd};\n\n#[derive(Debug, Clone, Copy, Default)]\npub struct Settings {\n /// Drop raw HTML from the input instead of passing it through.\n pub safe: bool,\n /// Rewrite relative links that end in `.md` so they point at the generated `.html` page.\n pub rewrite_md_links: bool,\n}\n\npub struct Rendered {\n pub body: String,\n pub toc: Vec<(HeadingLevel, String, String)>,\n}\n\npub fn options() -> Options {\n Options::ENABLE_TABLES\n | Options::ENABLE_FOOTNOTES\n | Options::ENABLE_STRIKETHROUGH\n | Options::ENABLE_TASKLISTS\n | Options::ENABLE_SMART_PUNCTUATION\n}\n\n/// `Rust & WebAssembly: a tour` becomes `rust-webassembly-a-tour`.\npub fn slug(text: &str) -> String {\n let mut out = String::new();\n for c in text.chars() {\n if c.is_alphanumeric() {\n out.extend(c.to_lowercase());\n } else if !out.ends_with('-') && !out.is_empty() {\n out.push('-');\n }\n }\n out.trim_end_matches('-').to_string()\n}\n\npub fn render(markdown: &str, settings: Settings) -> Rendered {\n // First pass: find the headings so each one can get an id and a table of contents entry.\n let mut toc = Vec::new();\n let mut seen: std::collections::HashMap<String, usize> = std::collections::HashMap::new();\n let mut current: Option<(HeadingLevel, String)> = None;\n for event in Parser::new_ext(markdown, options()) {\n match event {\n Event::Start(Tag::Heading { level, .. }) => current = Some((level, String::new())),\n Event::Text(t) | Event::Code(t) => {\n if let Some((_, text)) = current.as_mut() {\n text.push_str(&t);\n }\n }\n Event::End(TagEnd::Heading(_)) => {\n if let Some((level, text)) = current.take() {\n let base = slug(&text);\n // A repeated heading gets `-1`, `-2`, ... so anchors stay unique.\n let count = seen.entry(base.clone()).or_insert(0);\n let id = if *count == 0 { base } else { format!(\"{base}-{count}\") };\n *count += 1;\n toc.push((level, text, id));\n }\n }\n _ => {}\n }\n }\n\n // Second pass: transform the event stream, then hand it to the HTML writer.\n let mut next_heading = 0;\n let events = Parser::new_ext(markdown, options()).filter_map(|event| match event {\n Event::Html(_) | Event::InlineHtml(_) if settings.safe => None,\n Event::Start(Tag::Heading { level, classes, attrs, .. }) => {\n let id = toc.get(next_heading).map(|(_, _, id)| CowStr::from(id.clone()));\n next_heading += 1;\n Some(Event::Start(Tag::Heading { level, id, classes, attrs }))\n }\n Event::Start(Tag::Link { link_type, dest_url, title, id }) if settings.safe && !is_safe_url(&dest_url) => {\n Some(Event::Start(Tag::Link { link_type, dest_url: CowStr::from(\"#\"), title, id }))\n }\n Event::Start(Tag::Image { link_type, dest_url, title, id }) if settings.safe && !is_safe_url(&dest_url) => {\n Some(Event::Start(Tag::Image { link_type, dest_url: CowStr::from(\"\"), title, id }))\n }\n Event::Start(Tag::Link { link_type, dest_url, title, id }) if settings.rewrite_md_links => {\n let dest_url = match dest_url.strip_suffix(\".md\") {\n Some(stem) if !dest_url.contains(\"://\") => CowStr::from(format!(\"{stem}.html\")),\n _ => dest_url,\n };\n Some(Event::Start(Tag::Link { link_type, dest_url, title, id }))\n }\n other => Some(other),\n });\n\n let mut body = String::new();\n html::push_html(&mut body, events);\n Rendered { body, toc }\n}\n\n/// Allow web links, mail links and relative paths. Anything with another scheme, such as\n/// `javascript:` or `data:`, is replaced.\nfn is_safe_url(url: &str) -> bool {\n let trimmed = url.trim_start();\n match trimmed.split_once(':') {\n // A colon before any slash or question mark means the text before it is a scheme.\n Some((scheme, _)) if !scheme.contains(['/', '?', '#']) => {\n matches!(scheme.to_ascii_lowercase().as_str(), \"http\" | \"https\" | \"mailto\")\n }\n _ => true,\n }\n}\n\npub fn page(title: &str, rendered: &Rendered) -> String {\n let mut nav = String::new();\n for (level, text, id) in &rendered.toc {\n let depth = *level as usize;\n nav.push_str(&format!(\"<li class=\\\"h{depth}\\\"><a href=\\\"#{id}\\\">{}</a></li>\\n\", escape(text)));\n }\n format!(\n \"<!doctype html>\\n<html lang=\\\"en\\\">\\n<head>\\n<meta charset=\\\"utf-8\\\">\\n<title>{}</title>\\n</head>\\n<body>\\n<nav><ul>\\n{nav}</ul></nav>\\n<main>\\n{}</main>\\n</body>\\n</html>\\n\",\n escape(title),\n rendered.body\n )\n}\n\nfn escape(text: &str) -> String {\n text.replace('&', \"&\").replace('<', \"<\").replace('>', \">\").replace('\"', \""\")\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn slugs_drop_punctuation_and_collapse_runs() {\n assert_eq!(slug(\"Rust & WebAssembly: a tour\"), \"rust-webassembly-a-tour\");\n assert_eq!(slug(\" Spaces \"), \"spaces\");\n }\n\n #[test]\n fn raw_html_passes_through_by_default_and_is_removed_in_safe_mode() {\n let md = \"hello\\n\\n<script>alert(1)</script>\\n\\nworld <b onclick=\\\"x()\\\">bold</b>\\n\";\n assert!(render(md, Settings::default()).body.contains(\"<script>\"));\n let safe = render(md, Settings { safe: true, ..Settings::default() }).body;\n assert!(!safe.contains(\"<script>\") && !safe.contains(\"onclick\"));\n assert!(safe.contains(\"hello\") && safe.contains(\"world\") && safe.contains(\"bold\"));\n }\n\n #[test]\n fn safe_mode_neutralizes_script_urls_but_keeps_ordinary_links() {\n let md = \"[a](javascript:alert(1)) [b](JaVaScRiPt:x) [c](https://example.test) [d](page.html?q=a:b) \";\n let out = render(md, Settings { safe: true, ..Settings::default() }).body;\n assert!(!out.to_lowercase().contains(\"javascript:\"), \"{out}\");\n assert!(!out.contains(\"data:\"), \"{out}\");\n assert!(out.contains(\"https://example.test\") && out.contains(\"page.html?q=a:b\"));\n }\n\n #[test]\n fn markdown_links_are_rewritten_but_external_ones_are_not() {\n let md = \"[a](guide.md) and [b](https://example.test/x.md)\";\n let out = render(md, Settings { rewrite_md_links: true, ..Settings::default() }).body;\n assert!(out.contains(\"href=\\\"guide.html\\\"\"));\n assert!(out.contains(\"href=\\\"https://example.test/x.md\\\"\"));\n }\n\n #[test]\n fn every_heading_gets_a_unique_enough_id_and_a_toc_entry() {\n let r = render(\"# One\\n\\n## Two\\n\\n## Two\\n\", Settings::default());\n assert_eq!(r.toc.len(), 3);\n assert!(r.body.contains(\"id=\\\"one\\\"\"));\n assert!(r.body.contains(\"id=\\\"two\\\"\") && r.body.contains(\"id=\\\"two-1\\\"\"));\n }\n}The full converter is in that listing, and the safe-mode arm is one line near the top of the filter_map: Event::Html(_) | Event::InlineHtml(_) if settings.safe => None. A test locks it in: with default settings the output contains <script>; with safe mode it contains neither the script nor an onclick attribute on an inline tag, and the surrounding text (hello, world, bold) survives.
A converter that removes raw HTML is still unsafe, and this is the part that deserves attention. Markdown has its own syntax for links and images, and the URLs inside can carry executable schemes. A plain link and an image, converted in safe mode before the URL fix:
\n<p><a href=\"javascript:alert(1)\">click</a> and <img src=\"data:text/html;base64,AAAA\" alt=\"img\" /></p>The script tag is gone and the javascript: URL is not. Clicking the link runs the code. Safe mode therefore has a second job: inspect each link and image destination and replace anything that is not an ordinary web, mail or relative URL.
fn is_safe_url(url: &str) -> bool {\n let trimmed = url.trim_start();\n match trimmed.split_once(':') {\n // A colon before any slash or question mark means the text before it is a scheme.\n Some((scheme, _)) if !scheme.contains(['/', '?', '#']) => {\n matches!(scheme.to_ascii_lowercase().as_str(), \"http\" | \"https\" | \"mailto\")\n }\n _ => true,\n }\n}The function is an allow-list, not a block-list: it accepts three schemes by name and refuses everything else, so a scheme nobody has thought of yet is blocked by default. It lowercases the scheme because browsers do (JaVaScRiPt: runs), and it treats a colon after a slash or a question mark as part of the path, so a relative link like page.html?q=a:b survives. After the fix, the same input converts to:
<p><a href=\"#\">click</a> and <img src=\"\" alt=\"img\" /></p>\n<p><a href=\"https://example.test\">ok</a> <a href=\"page.html\">rel</a></p>The dangerous link now points at # and the image has an empty source, while the ordinary links are untouched. The test safe_mode_neutralizes_script_urls_but_keeps_ordinary_links asserts both halves, including that the mixed-case JaVaScRiPt: is caught.
Be honest about what this is. It is a reasonable defense for a small tool, and it is not a full HTML sanitizer. For content from untrusted users that will be shown on a website, convert with this tool and then pass the HTML through a real sanitizer such as the ammonia crate, which parses the output and applies an allow-list of tags and attributes. Layered defenses fail less often than any single filter, and a sanitizer is the layer that does not depend on you remembering every case.
Readers want to link to a section, and the table of contents needs somewhere to point. The converter makes two passes over the events. The first collects headings: when it sees the start of a heading it begins accumulating text, and at the end it computes an id from the text. The slug function lowercases, drops punctuation and joins words with hyphens, so Rust & WebAssembly: a tour becomes rust-webassembly-a-tour.
A naive slug collides when two headings share a title, and that happens constantly (Setup, Examples, Notes). The first version used the slug directly and a test that claimed to check uniqueness only checked the count of headings, which passed. Looking at the output showed the bug:
id=\"notes\"\nid=\"setup\"\nid=\"setup\"Two headings with the same id are invalid HTML, and the second link in the table of contents jumps to the first section. The fix counts occurrences of each slug and appends a suffix to repeats, the convention used by GitHub:
\nid=\"notes\"\nid=\"setup\"\nid=\"setup-1\"The test now asserts both id=\"two\" and id=\"two-1\" for a document with two Two headings. The lesson is about tests as much as slugs: a test that asserts the length of a list proves the list has that many entries and says nothing about the entries. Assert the thing you care about.
The second pass runs the events through filter_map and rewrites each heading start to carry its precomputed id, in order. Because the first pass and the second walk the same document in the same order, the nth heading in one corresponds to the nth in the other. That coupling is the weak point of the design, and it is acceptable here because both passes use the same options. Changing the options between passes would shift the indices, which is why the options live in one function.
Documentation written as a folder of Markdown files links between pages with guide.md. After conversion those links should point at guide.html. The event stream again makes this a local change: match Start(Tag::Link { dest_url, .. }), and if the destination ends in .md and has no :// in it, swap the suffix. Links to other sites that happen to end in .md are left alone, because the check for a scheme separator excludes them:
<a href=\"guide.html\">\n<a href=\"https://example.test/guide.md\">\n<a href=\"#1\">The third line is the footnote reference, untouched. The rewrite handles fragments poorly (guide.md#setup would not match the suffix check), which is a good first extension: split off the #fragment before testing and reattach it afterward.
A fragment of HTML is rarely what you want to publish. The page function wraps the body in a document with a charset, a title and a navigation list built from the table of contents, escaping the title and heading text so a heading containing < cannot break the markup. The title comes from the --title flag or, by default, the first heading:
<!doctype html>\n<html lang=\"en\">\n<head>\n<meta charset=\"utf-8\">\n<title>Release notes</title>\n</head>\n<body>\n<nav><ul>\n<li class=\"h1\"><a href=\"#release-notes\">Release notes</a></li>\n<li class=\"h2\"><a href=\"#features\">Features</a></li>\n</ul></nav>\n<main>\n<h1 id=\"release-notes\">Release notes</h1>The binary is a thin shell, as every tool in this series is: parse arguments with clap, call convert::render, write the result, and map errors to an exit code.
mod convert;\n\nuse std::io::{self, Read, Write};\nuse std::path::PathBuf;\nuse std::process::ExitCode;\n\nuse anyhow::{Context, Result};\nuse clap::Parser;\n\n/// Convert Markdown to HTML.\n#[derive(Parser)]\n#[command(name = \"mdhtml\", version)]\nstruct Cli {\n /// Markdown file to read. Reads standard input when omitted.\n input: Option<PathBuf>,\n /// Write the result here instead of standard output.\n #[arg(short, long)]\n output: Option<PathBuf>,\n /// Wrap the output in a full HTML page with a table of contents.\n #[arg(long)]\n page: bool,\n /// Remove raw HTML from the input. Use this for text you do not trust.\n #[arg(long)]\n safe: bool,\n /// Point links to `other.md` at `other.html`.\n #[arg(long)]\n rewrite_links: bool,\n /// Page title; defaults to the first heading.\n #[arg(long)]\n title: Option<String>,\n}\n\nfn run(cli: &Cli) -> Result<()> {\n let markdown = match &cli.input {\n Some(path) => std::fs::read_to_string(path).with_context(|| format!(\"reading {}\", path.display()))?,\n None => {\n let mut buf = String::new();\n io::stdin().read_to_string(&mut buf).context(\"reading standard input\")?;\n buf\n }\n };\n let settings = convert::Settings { safe: cli.safe, rewrite_md_links: cli.rewrite_links };\n let rendered = convert::render(&markdown, settings);\n let output = if cli.page {\n let title = cli.title.clone().or_else(|| rendered.toc.first().map(|(_, t, _)| t.clone())).unwrap_or_else(|| \"Untitled\".into());\n convert::page(&title, &rendered)\n } else {\n rendered.body\n };\n match &cli.output {\n Some(path) => std::fs::write(path, output).with_context(|| format!(\"writing {}\", path.display()))?,\n None => io::stdout().write_all(output.as_bytes()).context(\"writing standard output\")?,\n }\n Ok(())\n}\n\nfn main() -> ExitCode {\n match run(&Cli::parse()) {\n Ok(()) => ExitCode::SUCCESS,\n Err(e) => {\n eprintln!(\"mdhtml: {e:#}\");\n ExitCode::from(2)\n }\n }\n}Reading from a file, from standard input when none is given, writing to a file or standard output, and the three behavior flags (--page, --safe, --rewrite-links) cover most uses. A missing input file produces a one-line message with the path in it and exit status 2:
mdhtml: reading nope.md: No such file or directory (os error 2)Run mdhtml --safe docs/guide.md > guide.html in a shell script, and the pipeline works with curl, grep and xargs like any other filter. For a whole folder, find docs -name '*.md' | xargs -I{} mdhtml --page --safe --rewrite-links {} -o {}.html converts every page.
Static sites put metadata at the top of each page in a block fenced by ---, and tools often want to list or lint the code samples in a document. Neither needs a new parser. Front matter is text before Markdown starts, so you split it off first; code blocks are runs of events between Start(Tag::CodeBlock) and End(TagEnd::CodeBlock), so you collect them from the stream. This program does both on one document and prints what it found:
use std::collections::BTreeMap;\n\nuse pulldown_cmark::{CodeBlockKind, Event, Options, Parser, Tag, TagEnd};\n\n/// Split a leading `---` block of `key: value` lines from the Markdown that follows it.\nfn split_front_matter(text: &str) -> (BTreeMap<String, String>, &str) {\n let mut meta = BTreeMap::new();\n let Some(rest) = text.strip_prefix(\"---\\n\") else { return (meta, text) };\n let Some(end) = rest.find(\"\\n---\\n\") else { return (meta, text) };\n for line in rest[..end].lines() {\n if let Some((key, value)) = line.split_once(':') {\n meta.insert(key.trim().to_string(), value.trim().trim_matches('\"').to_string());\n }\n }\n (meta, &rest[end + 5..])\n}\n\n/// Every fenced code block with its language tag.\nfn code_blocks(markdown: &str) -> Vec<(String, String)> {\n let mut out = Vec::new();\n let mut current: Option<(String, String)> = None;\n for event in Parser::new_ext(markdown, Options::empty()) {\n match event {\n Event::Start(Tag::CodeBlock(kind)) => {\n let lang = match kind {\n CodeBlockKind::Fenced(info) => info.split_whitespace().next().unwrap_or(\"\").to_string(),\n CodeBlockKind::Indented => String::new(),\n };\n current = Some((lang, String::new()));\n }\n Event::Text(t) => {\n if let Some((_, code)) = current.as_mut() {\n code.push_str(&t);\n }\n }\n Event::End(TagEnd::CodeBlock) => out.extend(current.take()),\n _ => {}\n }\n }\n out\n}\n\nfn main() {\n let doc = \"---\\ntitle: \\\"Ownership notes\\\"\\ntags: rust, memory\\n---\\n# Intro\\n\\n```rust\\nlet s = String::from(\\\"hi\\\");\\n```\\n\\nText.\\n\\n```toml\\n[dependencies]\\nserde = \\\"1\\\"\\n```\\n\";\n let (meta, body) = split_front_matter(doc);\n println!(\"front matter: {meta:?}\");\n println!(\"body starts: {:?}\", body.lines().next());\n for (lang, code) in code_blocks(body) {\n println!(\"{lang:>5}: {} line(s), first = {:?}\", code.lines().count(), code.lines().next().unwrap_or(\"\"));\n }\n let (no_meta, same) = split_front_matter(\"# Just a heading\\n\");\n println!(\"without front matter: {} keys, body unchanged: {}\", no_meta.len(), same == \"# Just a heading\\n\");\n}front matter: {\"tags\": \"rust, memory\", \"title\": \"Ownership notes\"}\nbody starts: Some(\"# Intro\")\n rust: 1 line(s), first = \"let s = String::from(\\\"hi\\\");\"\n toml: 2 line(s), first = \"[dependencies]\"\nwithout front matter: 0 keys, body unchanged: trueSeveral details in split_front_matter are the sort that bite later. It returns the metadata and a slice of the original text, &str, borrowed from the input, so the body is not copied. It requires the opening fence at the very start and a closing fence on its own line, and if either is missing it returns the text unchanged with no metadata, which is the safe choice: a document that starts with a horizontal rule, ---, must not lose its first lines. It uses a BTreeMap so the keys print in a stable order, which matters for tests. And it handles only flat key: value lines. Real YAML has lists, nested maps and multi-line strings, and for those the right move is the serde_yaml crate (or TOML front matter with toml), not a longer hand-rolled parser.
code_blocks shows the standard technique for collecting text between two markers: keep an Option accumulator, set it on the start event, append Text events while it is Some, and take it on the end event. The language comes from the fence's info string, rust or toml here, and an indented block has none. A documentation checker uses this to find every rust block and compile it, which is exactly how cargo test checks doctests. The same loop with a different action (write each block to a file, count lines, run a formatter) covers most \"do something with the code samples\" tools.
A converter is a pure function from text to text, which is as testable as code gets, and its failures are specific: a link that did not rewrite, an id that collided, a script tag that survived. Keep one test per behavior and make each assertion about the property, not about the whole output string, because the exact HTML changes when the library updates. For example, assert that the output contains(\"href=\\\"guide.html\\\"\"), not that it equals a full document. Add one snapshot test of a complete sample document so that an upgrade shows up as a reviewable diff; the insta crate manages the stored snapshots. And add every bug you find to the suite as a failing case before fixing it: the duplicate-heading ids and the javascript: URL both entered the project that way.
Run the whole suite on a corpus of real documents once in a while. Parsers meet input that tests never imagine, such as a file with Windows line endings, a table with a missing cell, or a footnote reference with no definition. The event stream is forgiving, and the output should be too: render something sensible and never panic. A quick check is to run the converter over every Markdown file in a large repository and confirm that it exits zero for all of them.
\nThe pass structure is cheap. A generated document of 4,000 sections with tables, lists and links, 0.56 MB, converted with --safe --rewrite-links --page in about 10 milliseconds once the file was in the cache (the first cold run took 0.23 s, dominated by reading the file from disk), producing 1.48 MB of HTML. The pull parser's lack of an in-memory tree is the reason. If you ever need to convert thousands of files, the time is in process startup and disk I/O, not in the conversion, so convert in one process with a loop instead of launching mdhtml per file.
Each of these is a change to the event stream. Syntax highlighting for fenced code blocks: collect the text events inside Start(Tag::CodeBlock(...)), run them through a highlighter such as syntect, and emit one Event::Html with the colored markup. That is the one place where producing raw HTML inside safe mode is correct, because you generated it. Front matter: strip a leading --- block before parsing and return it separately. Heading permalinks: add an anchor Html event after each heading's text. Custom containers like :::note: the parser does not support them, so you recognize them in text events yourself, or reach for a parser that does.
For a documentation site with a theme, search and navigation, mdBook or a static site generator already exists and solves it. If you need strict GitHub-flavored Markdown output and a tree to manipulate, comrak builds an AST and mirrors GitHub's renderer. If you need many output formats, pandoc converts between dozens. Use pulldown-cmark directly when you want a fast, small, embeddable converter that you can bend through an iterator, as in a static site generator you are writing, a preview in an editor, or a build step. For the other half of that toolbox, building a Markdown link checker parses the same documents to verify their links.
A converter earns its place when it runs unattended. In a documentation repository the usual wiring is a build step that converts every page and fails the build if anything goes wrong. Two properties of the tool make that reliable. It exits non-zero with a message on any I/O failure, so a missing or unreadable file stops the job instead of producing a half-built site. And its output is deterministic: the same input and flags give byte-identical HTML, because ids depend only on the heading text and the order of the document, with no clock and no random state. That lets you commit the generated pages and review the diff when a source page changes, or cache the build by hashing the inputs.
\nPair it with the link checker from the Markdown link checker guide: convert with --rewrite-links so pages point at each other's HTML, then check that every internal link resolves to a file that exists. A pipeline that converts, checks links and sanitizes the result catches most documentation bugs before a reader sees them. Pin the versions of pulldown-cmark in Cargo.lock and update on purpose, since a parser upgrade can legitimately change the HTML for edge cases, and the snapshot test is where you notice.
jq is the tool people reach for when a JSON document is too big to read and too awkward for grep. It is also a small language: a query like .users[] | select(.role == \"dev\") | .name is a program that describes a path through the data, filters it and extracts a field. Building a miniature version in Rust teaches a surprising amount in about 250 lines. You write a tokenizer-free recursive descent parser, model a query as an enum, evaluate it with a function that returns many results, and wrap everything in a command line tool that behaves well in a pipeline.
The tool built here is called mjq. It supports field access, array indexing with negative indexes, iteration, pipes, length, keys, map, select with the six comparison operators, and a mode for streams of newline-delimited JSON. It was built and tested on Rust 1.99 (edition 2024) with serde_json 1.0.154 and clap 4.6.7, and every output below is real.
Start from the queries you want to run, because the grammar follows from them.
\n.team reads the field team of the input object..users[0].name goes into an array by index, then reads a field..users[-1] picks the last element..users[] yields every element of the array, as separate results..users | length feeds the array into the length builtin..users | map(.role) applies a filter to each element and collects the results..users[] | select(.role == \"dev\") keeps only the elements that match.Two ideas run through the list. A filter takes one JSON value and produces zero, one or many values; .users[] produces three, select may produce none. And filters compose with |, which feeds every output of the left into the right. Those two facts decide the design: a filter is an enum, and running one returns a Vec<Value>.
The parser's output is a tree of Filter values, and the evaluator walks it. Writing the enum first fixes what the language can say:
#[derive(Debug, Clone, PartialEq)]\npub enum Filter {\n /// `.` returns the input unchanged.\n Identity,\n /// `.name`\n Field(String),\n /// `.[3]`\n Index(i64),\n /// `.[]` yields every element of an array or every value of an object.\n Iterate,\n /// `length`\n Length,\n /// `keys`\n Keys,\n /// `select(<filter> <op> <json>)` keeps the input when the comparison holds.\n Select(Box<Filter>, Cmp, Value),\n /// `map(<filter>)` applies the filter to each element and collects the results.\n Map(Box<Filter>),\n /// `a | b` feeds every output of `a` into `b`.\n Pipe(Box<Filter>, Box<Filter>),\n}Notice the Box<Filter> in Select, Map and Pipe: a filter can contain filters, and a type that contains itself needs an indirection to have a finite size. The path .users[0].name does not need its own variant. The parser turns it into Pipe(Field(\"users\"), Pipe(Index(0), Field(\"name\"))), because a path is just a pipeline of steps. That is a common simplification: make the evaluator handle a few primitives and let the parser build composite structure out of them.
The evaluator is one function with a match over the enum. Its type tells the whole story: fn run(filter: &Filter, input: &Value) -> Result<Vec<Value>, String>. The Vec carries the \"zero, one or many\" semantics, and the Result carries failures such as indexing a string.
Filter::Select(inner, op, wanted) => {\n let found = run(inner, input)?;\n Ok(if found.iter().any(|v| op.holds(v, wanted)) { vec![input.clone()] } else { vec![] })\n}\nFilter::Map(inner) => {\n let items = match input {\n Value::Array(items) => items,\n other => return Err(format!(\"cannot map over {}\", kind(other))),\n };\n let mut out = Vec::new();\n for item in items {\n out.extend(run(inner, item)?);\n }\n Ok(vec![Value::Array(out)])\n}\nFilter::Pipe(left, right) => {\n let mut out = Vec::new();\n for mid in run(left, input)? {\n out.extend(run(right, &mid)?);\n }\n Ok(out)\n}Pipe is the heart. It runs the left filter, which may produce many values, and for each one runs the right filter, concatenating all the results. That single rule gives you everything jq users find magical: .users[] | .name first expands the array into three values, then takes .name of each, producing three names. Select returns either the input in a one-element vector or an empty vector, which is how filtering falls out of the same shape. Map is Iterate plus a collect, wrapped in an array. Nothing in the evaluator knows about jq specifically; it is three recursion rules and a handful of primitives.
Errors are values. Indexing a string returns Err(\"cannot index string\"), and the first ? stops the whole run. That is stricter than jq, which can continue past some errors, and it is the right default for a small tool: a query that does not match the data should say so.
The parser reads bytes with a position and builds the tree. A query is a pipeline of paths separated by |, and a path is a chain of steps:
fn pipeline(&mut self) -> Result<Filter, ParseError> {\n let mut left = self.path()?;\n while self.eat(b'|') {\n let right = self.path()?;\n left = Filter::Pipe(Box::new(left), Box::new(right));\n }\n Ok(left)\n}path handles .name, [3], [-1] and [] in a loop, or dispatches to builtin when the text starts with a letter (length, keys, map(...), select(...)). map calls pipeline recursively for its argument, which is why map(.a | .b) works. The helper eat(byte) skips whitespace and consumes a byte if it is next, which keeps the code free of manual index arithmetic. The parser never panics on bad input; each failure becomes a ParseError variant with enough context to write a good message.
The binary is thin, as every well-structured Rust tool is. clap supplies the arguments, run does the work and returns a Result, and main turns the result into an exit code:
mod query;\n\nuse std::io::{self, Read};\nuse std::path::PathBuf;\nuse std::process::ExitCode;\n\nuse anyhow::{Context, Result};\nuse clap::Parser;\n\n/// A small jq-style JSON query tool.\n#[derive(Parser)]\n#[command(name = \"mjq\", version)]\nstruct Cli {\n /// The query, for example `.users[].name`.\n query: String,\n /// JSON file to read. Reads standard input when omitted.\n file: Option<PathBuf>,\n /// Print raw strings without quotes.\n #[arg(short, long)]\n raw: bool,\n /// Print compact one-line JSON.\n #[arg(short, long)]\n compact: bool,\n /// Treat the input as a stream of JSON values, one per line, and run the query on each.\n #[arg(short, long)]\n lines: bool,\n}\n\nfn run(cli: &Cli) -> Result<()> {\n let text = match &cli.file {\n Some(path) => std::fs::read_to_string(path).with_context(|| format!(\"reading {}\", path.display()))?,\n None => {\n let mut buf = String::new();\n io::stdin().read_to_string(&mut buf).context(\"reading standard input\")?;\n buf\n }\n };\n let filter = query::parse(&cli.query).context(\"invalid query\")?;\n if cli.lines {\n for (n, line) in text.lines().enumerate().filter(|(_, l)| !l.trim().is_empty()) {\n let input: serde_json::Value = serde_json::from_str(line).with_context(|| format!(\"line {} is not valid JSON\", n + 1))?;\n emit(cli, &filter, &input)?;\n }\n } else {\n let input: serde_json::Value = serde_json::from_str(&text).context(\"the input is not valid JSON\")?;\n emit(cli, &filter, &input)?;\n }\n Ok(())\n}\n\nfn emit(cli: &Cli, filter: &query::Filter, input: &serde_json::Value) -> Result<()> {\n for value in query::run(filter, input).map_err(anyhow::Error::msg)? {\n match (&value, cli.raw, cli.compact) {\n (serde_json::Value::String(s), true, _) => println!(\"{s}\"),\n (_, _, true) => println!(\"{value}\"),\n _ => println!(\"{}\", serde_json::to_string_pretty(&value)?),\n }\n }\n Ok(())\n}\n\nfn main() -> ExitCode {\n match run(&Cli::parse()) {\n Ok(()) => ExitCode::SUCCESS,\n Err(e) => {\n eprintln!(\"mjq: {e:#}\");\n ExitCode::from(2)\n }\n }\n}It reads the JSON from a file or standard input, parses the query, evaluates it and prints each result on its own line. Flags handle output and input: pretty printing is the default, -c prints compact one-line JSON, -r prints strings without quotes so shell scripts can use them, and -l treats the input as one JSON value per line. Errors go to standard error with exit status 2, matching the convention in a command line tool from clap to a shipped release. The with_context calls are what make failures readable: \"reading nope.json\" tells you which step failed and the underlying operating system message says why.
Given a small document with a team name and three users:
\n{\n \"team\": \"platform\",\n \"users\": [\n {\"name\": \"ada\", \"role\": \"admin\", \"langs\": [\"rust\", \"c\"], \"active\": true},\n {\"name\": \"grace\", \"role\": \"dev\", \"langs\": [\"rust\", \"go\"], \"active\": true},\n {\"name\": \"linus\", \"role\": \"dev\", \"langs\": [\"c\"], \"active\": false}\n ]\n}These are the queries and their output, with -c for compact results:
mjq '.team' \"platform\"\nmjq '.users[0].name' \"ada\"\nmjq '.users[].name' \"ada\" \"grace\" \"linus\"\nmjq '.users | length' 3\nmjq '.users[1] | keys' [\"active\",\"langs\",\"name\",\"role\"]\nmjq '.users | map(.role)' [\"admin\",\"dev\",\"dev\"]\nmjq '.users[] | select(.role == \"dev\") | .name' \"grace\" \"linus\"\nmjq '.users[-1].langs[0]' \"c\"\nmjq '.missing' null\nmjq '.users[0].name.first' mjq: cannot read field \"first\" of stringEach line is worth reading. A missing field gives null, as in jq, and a field of a string is an error. The keys output is sorted alphabetically because serde_json keeps object keys in sorted order by default; enabling its preserve_order feature would keep the file's order. The select query chains three stages: expand the array, keep the developers, take the names.
The failure paths matter as much as the happy ones:
\nmjq '.users[' mjq: invalid query: the query ended too early\nmjq 'users' mjq: invalid query: unknown function \"users\" (try length, keys, map, select)\n$ echo 'not json' | mjq '.a'\nmjq: the input is not valid JSON: expected ident at line 1 column 2The second message was a bug in an earlier draft: the unknown function users was reported as \"not valid JSON\" because it reused the literal error variant. Writing the test that exercised every error case (errors_are_values_not_panics below) is what found it. Give each failure its own variant, so each gets its own message.
The first version of select supported only ==. Real queries need ordering: \"people over 30\", \"names after h\". Adding it touches three places and teaches how a small language grows. A Cmp enum names the six operators. The parser gets a comparison method that reads ==, !=, <=, >=, < or >, checking two-character operators before one-character ones so >= is not read as > followed by garbage. And Cmp::holds decides what each operator means for each pair of JSON types:
impl Cmp {\n fn holds(self, left: &Value, right: &Value) -> bool {\n use std::cmp::Ordering;\n let ordering = match (left, right) {\n (Value::Number(a), Value::Number(b)) => a.as_f64().partial_cmp(&b.as_f64()),\n (Value::String(a), Value::String(b)) => Some(a.cmp(b)),\n _ => None,\n };\n match self {\n Cmp::Eq => left == right,\n Cmp::Ne => left != right,\n // Ordering is defined only between two numbers or two strings.\n Cmp::Lt => ordering == Some(Ordering::Less),\n Cmp::Le => matches!(ordering, Some(Ordering::Less | Ordering::Equal)),\n Cmp::Gt => ordering == Some(Ordering::Greater),\n Cmp::Ge => matches!(ordering, Some(Ordering::Greater | Ordering::Equal)),\n }\n }\n}The design choice is in the _ => None arm: ordering a number against a string, or an object against anything, has no answer, so < and > are simply false, while == and != work on any pair. Real jq defines a total order across types; for a small tool, \"false when not comparable\" is easier to explain. partial_cmp on f64 returns None for NaN, which the same code handles without a special case, since a None ordering makes every ordered comparison false. The test that locks this behavior in was written to the wrong expectation first:
assertion `left == right` failed\n left: [Number(36)]\n right: [Number(41), Number(36)]The test asked for ages of people whose names are at least \"h\", and expected two. The program returned one, and the program was right: \"grace\" starts with g, which sorts before h, so only \"linus\" qualifies. The expectation was the bug. Failing tests are evidence, not verdicts; read both sides before editing either. With the expectation corrected, five tests pass, and the new queries work:
mjq -c '.people[] | select(.age > 30) | .name' \"grace\" \"linus\"Logs and event feeds are often JSON values one per line, not one big array. Reading such a file as a single document fails on the second line. The -l flag processes each line separately, running the same filter and printing results as they arrive, which also keeps memory bounded by the size of one line. The structure is a loop over text.lines(), a filter that skips blank lines, and the same emit function that handled the single-document case, extracted so both paths share it. Applied to a four-line event log:
mjq -c -l 'select(.level == \"error\") | .ms' events.ndjson 340 95The two error events had latencies of 340 and 95 milliseconds, and the two info events were filtered out. A malformed line reports its number and stops:
\n$ printf '{\"a\":1}\\noops\\n' | mjq -c -l '.'\n{\"a\":1}\nmjq: line 2 is not valid JSON: expected value at line 1 column 1The first line's output appeared before the failure, which is the behavior you want from a streaming tool: results for good lines are not held hostage by a bad one later. The exit status was 2. For truly unbounded streams, replace read_to_string with a BufReader and lines() so the file is never fully in memory. That is the version to ship if you point this at a log that grows forever.
A tool like this should be measured against the real thing. The test inputs were 300,000 generated user records: a 25.7 MB single document and the same records as 25.4 MB of newline-delimited JSON. The release build of mjq ran on an Apple silicon laptop, and jq is the version installed by the system package manager. Times are wall-clock seconds from time.
All times are the best of five runs, in wall-clock seconds, for the release build:
\n.team (parse the document, read one field): mjq 0.20 s..users | length: mjq 0.34 s, jq 1.6 0.51 s..users[] | select(.role == \"ops\") | .name, 59,603 matches: mjq 0.57 s, jq 1.6 0.62 s.-l 'select(.age > 60) | .id' over the newline-delimited file, 56,657 matches: mjq 0.26 s, one run.Three observations come out of it. Parsing dominates: just reading the document and returning one field takes 0.20 s, so more than half of the length query's 0.34 s is serde_json building the tree. The remaining 0.14 s is the cost of a design shortcut named earlier: Field clones the whole array so the next stage can own it. On the filtering query, mjq already matches jq 1.6, which is a respectable result for 300 lines of code and no tuning, since both spend most of their time in the parse. And the streaming mode is fast because each line is a small allocation that is dropped before the next, which is friendly to the cache.
CowThe shortcut is easy to remove, and removing it is a good lesson in how Rust expresses \"borrow when you can, own when you must\". The evaluator returns Vec<Value>, so every result is owned. Most filters do not need to own anything: Field, Index, Iterate and Identity return parts of the input, and Select returns the input itself. Only Length, Keys and Map build new values. The type that models \"either a borrow or an owned value\" is Cow:
pub fn run<'a>(filter: &Filter, input: &'a Value) -> Result<Vec<Cow<'a, Value>>, String> {\n match filter {\n Filter::Identity => Ok(vec![Cow::Borrowed(input)]),\n Filter::Field(name) => match input {\n Value::Object(map) => Ok(vec![Cow::Borrowed(map.get(name).unwrap_or(&NULL))]),\n Value::Null => Ok(vec![Cow::Borrowed(&NULL)]),\n other => Err(format!(\"cannot read field {name:?} of {}\", kind(other))),\n },\n Filter::Iterate => match input {\n Value::Array(items) => Ok(items.iter().map(Cow::Borrowed).collect()),\n Value::Object(map) => Ok(map.values().map(Cow::Borrowed).collect()),\n other => Err(format!(\"cannot iterate over {}\", kind(other))),\n },\n Filter::Length => Ok(vec![Cow::Owned(match input {\n Value::Array(a) => a.len().into(),\n Value::String(s) => s.chars().count().into(),\n _ => 0.into(),\n })]),\n // ... Index, Keys, Select and Map follow the same pattern\n Filter::Pipe(left, right) => {\n let mut out = Vec::new();\n for mid in run(left, input)? {\n match mid {\n Cow::Borrowed(r) => out.extend(run(right, r)?),\n Cow::Owned(v) => out.extend(run(right, &v)?.into_iter().map(|c| Cow::Owned(c.into_owned()))),\n }\n }\n Ok(out)\n }\n }\n}Two details carry the design. The lifetime 'a on the input and on the Cow says that borrowed results live as long as the input document, which the compiler checks. A missing field returns a borrow of a static NULL, so even \"not found\" allocates nothing. And Pipe shows the cost of mixing: if the left stage returned a borrow, the right stage keeps borrowing from the same input; if it returned an owned value, the right stage's results might point into that value, which dies at the end of the arm, so they are copied out with into_owned. The compiler insists on that copy; without it the borrow would outlive its owner, the E0597 family of errors from E0597. Five tests passed unchanged apart from converting results with into_owned in the helper.
The measured effect on the 25.7 MB file, best of five:
\n.users | length: 0.34 s before, 0.21 s after, which is the parse cost alone..users[] | select(.role == \"ops\") | .name: 0.57 s before, 0.29 s after, about twice as fast as jq 1.6 (0.62 s).Removing the clones took the length query to the floor set by parsing and cut the filter query nearly in half. The code change was one function. The lesson is the order of work: write the simple, cloning evaluator first, get the tests green, measure, and only then thread lifetimes through, with the tests as your safety net.
\nThe evaluator and parser are pure functions, which makes them cheap to test without a binary:
\n#[test]\nfn iteration_yields_many_outputs() {\n let v = json!([1, 2, 3]);\n assert_eq!(q(\".[]\", &v), [json!(1), json!(2), json!(3)]);\n}\n\n#[test]\nfn select_filters_and_map_collects() {\n let v = json!([{\"k\": 1}, {\"k\": 2}, {\"k\": 1}]);\n assert_eq!(q(\".[] | select(.k == 1)\", &v).len(), 2);\n assert_eq!(q(\"map(.k)\", &v), [json!([1, 2, 1])]);\n}test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00sFour tests cover paths and negative indexes, iteration, select and map, and the error cases. The json! macro builds Values inline, which keeps the test data next to the assertions. Add a test every time you find a bug by hand, and keep the case small.
It clones aggressively. Each stage copies its inputs into Vec<Value>, which is fine for documents of a few megabytes and wasteful for a 2 GB file. A faster design returns references (Vec<&Value>) for read-only filters and clones only when a filter must construct a new value, using Cow<Value> to keep both cases in one type. It parses the whole document into memory, where a streaming parser would handle files larger than RAM; the serde_json streaming deserializer reads a sequence of values from a reader, which is how line-delimited JSON is processed. And its language is small: no arithmetic, no if, no variables, no // alternatives, no .. recursive descent, no string interpolation. Each of those is a new Filter variant and a few lines in run, which is a pleasant extension path: add + and -, then comparison operators beyond ==, then if, and you have the core of jq.
Where the real jq is the better tool, use it. It has hundreds of builtins, handles streaming and has been tested against every odd corner of JSON. Build your own when you want a query language embedded in your program, a deliberately small and safe subset for users to write in a config file, or the education. The same recursive-descent structure powers the calculator in ten Rust project ideas, and the Result-returning style is the one explained in error handling in Rust.
Pick extensions by the Rust concept they teach. A --sort-keys flag teaches BTreeMap versus IndexMap and the preserve_order feature. A sort_by(.field) builtin teaches closures, sort_by and stable ordering of JSON values. An --arg name=value option with $name variables teaches environments threaded through the evaluator, which means passing a context struct alongside input. A real tokenizer in front of the parser teaches how to report error positions precisely. And a --stream mode built on a BufReader and serde_json::Deserializer::into_iter teaches iterator-based pipelines that never hold more than one value. Each is an afternoon of work, each ships with its own tests, and together they turn the toy into something you would use. The next project in the same family is a Markdown converter, which swaps JSON for a text format and the filter tree for an event stream.
A rolling update is a controlled outage of every pod, one at a time. If the old pod exits carelessly, a few requests fail every deploy. The sequence Kubernetes runs on shutdown is well defined, and a Rust service can follow it in about forty lines. Versions: Rust 1.90, Axum 0.8, Tokio 1.47, Kubernetes 1.31.
\nWhen a pod is deleted:
\nTerminating.preStop hook (if any) runs.SIGTERM.terminationGracePeriodSeconds, it receives SIGKILL.Step 2's removal takes time to reach kube-proxy, ingress controllers and external load balancers. Requests can arrive for several seconds after SIGTERM. An app that closes its listener immediately resets those connections.
SIGTERM, set a flag so /ready returns 503.use axum::{extract::State, http::StatusCode, routing::get, Router};\nuse std::sync::{\n atomic::{AtomicBool, Ordering},\n Arc,\n};\nuse std::time::Duration;\nuse tokio::signal::unix::{signal, SignalKind};\n\n#[derive(Clone)]\nstruct AppState {\n draining: Arc<AtomicBool>,\n}\n\nasync fn ready(State(s): State<AppState>) -> StatusCode {\n if s.draining.load(Ordering::SeqCst) {\n StatusCode::SERVICE_UNAVAILABLE\n } else {\n StatusCode::OK\n }\n}\n\nasync fn work() -> &'static str {\n tokio::time::sleep(Duration::from_millis(200)).await;\n \"done\"\n}\n\n#[tokio::main]\nasync fn main() {\n let state = AppState { draining: Arc::new(AtomicBool::new(false)) };\n let app = Router::new()\n .route(\"/work\", get(work))\n .route(\"/ready\", get(ready))\n .route(\"/live\", get(|| async { \"ok\" }))\n .with_state(state.clone());\n\n let listener = tokio::net::TcpListener::bind(\"0.0.0.0:8080\").await.unwrap();\n\n let shutdown = async move {\n let mut term = signal(SignalKind::terminate()).unwrap();\n let mut int = signal(SignalKind::interrupt()).unwrap();\n tokio::select! {\n _ = term.recv() => {}\n _ = int.recv() => {}\n }\n tracing::info!(\"shutdown signal received, failing readiness\");\n state.draining.store(true, Ordering::SeqCst);\n tokio::time::sleep(Duration::from_secs(5)).await;\n tracing::info!(\"draining connections\");\n };\n\n axum::serve(listener, app)\n .with_graceful_shutdown(shutdown)\n .await\n .unwrap();\n tracing::info!(\"clean exit\");\n}After the shutdown future completes, Axum stops accepting new connections and waits for existing ones to finish. Keep-alive connections that are idle close, and connections with a request in flight complete first. If a handler might run for minutes, wrap the whole serve in a timeout so a stuck request cannot hold shutdown forever (Axum graceful shutdown, Tokio graceful shutdown).
apiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: api\nspec:\n replicas: 3\n strategy:\n rollingUpdate:\n maxUnavailable: 0\n maxSurge: 1\n template:\n spec:\n terminationGracePeriodSeconds: 45\n containers:\n - name: api\n image: registry.example.test/api:1.4.0\n ports: [{ containerPort: 8080 }]\n readinessProbe:\n httpGet: { path: /ready, port: 8080 }\n periodSeconds: 2\n failureThreshold: 1\n livenessProbe:\n httpGet: { path: /live, port: 8080 }\n periodSeconds: 10Choose the grace period larger than: the readiness delay (5 s) plus your longest request plus a margin. periodSeconds: 2 and failureThreshold: 1 make the readiness failure take effect fast. The liveness probe must keep passing during drain, or Kubernetes will kill the pod early, which is why /live is separate (health checks).
If you do not want sleep logic in the app:
\nlifecycle:\n preStop:\n exec:\n command: [\"sleep\", \"8\"]This needs a sleep binary in the image, absent in distroless and scratch images. Newer Kubernetes versions offer a sleep action natively: preStop: { sleep: { seconds: 8 } }. The grace period includes the hook's time.
kubectl rollout restart deployment/api &\noha -z 60s -c 20 https://api.example.test/workCount non 2xx responses during the rollout. Zero is the target. Failures that are connection resets point to a too-short delay, and 503s from the app point to readiness flipping before the load balancer stops sending. Add long-lived connections and WebSockets to the test, since they need a policy to close them with a close frame (zero downtime deploys).
\nexec or tini.Observing a running Linux system usually means changing it (adding logging, restarting under a debugger) or accepting the cost of tools that slow it down. eBPF turns that around. You write a very small program, the kernel proves it safe, and it runs inside the kernel at a hook point such as \"a process called exec\", reporting what it sees into a shared table that an ordinary program reads. Aya lets you write both halves in Rust: the kernel program and the user-space loader. This guide builds the smallest useful example, a counter of how many times each program name was executed, and it was actually loaded into a Linux kernel and run. It also shows the most important thing about eBPF, the verifier, by deliberately writing a program the kernel refuses and reading its refusal. Versions: aya 0.16.0, aya-ebpf 0.3.0, bpf-linker 0.13.0, rustc 1.99.0 for user space and nightly 1.101.0 for the kernel program, on Linux 7.0.14 (the kernel Docker Desktop runs in its virtual machine) on an Apple M1 Max.
\nAn eBPF program is a function compiled to a restricted instruction set and loaded into the kernel with the bpf() system call. Before it runs, the kernel's verifier walks every possible path through it and rejects it unless it can prove the program terminates, never reads memory it should not, and never dereferences a pointer it has not checked. Once accepted, it is compiled to native code and attached to a hook: a tracepoint, a function entry, a network interface. Whenever the hook fires, the program runs. Programs share data with user space through maps, kernel-resident tables (hash maps, arrays, ring buffers) that both sides can read and write.
That gives a three-part shape to every Aya project. A kernel crate compiled for the bpfel-unknown-none target holds the programs and declares the maps. A user-space crate loads the compiled object, attaches the programs to hooks, and reads the maps. Optionally a shared crate holds types both sides use. The restricted nature of the kernel side is why it is #![no_std] and #![no_main]: there is no allocator, no standard library, no threads, only the helper functions the kernel provides.
The kernel crate needs two things most Rust projects do not. It needs the nightly compiler, because the eBPF target has no prebuilt standard library and must be built from source with -Z build-std=core, which is a nightly flag, plus the rust-src component. And it needs bpf-linker, a linker that turns Rust's LLVM output into a loadable eBPF object. Both went into one Docker image, because Linux is the only place this runs and Docker Desktop's virtual machine provides a Linux kernel on a Mac:
FROM rust:1.99-slim-bookworm\nRUN apt-get update && apt-get install -y --no-install-recommends build-essential pkg-config git curl ca-certificates zstd bpftool iproute2 \\\n && rm -rf /var/lib/apt/lists/*\nRUN rustup toolchain install nightly --profile minimal --component rust-src\nRUN curl -sL https://github.com/aya-rs/bpf-linker/releases/latest/download/bpf-linker-aarch64-unknown-linux-musl.tar.zst -o /tmp/bl.tar.zst \\\n && tar -xpf /tmp/bl.tar.zst -C /usr/local/bin && rm /tmp/bl.tar.zstTwo notes on that file. The linker is installed from the project's release tarball and not with cargo install bpf-linker: the project's own README advises against building it from source, because a source build is tied to one specific LLVM version, and the nightly compiler used here carries LLVM 23.1.5. The prebuilt binary reported bpf-linker 0.13.0 and worked on the first try. And the tarball name contains the CPU architecture (aarch64 here); on an x86-64 machine you would fetch the x86_64 one.
The kernel crate is small. It declares one map, a hash from a 16-byte process name to a 64-bit counter, and one program attached to a tracepoint:
\n#![no_std]\n#![no_main]\n\nuse aya_ebpf::{\n helpers::bpf_get_current_comm,\n macros::{map, tracepoint},\n maps::HashMap,\n programs::TracePointContext,\n};\n\n#[map]\nstatic EXEC_COUNTS: HashMap<[u8; 16], u64> = HashMap::with_max_entries(1024, 0);\n\n#[tracepoint]\npub fn count_exec(_ctx: TracePointContext) -> u32 {\n let comm = bpf_get_current_comm().unwrap_or([0u8; 16]);\n unsafe {\n match EXEC_COUNTS.get_ptr_mut(&comm) {\n Some(count) => *count += 1,\n None => {\n let _ = EXEC_COUNTS.insert(&comm, &1u64, 0);\n }\n }\n }\n 0\n}\n\n#[panic_handler]\nfn panic(_info: &core::panic::PanicInfo) -> ! {\n unsafe { core::hint::unreachable_unchecked() }\n}Read it as the kernel will. #[map] places the static into an ELF section the loader recognises as a map definition, with room for 1,024 distinct names. #[tracepoint] marks count_exec as a program of type tracepoint; its argument is the context the kernel passes (unused here, so the underscore). The body asks the kernel for the name of the current process with bpf_get_current_comm, a 16-byte buffer, then looks it up in the map. If the name is present, it increments the counter; if not, it inserts a 1. The lookup returns a raw pointer, which is why that block is unsafe, and the match on Some and None is not just style: the kernel's verifier insists on exactly that null check, as the failure test below proves. The panic handler must exist because there is no standard library; it tells the compiler that reaching it is impossible, which is acceptable because the verifier would have rejected any path that could panic.
The manifest sets the profile for size and speed, with debug information on for the loader:
\n[package]\nname = \"lab-ebpf\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n[dependencies]\naya-ebpf = \"0.3\"\n\n[[bin]]\nname = \"lab-ebpf\"\npath = \"src/main.rs\"\n\n[profile.release]\nopt-level = 3\nlto = true\npanic = \"abort\"\ncodegen-units = 1\ndebug = 2It is built for the eBPF target with the nightly flag described above:
\ncd ebpf\ncargo +nightly build --target=bpfel-unknown-none -Z build-std=core --releaseThe build finished in 13.6 seconds, most of it compiling core and the dependencies. The output is not a native executable. It is an ELF object of 2,192 bytes, and reading its section table shows what the loader will find:
readelf -S -W target/bpfel-unknown-none/release/lab-ebpf | grep -E \"tracepoint|maps\" [ 3] tracepoint PROGBITS 0000000000000000 0001a0 000300 00 AX 0 0 8\n [ 4] .reltracepoint REL 0000000000000000 0005a0 000050 10 I 7 3 8\n [ 5] maps PROGBITS 0000000000000000 0004a0 00001c 00 WA 0 0 4The program lives in a section named tracepoint, 0x300 (768) bytes of eBPF instructions. The maps section is 0x1c (28) bytes, the definition of the one map. The .reltracepoint section holds relocations: places in the program that must be patched with the map's real file descriptor at load time. This is what \"the loader\" does, and why a compiled eBPF program is a file you load and not a binary you execute.
The loader is an ordinary Rust program using the aya crate. It embeds the object file into its own binary at compile time, loads the program into the kernel, attaches it to the sched_process_exec tracepoint, does some work to observe, and then reads the map:
use anyhow::{Context, Result};\nuse aya::{Ebpf, maps::HashMap, programs::TracePoint};\nuse std::process::{Command, Stdio};\n\nfn main() -> Result<()> {\n let mut ebpf = Ebpf::load(aya::include_bytes_aligned!(concat!(\n env!(\"CARGO_MANIFEST_DIR\"),\n \"/../ebpf/target/bpfel-unknown-none/release/lab-ebpf\"\n )))\n .context(\"loading the eBPF object\")?;\n\n let program: &mut TracePoint = ebpf.program_mut(\"count_exec\").context(\"no such program\")?.try_into()?;\n program.load().context(\"the kernel verifier rejected the program\")?;\n program.attach(\"sched\", \"sched_process_exec\")?;\n println!(\"attached count_exec to sched:sched_process_exec\");\n\n // Generate some work to observe: three commands, a few times each.\n for (cmd, times) in [(\"true\", 5), (\"ls\", 3), (\"uname\", 2)] {\n for _ in 0..times {\n Command::new(cmd).stdout(Stdio::null()).status()?;\n }\n }\n\n let counts: HashMap<_, [u8; 16], u64> = HashMap::try_from(ebpf.map(\"EXEC_COUNTS\").context(\"no map\")?)?;\n let mut rows: Vec<(String, u64)> = counts\n .iter()\n .map(|kv| kv.map(|(k, v)| (String::from_utf8_lossy(&k).trim_end_matches('\\0').to_string(), v)))\n .collect::<Result<_, _>>()?;\n rows.sort();\n for (comm, n) in rows {\n println!(\"{comm:>8} {n}\");\n }\n Ok(())\n}The steps map one to one onto the concepts above. Ebpf::load parses the object. program_mut(\"count_exec\") finds the program by its function name, and load() is the point at which the kernel's verifier runs. attach(\"sched\", \"sched_process_exec\") connects it to the tracepoint named by category and event. The loader then launches known commands (five true, three ls, two uname) so the answer is predictable, reads the typed map and prints it. The key type is the same 16-byte array on both sides, and the typed HashMap<_, [u8; 16], u64> wrapper checks that at the boundary.
Loading eBPF needs privileges, and attaching a tracepoint needs the kernel's tracing file system mounted. In a container, that means --privileged and mounting tracefs. Here is the run:
docker run --rm --privileged -v \"$PWD:/w\" -w /w/app aya-lab bash -c \\\n 'mount -t tracefs nodev /sys/kernel/tracing; ./target/release/lab'attached count_exec to sched:sched_process_exec\n ls 3\n true 5\n uname 2The counts are exactly what the loader ran: three ls, five true, two uname. They are the kernel program's counts, not the loader's. Every exec in that window passed through count_exec and landed in the map. The total ran inside the kernel with no changes to ls and no tracing of the loader itself. (The loader's own start-up is not in the table because it started before the attach.)
Once a program is in the kernel, it belongs to the system, not to your loader, and the standard tool for inspecting it is bpftool. I changed the loader to wait a few seconds before reading the map (an environment variable, HOLD_SECS) and ran bpftool from the same container while it was waiting:
bpftool prog show | grep -A2 count_exec\nbpftool map show | grep -A1 EXEC_COUNTS\nbpftool prog dump xlated id \"$PID\"1597: tracepoint name count_exec tag 7cd66e66b72eecde gpl\n\tloaded_at 2026-11-07T19:30:42+0000 uid 0\n\txlated 848B jited 600B memlock 4096B map_ids 106,105\n106: hash name EXEC_COUNTS flags 0x0\n\tkey 16B value 8B max_entries 1024 memlock 91744B\n 0: (bf) r6 = r10\n 1: (07) r6 += -16\n 2: (bf) r1 = r6\n 3: (b7) r2 = 0\n 4: (b7) r3 = 16\n 5: (85) call pc+92#bpf_prog_4f9302ba65b90ccd_FThe first line is the kernel's record of your program: id 1597, type tracepoint, the name you gave the function, a hash of its instructions (tag) and the licence string gpl. That licence matters: the kernel only lets a program call certain helper functions if it declares a GPL-compatible licence, and Aya sets it for you. The size line is a useful number. The program the verifier accepted is 848 bytes of translated eBPF instructions, and the kernel compiled it to 600 bytes of native arm64 code (jited). The map is a hash with 16-byte keys, 8-byte values and room for 1,024 entries, and the kernel charged 91,744 bytes of locked memory for it. Those 91,744 bytes are the cost of reserving space for the full table, not for the three entries in use, which is why max_entries deserves a thought.
The dump of the translated program starts the way the verifier trace did: the stack pointer copy, the call that zeroes the name buffer (a call to a subprogram, call pc+92), and then bpf_get_current_comm. It is the same code in a more compact form. Being able to read it is mostly useful for one purpose: when the verifier rejects something, the instruction numbers in its message refer to exactly this listing, so you can find the line that failed.
The run with the wait was also instructive for what it counted. The first run, with no pause, saw only my three commands. This one printed the same three plus a number of other names:
\nattached count_exec to sched:sched_process_exec\n 6 16\n bpftool 4\n cut 1\n grep 3\n head 3\n ls 3\nmysqladmin 8\nredis-cli 7\n runc 16\n sh 1\n true 5\n uname 2\n wget 1The additional names are the shell commands I used to inspect (bpftool, grep, head, cut, sh) and, more tellingly, programs from other containers on the same virtual machine: mysqladmin, redis-cli, runc and wget, which are what health checks of unrelated containers execute. Even true and ls appear because the loader ran them. (The row whose name is just 6 is a process whose name is the digit 6; I did not identify it, and the row shows that names are whatever the kernel stored, not a vetted label.) That is the point and the hazard of this kind of tool in one table: a tracepoint is global. It sees every exec on the kernel, in every container and namespace, not just yours, which is why it needs privilege and why anything built on it should be treated as a security-sensitive component. It is also why filtering by cgroup or PID namespace inside the kernel program is a standard early improvement.
The verifier is the part of eBPF that Rust developers meet first, because it is a second type checker that you cannot argue with. To see it, I removed the null check and wrote what looks like reasonable code: look up the counter and increment it.
\nunsafe {\n // Missing the null check: the lookup can fail.\n *EXEC_COUNTS.get_ptr_mut(&comm).unwrap_unchecked() += 1;\n}It compiles. Rust has no objection, because the unsafe block says to trust the programmer. The kernel does not. Loading it printed a long verifier trace, instruction by instruction with the state of each register, and at the end:
80: (85) call bpf_map_lookup_elem#1 ; R0=map_value_or_null(id=4,map=EXEC_COUNTS,ks=16,vs=8)\n81: (79) r1 = *(u64 *)(r0 +0)\nR0 invalid mem access 'map_value_or_null'\nverification time 155 usec\nstack depth 48+0\nprocessed 159 insns (limit 1000000)Read the last lines like a compiler error. Instruction 80 called the map lookup helper, and the verifier tracked its return value as map_value_or_null: a pointer that might be null. Instruction 81 then read through it, and the verifier stopped: \"R0 invalid mem access\". The register R0 holds the helper's result, and the verifier will not allow a dereference until a null comparison has narrowed its type. The trace before this point (about 160 instructions in all) also contains an unrelated detail worth seeing once: the compiler unrolled the zeroing of the 16-byte name buffer into a small loop, and the verifier walked it iteration by iteration, which is how it proves loops terminate. Programs that loop over data of unknown size are the ones that get rejected for exceeding the verifier's limits, which is why eBPF code favours bounded loops and fixed-size buffers.
The error text is long and the fix is short: restore the match. The lesson is that the kernel enforces a property Rust's types do not express here, and the error arrives at load time, not at compile time. So test loading in CI, not just compiling.
When I introduced the broken program I rebuilt only the kernel crate and ran the loader, and it happily printed the correct counts. The loader had embedded the old object when it was compiled: include_bytes_aligned! copies the file into the user-space binary at build time, so changing the kernel crate does nothing until the loader is rebuilt. After touch app/src/main.rs and a rebuild the verifier error appeared. If a change to your kernel program seems to have no effect, this is the first thing to check, and a build script (or the aya-build crate) that rebuilds the kernel crate and triggers the loader is the usual cure. The same trap runs in the other direction: the loader reported a correct result from a program I had already broken.
Three details of the map access matter in real tools. First, map reads from user space are not atomic with respect to the kernel program, so a counter can be read while it is being incremented; for monotonic counts that is fine, for pairs of values that must agree it is not. Second, per-CPU variants of the map types (PerCpuHashMap) avoid contention by giving each CPU its own slot, at the cost of the loader summing them. Third, the map has a fixed capacity, 1,024 here; when it is full, an insert fails, and the kernel program above silently ignores that failure with let _ =. A real tool would count the failures in a second map so lost events are visible.
The measured outcome is short. A 2,192-byte object built in 13.6 seconds was accepted by a real kernel, attached to sched_process_exec, and counted ten process launches exactly (three, five and two). Removing one null check produced a rejection at instruction 81 with a precise reason. Aya made both halves ordinary Rust, and the verifier made one of them stricter than Rust ever will be.
Neovim ships with a language server client, and since version 0.11 it also ships the configuration API and the default key mappings that make a plugin-free Rust setup practical. rust-analyzer provides the intelligence: type-aware hover, go to definition, references, rename, diagnostics from the compiler and from clippy, and formatting through rustfmt. This guide sets up that stack with about twenty lines of Lua and no plugins at all, then drives it in a headless Neovim, a Neovim with no screen that runs a script, to show what each feature returns. Everything below was run, and the output is real. Versions: Neovim 0.12.6, rust-analyzer 1.99.0, rustc 1.99.0, on an Apple M1 Max with macOS. Completion menus and the visual look of diagnostics need a screen, so those parts are described from the documented behaviour and are marked.
\nThree programs cooperate, and knowing which one answers which question saves time when something breaks. Neovim is the editor and the LSP client: it starts a server, sends it the buffer's text and your requests, and shows the replies. rust-analyzer is the language server: it parses and type-checks your crate itself, quickly and incrementally, to answer hover and navigation requests. cargo check (or cargo clippy) is run by rust-analyzer in the background on save to produce the compiler's own diagnostics. So the editor never talks to rustc; the server does. That is why two sources of errors appear on screen, labelled rust-analyzer and clippy in the output further down: the first is instant and approximate, the second is slower and exact.
Install rust-analyzer with rustup, so it matches the toolchain, and confirm Neovim's version, because the configuration API used here does not exist before 0.11:
\nrustup component add rust-analyzer\nrust-analyzer --version\nnvim --version | head -1rust-analyzer 1.99.0 (b940084d 2026-09-28)\nNVIM v0.12.6Using the rustup component rather than a separate download has one useful property: rust-analyzer reports the same version string as the compiler it was built with, 1.99.0 here, so there is no mismatch between the analyser's idea of the standard library and the compiler's. A distribution package or a download from the project can lag behind a new compiler.
Put this in ~/.config/nvim/init.lua (or, to try it first, in any file passed with nvim -u). It defines the server once and turns it on. The first call describes how to start and attach; the second activates it for every Rust buffer:
vim.lsp.config('rust_analyzer', {\n cmd = { 'rust-analyzer' },\n filetypes = { 'rust' },\n root_markers = { 'Cargo.toml', '.git' },\n settings = { ['rust-analyzer'] = { check = { command = 'clippy' } } },\n})\nvim.lsp.enable('rust_analyzer')Each field has one job. cmd is the program to run. filetypes limits attachment to Rust buffers. root_markers is how Neovim decides what the project is: it walks up from the file until it finds a Cargo.toml, and that directory becomes the root passed to the server, which means a workspace with several crates should have its own Cargo.toml at the top. settings forwards options to rust-analyzer; here, check.command = 'clippy' makes the background check run clippy instead of plain cargo check, so lint warnings show up as diagnostics. There is no plugin manager and no lspconfig: in 0.11 and later, the configuration API is part of Neovim, and plugins like nvim-lspconfig mostly supply ready-made defaults for other languages.
A headless run is the way to test an editor setup without looking at it, and a script is a way to prove the configuration rather than assume it. I pointed Neovim at a small crate (the signed-integer parser from the VS Code guide, a library plus a binary) and ran a Lua script with nvim --headless -u init.lua -c \"luafile test.lua\". The script opens src/main.rs, waits for a client, and prints what it found:
local function log(...) io.stdout:write(table.concat(vim.tbl_map(tostring, { ... }), ' '), '\\n') end\nvim.cmd('edit src/main.rs')\nlocal ok = vim.wait(60000, function() return #vim.lsp.get_clients({ bufnr = 0 }) > 0 end, 200)\nlocal c = vim.lsp.get_clients({ bufnr = 0 })[1]\nlog('attached:', ok, c and c.name, c and c.root_dir and vim.fs.basename(c.root_dir))\nlog('server version:', c.server_info and c.server_info.version)\nlog('capabilities: hover', c.server_capabilities.hoverProvider ~= nil,\n 'rename', c.server_capabilities.renameProvider ~= nil,\n 'inlayHint', c.server_capabilities.inlayHintProvider ~= nil)attached: true rust_analyzer nv\nserver version: 1.99.0 (b940084d 2026-09-28)\ncapabilities: hover true rename true inlayHint trueThe client attached, named rust_analyzer, with the root directory nv, the folder holding Cargo.toml. The server identifies as 1.99.0. The capabilities line is the server telling the client what it can do; any feature listed as false would have no mapping to call. If attached is false in your own run, the usual causes are, in order: the file is not detected as Rust (:set filetype?), no Cargo.toml above the file so no root is found, or rust-analyzer is not on the PATH Neovim sees. The command :checkhealth vim.lsp reports the active clients and flags a missing executable.
From 0.11 on, Neovim defines default mappings for LSP actions when a server attaches, so a fresh configuration is already usable. I asked Neovim what each key is bound to in a Rust buffer after attach:
\nK vim.lsp.buf.hover()\ngrn vim.lsp.buf.rename()\ngra vim.lsp.buf.code_action()\ngrr vim.lsp.buf.references()\ngri vim.lsp.buf.implementation()\ngrt vim.lsp.buf.type_definition()\ngO vim.lsp.buf.document_symbol()\n<C-S> vim.lsp.buf.signature_help()Notice what is missing from that list: go to definition. In recent Neovim it is not a gd mapping but the editor's tag stack: for a buffer with an attached server, Ctrl-] jumps to the definition and Ctrl-t returns, using the server through the tagfunc option. I did not press those keys in the headless run; I called the underlying request instead, as the next section shows. Most people add gd as a habit, and it is one line in a callback.
The script places the cursor on the call to parse_signed in main.rs (line 7, column 21) and sends the same protocol requests the keys send. rust-analyzer answers null until it has indexed the project, so the helper retries until it gets a real answer; in an interactive session you notice this as hover that does nothing for the first seconds in a big workspace. These are the results.
Hover returned the crate, the full signature and the doc comment, as Markdown:
\n```rust\napp\n```\n```rust\npub fn parse_signed(input: &str) -> Result<i64, String>\n```\n---\nParses a signed decimal integer, allowing a leading + or -.That is the documentation comment from lib.rs, pulled across files by the server's index. In the editor it appears in a floating window when you press K. It is also a quick way to see an inferred type: hover over a let binding and you get the type the compiler assigned.
Definition returned lib.rs line 1, the function in the other file of the crate. References returned 4: the definition, the call in main.rs, and the two uses in the unit tests inside lib.rs. The count includes the declaration because the request asks for it (includeDeclaration = true); grr shows the list in the quickfix window, and the quickfix list is the way to review a rename before you make it.
Rename is the request that shows why a language server beats a search-and-replace. Asked to rename parse_signed to parse_number, the server returned a workspace edit touching two files:
rename parse_signed -> parse_number edits: lib.rs x3, main.rs x1Three edits in lib.rs (the definition and the two test calls) and one in main.rs, four in total, each at a real use of the function. Text search would also have rewritten a comment, a string and any other function that happens to contain the name; the server rewrites only the symbol. The edits are returned and not applied by the request itself, so in the editor grn asks for the new name and then applies them across all open and unopened files.
To see diagnostics, the script writes a bad line into the buffer, saves, and waits until the client reports something. First a type error, let unused_value: i32 = \"text\";:
diagnostic: line 5 rust-analyzer expected i32, found &'static strrust-analyzer's own type checker found it as soon as the text changed, without running cargo. That is the fast, approximate source. A second run replaced the error with code that compiles but that clippy dislikes, let v = vec![1]; let _n = v.len() == 0;. This time the diagnostics came from the background cargo clippy run triggered by the save, a couple of seconds later:
diagnostic: line 5 clippy length comparison to zero ... `#[warn(clippy::len_zero)]` on by default\ndiagnostic: line 5 clippy using `is_empty` is clearer and more explicit: `v.is_empty()`(The first message is shortened where marked; the full text includes a link to the lint's documentation.) Two things matter here. The source field says which tool spoke, which is how you tell an approximation from the compiler. And clippy's output is the reason for the setting at the top: with the default check.command = \"check\", the second case would have produced no diagnostic at all. A type error also hides clippy's findings, since clippy does not run on code that does not compile; fix the errors first and the lints appear.
rust-analyzer formats by calling rustfmt, and Neovim exposes that as vim.lsp.buf.format(). I replaced the buffer with deliberately cramped code and formatted it:
vim.api.nvim_buf_set_lines(0, 0, -1, false, { 'fn main(){let x=1;println!(\"{x}\");}' })\nvim.lsp.buf.format({ async = false, timeout_ms = 20000 })fn main() {\n let x = 1;\n println!(\"{x}\");\n}That is rustfmt's output, produced through the language server, using the same rustfmt.toml the command line would use. Because it goes through the protocol, the same call formats other languages with their servers, which is the point of having one mapping.
The defaults are enough to start. A few additions make daily use smoother. This is the full file I would keep: it adds format on save, a gd mapping, inlay hints and a diagnostic style, all through LspAttach, which fires once for each buffer that gets a client. I ran the first block of this file in the headless tests above; the extra mappings and the visual settings below are standard Neovim API that I did not exercise in a headless session, so check them in your editor:
vim.lsp.config('rust_analyzer', {\n cmd = { 'rust-analyzer' },\n filetypes = { 'rust' },\n root_markers = { 'Cargo.toml', '.git' },\n settings = {\n ['rust-analyzer'] = {\n check = { command = 'clippy' },\n cargo = { allFeatures = true },\n },\n },\n})\nvim.lsp.enable('rust_analyzer')\n\nvim.diagnostic.config({ virtual_text = true, severity_sort = true })\n\nvim.api.nvim_create_autocmd('LspAttach', {\n callback = function(args)\n local client = vim.lsp.get_client_by_id(args.data.client_id)\n local opts = { buffer = args.buf }\n vim.keymap.set('n', 'gd', vim.lsp.buf.definition, opts)\n vim.keymap.set('n', '<leader>f', function() vim.lsp.buf.format({ async = true }) end, opts)\n if client and client:supports_method('textDocument/inlayHint') then\n vim.lsp.inlay_hint.enable(true, { bufnr = args.buf })\n end\n if client and client:supports_method('textDocument/completion') then\n vim.lsp.completion.enable(true, client.id, args.buf, { autotrigger = true })\n end\n end,\n})\n\nvim.api.nvim_create_autocmd('BufWritePre', {\n pattern = '*.rs',\n callback = function() vim.lsp.buf.format({ async = false }) end,\n})Walk through the additions. cargo.allFeatures = true makes the analyser (and the check) build with every feature enabled, which stops code under #[cfg(feature = \"x\")] from being greyed out, at the cost of checking features you may not ship. vim.diagnostic.config puts the message at the end of the line and sorts errors above warnings. In the LspAttach callback, client:supports_method guards each feature, so the same file works with servers that lack them. Inlay hints draw inferred types and parameter names as faint text, and are turned on per buffer. Completion uses the built-in client: with autotrigger the menu opens as you type. The BufWritePre autocommand runs the server's formatter before each save of a Rust file, so formatting is never a separate step. When I queried inlay hints for the first twenty lines of main.rs in the headless run, the server advertised support but the response was an empty list, and I did not investigate why; I cannot show you hint output from this guide.
The first request after opening a project is slow, and everything after is fast, because rust-analyzer must load the project's metadata, index the standard library and your dependencies, and then run a first cargo clippy. The helper in my script waited with retries for that reason, and your editor does the same when hover returns nothing in the first seconds. On large workspaces three settings help. Set the check command to check rather than clippy if the lint run is slow, and run clippy from a terminal or CI instead. Keep the target directory used by the analyser separate from your terminal builds with rust-analyzer.cargo.targetDir = true, so the two do not invalidate each other's cache and block each other on the cargo lock. And exclude generated directories with files.excludeDirs if the index feels heavy. I did not benchmark these on a large project; they are the commonly suggested knobs, and your own measurements decide.
Most failures in an LSP setup are one of three things, and the editor can tell you which. The health check lists every active client, the command it was started with, the root directory it chose and the settings it was sent, which is the whole configuration as the server received it, not as you remember writing it. This is the report from the headless session, trimmed to the parts that matter:
\nvim.lsp:\n- LSP log level : WARN\n- Log path: /Users/petehouston/.local/state/nvim/lsp.log\nvim.lsp: Active Clients ~\n- rust_analyzer (id: 1)\n - Version: 1.99.0 (b940084d 2026-09-28)\n - Root directory: .../scratchpad/nv\n - Command: { \"rust-analyzer\" }\n - Settings: { [\"rust-analyzer\"] = { check = { command = \"clippy\" } } }\n - Attached buffers: 1\nvim.lsp: Enabled Configurations ~\n- rust_analyzer:\n - filetypes: rust\n - root_markers: { \"Cargo.toml\", \".git\" }Read it as a checklist. If Active Clients is empty, the server never started or never attached: check that the file's filetype matches filetypes, that a root marker exists above the file and that the command is on the PATH Neovim inherited (a GUI launcher often has a shorter PATH than your shell). If the client is listed but features are missing, compare Settings with what you intended; a typo in a nested key is silently ignored by the server. If the Root directory is not the folder you expected, the analyser is indexing the wrong project, which shows up as \"cannot find module\" errors on code that compiles: in a workspace with several Cargo.toml files, the nearest one wins unless a root marker above it matches first. The log file named in the report holds the protocol traffic at higher levels. Turn it up with vim.lsp.log.set_level('debug') for one session, reproduce the problem, read the file, and turn it back down, because debug logging records every document change and grows quickly.
Two habits keep the setup healthy over time. Update rust-analyzer with the toolchain (rustup update), since the component is versioned with the compiler and a mismatched analyser is the most common cause of wrong \"cannot find\" errors after a compiler upgrade. And keep the editor configuration in version control next to your other dotfiles; the entire Rust setup here is under thirty lines of Lua, so a clean machine reaches a working editor in the time it takes to install the component and copy one file.
A repository with several crates under one [workspace] is where the root choice matters most. If the top-level Cargo.toml lists the members, open any file and the analyser should load the whole workspace, so go to definition and rename work across crates. Neovim picks the root by the markers in order, taking the first directory upward that contains any of them. With Cargo.toml listed first and each member crate having its own, the nearest member wins, and the analyser then sees only that crate and its dependencies, not its siblings. For rust-analyzer this is usually harmless, because it reads the workspace from the manifest and discovers the other members itself, but if rename misses a sibling crate, the root is the first thing to check in the health report. The reliable fix is to put the workspace root marker first by testing for [workspace] in a function passed as root_dir, and the lazy fix is to start Neovim from the workspace root.
nvim-dap, which this guide does not cover.gra but has no lens by default; running a single test is usually a terminal command or a plugin.vim.lsp.start or a plugin.cmd and nothing else.The measured result is a plugin-free setup that attached on the first try, returned a hover with the doc comment, found four references, planned a four-edit rename across two files, reported a type error and two clippy findings, and formatted through rustfmt. The editing experience comes from a few lines of configuration and a server that already knows your crate.
","date_published":"2026-11-03T10:20:00+00:00","tags":["Tooling"]},{"id":"https://rusthacks.dev/library/rust-dev-containers","url":"https://rusthacks.dev/library/rust-dev-containers","title":"Rust Dev Containers for VS Code and Reproducible Setup","summary":"A dev container puts the toolchain, system libraries and services in a Docker image that the editor attaches to. A new contributor opens the folder and everything works.","content_html":"A dev container is a development environment described in files and run in a container: the compiler, the system libraries, the debugger and the editor extensions all come from the repository, so a new contributor, a CI job and your laptop in a year all start from the same place. For Rust the payoff is concrete. The toolchain version stops drifting, native dependencies such as OpenSSL headers are installed once in a Dockerfile rather than in a wiki page, and \"works on my machine\" turns into a question about one image. This guide builds a Rust dev container for a small crate, runs the toolchain inside it, and reports what happened, including a permissions failure on a fresh cache volume that cost a build and the fix. Versions: Docker 29.9.2, the official rust:1.99-slim-bookworm image (rustc 1.99.1), on an Apple M1 Max with Docker Desktop. One limit up front: VS Code and the Dev Containers extension are not available on this machine, so I did not open the folder in an editor. Everything the editor would do with devcontainer.json (build the image, mount the volumes, run the commands) I ran by hand with the Docker CLI, and the guide marks the one place that depends on the editor.
The specification is a folder called .devcontainer with two files. The Dockerfile builds the image, and devcontainer.json tells the tooling how to run it: which user, which mounts, which extensions, which command to run after creation. Keeping the image definition separate from the runtime settings matters, because the Dockerfile can then be used unchanged by CI.
FROM rust:1.99-slim-bookworm\n\nARG USERNAME=dev\nARG UID=1000\n\nRUN apt-get update \\\n && apt-get install -y --no-install-recommends git pkg-config libssl-dev gdb ca-certificates \\\n && rm -rf /var/lib/apt/lists/* \\\n && useradd --create-home --uid ${UID} --shell /bin/bash ${USERNAME} \\\n && mkdir -p /usr/local/cargo/registry /workspace /workspace-target \\\n && chown -R ${USERNAME}:${USERNAME} /usr/local/cargo /workspace /workspace-target\n\nUSER ${USERNAME}\nENV CARGO_TARGET_DIR=/workspace-target\nRUN rustup component add rustfmt clippyRead it top to bottom. The base is the official Rust image, pinned to a minor version, so the compiler is the one in the tag and not whatever \"latest\" meant on the day of the build. slim drops compilers and tools you do not need, but it also drops pkg-config and the C library headers that many crates need at build time, so those are installed explicitly: libssl-dev for anything linking OpenSSL, gdb for debugging, git because the editor and cargo both want it. The package lists are deleted in the same RUN step so they do not stay in a layer. A non-root user is created, because a container running as root writes root-owned files into your project folder on Linux hosts, which is the most common complaint about dev containers. The last two lines set the cargo target directory outside the project and add the formatter and linter, which the slim image does not include. The reason for the target directory is covered below, after the bug it caused.
{\n \"name\": \"app\",\n \"build\": { \"dockerfile\": \"Dockerfile\" },\n \"remoteUser\": \"dev\",\n \"mounts\": [\n \"source=app-cargo-registry,target=/usr/local/cargo/registry,type=volume\",\n \"source=app-target,target=/workspace-target,type=volume\"\n ],\n \"runArgs\": [\"--cap-add=SYS_PTRACE\", \"--security-opt\", \"seccomp=unconfined\"],\n \"customizations\": {\n \"vscode\": {\n \"extensions\": [\"rust-lang.rust-analyzer\", \"vadimcn.vscode-lldb\", \"tamasfe.even-better-toml\"],\n \"settings\": { \"rust-analyzer.check.command\": \"clippy\" }\n }\n },\n \"postCreateCommand\": \"cargo fetch\"\n}I checked that the file parses as JSON; the Dev Containers specification also allows comments in it, and the tooling reads the keys shown. remoteUser makes the editor's terminal and extensions run as dev. The two mounts create named Docker volumes for the cargo registry and the build output, which persist across rebuilds of the container. runArgs adds the capabilities a debugger needs (more on that below). customizations.vscode lists the extensions that are installed inside the container, so the language server runs next to the compiler it needs. And postCreateCommand fetches dependencies once after creation.
The Dev Containers tooling builds from the Dockerfile; the equivalent by hand is one command:
\ndocker build -t app-dev -f .devcontainer/Dockerfile .devcontainerThe build took about 18 seconds on this machine with the base image already pulled. The result is 1.05 GB, against 817 MB for the base image: the extra 230 MB is gdb, git, the OpenSSL headers and the two rustup components. That is a one-time cost per developer, not per build, and layers are shared between images, but it is worth knowing that \"slim\" is not small once you add a debugger. A first command inside confirms who you are and what you have:
\ndocker run --rm app-dev bash -c 'id -un; id -u; rustc --version; cargo --version'dev\n1000\nrustc 1.99.1 (58cf8f9c3 2026-10-13)\ncargo 1.99.1 (5f94df478 2026-08-27)The user is dev with uid 1000 and the toolchain is 1.99.1. Note that the tag 1.99-slim-bookworm floats across patch releases: it was 1.99.1 the day I pulled it, and a later pull could carry 1.99.2. If you need byte-for-byte reproducibility, pin the full version (rust:1.99.1-slim-bookworm) or the image digest, and bump it deliberately.
My first devcontainer.json sent the build output to a named volume, app-target, mounted at /workspace-target, so that thousands of build files would not cross a slow host file-sharing layer. The first Dockerfile did not create that directory. I ran the build as the non-root user and got:
error: failed to create directory `/workspace-target/debug`\n\nCaused by:\n Permission denied (os error 13)Listing the mount point showed why: drwxr-xr-x 2 root root. When Docker creates a named volume at a path that does not exist in the image, it creates it as root. If the path does exist in the image, Docker copies that directory's ownership and contents into the empty volume on first use, so the cure is to create the directory in the Dockerfile and give it to the user before the user is switched. That is the mkdir -p ... /workspace-target and the extended chown in the Dockerfile shown above. Do the same for any directory you mount a volume onto. The registry volume worked from the start for exactly this reason: /usr/local/cargo/registry existed in the image and was owned by dev after the chown.
A smaller lesson hides in the same failure: my first timing script piped cargo's output through grep real, so the failure produced no visible error and a believable \"0.93 seconds\". Whenever you time a command, look at its exit status and output once before trusting the number.
With the fix in place I scripted the sequence the volumes exist for. The project depends on serde_json, so the first build has real work to do: download and compile it. Each step ran in a throwaway container, docker run --rm, with the same two volumes:
RUN=\"docker run --rm -v $PWD:/workspace -w /workspace \\\n -v app-cargo-registry:/usr/local/cargo/registry \\\n -v app-target:/workspace-target app-dev\"\n$RUN cargo build # cold: volumes empty\n$RUN cargo build # new container, volumes kept\ntouch src/main.rs\n$RUN cargo build # one source file changedcold (empty volumes) 2.92 s\nnew container, volumes kept 0.27 s\none source file touched 0.36 sThe cold build, including the dependency download, took 2.92 seconds. Throw the container away, start another, and the build finished in 0.27 seconds, because the registry and the compiled dependencies were still in the volumes. The container was disposable; the cache was not. That is the design to aim for: containers you can delete without a second thought, and a small set of named volumes that hold the expensive state. This crate is tiny, so the absolute numbers are small; on a project with hundreds of crates the same ratio is the difference between a minute and ten. In the same session cargo test reported 2 passed; 0 failed, cargo fmt --check passed and cargo clippy printed nothing, so the formatter and linter installed in the image work for the non-root user.
I did not measure bind-mounted versus volume build directories on a large project, and the claim that keeping target off the host file share is faster is widely reported for Docker Desktop on macOS and Windows but is not something this guide timed. On a Linux host, where bind mounts are native, putting target in the project folder is fine and lets host tools see it.
The source tree is a bind mount of the host folder, and anything the container writes there belongs to someone on the host. After the run above, the lockfile written through the mount was owned by uid 501 on the host, which is my macOS user, even though the process in the container ran as uid 1000. Docker Desktop for macOS translates ownership across its file-sharing layer, so it just works. On a Linux host there is no translation: a container user with uid 1000 writes files as uid 1000, and if your login is uid 1001 you will find files you cannot edit. The Dockerfile accepts a UID build argument for that, and the Dev Containers tooling has an updateRemoteUserUID option (on by default on Linux) that remaps the container user to match the host at start-up. I could not test the Linux case here, so treat it as the documented behaviour.
A project that already pins its compiler in rust-toolchain.toml meets the image's compiler as a second source of truth. To see what happens, I added a file requesting 1.99.0 and ran cargo --version in the container, whose installed version is 1.99.1:
info: syncing channel updates for 1.99.0-aarch64-unknown-linux-gnu\ninfo: latest update on 2026-10-01 for version 1.99.0 (b940084d7 2026-09-28)\ninfo: downloading 3 components\nwarn: the missing active toolchain `1.99.0-aarch64-unknown-linux-gnu` has been auto-installedrustup obeyed the project file and downloaded a second toolchain into the container's home directory, at run time, in the container's writable layer. It worked, and it is the wrong place for it: the download repeats in every fresh container, it is invisible in the image, and the editor may start before it finishes. Pick one source of truth. For a team that wants the container to be the environment, set the Dockerfile's base tag to the version in rust-toolchain.toml and keep them equal in review; or install the toolchain from the file during the build with RUN rustup show in the project directory so it is baked in. For a team that supports contributors who do not use the container, keep rust-toolchain.toml authoritative and let the image follow it.
The package list in the Dockerfile is the part that grows. A useful test for each line is whether a build or a test fails without it. pkg-config and libssl-dev pass that test for any crate that links OpenSSL, and a missing one shows up as a build-script error naming the library, which is easy to trace back to a package. Developer conveniences such as a shell prompt, ripgrep or jq fail the test, and they belong either in a personal dotfiles setup that the editor can apply to every container, or in a second image layer that only developers use. Keeping the base image to what the build needs also means the CI image, which is the same file, stays small and its cache stays warm. When you do add a tool, add it with a pinned version where the tool allows, such as cargo install --locked cargo-deny --version 0.20.2, so the environment is reproducible and a new release of a helper does not change what your contributors see.
Prebuilt features are the alternative to writing a Dockerfile. The Dev Containers specification lets devcontainer.json list features such as a Rust toolchain or Git, which the tooling layers onto a base image. They are convenient for a quick start and give you less control over exactly what is installed. I used a Dockerfile because it is the more transparent choice and the one a CI job can build without the Dev Containers tooling, and I did not test the features mechanism.
A debugger uses ptrace, and Docker's default seccomp profile and capability set have historically blocked it, which is why the example devcontainer.json adds --cap-add=SYS_PTRACE and seccomp=unconfined. I ran gdb in batch mode against the built binary twice, once with the defaults and once with those flags. In both runs the program ran under gdb and exited normally, with no ptrace error:
total = 0\n[Inferior 1 (process 18) exited normally]\nNo stack.So on Docker Desktop for Mac the flags were not needed for gdb to launch a process. I would not conclude that they are never needed: that Docker runs containers inside a Linux virtual machine with its own defaults, and a Linux host with Docker Engine and the standard seccomp profile is the case where debuggers have been reported to fail. The program in that test also ran without arguments and no breakpoint was hit, which means this was a check that gdb can start a process, not a debugging session. Keep the flags if your team includes Linux hosts, and note that seccomp=unconfined weakens the container's isolation, which matters if you run untrusted code in it.
A dev container pays for itself twice if CI runs the same image. The commands in the container are ordinary cargo, so a CI job needs no special tooling beyond running them in the image built from the same Dockerfile:
\ndocker build -t app-dev -f .devcontainer/Dockerfile .devcontainer\ndocker run --rm -v \"$PWD:/workspace\" -w /workspace app-dev \\\n bash -c 'cargo fmt --check && cargo clippy --all-targets -- -D warnings && cargo test'When a check fails in CI and passes locally, the usual cause is a different environment; here there is none to differ. The cost is image build time, which a registry cache or a prebuilt image removes. Most CI systems can also run a job directly in a container image, which skips the build step if you publish the image.
\ndevcontainer.json. Volumes survive a rebuild; only the image is replaced.docker volume rm app-cargo-registry app-target. The next build is cold, and nothing else is lost.devcontainer.json shape, which this guide does not cover.The measured facts are these: the image builds in about 18 seconds to 1.05 GB, a fresh volume mounted at a path the image does not contain is owned by root and breaks a non-root build until the Dockerfile creates and owns that path, a named volume turns a 2.92-second cold build into a 0.27-second one in a new container, and a conflicting rust-toolchain.toml silently downloads a second compiler at run time. Each of those is a line in a file you commit.
A Rust continuous integration pipeline is a short list of commands that must pass before code merges, run in the same order on every push. The commands are not the hard part; any team can write cargo test into a workflow file. The hard part is choosing the set that catches real problems without becoming a twenty-minute wait that people learn to ignore, and knowing which failure each step exists to catch. This guide builds that pipeline around a small crate: formatting, clippy, tests, documentation, a minimum supported Rust version (MSRV) check and a dependency policy check. Every step was run locally, a failure was provoked on purpose for each one, and the real output is shown, so you know what a red build will look like before you meet one. The toolchain is rustc 1.99.0, cargo-deny 0.20.2, edition 2024, on an Apple M1 Max. The YAML for GitHub Actions at the end was not run on GitHub's servers; the commands inside it were all run here, and the guide says where that distinction matters.
The subject is a tiny library that counts words. It has a unit test, a doctest, and a documented public function, which is enough to give every check something to do:
\n//! Word frequency counting.\n\nuse std::collections::HashMap;\n\n/// Counts how often each lowercase word appears.\n///\n/// ```\n/// let counts = ci::count_words(\"the cat the hat\");\n/// assert_eq!(counts[\"the\"], 2);\n/// ```\npub fn count_words(text: &str) -> HashMap<String, usize> {\n let mut counts = HashMap::new();\n for word in text.split_whitespace() {\n *counts.entry(word.to_lowercase()).or_insert(0) += 1;\n }\n counts\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn counts_case_insensitively() {\n let c = count_words(\"Go go GO\");\n assert_eq!(c[\"go\"], 3);\n }\n}Two files outside the source pin the environment. A rust-toolchain.toml names the compiler and its components, so the version on a laptop and the version in CI cannot drift apart. And the manifest declares rust-version, the oldest compiler the crate promises to support:
[toolchain]\nchannel = \"1.99.0\"\ncomponents = [\"rustfmt\", \"clippy\"][package]\nname = \"ci\"\nversion = \"0.1.0\"\nedition = \"2024\"\nlicense = \"MIT\"\nrust-version = \"1.85\"\n\n[dependencies]\nserde = { version = \"1.0.229\", features = [\"derive\"] }\nserde_json = \"1.0.154\"Pinning the toolchain to an exact version is the single most useful decision in the file. A pipeline on stable fails on a day nobody touched the code, because a new compiler release added a lint. That failure is real information, but it should arrive as a deliberate update pull request, not as a surprise on someone else's unrelated change.
cargo fmt --check exits non-zero and prints a diff if any file differs from rustfmt's output. It changes nothing, which is why it belongs in CI; the fix is for a developer to run cargo fmt locally. To see what the failure looks like I removed the spaces around one =:
Diff in src/lib.rs:9:\n /// assert_eq!(counts[\"the\"], 2);\n /// ```\n pub fn count_words(text: &str) -> HashMap<String, usize> {\n- let mut counts=HashMap::new();\n+ let mut counts = HashMap::new();\n for word in text.split_whitespace() {\n *counts.entry(word.to_lowercase()).or_insert(0) += 1;\n }The exit code was 1. Run this first. It takes under a second, it needs no compilation, and a formatting failure should never cost a contributor a full build before they learn about it. The same ordering logic applies to the rest of the list: cheapest and most likely to fail goes first.
\nClippy catches code that compiles but is probably not what you meant. In CI it should fail the build, otherwise warnings accumulate until nobody reads them. The flags matter. --all-targets includes tests, examples and benches, which a plain cargo clippy skips. -- -D warnings promotes every warning to an error. I introduced a classic: calling .into_iter() on something that is already an iterator.
error: useless conversion to the same type: `std::str::SplitWhitespace<'_>`\n --> src/lib.rs:13:17\n |\n13 | for word in text.split_whitespace().into_iter() {\n | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: consider removing `.into_iter()`: `text.split_whitespace()`\n |\n = help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.99.0/index.html#useless_conversion\n = note: `-D clippy::useless-conversion` implied by `-D warnings`\n = help: to override `-D warnings` add `#[allow(clippy::useless_conversion)]`\n\nerror: could not compile `ci` (lib test) due to 1 previous error\nA note on the pinned toolchain and clippy: the help link in that message points at rust-1.99.0 documentation, because the lint list is versioned with the compiler. Another reason to pin. The output above is trimmed to the first error; clippy printed a second, identical one for the library build.
cargo test runs unit tests, integration tests and doctests in one command, which is why nobody should replace it with a custom script. Doctests are the part teams forget: they compile your documentation examples as tiny programs, so a README-style example that no longer builds fails here. I changed the expected count in the unit test from 3 to 2:
failures:\n\n---- tests::counts_case_insensitively stdout ----\n\nthread 'tests::counts_case_insensitively' panicked at src/lib.rs:26:9:\nassertion `left == right` failed\n left: 3\n right: 2\nnote: run with `RUST_BACKTRACE=1` environment variable to display a backtrace\n\nfailures:\n tests::counts_case_insensitively\n\ntest result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00sTwo flags are worth adding in CI. --locked makes cargo fail if Cargo.lock would need to change, which catches a dependency edit committed without its lockfile update. And for a library crate you also want cargo test --no-default-features and --all-features runs if the crate has features, because the default feature set is the one combination nobody forgets to test.
The --locked flag deserves a demonstration, because the failure is quiet in its absence. I added a new line to the dependency table, fastrand = \"2\", without regenerating the lockfile, then ran the tests the way CI would:
Updating crates.io index\nerror: cannot update the lock file Cargo.lock because --locked was passed to prevent this\nhelp: to generate the lock file without accessing the network, remove the --locked flag and use --offline instead.Without the flag, cargo would have resolved the new crate on the build machine, written a lockfile nobody committed, and gone green. The next developer to pull the branch would then resolve differently. With the flag, the pull request fails until the manifest and the lockfile agree. My first attempt at this experiment, adding itoa, did not fail, and the reason is a useful caution: itoa was already in the lockfile as a dependency of serde_json, so the lock did not need to change. The flag only objects when resolution would produce a different file.
After the runs above the target directory of this tiny crate was 79 MB. Almost all of it is the compiled output of serde, serde_json, syn, quote and proc-macro2, none of which changed between runs. That is what a CI cache restores, and it is why the 6.47 second cold run collapsed to 0.74 seconds on the second run. Three habits keep the cache honest:
RUSTFLAGS=\"-D warnings\" and a job that does not will not share compiled dependencies, so a mismatch silently halves the benefit of the cache.The cache has one limit that no setting fixes: the first build after the lockfile changes is cold. A pull request that updates a core dependency pays the full price once, and every later run is warm again. Plan around it and do not read one slow run as a regression.
\nA workflow file is code that runs with credentials, and a pipeline for an open-source crate runs on pull requests from strangers. The YAML above is the functional part; these changes harden it, and none of them alter what the checks do:
\npermissions:\n contents: read\n\nconcurrency:\n group: ci-${{ github.ref }}\n cancel-in-progress: truepermissions: contents: read reduces the default token to read-only, so a compromised step cannot push code or edit releases. concurrency with cancel-in-progress stops the previous run when a new commit arrives on the same branch, which saves minutes on a busy pull request. For stricter supply-chain hygiene, pin third-party actions to a full commit SHA rather than a tag, since a tag can be moved. Only publish jobs should ever see a secret; a test job needs none, and a pull request from a fork does not receive them at all.
The value of a pipeline is decided on the day it fails, so shape the output for that day. Keep each check as its own step with its own name, so the first red line in the summary says \"clippy\" or \"docs\" without anyone opening a log. Print the diff for formatting (the default does). Keep test output on failure only; the standard harness already hides the output of passing tests. And resist the pull to put every command into one make ci line: a single step that does seven things fails with seven possible meanings. Separate steps cost nothing in time because they share the same machine and the same cache.
When a check fails only in CI, the first suspects are always the same three: a different compiler version (check the pin), a different feature set (check --all-features and the default features), and a different operating system. A pipeline that logs rustc --version --verbose and the exact test command at the top of the job makes the first of those a ten-second check. Add that line; it is one line of YAML and it ends more arguments than any other.
Broken intra-doc links are silent in a normal build and show up as dead text on docs.rs. cargo doc with RUSTDOCFLAGS=\"-D warnings\" turns them into failures. --no-deps skips documenting every dependency, which is slow and not your problem. I linked to a function that does not exist:
error: unresolved link to `missing_fn`\n --> src/lib.rs:5:57\n |\n5 | /// Counts how often each lowercase word appears, see [`missing_fn`].\n | ^^^^^^^^^^ no item named `missing_fn` in scope\n |\n = note: `-D rustdoc::broken-intra-doc-links` implied by `-D warnings`\n = help: to override `-D warnings` add `#[allow(rustdoc::broken_intra_doc_links)]`\n\nerror: could not document `ci`If the manifest says rust-version = \"1.85\", something must check it, or the number is a guess. The check is to build with exactly that compiler. The first instinct, to set the MSRV to whatever is oldest that still works, runs into a floor set by the language edition. I tried 1.84.0 on this crate:
error: failed to parse manifest at `Cargo.toml`\n\nCaused by:\n feature `edition2024` is requiredEdition 2024 shipped in 1.85, so no compiler older than that can read the manifest, and 1.85 is the honest floor for this crate. With cargo +1.85.0 check the build finished cleanly in 5.49 seconds on a cold target directory. In CI, install that exact toolchain in a separate job and run cargo check --locked --all-features with it. Use check, not test: dev-dependencies are allowed to need newer compilers than the library itself, and a compile check is enough to prove the promise you made to users. Raising the MSRV is a normal, deliberate change, but it should be a change someone made on purpose, which is what the job enforces.
Everything so far checks your code. cargo deny checks what you depend on: licences, known security advisories, banned crates and duplicate versions. A minimal deny.toml that allows only two licences looks reasonable and fails on the first run:
[licenses]\nallow = [\"MIT\", \"Apache-2.0\", \"Unicode-3.0\"]\n\n[bans]\nmultiple-versions = \"warn\"With only MIT and Apache-2.0 on the list, cargo deny check licenses bans printed bans ok, licenses FAILED. One cause was my own crate: it had no license field, and cargo-deny reports \"ci = 0.1.0 is unlicensed\" as an error. The other was a transitive dependency of serde under Unicode-3.0, which I had not allowed. After adding the field and the licence to the list:
bans ok, licenses okThis is the most useful lesson from the step: a policy you write from memory is wrong the first time you run it, and the failure tells you something about your real dependency tree. Run it once locally, fix the list, then commit it. The advisory check (cargo deny check advisories) needs to fetch the RustSec database, so it is the one check that fails for network reasons, not code reasons; schedule it nightly in addition to running it on pull requests, because a new advisory can appear against code that has not changed.
The local version of the whole pipeline is a short script. Order is by cost and by likelihood of failure:
\nset -e\ncargo fmt --check\ncargo clippy --all-targets --locked -- -D warnings\ncargo test --locked\nRUSTDOCFLAGS=\"-D warnings\" cargo doc --no-deps --lockedI timed it twice on this crate, after adding serde and serde_json as dependencies. With an empty target directory the whole script took 6.47 seconds. Run again immediately, with the build artefacts in place, it took 0.74 seconds. The gap between those two numbers is the entire argument for caching in CI: almost all of a Rust pipeline's time is compiling dependencies that did not change. The crate here is tiny, so the cold run is short; on a project with a few hundred crates the same ratio turns six seconds into ten minutes.
\nThis is the GitHub Actions version. I did not run it on GitHub; each command in it is one I ran above, and the structure follows the actions' documented inputs. The decisions to notice are the pinned toolchain, one cache action, a separate MSRV job and --locked everywhere:
name: ci\non:\n push:\n branches: [main]\n pull_request:\n\nenv:\n CARGO_TERM_COLOR: always\n RUSTFLAGS: \"-D warnings\"\n\njobs:\n check:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: dtolnay/rust-toolchain@master\n with:\n toolchain: 1.99.0\n components: rustfmt, clippy\n - uses: Swatinem/rust-cache@v2\n - run: cargo fmt --check\n - run: cargo clippy --all-targets --locked -- -D warnings\n - run: cargo test --locked\n - run: cargo doc --no-deps --locked\n env:\n RUSTDOCFLAGS: \"-D warnings\"\n\n msrv:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: dtolnay/rust-toolchain@master\n with:\n toolchain: 1.85.0\n - uses: Swatinem/rust-cache@v2\n - run: cargo check --locked --all-features\n\n deny:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: EmbarkStudios/cargo-deny-action@v2Why each choice. RUSTFLAGS: \"-D warnings\" at the job level makes the compiler's own warnings fatal, so an unused import fails the build the same way a clippy lint does. The cache action keys on the lockfile and the compiler version, restoring ~/.cargo and target, and it prunes workspace crates so the cache does not grow without bound. Running msrv and deny as separate jobs lets them run in parallel with the main job instead of extending it, and gives each a clear name when it goes red. One more change to avoid is putting cargo build --release in the pull-request path; a release build is for the deploy job, and it doubles the wall time without testing anything the debug build did not.
A pipeline that only runs in the cloud trains people to push and wait. The commands in this guide are all plain cargo, so the cheapest improvement is to make the local and remote versions identical. Keep the four-line script from earlier in the repository and call it from both the workflow and a git pre-push hook, or from a task runner target everyone knows. The point is not enforcement. It is that a contributor who sees a red clippy error on their own machine, in under a second on a warm build, fixes it before it becomes a review comment, and a pull request that arrives already green spends its reviewers' attention on the design instead of on whitespace. When the local script and the workflow ever disagree, treat that as a bug in one of them, because the script is the definition of passing and the workflow is only the place it is enforced.
\nA pipeline of these six steps is a floor, not a ceiling, and some advice does not fit every project:
\nstable and beta to see new lints coming; that job should be allowed to fail without blocking merges.--no-default-features and --all-features, and should not commit Cargo.lock to be consumed. An application should commit it and use --locked everywhere, which is why the script above uses it.The measurements that justify the shape are the ones above: formatting fails in under a second with a readable diff, a clippy error tells you the lint and the replacement, a failing test prints both values, a broken doc link names the missing item, and a pinned MSRV makes a language-edition floor visible instead of a surprise. The cold run of 6.47 seconds against a warm run of 0.74 shows why the cache step pays for itself, and the failed licence check on the first attempt shows why the dependency policy should be run on your real tree before you commit it.
","date_published":"2026-11-03T10:02:00+00:00","tags":["Testing"]},{"id":"https://rusthacks.dev/library/rust-vector-search-pgvector","url":"https://rusthacks.dev/library/rust-vector-search-pgvector","title":"Vector Search From Rust With pgvector and Embeddings","summary":"pgvector adds a vector type and nearest-neighbor operators to Postgres. From Rust you store embeddings as arrays, index them with HNSW and query with an ORDER BY on distance.","content_html":"A search box that understands meaning, a \"related posts\" panel, a retrieval step for a question-answering bot: all of them reduce to the same operation. Turn each document into a list of numbers (an embedding) such that similar documents get nearby lists, store the lists, and when a query arrives, turn it into a list too and ask for the closest stored ones. PostgreSQL does this with the pgvector extension, which adds a vector column type, distance operators and approximate indexes, so the vectors live next to the rest of your data, in transactions, with your backups. This guide builds the whole path from Rust: an extension in a container, a table, inserts, the three distance operators and how they relate, a measured comparison of exact search against an HNSW index (including how much accuracy you give up for speed), the filtered-search trap that returns zero rows, and the operational facts that decide whether it stays fast. Everything ran with Rust 1.99, SQLx 0.9.0, the pgvector crate 0.4.2 and PostgreSQL 17 with pgvector 0.8.7, on an Apple M1 Max with the database in a container. One piece is deliberately a stand-in, the embedding model, and the text says where.
An embedding model maps text (or images, or audio) to a vector with a fixed number of dimensions, say 384 or 1,536, so that the geometric distance between two vectors reflects how similar their meanings are. \"Rust async runtime\" lands close to \"tokio executor\" and far from \"sourdough starter\". The database does not know anything about meaning. It stores vectors and answers \"which stored vectors are nearest to this one?\", and the quality of the answers is entirely the model's. That separation is useful for design: the database layer (this guide) is the same whichever model you pick, and you can swap models by re-computing the vectors.
\nNearest means a choice of distance. pgvector offers three operators that you will use:
\n| Operator | Distance | Use when |
|---|---|---|
<=> | Cosine distance (1 minus cosine similarity) | The default for text embeddings; direction matters, length does not |
<-> | Euclidean (L2) distance | Vectors where magnitude is meaningful, or the model says so |
<#> | Negative inner product | Vectors already normalised to length 1 (fastest, and equivalent to cosine for them) |
The negation in <#> exists because Postgres indexes return the smallest value first, so a larger inner product (more similar) has to sort as a smaller number. Follow the model's documentation: most text embedding models are trained for cosine similarity.
Run PostgreSQL in a container, add the extension and create it in your database. On the official Debian-based image the extension is a package from the Postgres apt repository that the image already trusts; there are also ready-made images with it built in:
\ndocker run -d --name pg --shm-size=1g -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=app -p 55433:5432 postgres:17\ndocker exec pg bash -c \"apt-get update -qq && apt-get install -y -qq postgresql-17-pgvector\"\ndocker exec pg psql -U postgres -d app -c \"CREATE EXTENSION vector;\" -c \"SELECT extversion FROM pg_extension WHERE extname = 'vector';\"Two details come from running it. --shm-size=1g is not decoration: Docker gives a container 64 MB of shared memory by default, and building a vector index in parallel needs more. With the default, the index build in this guide failed with could not resize shared memory segment ... No space left on device, an error that mentions disk and is actually about /dev/shm. The other is the extension version, 0.8.7 here; features such as iterative index scans (used below) need pgvector 0.8 or newer, so check extversion before you rely on them.
[dependencies]\nsqlx = { version = \"0.9\", features = [\"runtime-tokio\", \"postgres\"] }\npgvector = { version = \"0.4\", features = [\"sqlx\"] }\ntokio = { version = \"1\", features = [\"full\"] }The pgvector crate provides a Vector type with SQLx Encode and Decode implementations behind its sqlx feature, so you can bind and read vectors like any other column type; it supports SQLx 0.8 and 0.9. The first program creates a table, embeds seven short documents, and queries with each operator:
use pgvector::Vector;\nuse sqlx::{AssertSqlSafe, PgPool};\n\nconst DIMS: usize = 64;\n\n/// A toy embedding: hash each word into one of 64 buckets with a sign, then L2-normalise.\n/// A real system would call an embedding model here; everything after this function is unchanged.\nfn embed(text: &str) -> Vector {\n let mut v = vec![0f32; DIMS];\n for word in text.split(|c: char| !c.is_alphanumeric()).filter(|w| !w.is_empty()) {\n let mut h: u64 = 0xcbf29ce484222325;\n for b in word.to_lowercase().bytes() {\n h = (h ^ u64::from(b)).wrapping_mul(0x100000001b3);\n }\n let sign = if (h >> 63) & 1 == 0 { 1.0 } else { -1.0 };\n v[(h % DIMS as u64) as usize] += sign;\n }\n let norm = v.iter().map(|x| x * x).sum::<f32>().sqrt().max(1e-9);\n Vector::from(v.into_iter().map(|x| x / norm).collect::<Vec<f32>>())\n}\n\n#[tokio::main]\nasync fn main() -> Result<(), sqlx::Error> {\n let pool = PgPool::connect(&std::env::var(\"DATABASE_URL\").unwrap()).await?;\n sqlx::query(\"CREATE EXTENSION IF NOT EXISTS vector\").execute(&pool).await?;\n sqlx::query(\"DROP TABLE IF EXISTS docs\").execute(&pool).await?;\n sqlx::query(\"CREATE TABLE docs (id bigserial PRIMARY KEY, body text NOT NULL, embedding vector(64) NOT NULL)\").execute(&pool).await?;\n\n let docs = [\n \"tokio is an async runtime for rust network services\",\n \"async await in rust compiles to state machines\",\n \"how to bake sourdough bread with a long cold ferment\",\n \"rust ownership and borrowing explained with examples\",\n \"postgres index types btree gin gist and hnsw\",\n \"a recipe for slow roasted tomato soup with basil\",\n \"vector search finds nearest neighbours by distance\",\n ];\n for body in docs {\n sqlx::query(\"INSERT INTO docs (body, embedding) VALUES ($1, $2)\").bind(body).bind(embed(body)).execute(&pool).await?;\n }\n\n for (label, op) in [(\"cosine distance <=>\", \"<=>\"), (\"euclidean (L2) <->\", \"<->\"), (\"negative inner <#>\", \"<#>\")] {\n let q = embed(\"async rust runtime\");\n let sql = format!(\"SELECT body, embedding {op} $1 AS dist FROM docs ORDER BY embedding {op} $1 LIMIT 3\");\n let rows: Vec<(String, f64)> = sqlx::query_as(AssertSqlSafe(sql)).bind(&q).fetch_all(&pool).await?;\n println!(\"{label}\");\n for (body, dist) in rows {\n println!(\" {dist:>7.4} {body}\");\n }\n }\n Ok(())\n}\ncosine distance <=>\n 0.4226 tokio is an async runtime for rust network services\n 0.5918 async await in rust compiles to state machines\n 0.7818 rust ownership and borrowing explained with examples\neuclidean (L2) <->\n 0.9194 tokio is an async runtime for rust network services\n 1.0879 async await in rust compiles to state machines\n 1.2504 rust ownership and borrowing explained with examples\nnegative inner <#>\n -0.5774 tokio is an async runtime for rust network services\n -0.4082 async await in rust compiles to state machines\n -0.2182 rust ownership and borrowing explained with examplesLook at the structure before the numbers. The embed function is the only model-dependent code; it takes text and returns a Vector. Here it is a toy: it hashes each word into one of 64 buckets with a random sign and normalises the result, which makes texts that share words land close together but knows nothing about synonyms. It exists so the example runs with no API key and no model download, and it is not a substitute: in a real system you replace the body with a call to an embedding service or a local model, keep the dimensions in sync with the column (vector(64) becomes vector(1536), say), and everything else in this guide is unchanged. The results are still sensible, because the query \"async rust runtime\" shares words with the first two documents, and the bread and soup recipes do not appear.
Now the numbers. All three operators ranked the same three documents in the same order, which is expected for unit vectors, and the values relate exactly: the first result has cosine distance 0.4226, so its inner product is 1 - 0.4226 = 0.5774, which is the (negated) value in the third block, and its Euclidean distance is the square root of 2 x 0.4226, which is 0.9194. For normalised vectors the three metrics are the same ranking in different units, so choose by speed and by what your index was built with. A second Rust detail from this build: SQLx 0.9 refuses a runtime-built SQL string at compile time (\"dynamic SQL strings should be audited for possible injections\"), because query() now only takes literals. The demo builds its query text from a fixed list of operators, so it wraps it in AssertSqlSafe(...), which is you promising that nothing from outside went into the string. Values always go through bind, never through the string.
Without an index, ORDER BY embedding <=> $1 LIMIT 10 compares the query with every row: an exact answer, and a cost that grows with the table. The benchmark loads 100,000 vectors of 128 dimensions, drawn from 100 clusters (real embeddings cluster; uniformly random points in high dimensions are a misleading test), and times 30 queries:
use pgvector::Vector;\nuse sqlx::{AssertSqlSafe, PgPool, QueryBuilder, Postgres, Row};\nuse std::time::Instant;\n\nconst DIMS: usize = 128;\nconst N: usize = 100_000;\nconst CLUSTERS: usize = 100;\n\nstruct Rng(u64);\nimpl Rng {\n fn next(&mut self) -> f32 {\n self.0 = self.0.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);\n ((self.0 >> 40) as f32 / (1u64 << 24) as f32) * 2.0 - 1.0\n }\n}\n\nfn normalise(mut v: Vec<f32>) -> Vec<f32> {\n let n = v.iter().map(|x| x * x).sum::<f32>().sqrt();\n v.iter_mut().for_each(|x| *x /= n);\n v\n}\n\nfn point(rng: &mut Rng, centers: &[Vec<f32>]) -> Vec<f32> {\n let c = ¢ers[((rng.next() + 1.0) * 0.5 * CLUSTERS as f32) as usize % CLUSTERS];\n normalise(c.iter().map(|x| x + 0.35 * rng.next()).collect())\n}\n\nasync fn top10(pool: &PgPool, q: &Vector) -> Result<Vec<i64>, sqlx::Error> {\n let rows = sqlx::query(\"SELECT id FROM items ORDER BY embedding <=> $1 LIMIT 10\").bind(q).fetch_all(pool).await?;\n Ok(rows.iter().map(|r| r.get::<i64, _>(0)).collect())\n}\n\n#[tokio::main]\nasync fn main() -> Result<(), sqlx::Error> {\n let pool = PgPool::connect(&std::env::var(\"DATABASE_URL\").unwrap()).await?;\n sqlx::query(\"CREATE EXTENSION IF NOT EXISTS vector\").execute(&pool).await?;\n sqlx::query(\"DROP TABLE IF EXISTS items\").execute(&pool).await?;\n sqlx::query(\"CREATE TABLE items (id bigserial PRIMARY KEY, embedding vector(128) NOT NULL)\").execute(&pool).await?;\n\n let mut rng = Rng(42);\n let centers: Vec<Vec<f32>> = (0..CLUSTERS).map(|_| normalise((0..DIMS).map(|_| rng.next()).collect())).collect();\n\n let t = Instant::now();\n for _ in 0..N / 1000 {\n let mut qb: QueryBuilder<Postgres> = QueryBuilder::new(\"INSERT INTO items (embedding) \");\n qb.push_values(0..1000, |mut b, _| {\n b.push_bind(Vector::from(point(&mut rng, ¢ers)));\n });\n qb.build().execute(&pool).await?;\n }\n println!(\"inserted {N} vectors of {DIMS} dimensions in {:.1} s\", t.elapsed().as_secs_f64());\n sqlx::query(\"ANALYZE items\").execute(&pool).await?;\n\n let queries: Vec<Vector> = (0..30).map(|_| Vector::from(point(&mut rng, ¢ers))).collect();\n\n let t = Instant::now();\n let mut exact = Vec::new();\n for q in &queries {\n exact.push(top10(&pool, q).await?);\n }\n let seq = t.elapsed().as_secs_f64() * 1000.0 / queries.len() as f64;\n println!(\"exact (sequential scan): {seq:.1} ms per query\");\n\n let t = Instant::now();\n sqlx::query(\"CREATE INDEX items_hnsw ON items USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 64)\").execute(&pool).await?;\n println!(\"HNSW index built in {:.1} s\", t.elapsed().as_secs_f64());\n let size: i64 = sqlx::query_scalar(\"SELECT pg_relation_size('items_hnsw')\").fetch_one(&pool).await?;\n let table: i64 = sqlx::query_scalar(\"SELECT pg_relation_size('items')\").fetch_one(&pool).await?;\n println!(\"table {:.0} MB, index {:.0} MB\", table as f64 / 1048576.0, size as f64 / 1048576.0);\n\n for ef in [10, 40, 100, 400] {\n let mut conn = pool.acquire().await?;\n sqlx::query(AssertSqlSafe(format!(\"SET hnsw.ef_search = {ef}\"))).execute(&mut *conn).await?;\n let t = Instant::now();\n let mut hits = 0;\n for (q, truth) in queries.iter().zip(&exact) {\n let rows = sqlx::query(\"SELECT id FROM items ORDER BY embedding <=> $1 LIMIT 10\").bind(q).fetch_all(&mut *conn).await?;\n hits += rows.iter().filter(|r| truth.contains(&r.get::<i64, _>(0))).count();\n }\n let ms = t.elapsed().as_secs_f64() * 1000.0 / queries.len() as f64;\n println!(\"hnsw.ef_search = {ef:>3}: {ms:>5.2} ms per query, recall@10 = {:.1}%\", hits as f64 * 100.0 / (queries.len() * 10) as f64);\n }\n Ok(())\n}\ninserted 100000 vectors of 128 dimensions in 0.7 s\nexact (sequential scan): 12.8 ms per query\nHNSW index built in 35.4 s\ntable 56 MB, index 79 MB\nhnsw.ef_search = 10: 0.85 ms per query, recall@10 = 39.3%\nhnsw.ef_search = 40: 1.05 ms per query, recall@10 = 67.3%\nhnsw.ef_search = 100: 1.58 ms per query, recall@10 = 78.7%\nhnsw.ef_search = 400: 4.74 ms per query, recall@10 = 91.7%Exact search over 100,000 vectors took 12.8 ms per query. That is perfectly acceptable for a small table and does not scale: ten million vectors would be about 1.3 seconds. The remedy is an approximate nearest neighbour index, and the numbers show the cost of approximation honestly, which most introductions leave out. The HNSW index (hierarchical navigable small world, a layered proximity graph) answered in under a millisecond to 4.7 ms depending on one setting, hnsw.ef_search, the size of the candidate list kept during the graph walk. At 10 it was 15 times faster than the scan and found only 39 percent of the true ten nearest neighbours; at 400 it was 2.7 times faster and found 92 percent. This is recall, the fraction of the true top ten that the index returned, measured against the exact answers computed first. No setting reaches 100 percent for free. Choose the recall your product needs (a \"related posts\" widget tolerates 70 percent; a retrieval step feeding an answer-generation model usually wants 90 or more), measure it on your own data, and set ef_search to the cheapest value that meets it. The index also costs to build and store: 35 seconds and 79 MB here, larger than the 56 MB table, so budget memory for it (a vector index is fast when it fits in cache).
The index in the benchmark was created with:
\nCREATE INDEX items_hnsw ON items USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 64);The operator class (vector_cosine_ops) must match the operator in your queries: an index built for cosine distance is not used for <->. m is the number of connections per node in the graph (16 is the default) and ef_construction is the candidate list size while building (64 is the default); raising either improves recall at a given ef_search and costs build time and index size. They are fixed at build time, while ef_search is a per-session setting you can change per query, which is why tuning starts there. pgvector also offers IVFFlat indexes, which cluster the vectors into lists and search the nearest few lists: they build faster and use less memory, and generally give lower recall at the same speed; they must be built after the table has data, since the clusters come from it. HNSW is the usual default.
Real queries have a filter: nearest documents in this category, for this tenant. With an approximate index, the filter is applied after the index returns its candidates, and that interacts badly with selective filters. On the same table, with a category column that splits the rows into 100 groups of 1,000, a search for the ten nearest items in one category with ef_search = 40:
-- filtered search, default settings\n count\n-------\n 0\n\n-- same with iterative scan\n count\n-------\n 10The default returned zero rows. The index produced its 40 nearest neighbours of the query vector across the whole table, and none of them happened to be in the requested category, so the filter discarded all of them. Nothing failed; the query simply answered \"no results\". The fix in pgvector 0.8 and newer is iterative index scans: SET hnsw.iterative_scan = relaxed_order; tells the scan to keep walking the graph until it has enough rows that pass the filter (or reaches a limit), and the same query returned 10. The mode name states the trade-off: results are close to, but not strictly in, distance order, and you can re-sort the final ten in SQL. Other remedies when the filter is very selective: a partial index per tenant, a table partition per tenant, or an exact search when the filter leaves few rows (the planner should choose it if statistics are fresh, so run ANALYZE). The worst habit is to test with an unfiltered query and ship a filtered one. Add a test that asserts the filtered query returns the requested number of rows for a selective filter.
Limit\n InitPlan 1\n -> Index Scan using items_pkey on items items_1\n Index Cond: (id = 5000)\n -> Index Scan using items_hnsw on items\n Order By: (embedding <=> (InitPlan 1).col1)That is the plan to look for in EXPLAIN: an Index Scan using items_hnsw with an Order By on the distance expression. If you see Seq Scan and a Sort, the index is not used. The usual causes are a mismatched operator class, a missing LIMIT (the index serves ordered top-k queries, not unbounded ones), a cast that changes the type, or stale statistics on a small table where the planner correctly judges the scan cheaper. Reading plans is the subject of PostgreSQL EXPLAIN and indexes.
The toy embed function is the seam where the real model goes, and the way to keep that seam clean is a small trait that also enforces the two invariants that matter when you swap models: batch sizes match, and the dimensions equal the column's. A real implementation of this trait would make an HTTP request to an embedding service (or run a local model) in embed; the pipeline around it does not change.
use anyhow::{bail, Result};\n\ntrait Embedder {\n fn model(&self) -> &str;\n fn dimensions(&self) -> usize;\n async fn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>>;\n}\n\nstruct HashEmbedder;\n\nimpl Embedder for HashEmbedder {\n fn model(&self) -> &str {\n \"toy-hash-64\"\n }\n\n fn dimensions(&self) -> usize {\n 64\n }\n\n async fn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>> {\n Ok(texts\n .iter()\n .map(|t| {\n let mut v = vec![0f32; 64];\n for w in t.split_whitespace() {\n let h = w.bytes().fold(0xcbf29ce484222325u64, |h, b| (h ^ u64::from(b)).wrapping_mul(0x100000001b3));\n v[(h % 64) as usize] += 1.0;\n }\n let n = v.iter().map(|x| x * x).sum::<f32>().sqrt().max(1e-9);\n v.iter().map(|x| x / n).collect()\n })\n .collect())\n }\n}\n\nstruct Row {\n body: String,\n model: String,\n vector: Vec<f32>,\n}\n\nasync fn embed_all<E: Embedder>(embedder: &E, texts: Vec<String>, batch: usize) -> Result<Vec<Row>> {\n let mut rows = Vec::with_capacity(texts.len());\n for chunk in texts.chunks(batch) {\n let vectors = embedder.embed(chunk).await?;\n if vectors.len() != chunk.len() {\n bail!(\"model returned {} vectors for {} inputs\", vectors.len(), chunk.len());\n }\n for (body, vector) in chunk.iter().zip(vectors) {\n if vector.len() != embedder.dimensions() {\n bail!(\"expected {} dimensions, got {}\", embedder.dimensions(), vector.len());\n }\n rows.push(Row { body: body.clone(), model: embedder.model().to_string(), vector });\n }\n }\n Ok(rows)\n}\n\n#[tokio::main]\nasync fn main() -> Result<()> {\n let texts: Vec<String> = (0..150).map(|i| format!(\"document number {i} about rust\")).collect();\n let rows = embed_all(&HashEmbedder, texts, 64).await?;\n println!(\"{} rows, model {}, {} dimensions, first body {:?}\", rows.len(), rows[0].model, rows[0].vector.len(), rows[0].body);\n Ok(())\n}\n150 rows, model toy-hash-64, 64 dimensions, first body \"document number 0 about rust\"Each row carries the model name, which pays off the first time you change models. The checks in embed_all catch the failures that otherwise surface as a confusing database error (\"expected 1536 dimensions, not 768\") or, worse, silently mismatched results: a service that returns fewer vectors than inputs when a text is dropped, or a model whose dimension was changed in a new release. Batch the calls (here 64 texts at a time), since embedding services charge and rate-limit per request, and wrap the call with the retry and timeout patterns from async resilience patterns, since a bulk re-embedding job runs for hours and will meet transient failures.
HNSW makes reads cheap by doing work at write time: each insert has to find its place in the graph. The same 10,000 vectors that loaded in a fraction of a second into the unindexed table took 8.8 seconds to insert into the table that already had the 79 MB index, about 0.88 ms per row against roughly 7 microseconds without it, a factor of 125. Deletes are cheap at the moment and leave dead entries until VACUUM (about 180 ms for this table after deleting 10,000 rows). Three rules follow. For a bulk load, insert first and build the index afterwards (35 seconds for 100,000 rows here, for a table you then query in a millisecond). For a steady trickle of writes, accept the per-row cost, which is small in absolute terms, and batch inserts in a transaction. For a re-embedding migration that touches every row, build a second column or table with the new vectors, index it, and switch queries over, instead of updating in place under a live index.
embedding_model text lets you re-embed gradually and query only vectors from the current model.vector type holds many thousands of dimensions, but indexes are limited to about 2,000; the halfvec type (16-bit floats) halves the storage and indexes up to about 4,000. Check the extension's documentation for the exact figures in your version.maintenance_work_mem large enough for the graph during CREATE INDEX, or the build slows sharply once it spills; building the index before loading data is much slower than loading, then indexing.A vector(n) value takes 4 bytes per dimension plus an 8 byte header, so 128 dimensions is 520 bytes, and 100,000 of them is 52 million bytes: the benchmark table measured 56 MB once page overhead and the bigserial key are included. Scale it for your case: a million 1,536-dimension vectors is about 6.1 GB of table, and an HNSW index on them can be larger than that again, so the working set that must fit in memory for fast queries is a multiple of the raw data. Do this arithmetic first. If it does not fit, the options are fewer dimensions (some models can be truncated), halfvec storage at half the size, or a different store. An hour of arithmetic here is cheaper than discovering at launch that the index no longer fits in RAM and every query is a disk read.
Use it when your data is already in PostgreSQL, the table has up to a few million vectors and you value transactions, joins and one system to operate. Reach for a dedicated vector database when you have hundreds of millions of vectors, very high query rates with tight latency, or need features such as built-in sharding for vectors. Skip vectors entirely when keyword search already answers the question; PostgreSQL full-text search and trigram indexes are cheaper to run and easier to explain (see full-text search from Rust). The SQLx patterns used here, pooling, transactions and the compile-time macros, are covered in SQLx explained, and the broader cache-and-index thinking behind choosing a store is in PostgreSQL versus SQLite.
","date_published":"2026-11-03T10:00:00+00:00","tags":["Databases"]},{"id":"https://rusthacks.dev/library/node-addons-napi-rs","url":"https://rusthacks.dev/library/node-addons-napi-rs","title":"Node Addons in Rust With Napi rs for JavaScript Speed","summary":"napi-rs generates a Node-API addon from annotated Rust. Functions become JavaScript functions, async work runs on a thread pool, and TypeScript types come for free.","content_html":"Node.js is excellent at I/O and a poor place to run a CPU-heavy loop: one thread executes your JavaScript, so a 250 ms computation freezes every request on the process for 250 ms. A native addon written in Rust fixes the freezing, not necessarily the speed, and it is worth being clear about that difference before writing any code. With napi-rs you write ordinary Rust functions, annotate them with #[napi], and get a .node binary that require() loads, with TypeScript declarations generated for you. This guide builds an addon from an empty directory with a synchronous function, a function returning structured data, a function that runs on Node's thread pool and returns a promise, an async fn, and a callback. It measures what the addon is good for (keeping the event loop free, and using several cores) and what it is not (beating V8 at string work), and it records two conversion traps that silently produce wrong answers. Everything ran on Node 24.1.0 with Rust 1.99, napi 3.16, napi-derive 3.7, napi-build 2.6 and the @napi-rs/cli 3.x tool, on an Apple M1 Max.
Node loads native code through N-API, a stable C interface for addons: functions to read arguments, create strings and objects, throw errors and schedule work, with a promise that binaries compiled against one version keep working on later Node versions. Writing against it by hand in C is tedious and unsafe. napi-rs wraps the interface: the #[napi] macro generates the glue that converts JavaScript values to Rust types and back, the napi-build crate configures the linker, and the @napi-rs/cli npm package drives the build, names the output file for the platform and generates the TypeScript declarations and the loader script. Because the interface is stable, you build one binary per operating system and CPU, not one per Node version.
[package]\nname = \"fastnode\"\nversion = \"0.1.0\"\nedition = \"2024\"\n\n[lib]\ncrate-type = [\"cdylib\"]\n\n[dependencies]\nnapi = { version = \"3\", default-features = false, features = [\"napi6\", \"async\", \"tokio_rt\"] }\nnapi-derive = \"3\"\ntokio = { version = \"1\", features = [\"time\"] }\n\n[build-dependencies]\nnapi-build = \"2\"\n\n[profile.release]\nlto = true\nfn main() {\n napi_build::setup();\n}\n{\n \"name\": \"fastnode\",\n \"version\": \"0.1.0\",\n \"main\": \"index.js\",\n \"types\": \"index.d.ts\",\n \"napi\": {\n \"binaryName\": \"fastnode\",\n \"targets\": [\"aarch64-apple-darwin\"]\n },\n \"scripts\": {\n \"build\": \"napi build --platform --release\"\n },\n \"devDependencies\": {\n \"@napi-rs/cli\": \"^3.0.0\"\n }\n}\nThe crate type cdylib makes a dynamic library, which the CLI renames to fastnode.darwin-arm64.node, because Node loads files with the .node extension. The napi6 feature selects the N-API version you require (version 6 is old enough to be available on every supported Node), async enables the thread-pool task API and tokio_rt lets async fn exports run on a Tokio runtime that napi-rs manages; a direct tokio dependency with the time feature is still needed because your own code names tokio::time::sleep. default-features = false keeps the binary small. The build.rs is two lines and mandatory: napi_build::setup() passes the platform-specific linker flags (on macOS, the undefined-symbol flags that let the addon resolve N-API functions from the Node process that loads it). In package.json, napi.binaryName sets the output name and targets lists the platforms the project builds for, and the one script, napi build --platform --release, is the whole build. Run npm install once to fetch the CLI.
use napi::bindgen_prelude::*;\nuse napi_derive::napi;\nuse std::collections::HashMap;\n\n/// Iterative Fibonacci, wrapping like a u64.\n#[napi]\npub fn fib(n: u32) -> BigInt {\n let (mut a, mut b) = (0u64, 1u64);\n for _ in 0..n {\n (a, b) = (b, a.wrapping_add(b));\n }\n BigInt::from(a)\n}\n\n#[napi(object)]\npub struct WordCount {\n pub word: String,\n pub count: u32,\n}\n\n#[napi]\npub fn top_words(text: String, n: u32) -> Result<Vec<WordCount>> {\n if n == 0 {\n return Err(Error::new(Status::InvalidArg, \"n must be at least 1\"));\n }\n let mut counts: HashMap<String, u32> = HashMap::new();\n for w in text.split(|c: char| !c.is_alphanumeric()).filter(|w| !w.is_empty()) {\n *counts.entry(w.to_lowercase()).or_default() += 1;\n }\n let mut pairs: Vec<_> = counts.into_iter().collect();\n pairs.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0)));\n pairs.truncate(n as usize);\n Ok(pairs.into_iter().map(|(word, count)| WordCount { word, count }).collect())\n}\n\npub struct Churn {\n values: Vec<u32>,\n rounds: u32,\n}\n\n#[napi]\nimpl Task for Churn {\n type Output = u32;\n type JsValue = u32;\n\n fn compute(&mut self) -> Result<Self::Output> {\n Ok(self.values.iter().fold(0u32, |acc, v| {\n let mut h = *v ^ 0x811c9dc5;\n for _ in 0..self.rounds {\n h = (h ^ (h >> 15)).wrapping_mul(0x01000193);\n }\n acc ^ h\n }))\n }\n\n fn resolve(&mut self, _env: Env, output: Self::Output) -> Result<Self::JsValue> {\n Ok(output)\n }\n}\n\n#[napi(ts_return_type = \"Promise<number>\")]\npub fn churn(values: Vec<u32>, rounds: u32) -> AsyncTask<Churn> {\n AsyncTask::new(Churn { values, rounds })\n}\n\n#[napi]\npub async fn delayed_double(n: u32, ms: u32) -> Result<u32> {\n tokio::time::sleep(std::time::Duration::from_millis(u64::from(ms))).await;\n n.checked_mul(2).ok_or_else(|| Error::new(Status::GenericFailure, format!(\"{n} * 2 overflows u32\")))\n}\n\n#[napi]\npub fn for_each_line(text: String, callback: Function<'_, FnArgs<(u32, String)>, ()>) -> Result<u32> {\n let mut n = 0;\n for (i, line) in text.lines().enumerate() {\n callback.call(FnArgs::from((i as u32, line.to_string())))?;\n n += 1;\n }\n Ok(n)\n}\nEach export shows a different idea. fib returns BigInt because a JavaScript number cannot hold every 64-bit integer: above 2 to the power of 53 it loses precision silently, and BigInt is the faithful type. #[napi(object)] on a struct makes a plain JavaScript object with typed fields, which top_words returns as an array. Returning Result<T> turns an Err into a thrown exception whose code is the Status you chose. The Churn struct implements Task: compute runs on a worker thread from libuv's pool (never on the JavaScript thread), and resolve runs back on the main thread to turn the result into a JavaScript value; wrapping it in AsyncTask makes the exported function return a promise. delayed_double is an ordinary async fn, run on the managed Tokio runtime. for_each_line receives a JavaScript function and calls it synchronously for each line.
npm run buildThe release build took 12 seconds the first time and produced a 786 KB fastnode.darwin-arm64.node (the size includes the Tokio pieces), plus two generated files you commit: index.js, an 840-line loader that picks the right binary for the current platform and falls back with a clear error, and index.d.ts, the declarations. The declarations are the payoff for the annotations; this is the generated content for the exports:
export declare function churn(values: Array<number>, rounds: number): Promise<number>\n\nexport declare function delayedDouble(n: number, ms: number): Promise<number>\n\n/** Iterative Fibonacci, wrapping like a u64. */\nexport declare function fib(n: number): bigint\n\nexport declare function forEachLine(text: string, callback: (arg0: number, arg1: string) => void): number\n\nexport declare function topWords(text: string, n: number): Array<WordCount>\n\nexport interface WordCount {\n word: string\n count: number\n}\nRust names are converted to camelCase (top_words became topWords), doc comments become JSDoc, the struct became an interface, and the tuple callback parameters became two positional arguments. That last one deserves a note: the first version of for_each_line declared its callback as Function<'_, (u32, String), ()>, and the generated type was (arg: [number, string]) => void. The callback was called with one array argument, so a JavaScript function written as (i, line) => ... received the array in i and undefined in line, and the output read 0,alpha:undefined. Wrapping the argument type in FnArgs<(u32, String)>, as the final source does, spreads the tuple into positional arguments. Read the generated .d.ts after every signature change; it is the contract JavaScript sees.
const { fib, topWords, churn } = require('./index.js');\n\nconsole.log(fib(50), typeof fib(50));\nconsole.log(topWords('the cat and the hat and THE bat', 2));\ntry {\n topWords('x', 0);\n} catch (e) {\n console.log(e.code, '-', e.message);\n}\ntry {\n fib(-1);\n} catch (e) {\n console.log(e.code, '-', e.message);\n}\nchurn([1, 2, 3], 10).then((v) => console.log('churn resolved', v));\nconsole.log('this line prints before the promise resolves');\n12586269025n bigint\n[ { word: 'the', count: 3 }, { word: 'and', count: 2 } ]\nInvalidArg - n must be at least 1\nthis line prints before the promise resolves\nchurn resolved 806291124The results are what the signatures promised: fib(50) is a bigint, topWords returns real objects, a bad argument throws with the code and message from Rust, and the promise resolves after the synchronous line has already printed, proving that churn ran off the main thread. The remaining exports behave the same way:
const { delayedDouble, forEachLine } = require('./index.js');\n\n(async () => {\n const t = Date.now();\n const [a, b] = await Promise.all([delayedDouble(21, 100), delayedDouble(5, 100)]);\n console.log(a, b, 'in about', Math.round((Date.now() - t) / 100) * 100, 'ms');\n try {\n await delayedDouble(3_000_000_000, 1);\n } catch (e) {\n console.log(e.code, '-', e.message);\n }\n const seen = [];\n const count = forEachLine('alpha\\nbeta\\ngamma', (i, line) => seen.push(`${i}:${line}`));\n console.log(count, seen);\n})();\n42 10 in about 100 ms\nGenericFailure - 3000000000 * 2 overflows u32\n3 [ '0:alpha', '1:beta', '2:gamma' ]Two concurrent calls with a 100 ms Rust-side sleep completed in about 100 ms in total, not 200, because the async function is truly asynchronous. The overflow error arrived as a rejected promise with the code and message from Rust. And the callback saw positional arguments.
\nPyO3 raises OverflowError for a negative number passed to an unsigned parameter. N-API's number conversion does not. JavaScript has one numeric type, a double, and napi-rs converts it to u32 by the engine's rules, which wrap:
const { fib } = require('./index.js');\nlet t = process.hrtime.bigint();\nconst r = fib(-1);\nconsole.log('fib(-1) =', r, 'in', Number(process.hrtime.bigint() - t) / 1e6, 'ms');\ntry { fib(1.5); console.log('fib(1.5) ok'); } catch (e) { console.log('fib(1.5):', e.code, e.message); }\ntry { fib('7'); console.log('fib(\"7\") ok'); } catch (e) { console.log('fib(\"7\"):', e.code, e.message); }\ntry { fib(2 ** 40); console.log('fib(2**40) ok'); } catch (e) { console.log('fib(2**40):', e.code, e.message); }\nfib(-1) = 6337557995341940450n in 1387.272208 ms\nfib(1.5) ok\nfib(\"7\"): NumberExpected Failed to convert napi value String into rust type `u32`\nfib(2**40) okfib(-1) did not throw. The value became 4,294,967,295, and the function ran four billion iterations for 1.39 seconds, blocking the event loop for the whole time, then returned a plausible-looking number. fib(1.5) truncated to 1, fib(2**40) wrapped to 0, and only a string was rejected. If a function takes an unsigned or bounded integer from untrusted JavaScript, accept i64 or f64 and validate the range yourself (if !(0..=1000).contains(&n) { return Err(...) }), or use BigInt for exact 64-bit values. A request handler that passes req.query.n to a Rust function would otherwise hand any client a denial-of-service switch.
const { topWords, churn } = require('./index.js');\n\nfunction jsTopWords(text, n) {\n const counts = new Map();\n for (const w of text.split(/[^\\p{L}\\p{N}]+/u)) {\n if (!w) continue;\n const k = w.toLowerCase();\n counts.set(k, (counts.get(k) || 0) + 1);\n }\n return [...counts.entries()]\n .sort((a, b) => b[1] - a[1] || (a[0] < b[0] ? -1 : 1))\n .slice(0, n)\n .map(([word, count]) => ({ word, count }));\n}\n\nfunction jsChurn(values, rounds) {\n let acc = 0;\n for (const v of values) {\n let h = (v ^ 0x811c9dc5) >>> 0;\n for (let i = 0; i < rounds; i++) {\n h = Math.imul((h ^ (h >>> 15)) >>> 0, 0x01000193) >>> 0;\n }\n acc = (acc ^ h) >>> 0;\n }\n return acc;\n}\n\nfunction best(f, runs = 5) {\n let b = Infinity;\n for (let i = 0; i < runs; i++) {\n const t = process.hrtime.bigint();\n f();\n b = Math.min(b, Number(process.hrtime.bigint() - t) / 1e6);\n }\n return b;\n}\n\nasync function main() {\n const text = 'the quick brown fox jumps over the lazy dog and the cat '.repeat(20000);\n const js = best(() => jsTopWords(text, 3));\n const rs = best(() => topWords(text, 3));\n console.log(`topWords on ${(text.length / 1e6).toFixed(1)} MB: js ${js.toFixed(1)} ms | rust ${rs.toFixed(1)} ms | ${(js / rs).toFixed(1)}x`);\n console.log(JSON.stringify(jsTopWords(text, 3)) === JSON.stringify(topWords(text, 3)) ? 'same answer' : 'DIFFERENT');\n\n const values = Array.from({ length: 2000 }, (_, i) => i);\n const rounds = 20000;\n if (jsChurn(values, rounds) !== (await churn(values, rounds))) throw new Error('churn mismatch');\n\n let t = process.hrtime.bigint();\n for (let i = 0; i < 4; i++) jsChurn(values, rounds);\n const jsSeq = Number(process.hrtime.bigint() - t) / 1e6;\n\n t = process.hrtime.bigint();\n await churn(values, rounds);\n const rsOne = Number(process.hrtime.bigint() - t) / 1e6;\n\n t = process.hrtime.bigint();\n await Promise.all([1, 2, 3, 4].map(() => churn(values, rounds)));\n const rsFour = Number(process.hrtime.bigint() - t) / 1e6;\n console.log(`churn: js x4 on the main thread ${jsSeq.toFixed(0)} ms | rust x1 ${rsOne.toFixed(0)} ms | rust x4 in parallel ${rsFour.toFixed(0)} ms`);\n\n // event loop lag while the work runs\n async function lag(work) {\n let worst = 0, last = performance.now();\n const timer = setInterval(() => {\n const now = performance.now();\n worst = Math.max(worst, now - last - 10);\n last = now;\n }, 10);\n await new Promise((r) => setTimeout(r, 30));\n await work();\n await new Promise((r) => setTimeout(r, 30));\n clearInterval(timer);\n return worst;\n }\n const lagJs = await lag(async () => { for (let i = 0; i < 4; i++) jsChurn(values, rounds); });\n const lagRs = await lag(() => Promise.all([1, 2, 3, 4].map(() => churn(values, rounds))));\n console.log(`worst timer delay while working: js ${lagJs.toFixed(0)} ms | rust AsyncTask ${lagRs.toFixed(0)} ms`);\n}\nmain();\ntopWords on 1.1 MB: js 17.5 ms | rust 11.9 ms | 1.5x\nsame answer\nchurn: js x4 on the main thread 258 ms | rust x1 64 ms | rust x4 in parallel 66 ms\nworst timer delay while working: js 258 ms | rust AsyncTask 2 msRead it honestly. Raw speed on string work is modest: counting words in 1.1 MB took 17.5 ms in V8 and 11.9 ms in Rust, a 1.5x difference. V8's string and map code is very good, and it takes a lot of Rust to beat. A tight numeric loop is a tie: the JavaScript version of the hash loop ran at the same speed as the Rust one (about 64 ms per pass), because V8 compiles integer loops with Math.imul to near-native code. If a profile says your slow function is a loop like this, an addon will not make a single call faster. What the addon changes is where the work runs. Four passes in JavaScript took 258 ms on the main thread, during which a 10 ms timer was delayed by up to 258 ms, which is a stalled server. The four Rust tasks ran on the thread pool in parallel and finished in 66 ms, and the worst timer delay was 2 ms: the event loop stayed responsive and the work used four cores. For comparison, Node's own answer to the same problem is worker_threads, which can run the same JavaScript in parallel without any Rust; the addon wins when the code is faster in Rust, when you want to share memory without copying, or when you need a Rust library that has no JavaScript equivalent.
Three more exports cover the rest of what real addons need. Buffers: a Uint8Array parameter is a view onto the memory of a Node Buffer, not a copy, so passing a 100 MB buffer costs nothing and the function reads it in place. Classes: #[napi] on a struct and its impl produces a JavaScript class whose instances own a Rust value; #[napi(constructor)] is the constructor (an Option<u32> parameter makes an optional argument) and #[napi(getter)] makes a property. Threadsafe functions: JavaScript functions may only be called on the main thread, so a Rust thread that wants to report progress wraps the callback in a ThreadsafeFunction, which queues the call and lets Node run it on the event loop.
#[napi]\npub fn checksum(data: Uint8Array) -> BigInt {\n let h = data.iter().fold(0xcbf29ce484222325u64, |h, b| (h ^ u64::from(*b)).wrapping_mul(0x100000001b3));\n BigInt::from(h)\n}\n\n#[napi]\npub struct Counter {\n count: u32,\n step: u32,\n}\n\n#[napi]\nimpl Counter {\n #[napi(constructor)]\n pub fn new(step: Option<u32>) -> Result<Self> {\n let step = step.unwrap_or(1);\n if step == 0 {\n return Err(Error::new(Status::InvalidArg, \"step must be non-zero\"));\n }\n Ok(Counter { count: 0, step })\n }\n\n #[napi]\n pub fn bump(&mut self) -> u32 {\n self.count += self.step;\n self.count\n }\n\n #[napi(getter)]\n pub fn count(&self) -> u32 {\n self.count\n }\n}\n\n#[napi]\npub fn count_to(n: u32, on_tick: ThreadsafeFunction<u32>) {\n std::thread::spawn(move || {\n for i in 1..=n {\n on_tick.call(Ok(i), ThreadsafeFunctionCallMode::NonBlocking);\n std::thread::sleep(std::time::Duration::from_millis(20));\n }\n });\n}\nconst { checksum, Counter, countTo } = require('./index.js');\n\nconst data = Buffer.alloc(100 * 1024 * 1024, 7);\nconst t = process.hrtime.bigint();\nconst sum = checksum(data);\nconsole.log(`checksum of ${data.length / 1048576} MB buffer: ${sum.toString(16)} in ${Math.round(Number(process.hrtime.bigint() - t) / 1e6)} ms`);\n\nconst c = new Counter(5);\nconsole.log(c.bump(), c.bump(), c.count);\ntry { new Counter(0); } catch (e) { console.log(e.code, '-', e.message); }\n\nconst ticks = [];\nconst start = Date.now();\ncountTo(3, (err, i) => {\n ticks.push(i);\n if (i === 3) console.log('ticks', ticks, 'delivered over about', Math.round((Date.now() - start) / 10) * 10, 'ms');\n});\nconsole.log('countTo returned immediately');\nchecksum of 100 MB buffer: 5f17c1a5b1622325 in 138 ms\n5 10 10\nInvalidArg - step must be non-zero\ncountTo returned immediately\nticks [ 1, 2, 3 ] delivered over about 50 msThe checksum of a 100 MB buffer took 138 ms for a byte-at-a-time hash, with no copying, about 720 MB per second. The class kept its state in Rust between calls and threw the Rust-defined error from the constructor. countTo returned immediately, as the output order shows, and the three ticks arrived over about 40 ms from a background thread, each delivered as a normal callback on the event loop with the Node convention of an error first and the value second. The generated declarations mirror that: the callback is typed (err: Error | null, arg: number) => any. A ThreadsafeFunction keeps the process alive while it exists, which is right for a live subscription and wrong for a leaked one; call its abort method (or drop it in Rust once the thread finishes) so that Node can exit.
The rules of thumb that fall out: use a Buffer or Uint8Array for anything large, because converting a JavaScript array of numbers to a Vec copies and converts every element; keep objects crossing the boundary small and flat, because every property read is a call into the engine; and do not hold JavaScript values on another thread, because they are tied to the thread and the realm that created them. If you need a result back from background work, return a promise (as churn does) instead of passing a callback, which composes with async/await and error handling.
An addon is a native library with a JavaScript API, so test it the way you would test either: the Rust logic with cargo test, and the JavaScript surface with the runner you already use. Node's built-in node:test is enough and adds no dependency. The most useful tests compare the Rust result with a plain JavaScript reference over the edge cases, and check the properties that are the reason for the addon in the first place, such as keeping the event loop alive:
import { test } from 'node:test';\nimport assert from 'node:assert/strict';\nimport { createRequire } from 'node:module';\n\nconst { fib, topWords, churn, Counter } = createRequire(import.meta.url)('./index.js');\n\nfunction jsFib(n) {\n let a = 0n, b = 1n;\n const mask = (1n << 64n) - 1n;\n for (let i = 0; i < n; i++) [a, b] = [b, (a + b) & mask];\n return a;\n}\n\ntest('fib matches a BigInt reference, including the 64-bit wrap', () => {\n for (const n of [0, 1, 2, 10, 50, 93, 94, 1000]) {\n assert.equal(fib(n), jsFib(n), `fib(${n})`);\n }\n});\n\ntest('topWords sorts by count, then alphabetically', () => {\n assert.deepEqual(topWords('b a b a c', 3), [\n { word: 'a', count: 2 },\n { word: 'b', count: 2 },\n { word: 'c', count: 1 },\n ]);\n});\n\ntest('topWords rejects n = 0 with InvalidArg', () => {\n assert.throws(() => topWords('x', 0), { code: 'InvalidArg' });\n});\n\ntest('churn resolves off the main thread', async () => {\n let ticked = false;\n const timer = setTimeout(() => (ticked = true), 0);\n const result = churn(Array.from({ length: 1000 }, (_, i) => i), 20000);\n await new Promise((r) => setTimeout(r, 5));\n assert.equal(ticked, true, 'the event loop kept running');\n assert.equal(typeof (await result), 'number');\n clearTimeout(timer);\n});\n\ntest('Counter keeps state in Rust', () => {\n const c = new Counter(3);\n c.bump();\n assert.equal(c.bump(), 6);\n assert.equal(c.count, 6);\n});\n✔ fib matches a BigInt reference, including the 64-bit wrap (1.138375ms)\n✔ topWords sorts by count, then alphabetically (0.551042ms)\n✔ topWords rejects n = 0 with InvalidArg (0.249291ms)\n✔ churn resolves off the main thread (32.71325ms)\n✔ Counter keeps state in Rust (0.149541ms)\nℹ tests 5\nℹ pass 5\nℹ fail 0The first test pins the 64-bit wrap: the reference uses BigInt with a mask, and the case list includes 93 and 94, the boundary where a Fibonacci number stops fitting in 64 bits. The third test asserts the error code, not just that something threw, since the code is the part of the contract that JavaScript callers branch on. The fourth proves the property: while the Rust task ran, a zero-delay timer fired, so the event loop was not blocked. Put the build in the test script (npm run build && node --test), and run the suite in CI against the built binary on every platform you ship, since a native addon can pass on Linux and fail to load on Windows.
An addon is not the only way to move work off the JavaScript thread, and it is not always the best. The options differ in what they cost to build, what they can reach, and where they run:
\n| Native addon (napi-rs) | WebAssembly | worker_threads | |
|---|---|---|---|
| Runs in | Node only | Node and browsers | Node only |
| Speed | Native, with threads and SIMD | Close to native, sandboxed, no direct OS access | JavaScript speed, one thread each |
| Can use system libraries, files, sockets | Yes | Only through the host | Yes |
| Distribution | One binary per OS and CPU | One .wasm file | Plain JavaScript |
| Failure mode | A crash takes down the process | A trap is an exception | A worker error is an event |
| Best for | Heavy compute, existing Rust crates, OS access | Portable compute, running in the browser too | Parallelising code that is already fast enough JavaScript |
The decision usually falls out of two questions. Does the code need to run in a browser as well? Then WebAssembly. Is the code already as fast in JavaScript as it would be in Rust, as the hash loop was here? Then worker_threads gets you the parallelism with no native toolchain to maintain. The addon is the answer when the work is both heavy and needs something only native code has: a Rust crate with no JavaScript counterpart, a system API, shared memory across threads, or the last factor of two or ten in speed. The crash mode in the table deserves weight: a segfault or abort in an addon kills the whole Node process, including every request in flight, where a bug in a worker thread only fails that worker. Keep the unsafe surface of the addon small, return errors instead of panicking, and do not accept inputs you have not validated, as the fib(-1) case showed.
An addon is a platform-specific binary, so distribution means building for each platform you support and getting the right one onto the user's machine. The napi-rs convention is a main package plus one small package per platform (@yourscope/fastnode-darwin-arm64 and so on), listed as optionalDependencies of the main package so that npm install fetches only the one that matches. The CLI has commands for generating those packages and for the GitHub Actions workflow that builds the matrix on real runners (macOS, Windows, Linux with glibc and with musl) and publishes. Build Linux binaries in a container with an old glibc, for the same reason as Python manylinux wheels, or users on older distributions will see GLIBC_2.xx not found at require. Napi-rs 3 can also target WebAssembly (wasm32-wasip1-threads) as a fallback for platforms without a prebuilt binary, at a speed and compatibility cost. I built only for aarch64-apple-darwin here; the cross-platform matrix is configuration of the targets list and the workflow, and should be verified in CI on each target before release.
| Symptom | Cause | Fix |
|---|---|---|
Cannot find module './fastnode.darwin-arm64.node' | Not built for this platform, or the binary name differs from binaryName | Run npm run build on that platform; check the name |
Linker errors about undefined napi_* symbols | build.rs missing or does not call napi_build::setup() | Add it, with napi-build in build-dependencies |
| Callback receives one array argument | Tuple type used for the callback arguments | Wrap in FnArgs<(...)> |
| Server stalls during a call | Heavy work in a synchronous #[napi] function | Use AsyncTask or an async fn |
| Negative or large numbers produce odd results | Integer conversion wraps | Accept a wider type and validate |
cannot find crate tokio in your own code | The tokio_rt feature does not re-export tokio to you | Add tokio as a direct dependency with the features you use |
| Process aborts on a Rust panic | A panic unwound across the N-API boundary | Return Result for expected failures; treat panics as bugs |
If the slow part is I/O, an addon does not help; Node already does that well. If the function is cheap and called millions of times, the cost of crossing the boundary (argument conversion, a handle scope per call) can exceed the saving, so batch: pass an array or a Buffer and process it in one call. If you want to run the same code in the browser as well, compile to WebAssembly instead, as in Rust to WebAssembly, since an addon runs only in Node. And if you only need a CLI tool in your build pipeline, ship a binary and call it. For the Python equivalent of this guide, with the same shape and different gotchas, see Python extensions with PyO3 and maturin, and for the C interface under both, calling Rust from C.
Python is the right language for gluing things together and the wrong one for tight numeric loops, text crunching or anything you would like to run on all your cores at once. The standard fix is to write the hot function in a compiled language and call it from Python. In Rust, that is PyO3 for the bindings and maturin for building and packaging, and the result is a normal Python package: pip install, import, call. This guide builds one from an empty directory. You will write four Rust items (a function, a function that returns structured data, a function that releases Python's global interpreter lock, and a class), see Rust errors turn into Python exceptions, measure the speedup honestly, including the case where it is only 4 times and the case where it is 165 times, and build a wheel that installs on any Python from 3.9 up. Everything ran with Rust 1.99, PyO3 0.30.1, maturin 1.16.0 and CPython 3.14.6 on an Apple M1 Max. The performance numbers are from that machine; yours will differ in size, not in shape.
Decide before you write any Rust, because the boundary has a cost. A function call from Python into Rust takes tens of nanoseconds and converting arguments takes more, so wrapping a function that does a microsecond of work buys little. The win comes from work that stays inside Rust for a while: a loop over a million items, parsing a large string, compressing, hashing, geometry, a state machine, a CPU-bound task you want on several threads. If a profile of your Python shows one function eating most of the time and that function takes simple inputs (numbers, strings, byte buffers, lists of numbers) and returns a simple result, it is a good candidate. If the time goes into many small calls with Python objects, restructure first so that one call does a batch.
\npython3 -m venv .venv\n. .venv/bin/activate\npip install maturin\nmaturin --versionThe virtual environment matters for a practical reason: maturin develop installs the extension into the active environment and refuses to run without one. Outside a venv it stops with \"Couldn't find a virtualenv or conda environment\", which is a feature, because it keeps a half-built development package out of your system Python. pip install maturin gives you version 1.16.0; any 1.x release works with the configuration below.
[package]\nname = \"fastlib\"\nversion = \"0.1.0\"\nedition = \"2024\"\n\n[lib]\nname = \"fastlib\"\ncrate-type = [\"cdylib\"]\n\n[dependencies]\npyo3 = { version = \"0.30\", features = [\"abi3-py39\"] }[build-system]\nrequires = [\"maturin>=1.7,<2.0\"]\nbuild-backend = \"maturin\"\n\n[project]\nname = \"fastlib\"\nrequires-python = \">=3.9\"\ndynamic = [\"version\"]\n\n[tool.maturin]\nfeatures = [\"pyo3/extension-module\"]Each setting has a reason. crate-type = [\"cdylib\"] builds a C-compatible dynamic library, which is what Python loads as an extension module; the [lib] name must match the Python module name you will import. The abi3-py39 feature builds against CPython's stable ABI, so one compiled wheel works on Python 3.9 and every later version, instead of one wheel per interpreter release. The cost is a restricted API (some PyO3 conveniences are unavailable and a few operations are slightly slower), and for most extensions it is the right trade. The extension-module feature, set through maturin, tells PyO3 not to link against libpython: the extension runs inside an already-running interpreter, which supplies the symbols. Putting it in pyproject.toml and not in Cargo.toml keeps cargo test working, since a test binary has no interpreter to provide them. The file pyproject.toml is what makes pip install . and maturin build the project; dynamic = [\"version\"] takes the version from Cargo.toml so it is written once.
use pyo3::exceptions::PyValueError;\nuse pyo3::prelude::*;\nuse std::collections::HashMap;\n\n#[pymodule]\nmod fastlib {\n use super::*;\n\n /// Iterative Fibonacci, wrapping on overflow like a u64.\n #[pyfunction]\n fn fib(n: u32) -> u64 {\n let (mut a, mut b) = (0u64, 1u64);\n for _ in 0..n {\n (a, b) = (b, a.wrapping_add(b));\n }\n a\n }\n\n /// Count words in a string. Returns the 'n' most common as (word, count).\n #[pyfunction]\n fn top_words(text: &str, n: usize) -> PyResult<Vec<(String, usize)>> {\n if n == 0 {\n return Err(PyValueError::new_err(\"n must be at least 1\"));\n }\n let mut counts: HashMap<String, usize> = HashMap::new();\n for w in text.split(|c: char| !c.is_alphanumeric()).filter(|w| !w.is_empty()) {\n *counts.entry(w.to_lowercase()).or_default() += 1;\n }\n let mut pairs: Vec<_> = counts.into_iter().collect();\n pairs.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0)));\n pairs.truncate(n);\n Ok(pairs)\n }\n\n /// Mix every value through a hash-like loop. The heavy part runs with the GIL released.\n #[pyfunction]\n fn churn(py: Python<'_>, values: Vec<u64>, rounds: u32) -> u64 {\n py.detach(|| {\n values.iter().fold(0u64, |acc, v| {\n let mut h = *v ^ 0xcbf29ce484222325;\n for _ in 0..rounds {\n h = (h ^ (h >> 29)).wrapping_mul(0x100000001b3);\n }\n acc ^ h\n })\n })\n }\n\n /// Same work, but holding the GIL the whole time.\n #[pyfunction]\n fn churn_locked(values: Vec<u64>, rounds: u32) -> u64 {\n values.iter().fold(0u64, |acc, v| {\n let mut h = *v ^ 0xcbf29ce484222325;\n for _ in 0..rounds {\n h = (h ^ (h >> 29)).wrapping_mul(0x100000001b3);\n }\n acc ^ h\n })\n }\n\n #[pyclass]\n struct Counter {\n count: u64,\n step: u64,\n }\n\n #[pymethods]\n impl Counter {\n #[new]\n #[pyo3(signature = (step = 1))]\n fn new(step: u64) -> PyResult<Self> {\n if step == 0 {\n return Err(PyValueError::new_err(\"step must be non-zero\"));\n }\n Ok(Counter { count: 0, step })\n }\n\n fn bump(&mut self) -> u64 {\n self.count += self.step;\n self.count\n }\n\n #[getter]\n fn count(&self) -> u64 {\n self.count\n }\n }\n}\nPyO3 0.30 uses a declarative style: a #[pymodule] mod fastlib { ... } block whose items become the module's contents. #[pyfunction] exposes a function, and PyO3 converts arguments and results automatically: u32 arrives from a Python int (out-of-range values raise OverflowError for you), &str borrows a Python string without copying, Vec<u64> converts from any list of ints, and a returned Vec<(String, usize)> becomes a Python list of tuples. Returning PyResult<T> lets a function raise: PyValueError::new_err(\"...\") becomes a ValueError in Python with that message. The doc comment (///) becomes the Python docstring. #[pyclass] and #[pymethods] define a Python class backed by a Rust struct: #[new] is __init__, #[pyo3(signature = (step = 1))] gives the argument a default, and #[getter] exposes a read-only attribute.
maturin develop --releaseThat compiles in release mode and installs the module into the active environment as an editable package; the first build took about eight seconds. Without --release you get a debug build that can be 10 to 100 times slower, which is the most common reason for \"my Rust extension is not faster\". Now drive it from Python:
import fastlib\nimport threading\nimport time\n\nprint(fastlib.fib(50), fastlib.fib(10))\nprint(fastlib.top_words(\"the cat and the hat and THE bat\", 2))\n\ntry:\n fastlib.top_words(\"x\", 0)\nexcept ValueError as e:\n print(\"ValueError:\", e)\n\ntry:\n fastlib.fib(-1)\nexcept OverflowError as e:\n print(\"OverflowError:\", e)\n\nc = fastlib.Counter(step=5)\nprint(c.bump(), c.bump(), c.count)\ntry:\n fastlib.Counter(0)\nexcept ValueError as e:\n print(\"ValueError:\", e)\n\nprint(fastlib.churn([1, 2, 3], 10))\nprint(type(fastlib.fib), fastlib.fib.__doc__)\n12586269025 55\n[('the', 3), ('and', 2)]\nValueError: n must be at least 1\nOverflowError: number too small to fit in target type\n5 10 10\nValueError: step must be non-zero\n13593663989309771750\n<class 'builtin_function_or_method'> Iterative Fibonacci, wrapping on overflow like a u64.Walk through the output. fib(50) is 12,586,269,025, and the wrapping arithmetic documented in the docstring is visible through fastlib.fib.__doc__. The word count returned the two most common words as tuples. The three exception lines show the conversion in action: top_words(\"x\", 0) raised the ValueError from Rust, fib(-1) raised OverflowError without any code of ours, because a negative number does not fit a u32, and a Counter(0) raised ValueError from the constructor. The counter held its state in a Rust struct across calls (5, 10, then the count getter reports 10).
The benchmark compares each function with the obvious pure-Python version, taking the best of five runs:
\nimport fastlib, time, collections, re, threading\n\ndef py_fib(n):\n a, b = 0, 1\n for _ in range(n):\n a, b = b, (a + b) & 0xFFFFFFFFFFFFFFFF\n return a\n\ndef timeit(f, repeat=5):\n best = 1e9\n for _ in range(repeat):\n t = time.perf_counter(); f(); best = min(best, time.perf_counter() - t)\n return best\n\nn = 1_000_000\np = timeit(lambda: py_fib(n)); r = timeit(lambda: fastlib.fib(n))\nprint(f\"fib({n}): python {p*1e3:8.2f} ms | rust {r*1e3:8.3f} ms | {p/r:6.0f}x\")\n\n# tiny call: overhead dominated\ntp = timeit(lambda: [py_fib(10) for _ in range(100000)]); tr = timeit(lambda: [fastlib.fib(10) for _ in range(100000)])\nprint(f\"100k calls of fib(10): python {tp*1e3:7.1f} ms | rust {tr*1e3:7.1f} ms | {tp/tr:4.1f}x (per call {tr/100000*1e9:.0f} ns)\")\n\ntext = (\"the quick brown fox jumps over the lazy dog and the cat \" * 20000)\ndef py_top():\n c = collections.Counter(w.lower() for w in re.split(r\"[^\\w]+\", text) if w)\n return sorted(c.items(), key=lambda kv: (-kv[1], kv[0]))[:3]\na = timeit(py_top); b = timeit(lambda: fastlib.top_words(text, 3))\nprint(f\"top_words on {len(text)/1e6:.1f} MB: python {a*1e3:7.1f} ms | rust {b*1e3:7.1f} ms | {a/b:4.1f}x\")\nassert [list(x) for x in py_top()] == [list(x) for x in fastlib.top_words(text, 3)]\n\nvals = list(range(2000))\ndef run_threads(k, fn=fastlib.churn):\n res = [0]*k; ts = [threading.Thread(target=lambda i=i: res.__setitem__(i, fn(vals, 20000))) for i in range(k)]\n t = time.perf_counter(); [x.start() for x in ts]; [x.join() for x in ts]; return time.perf_counter() - t\nt1 = min(run_threads(1) for _ in range(3)); t4 = min(run_threads(4) for _ in range(3))\nprint(f\"churn with the GIL released: 1 thread {t1*1e3:.0f} ms, 4 threads {t4*1e3:.0f} ms for 4x the work\")\nl1 = min(run_threads(1, fastlib.churn_locked) for _ in range(3)); l4 = min(run_threads(4, fastlib.churn_locked) for _ in range(3))\nprint(f\"churn_locked (GIL held): 1 thread {l1*1e3:.0f} ms, 4 threads {l4*1e3:.0f} ms for 4x the work\")\nfib(1000000): python 53.26 ms | rust 0.322 ms | 165x\n100k calls of fib(10): python 37.3 ms | rust 6.2 ms | 6.0x (per call 62 ns)\ntop_words on 1.1 MB: python 52.5 ms | rust 12.1 ms | 4.3x\nchurn with the GIL released: 1 thread 64 ms, 4 threads 66 ms for 4x the work\nchurn_locked (GIL held): 1 thread 64 ms, 4 threads 258 ms for 4x the workFour results, four lessons. Pure compute is where the gain is: a million iterations of Fibonacci took 53 ms in Python and 0.32 ms in Rust, 165 times faster, because the Python version pays interpreter overhead on every iteration. Tiny calls are not free: a hundred thousand calls of fib(10) was only 6 times faster, with a call costing about 62 ns of which almost none is the Fibonacci. Crossing the boundary is the cost; batch work so that each call does a lot. Library-heavy code shrinks the gain: counting words in a 1.1 MB string was 4.3 times faster, a modest win, because Python's re and Counter are themselves compiled C code, so you are comparing Rust with C plus some interpreter glue. The assertion in the script checks that both versions return the same answer, which is the check you always want in a benchmark. The GIL is the multiplier: see the next section.
CPython's global interpreter lock lets only one thread run Python code at a time. A Rust function that holds the lock while it works blocks every other Python thread, exactly as a slow Python function would. Inside py.detach(|| ...) the lock is released, and other threads run, including other calls into your Rust code. The rule: copy what you need out of Python objects first (the Vec<u64> argument is already a Rust-owned copy by the time the function body starts), then detach for the heavy part. The last two benchmark lines run the same hashing loop, once with detach and once without, on four Python threads each doing the same work as a single thread:
| Variant | 1 thread | 4 threads (4x the work) |
|---|---|---|
churn, GIL released | 64 ms | 66 ms |
churn_locked, GIL held | 64 ms | 258 ms |
With the lock released, four threads did four times the work in the same wall time: real parallelism, from a language with a global lock. With the lock held, the same work took four times as long, because the threads queued behind one another. That difference is the strongest reason to use Rust from Python for CPU-bound work, and it is a one-line change. A caution from the first attempt at this benchmark: an earlier version detached only around a trivial sum and converted a two-million-element list while holding the lock, and four threads took four times as long. The conversion from a Python list is itself work under the lock, so a function whose cost is mostly argument conversion gains nothing from detach. Pass big data as a buffer (bytes, or a NumPy array through the numpy crate) that Rust can read without converting element by element. Older PyO3 code spells it py.allow_threads; the name changed to detach in recent releases, along with with_gil becoming attach.
Three more behaviours decide whether an extension feels like a real Python library. First, large inputs. A &[u8] parameter borrows the contents of a Python bytes object directly, with no copy, so a checksum over 100 MB costs only the checksum. It accepts bytes and nothing else; a bytearray or str is a TypeError with an accurate message. Second, long loops should stay interruptible: a Rust loop that never returns to Python ignores Ctrl-C, so call py.check_signals()? every few thousand iterations, which raises KeyboardInterrupt when a signal is pending. Third, panics: a Rust panic must not unwind into the interpreter, so PyO3 catches it and raises PanicException, deriving from BaseException rather than Exception so that ordinary except Exception handlers do not swallow it. Three small functions added to the module show all three:
/// FNV-1a over a bytes-like object, without copying it out of Python.\n#[pyfunction]\nfn checksum(data: &[u8]) -> u64 {\n data.iter().fold(0xcbf29ce484222325u64, |h, b| (h ^ u64::from(*b)).wrapping_mul(0x100000001b3))\n}\n\n/// Busy loop that stays interruptible from Python.\n#[pyfunction]\nfn spin(py: Python<'_>, seconds: f64) -> PyResult<u64> {\n let end = std::time::Instant::now() + std::time::Duration::from_secs_f64(seconds);\n let mut n = 0u64;\n while std::time::Instant::now() < end {\n n += 1;\n if n % 1000 == 0 {\n py.check_signals()?;\n }\n }\n Ok(n)\n}\n\n/// Always panics, to show what Python sees.\n#[pyfunction]\nfn explode() -> u32 {\n let v: Vec<u32> = Vec::new();\n v[3]\n}import fastlib, signal, time\n\ndata = bytes(range(256)) * 400_000\nt = time.perf_counter(); c = fastlib.checksum(data); dt = time.perf_counter() - t\nprint(f\"checksum of {len(data)/1e6:.0f} MB bytes: {c:#018x} in {dt*1e3:.0f} ms\")\nfor bad in (bytearray(b\"abc\"), \"text\"):\n try:\n fastlib.checksum(bad)\n except TypeError as e:\n print(f\"TypeError for {type(bad).__name__}:\", str(e).replace(chr(10), \" \"))\n\nsignal.signal(signal.SIGALRM, lambda *a: signal.raise_signal(signal.SIGINT))\nsignal.setitimer(signal.ITIMER_REAL, 0.2)\nt = time.perf_counter()\ntry:\n fastlib.spin(5.0)\nexcept KeyboardInterrupt:\n print(f\"spin(5.0) interrupted after {time.perf_counter() - t:.1f} s: KeyboardInterrupt\")\n\ntry:\n fastlib.explode()\nexcept BaseException as e:\n print(type(e).__name__, \"-\", e, \"| is an Exception:\", isinstance(e, Exception))\nchecksum of 102 MB bytes: 0x07ec59baf399a325 in 134 ms\nTypeError for bytearray: 'bytearray' object is not an instance of 'bytes'\nTypeError for str: 'str' object is not an instance of 'bytes'\nspin(5.0) interrupted after 0.2 s: KeyboardInterrupt\nPanicException - index out of bounds: the len is 0 but the index is 3 | is an Exception: FalseThe checksum ran over 102 MB in 134 ms with no copy, about 760 MB per second for a byte-at-a-time hash. The interrupt test schedules a SIGINT after 0.2 s while Rust spins for five: the call stopped after 0.2 s with KeyboardInterrupt, exactly as a Python loop would. The panic produced PanicException with Rust's message and isinstance(e, Exception) was false; the panic message is also printed to standard error by Rust's hook. The practical rules follow directly: expected failures return PyResult (and become precise Python exceptions), panics are for bugs, and any loop longer than a few milliseconds checks for signals.
Everything above sends work into Rust. The reverse, a Rust function that calls a Python callable for each item, is worth measuring before you design around it, because it is the case where the extension is slower than the code it replaced:
\n#[pyfunction]\nfn apply(f: &Bound<'_, PyAny>, values: Vec<i64>) -> PyResult<Vec<i64>> {\n values.into_iter().map(|v| f.call1((v,))?.extract::<i64>()).collect()\n}1M callbacks: python list comprehension 47 ms | Rust calling the same lambda 97 ms\nexception from the callback propagates: division by zeroA million calls of lambda v: v * 2 took 47 ms in a Python list comprehension and 97 ms from Rust: about twice as slow, because each callback builds an argument tuple, enters the interpreter's call machinery and converts the result back, while the comprehension runs on a fast path inside the interpreter. The error handling is the good news: an exception raised in the callback (ZeroDivisionError) came out of the Rust function as itself, because ? on a PyResult propagates it. The design rule is that the callback boundary is expensive, so give Rust data and let it work without calling back. If the user must supply behaviour, accept a small description of it (an enum or a string naming an operation) instead of a function, or make the callback process a whole batch per call.
A pure extension module is fine for a small helper. A real library usually wants a Python layer on top: a friendlier API, docstrings, type hints, convenience functions that do not need Rust. Maturin supports that with a source directory for the Python code and a private compiled module underneath, conventionally named _core. The package imports from it, and users only see the public names:
[build-system]\nrequires = [\"maturin>=1.7,<2.0\"]\nbuild-backend = \"maturin\"\n\n[project]\nname = \"fastlib\"\nrequires-python = \">=3.9\"\ndynamic = [\"version\"]\n\n[tool.maturin]\npython-source = \"python\"\nmodule-name = \"fastlib._core\"\nfeatures = [\"pyo3/extension-module\"]\nuse pyo3::prelude::*;\n\n#[pymodule]\nmod _core {\n use super::*;\n\n #[pyfunction]\n fn fib(n: u32) -> u64 {\n let (mut a, mut b) = (0u64, 1u64);\n for _ in 0..n {\n (a, b) = (b, a.wrapping_add(b));\n }\n a\n }\n}\n\"\"\"Fast helpers backed by Rust.\"\"\"\nfrom ._core import fib\n\n__all__ = [\"fib\", \"fib_many\"]\n\n\ndef fib_many(ns):\n \"\"\"Pure Python convenience layer on top of the Rust core.\"\"\"\n return [fib(n) for n in ns]\nThree names must agree: the Cargo library name (_core), the #[pymodule] module name (_core) and the last component of module-name in pyproject.toml (fastlib._core). After maturin develop the package imported correctly and fastlib.fib_many([10, 20, 30]) returned [55, 6765, 832040], with the compiled module landing at fastlib/_core.abi3.so next to the Python files. Editing a Python file takes effect immediately in the editable install; editing Rust needs another maturin develop. This layout also gives you a place for the .pyi stub file and the py.typed marker, which together make type checkers understand the package.
maturin build --release\nls target/wheels\npip install target/wheels/fastlib-0.1.0-cp39-abi3-macosx_11_0_arm64.whlThe wheel produced here was named fastlib-0.1.0-cp39-abi3-macosx_11_0_arm64.whl and was 251 KB. The tags say it all: cp39-abi3 means \"CPython 3.9 or newer, stable ABI\", and the platform part says macOS on ARM. You need one wheel per operating system and CPU architecture, not per Python version. Build them in CI with the PyO3/maturin-action GitHub Action, which runs inside the manylinux containers so Linux wheels work on every distribution, and publish with maturin publish or twine. Linux wheels must be built against an old glibc (the manylinux tag), which is exactly what the action's containers provide; a wheel built on your newest laptop distribution will fail to import on older servers with a GLIBC_2.xx not found error.
| Symptom | Cause | Fix |
|---|---|---|
| \"Couldn't find a virtualenv or conda environment\" | maturin develop outside a venv | Create and activate one, or use maturin build plus pip install |
ImportError: dynamic module does not define module export function | The [lib] name or module name differs from the import name | Make [lib] name, the #[pymodule] name and the import identical |
| Rust version is not faster | Debug build | maturin develop --release |
Linker errors about _Py* symbols in cargo test | extension-module set in Cargo.toml | Put it in pyproject.toml as shown |
pyo3_runtime.PanicException | A Rust panic crossed into Python | Return PyResult for expected failures; a panic is a bug, and the exception derives from BaseException |
| Ctrl-C does nothing during a long call | The Rust loop never checks for signals | Call py.check_signals()? periodically in long loops |
| Threads do not speed up | The heavy part runs with the GIL held, or conversion dominates | py.detach around the compute; pass buffers, not lists |
Wheel fails on old Linux: GLIBC_2.xx not found | Built outside manylinux | Build with maturin-action or a manylinux container |
Test the Rust logic with cargo test on plain functions (keep the algorithm in an ordinary function and make the #[pyfunction] a thin wrapper), and test the Python surface with pytest: import the module, call every function, assert on results and on the exact exception types. The most valuable kind of test compares the Rust implementation against a slow, obviously correct Python one over many inputs, including the edges:
import pytest\nimport fastlib\n\n\ndef python_fib(n):\n a, b = 0, 1\n for _ in range(n):\n a, b = b, (a + b) & 0xFFFFFFFFFFFFFFFF\n return a\n\n\n@pytest.mark.parametrize(\"n\", [0, 1, 2, 10, 50, 93, 94, 1000])\ndef test_matches_python(n):\n assert fastlib.fib(n) == python_fib(n)\n\n\ndef test_negative_is_overflow_error():\n with pytest.raises(OverflowError):\n fastlib.fib(-1)\n\n\ndef test_fib_many_keeps_order():\n assert fastlib.fib_many([5, 1, 10]) == [5, 1, 55]\n.......... [100%]\n10 passed in 0.01sThe parametrised cases include 93 and 94, the boundary where a Fibonacci number stops fitting in 64 bits and the Rust code wraps; the Python reference masks to 64 bits to match, so the test pins down that the behaviour is intentional and the same on both sides. The OverflowError test pins the conversion behaviour that PyO3 gives for free, which protects you from a future change of the argument type. Ship a fastlib.pyi stub file next to the module (or inside the package, with an empty py.typed marker) so editors and type checkers know the signatures; PyO3 does not generate stubs for you, and a hand-written 15-line file is enough. Add one benchmark to the suite that fails if the Rust path becomes slower than the Python fallback, so the reason for the dependency stays true.
If the bottleneck is I/O (network, disk), Rust will not help; use async Python. If it is a numeric array operation that NumPy, Polars or JAX already do, call those. If you need a command-line tool or a service, write a Rust binary and call it, avoiding the boundary altogether. And when the Python side needs to call back into Python many times from Rust, the cost of crossing in both directions can erase the gain, so keep the callbacks coarse. The same approach for JavaScript is in Node addons with napi-rs, and the C interface underneath both is explained in calling Rust from C.
","date_published":"2026-11-03T09:56:00+00:00","tags":["Systems"]},{"id":"https://rusthacks.dev/library/safe-rust-bindings-c-library","url":"https://rusthacks.dev/library/safe-rust-bindings-c-library","title":"Safe Rust Bindings for a C Library a Worked Example","summary":"A good binding is two crates. The sys crate mirrors the C header exactly, and the safe crate hides the pointers behind types that cannot be misused.","content_html":"Raw bindings from bindgen compile and call C correctly, and are almost unusable. Every function is unsafe, every string is a *const c_char, every handle must be freed by hand, and every rule about who owns what lives in a C comment that the compiler never reads. A safe binding turns those comments into types. When it is done, a Rust program can use the C library with no unsafe of its own, and the mistakes that C makes easy, a use after free, a leak, a double free, a call from the wrong thread, become compile errors.
This guide wraps the small C key-value store from the previous guide (calling C from Rust) in a safe crate, one decision at a time. Each decision is the answer to a rule in the C header, and the compiler's refusals at the end prove the rules are enforced. It was built and tested on Rust 1.99 (edition 2024) with cc 1.11.1, and every output block is real.
Wrapping a library starts with reading its contract, because a safe API is only as good as your understanding of the rules it encodes. The store's header makes these promises:
\nkv_open returns a handle or NULL, and the handle must be freed exactly once with kv_close.kv_set copies the key and value, so the caller keeps ownership of what it passed. It returns 0 or -1.kv_get returns a pointer owned by the store, valid until the next kv_set of that key or kv_close, or NULL if absent.kv_foreach calls a function pointer with an opaque void *.Each of them maps to a Rust feature. Promise 1 becomes ownership and Drop. Promise 2 becomes &str arguments converted to temporary CString values. Promise 3 becomes a lifetime. Promise 4 becomes a generic closure parameter. Promise 5 becomes the absence of Send and Sync.
sys moduleThe raw declarations live in their own module and are not public. Nothing outside the crate sees an unsafe function.
//! Raw declarations of the C API. Nothing outside this crate sees them.\n#![allow(non_camel_case_types)]\nuse std::ffi::{c_char, c_int, c_void};\n\n#[repr(C)]\npub struct kv_t {\n _private: [u8; 0],\n}\n\npub type kv_visit_fn = Option<unsafe extern \"C\" fn(*const c_char, *const c_char, *mut c_void)>;\n\nunsafe extern \"C\" {\n pub fn kv_open(name: *const c_char) -> *mut kv_t;\n pub fn kv_set(store: *mut kv_t, key: *const c_char, value: *const c_char) -> c_int;\n pub fn kv_get(store: *const kv_t, key: *const c_char) -> *const c_char;\n pub fn kv_len(store: *const kv_t) -> usize;\n pub fn kv_foreach(store: *const kv_t, visit: kv_visit_fn, user_data: *mut c_void);\n pub fn kv_close(store: *mut kv_t);\n}You can write this by hand for a small library or generate it with bindgen, as in the previous guide. Either way, it should be dull and mechanical. All the thinking happens one layer up. In a published binding, this layer would be a separate -sys crate with links = \"kv\" in its manifest, but a private module keeps the example in one file.
//! A safe interface to the C `kv` store.\n\nmod sys;\n\nuse std::ffi::{c_char, c_void, CStr, CString, NulError};\nuse std::ptr::NonNull;\n\n/// Everything that can go wrong, as a normal Rust error.\n#[derive(Debug, PartialEq, Eq)]\npub enum Error {\n /// A key, value or name contained a NUL byte, which a C string cannot hold.\n InteriorNul { position: usize },\n /// The C library reported failure, in practice that it ran out of memory.\n Failed,\n /// The library returned text that is not valid UTF-8.\n NotUtf8,\n}\n\nimpl From<NulError> for Error {\n fn from(e: NulError) -> Self {\n Error::InteriorNul { position: e.nul_position() }\n }\n}\n\nimpl std::fmt::Display for Error {\n fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n match self {\n Error::InteriorNul { position } => write!(f, \"text contains a NUL byte at position {position}\"),\n Error::Failed => f.write_str(\"the kv library reported an error\"),\n Error::NotUtf8 => f.write_str(\"the kv library returned text that is not UTF-8\"),\n }\n }\n}\n\nimpl std::error::Error for Error {}\n\n/// An owned handle to a store. The C store is freed when this value is dropped.\n///\n/// Holding a `NonNull` makes the type `!Send` and `!Sync` automatically. The C library\n/// does not document thread safety, so we do not claim it.\npub struct Kv {\n raw: NonNull<sys::kv_t>,\n}\n\nimpl Kv {\n pub fn open(name: &str) -> Result<Kv, Error> {\n let name = CString::new(name)?;\n // SAFETY: `name` is a valid NUL-terminated string that outlives the call.\n let raw = unsafe { sys::kv_open(name.as_ptr()) };\n NonNull::new(raw).map(|raw| Kv { raw }).ok_or(Error::Failed)\n }\n\n pub fn set(&mut self, key: &str, value: &str) -> Result<(), Error> {\n let (key, value) = (CString::new(key)?, CString::new(value)?);\n // SAFETY: `self.raw` is live (we own it), both strings are valid for the call,\n // and `kv_set` copies them. `&mut self` rules out a concurrent reader.\n let rc = unsafe { sys::kv_set(self.raw.as_ptr(), key.as_ptr(), value.as_ptr()) };\n if rc == 0 { Ok(()) } else { Err(Error::Failed) }\n }\n\n /// The value is borrowed straight from the C store, with no copy. The `&self` borrow\n /// lasts as long as the returned `&str`, so `set` (which needs `&mut self`) cannot run\n /// while it is in use. That is exactly the validity rule the C header documents.\n pub fn get(&self, key: &str) -> Result<Option<&str>, Error> {\n let key = CString::new(key)?;\n // SAFETY: `self.raw` is live; `key` is valid for the call.\n let ptr = unsafe { sys::kv_get(self.raw.as_ptr(), key.as_ptr()) };\n if ptr.is_null() {\n return Ok(None);\n }\n // SAFETY: the C library returns a NUL-terminated string owned by the store, valid\n // until the next `kv_set` of this key or `kv_close`. Both need `&mut self` or\n // ownership, which the returned lifetime (tied to `&self`) forbids.\n let text = unsafe { CStr::from_ptr(ptr) };\n text.to_str().map(Some).map_err(|_| Error::NotUtf8)\n }\n\n pub fn len(&self) -> usize {\n // SAFETY: `self.raw` is live.\n unsafe { sys::kv_len(self.raw.as_ptr()) }\n }\n\n pub fn is_empty(&self) -> bool {\n self.len() == 0\n }\n\n /// Snapshot of every entry, copied into owned strings so it can outlive later `set` calls.\n ///\n /// ```\n /// use safekv::Kv;\n ///\n /// # fn main() -> Result<(), safekv::Error> {\n /// let mut kv = Kv::open(\"doc\")?;\n /// kv.set(\"a\", \"1\")?;\n /// kv.set(\"b\", \"2\")?;\n /// let snapshot = kv.entries();\n /// kv.set(\"a\", \"changed\")?; // fine: the snapshot owns its data\n /// assert_eq!(snapshot, [(\"a\".to_string(), \"1\".to_string()), (\"b\".to_string(), \"2\".to_string())]);\n /// # Ok(())\n /// # }\n /// ```\n pub fn entries(&self) -> Vec<(String, String)> {\n let mut out = Vec::with_capacity(self.len());\n self.for_each(|k, v| out.push((k.to_owned(), v.to_owned())));\n out\n }\n\n /// Visit every entry in insertion order. The closure may capture and mutate local state.\n pub fn for_each<F: FnMut(&str, &str)>(&self, mut f: F) {\n unsafe extern \"C\" fn trampoline<F: FnMut(&str, &str)>(k: *const c_char, v: *const c_char, user: *mut c_void) {\n // SAFETY: `user` is the `&mut F` created below and is live for the whole call.\n let f = unsafe { &mut *user.cast::<F>() };\n let (k, v) = unsafe { (CStr::from_ptr(k), CStr::from_ptr(v)) };\n // A panic must not unwind into C. Dropping it here loses the panic, so record it\n // instead in a real wrapper; this version simply stops propagating.\n let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| f(&k.to_string_lossy(), &v.to_string_lossy())));\n }\n // SAFETY: `self.raw` is live and `f` outlives the call, which is the only time C uses the pointer.\n unsafe { sys::kv_foreach(self.raw.as_ptr(), Some(trampoline::<F>), (&mut f as *mut F).cast()) };\n }\n}\n\nimpl Drop for Kv {\n fn drop(&mut self) {\n // SAFETY: we own the handle and this is the only place it is freed.\n unsafe { sys::kv_close(self.raw.as_ptr()) };\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn set_get_and_overwrite() {\n let mut kv = Kv::open(\"t\").unwrap();\n assert!(kv.is_empty());\n kv.set(\"host\", \"a.test\").unwrap();\n kv.set(\"port\", \"80\").unwrap();\n kv.set(\"host\", \"b.test\").unwrap();\n assert_eq!(kv.len(), 2);\n assert_eq!(kv.get(\"host\").unwrap(), Some(\"b.test\"));\n assert_eq!(kv.get(\"absent\").unwrap(), None);\n }\n\n #[test]\n fn interior_nul_is_an_error_not_a_crash() {\n let mut kv = Kv::open(\"t\").unwrap();\n assert_eq!(kv.set(\"ba\\0d\", \"x\"), Err(Error::InteriorNul { position: 2 }));\n assert!(matches!(Kv::open(\"na\\0me\"), Err(Error::InteriorNul { .. })));\n }\n\n #[test]\n fn for_each_can_capture_state() {\n let mut kv = Kv::open(\"t\").unwrap();\n for (k, v) in [(\"a\", \"1\"), (\"b\", \"22\")] {\n kv.set(k, v).unwrap();\n }\n let mut seen = Vec::new();\n kv.for_each(|k, v| seen.push(format!(\"{k}={v}\")));\n assert_eq!(seen, [\"a=1\", \"b=22\"]);\n }\n\n #[test]\n fn entries_snapshot_survives_a_later_set() {\n let mut kv = Kv::open(\"t\").unwrap();\n kv.set(\"k\", \"old\").unwrap();\n let snapshot = kv.entries();\n kv.set(\"k\", \"new\").unwrap();\n assert_eq!(snapshot, [(\"k\".to_string(), \"old\".to_string())]);\n assert_eq!(kv.get(\"k\").unwrap(), Some(\"new\"));\n }\n\n #[test]\n fn many_open_and_close_cycles_do_not_crash() {\n for i in 0..2_000 {\n let mut kv = Kv::open(\"cycle\").unwrap();\n kv.set(&format!(\"key{i}\"), \"value\").unwrap();\n }\n }\n\n #[test]\n fn unicode_round_trips() {\n let mut kv = Kv::open(\"t\").unwrap();\n kv.set(\"city\", \"東京\").unwrap();\n assert_eq!(kv.get(\"city\").unwrap(), Some(\"東京\"));\n }\n}Read the file against the five promises.
\nKv owns the handleKv holds a NonNull<kv_t>, a pointer that the type guarantees is not null. Kv::open checks the C return value once, converting a NULL into Err(Error::Failed). From then on every method can assume the handle is valid, and the check cannot be forgotten, because it lives in the only constructor. The fields are private, so no one can build a Kv around a bad pointer.
The Drop impl calls kv_close. That single line eliminates the leak, because the handle is freed when the value goes out of scope on every path, including early returns and panics. It eliminates the double free, because Rust drops a value once and the move rules stop anyone from using it afterward. And it eliminates the use after free, as the third compile error below shows. Of the three bugs C programmers fear most, ownership types remove them entirely.
The C side wants NUL-terminated strings, and Rust strings can contain NUL bytes. Every method that takes text converts it through CString::new, which returns an error for an embedded NUL. The wrapper does not unwrap that error; it turns it into Error::InteriorNul { position } through a From impl so the ? operator does the conversion. The demo exercises it:
error: text contains a NUL byte at position 3A caller who passes user input with a stray NUL gets a typed error they can handle, rather than a truncated key, which is what C would have silently stored.
\nThis is the most important design decision in the file, and it is worth slowing down for. The header says kv_get's result is valid until the next kv_set of that key or kv_close. There are two ways to wrap it.
The cautious way copies the text into an owned String before returning, at the cost of an allocation on every read. It is always safe, and the right choice if reads are rare or the values small. The zero-copy way, used here, returns &str borrowed directly from the C store, and lets the borrow checker enforce the validity rule. The signature fn get(&self, key: &str) -> Result<Option<&str>, Error> has an elided lifetime that ties the returned &str to the borrow of self. While that reference is alive, self is borrowed, and set needs &mut self, so the compiler refuses to compile a program that would invalidate the pointer. The safe interface is a precise transcription of the C comment.
Here is a program that violates the rule, and what the compiler said:
\nuse safekv::Kv;\n\nfn main() {\n let mut kv = Kv::open(\"s\").unwrap();\n kv.set(\"host\", \"a.test\").unwrap();\n let host = kv.get(\"host\").unwrap().unwrap(); // borrowed from the C store\n kv.set(\"host\", \"b.test\").unwrap(); // C frees the old value here\n println!(\"{host}\");\n}error[E0502]: cannot borrow `kv` as mutable because it is also borrowed as immutable\n --> examples/bad1.rs:7:5\n |\n6 | let host = kv.get(\"host\").unwrap().unwrap(); // borrowed from the C store\n | -- immutable borrow occurs here\n7 | kv.set(\"host\", \"b.test\").unwrap(); // C frees the old value here\n | ^^^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here\n8 | println!(\"{host}\");\n | ---- immutable borrow later used hereIn C the same sequence compiles, runs, and reads freed memory. In Rust it does not compile, and the message points at the exact line where the C library would free the string. The cost of the guarantee is that code which holds a value across a set must copy it first (.to_string()), which is exactly what the C comment told callers to do.
&self and &mut selfThe split between get, len and for_each taking &self, and set taking &mut self, is the other half of the lifetime rule. It also happens to match the semantics: reading never changes the store, writing does. In C every function takes the same plain pointer, so the distinction is invisible; the wrapper adds it, and with it the ability to have many readers or one writer, checked at compile time.
kv_foreach takes a bare C function pointer and a void *. The wrapper's for_each accepts any FnMut(&str, &str), so the caller can capture variables. Inside is the trampoline pattern: a nested generic extern \"C\" function that casts the void * back to the closure and calls it. A panic inside the closure must not unwind into C, so the call sits in catch_unwind. The comment in the code is honest about the limit: the panic is swallowed rather than re-raised in the caller, and a production wrapper would record it and resume it after kv_foreach returns. The trampoline is unsafe code that no user of the crate has to read or write.
With the unsafe code behind the interface, the demo program has none:
\nuse safekv::Kv;\n\nfn main() -> Result<(), safekv::Error> {\n let mut kv = Kv::open(\"settings\")?;\n kv.set(\"host\", \"example.test\")?;\n kv.set(\"port\", \"8080\")?;\n\n println!(\"host = {:?}\", kv.get(\"host\")?);\n println!(\"nope = {:?}\", kv.get(\"nope\")?);\n\n let mut total = 0;\n kv.for_each(|key, value| {\n total += key.len() + value.len();\n println!(\" {key}: {value}\");\n });\n println!(\"{} entries, {} bytes of text\", kv.len(), total);\n\n match kv.set(\"bad\\0key\", \"x\") {\n Err(e) => println!(\"error: {e}\"),\n Ok(()) => unreachable!(),\n }\n Ok(())\n}host = Some(\"example.test\")\nnope = None\n host: example.test\n port: 8080\n2 entries, 24 bytes of text\nerror: text contains a NUL byte at position 3The ? operators carry errors up through main, absent keys are None rather than a null pointer to be checked, the closure updates total as a normal local, and the handle is freed when kv goes out of scope, with no call to remember. This is the point of the exercise: the demo reads like a program in any other safe language.
Send offThe compiler decided something on the crate's behalf. NonNull<T> is neither Send nor Sync, so Kv is neither, automatically. The header was silent about thread safety, so silence is the correct default: a library that does not promise to be thread-safe should be assumed not to be. The effect is that moving a Kv into another thread fails to compile:
error[E0277]: `NonNull<safekv::sys::kv_t>` cannot be sent between threads safely\n --> examples/bad2.rs:5:24\n |\n 5 | std::thread::spawn(move || {\n | ------------------ ^------\n | | |\n | _____|__________________within this `{closure@examples/bad2.rs:5:24: 5:31}`\n | | |\n | | required by a bound introduced by this call\n 6 | | println!(\"{}\", kv.len());\n 7 | | })\n | |_____^ `NonNull<safekv::sys::kv_t>` cannot be sent between threads safely\n |\n = help: within `{closure@examples/bad2.rs:5:24: 5:31}`, the trait `Send` is not implemented for `NonNull<safekv::sys::kv_t>`\nnote: required because it appears within the type `Kv`If you read the library's source or documentation and learn that a handle may be used from one thread at a time but moved between threads, you can opt in deliberately with unsafe impl Send for Kv {} and a comment citing the evidence. If the library is fully thread-safe internally, you can add unsafe impl Sync too. Both are claims about code you did not write, so make them only with documentation or source in hand. If you want sharing without evidence, wrap the handle in a Mutex<Kv> in your application, and the mutex serializes access, which is safe for any library that tolerates use from different threads one at a time.
Since Kv owns the handle, destroying it takes the value away. The compiler does the bookkeeping that C leaves to the programmer:
error[E0382]: borrow of moved value: `kv`\n --> examples/bad3.rs:6:20\n |\n4 | let kv = Kv::open(\"s\").unwrap();\n | -- move occurs because `kv` has type `Kv`, which does not implement the `Copy` trait\n5 | drop(kv);\n | -- value moved here\n6 | println!(\"{}\", kv.len());\n | ^^ value borrowed here after moveTwo lines of C do this without complaint and crash later. Here the second line is rejected, and the message names the move. Because Kv is not Copy or Clone, there cannot be two owners, and the handle is freed exactly once.
The entries method added during testing shows a pattern worth adopting wherever a wrapper hands out pointers the C library owns. Keep the zero-copy accessor, get, for callers who read and move on, and add an owned snapshot, entries, for callers who need data to outlive the next mutation. The snapshot is built on the same for_each machinery, copying each key and value into a String, so it adds no new unsafe code. The doctest demonstrates the difference: after taking a snapshot, a later set is allowed, because the snapshot borrows nothing from kv. The borrow checker's refusal in the first error message turns into a documented escape hatch.
Resist the urge to make the safe type clever. An iterator that yields borrowed pairs straight from the C list would be faster, and the C library gives no guarantee about the list's stability while you hold a pointer into it, so writing one means reading the C source to find out. The callback-based for_each is the access path the header documents, so build on that, and offer an iterator only if profiling shows the snapshot's allocations matter.
This C library signals failure with a single -1, so Error::Failed is enough. Real libraries are less kind, and the mapping is where wrappers earn their keep. A library that returns negative errno values becomes std::io::Error::from_raw_os_error(-rc), which gives callers kind() and a readable message for free. A library with its own error codes becomes a #[non_exhaustive] enum with one variant per documented code, and an Unknown(i32) for codes added later, so a new library version does not turn into a panic in your match. A library with a thread-local \"last error\" function, like OpenSSL's, needs the message fetched immediately after the failing call, on the same thread, before any other call clobbers it; put the failing call and the fetch in one private helper so nobody can separate them. In every case the principle matches the rest of the guide: the C caller had to know the convention, and the Rust caller should not.
Many C libraries need lib_init() before the first call and lib_cleanup() at exit. The first is easy to forget and the second is dangerous to call twice. Wrap the init in a static INIT: std::sync::Once that Kv::open runs through INIT.call_once(...), so the first handle created initializes the library and later ones do nothing, from any thread. Skip the cleanup call unless the library demands it; process exit reclaims memory, and a cleanup that races with a handle still in use on another thread is a crash that only appears at shutdown, in production, once a month. If cleanup is mandatory, tie it to a reference-counted private guard that every handle holds, so it runs when the last handle drops and not before.
The six unit tests and one documentation test in the file cover the behaviors that map to the header's promises. Set and overwrite checks that the second set of host replaces the first. A key containing a NUL byte becomes an error and not a crash. A closure captures local state. Non-ASCII text round-trips. The entries snapshot survives a later set, which is the copy-versus-borrow decision made visible. And two thousand open and close cycles run without a crash, a cheap smoke test for the Drop path. The doctest on entries doubles as the documentation example and as a test that the documented usage compiles. They all passed:
running 6 tests\ntest tests::entries_snapshot_survives_a_later_set ... ok\ntest tests::interior_nul_is_an_error_not_a_crash ... ok\ntest tests::for_each_can_capture_state ... ok\ntest tests::set_get_and_overwrite ... ok\ntest tests::unicode_round_trips ... ok\ntest tests::many_open_and_close_cycles_do_not_crash ... ok\ntest result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s\n\nDoc-tests safekv\ntest src/lib.rs - Kv::entries (line 89) ... ok\ntest result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00sTests have limits at this boundary. Miri cannot execute C code, so it will not check the foreign side. A use after free inside the C library, or a leak, needs a sanitizer: build the C file with -fsanitize=address and run the tests under it, or use valgrind on Linux. Run those on CI for the wrapper crate, since a wrapper that is sound on paper still depends on the C library behaving as documented. Include tests for the awkward inputs: empty strings, a very long value, keys that differ only in case, and the repeated open and close of many stores to expose leaks.
A few habits distinguish a binding you would depend on from one you would fork. Name the crate foo and its raw layer foo-sys, and pin the native library's version in the -sys crate's build script. Implement Debug for Kv without printing the pointer, or print the name only. Provide Error with Display and std::error::Error, as here, so it composes with anyhow and thiserror. Document every public method with a # Errors section. Offer a Result-returning API rather than panics; panicking on out-of-memory is the choice of the standard library, not of a binding. And if the C library has global state, such as an initialization call, wrap it in a std::sync::Once inside Kv::open so the user cannot forget it.
If the library has a maintained Rust binding, use it and spend the time elsewhere; check crates.io for a -sys crate before writing declarations. If you call one function once, an unsafe block with a SAFETY comment at the call site is simpler than a crate. If the C library is so small that porting it is an afternoon's work, a Rust port removes the foreign code, the build step and the sanitizer runs. And if the library is C++, plain bindgen struggles with templates and exceptions, so reach for cxx. Everywhere else, the process in this guide, read the contract, own the handle, borrow the pointer, convert the errors, leave out the auto traits you cannot justify, is the path to a binding where the compiler does the reviewing. For the complementary direction, see calling Rust from C with cbindgen, and for the language rules underneath, writing a safe wrapper around unsafe code.
Walk the public surface once with a hostile caller in mind. Can any safe sequence of calls reach a use after free, a double free or a data race? The compile errors above argue no for the three most common, and a reviewer should try to find a fourth. Is every unsafe block short, commented and local to one function? Does every public function either return a Result or cannot fail? Are the Send and Sync decisions written down in the type's documentation, with the evidence? If you can answer yes to all four, the crate is ready for someone else to build on.
Memory bugs at an FFI boundary are rarely about the languages. They are about nobody agreeing who is responsible. The rule that settles most questions is short: whoever allocates, frees, using the same allocator. Examples compile on Rust 1.90, edition 2024.
\nAn allocator keeps hidden bookkeeping next to each block, and a free implementation expects its own format. Rust's default allocator on many platforms is the system malloc, but you can replace it with jemalloc or mimalloc, and on Windows the C runtime and the Rust runtime can each own a heap. Passing a pointer to the wrong free corrupts the heap, sometimes silently and much later (custom allocators).
use std::ffi::{c_char, CString};\n\npub struct Config {\n pub retries: u32,\n}\n\n#[unsafe(no_mangle)]\npub extern \"C\" fn config_new() -> *mut Config {\n Box::into_raw(Box::new(Config { retries: 3 }))\n}\n\n#[unsafe(no_mangle)]\npub extern \"C\" fn config_free(p: *mut Config) {\n if !p.is_null() {\n // SAFETY: p came from config_new and has not been freed\n drop(unsafe { Box::from_raw(p) });\n }\n}\n\n#[unsafe(no_mangle)]\npub extern \"C\" fn config_describe(p: *const Config) -> *mut c_char {\n if p.is_null() {\n return std::ptr::null_mut();\n }\n // SAFETY: p is non-null and points to a live Config per the API contract\n let c = unsafe { &*p };\n match CString::new(format!(\"retries={}\", c.retries)) {\n Ok(s) => s.into_raw(),\n Err(_) => std::ptr::null_mut(),\n }\n}\n\n#[unsafe(no_mangle)]\npub extern \"C\" fn string_free(s: *mut c_char) {\n if !s.is_null() {\n // SAFETY: s came from CString::into_raw in this library\n drop(unsafe { CString::from_raw(s) });\n }\n}Each constructor has a destructor, and the string function names its partner in documentation. A C caller that calls free(str) instead of string_free(str) is the bug to prevent with comments and tests.
use std::ffi::c_void;\n\nunsafe extern \"C\" {\n fn make_buffer(len: usize) -> *mut u8; // returns malloc'd memory\n fn free_buffer(p: *mut u8); // the library's own free\n}\n\npub struct Buffer {\n ptr: *mut u8,\n len: usize,\n}\n\nimpl Buffer {\n pub fn new(len: usize) -> Option<Self> {\n // SAFETY: plain call, returns null on failure\n let ptr = unsafe { make_buffer(len) };\n if ptr.is_null() { None } else { Some(Self { ptr, len }) }\n }\n\n pub fn as_slice(&self) -> &[u8] {\n // SAFETY: ptr is valid for len bytes until Drop\n unsafe { std::slice::from_raw_parts(self.ptr, self.len) }\n }\n}\n\nimpl Drop for Buffer {\n fn drop(&mut self) {\n // SAFETY: ptr came from make_buffer and is freed once\n unsafe { free_buffer(self.ptr) };\n }\n}Never convert such a pointer into a Vec or Box with from_raw_parts. Rust would later free it with its allocator. Keep the foreign pointer in a type whose Drop calls the foreign free. Do not call std::mem::forget on guard types to \"hand off\" without recording who frees.
A function like const char *config_name(const config *) returns a pointer into the object. Document it as borrowed, valid until the object is freed or modified. In Rust, wrap it as &'a CStr tied to the owner's lifetime:
pub fn name(&self) -> &std::ffi::CStr {\n // SAFETY: the library guarantees a NUL terminated string valid as long as self\n unsafe { std::ffi::CStr::from_ptr(config_name(self.raw)) }\n}The returned reference borrows &self, so the compiler prevents it outliving the object.
| Mistake | Result | |---|---| | free() on a Box::into_raw pointer | heap corruption | | Box::from_raw on a malloc pointer | allocator mismatch | | Double free (calling destroy twice) | undefined behavior | | Using a pointer after destroy | use after free | | Returning a pointer to a local variable | dangling | | Forgetting the free | leak, found by sanitizers |
Use AddressSanitizer or Valgrind on the combined program, and Miri for the Rust half (sanitizers).
\nDrop.Next steps: callbacks, panics across FFI and cbindgen exports.
","date_published":"2026-11-03T09:50:00+00:00","tags":["Systems"]},{"id":"https://rusthacks.dev/library/call-rust-from-c-cbindgen","url":"https://rusthacks.dev/library/call-rust-from-c-cbindgen","title":"Calling Rust From C With Cbindgen and Staticlib Step by Step","summary":"To expose Rust to C, mark functions extern \"C\" with a stable name, build a staticlib or cdylib and let cbindgen write the header. The rules are about layout, ownership and panics.","content_html":"The usual direction of FFI talk is Rust calling C, but the reverse is how Rust gets adopted in the real world. A team does not rewrite a 400,000 line C application. It replaces one hot or hazardous module, a parser that keeps producing security advisories, a compression routine, a protocol decoder, with Rust code that the rest of the application calls through its existing C interface. The C side sees a header and a library. The Rust side keeps its type system, its tests and its borrow checker everywhere except the thin layer at the border.
\nThis guide builds that border. It exports a small word-counting library from Rust, generates the C header with cbindgen, compiles a C program against it and runs it. It also covers the rules the border must enforce: no panic may escape, memory must be freed by the side that allocated it, and errors must become integers. The library was built with Rust 1.99 (edition 2024) and cbindgen 0.29.4, the C program was compiled with the system cc on an Apple silicon Mac, and the outputs below are what they printed.
C has no generics, no ownership and no exceptions. Whatever you export must be expressible in those terms, which constrains the design more than any syntax does. Five rules cover it.
\nextern \"C\" plus #[unsafe(no_mangle)].#[repr(C)] structs whose layout C can reproduce. Everything else, a String, a Vec, a trait object, stays on the Rust side behind an opaque pointer.free it.The library below follows them one at a time.
\nStart with the manifest. The crate type decides what C links against: staticlib produces a .a archive that is linked into the final executable, and cdylib produces a shared library (.so, .dylib or .dll). Build both and ship whichever your C build prefers.
[package]\nname = \"wordstat\"\nversion = \"0.1.0\"\nedition = \"2024\"\n\n[lib]\ncrate-type = [\"staticlib\", \"cdylib\"]//! A word counter with a C API.\n\nuse std::ffi::{c_char, c_int, CStr, CString};\nuse std::panic::{catch_unwind, AssertUnwindSafe};\n\n/// Opaque to C: it only ever sees a `WordStats *`.\npub struct WordStats {\n words: usize,\n lines: usize,\n}\n\n/// Plain data that C can receive by value.\n#[repr(C)]\npub struct WordStatsSummary {\n pub words: usize,\n pub lines: usize,\n}\n\npub const WORDSTAT_OK: c_int = 0;\npub const WORDSTAT_NULL_ARGUMENT: c_int = -1;\npub const WORDSTAT_INVALID_UTF8: c_int = -2;\npub const WORDSTAT_PANIC: c_int = -3;\n\n/// Create a counter. Free it with `wordstat_free`.\n#[unsafe(no_mangle)]\npub extern \"C\" fn wordstat_new() -> *mut WordStats {\n Box::into_raw(Box::new(WordStats { words: 0, lines: 0 }))\n}\n\n/// Add the words and lines in a NUL-terminated UTF-8 string.\n///\n/// # Safety\n/// `stats` must come from `wordstat_new` and not have been freed. `text` must point to a\n/// NUL-terminated string that stays valid for the duration of the call.\n#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn wordstat_feed(stats: *mut WordStats, text: *const c_char) -> c_int {\n if stats.is_null() || text.is_null() {\n return WORDSTAT_NULL_ARGUMENT;\n }\n let result = catch_unwind(AssertUnwindSafe(|| {\n // SAFETY: both pointers were checked for null; validity is the caller's contract.\n let (stats, text) = unsafe { (&mut *stats, CStr::from_ptr(text)) };\n match text.to_str() {\n Ok(s) => {\n stats.words += s.split_whitespace().count();\n stats.lines += s.lines().count();\n WORDSTAT_OK\n }\n Err(_) => WORDSTAT_INVALID_UTF8,\n }\n }));\n result.unwrap_or(WORDSTAT_PANIC)\n}\n\n/// Totals so far. Returns zeros for a null pointer.\n///\n/// # Safety\n/// `stats` must be null or a live pointer from `wordstat_new`.\n#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn wordstat_summary(stats: *const WordStats) -> WordStatsSummary {\n match unsafe { stats.as_ref() } {\n Some(s) => WordStatsSummary { words: s.words, lines: s.lines },\n None => WordStatsSummary { words: 0, lines: 0 },\n }\n}\n\n/// A human-readable report. The returned string is allocated by Rust and must be\n/// released with `wordstat_string_free`, never with C's `free`.\n///\n/// # Safety\n/// `stats` must be null or a live pointer from `wordstat_new`.\n#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn wordstat_report(stats: *const WordStats) -> *mut c_char {\n let Some(s) = (unsafe { stats.as_ref() }) else { return std::ptr::null_mut() };\n let text = format!(\"{} words on {} lines\", s.words, s.lines);\n CString::new(text).map_or(std::ptr::null_mut(), CString::into_raw)\n}\n\n/// # Safety\n/// `s` must be null or a pointer returned by `wordstat_report`, not yet freed.\n#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn wordstat_string_free(s: *mut c_char) {\n if !s.is_null() {\n drop(unsafe { CString::from_raw(s) });\n }\n}\n\n/// # Safety\n/// `stats` must be null or a pointer from `wordstat_new`, not yet freed.\n#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn wordstat_free(stats: *mut WordStats) {\n if !stats.is_null() {\n drop(unsafe { Box::from_raw(stats) });\n }\n}\n\n/// Write the report into a buffer the caller owns, `snprintf` style.\n///\n/// Returns the length the full report needs, not counting the NUL, or a negative error code.\n/// At most `cap` bytes are written (including the NUL), so a return value `>= cap` means\n/// the output was truncated. Passing `buf = NULL` with `cap = 0` asks only for the length.\n///\n/// # Safety\n/// `stats` must be a live pointer from `wordstat_new`. `buf` must be null or valid for\n/// writes of `cap` bytes.\n#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn wordstat_report_into(stats: *const WordStats, buf: *mut c_char, cap: usize) -> isize {\n let Some(s) = (unsafe { stats.as_ref() }) else { return WORDSTAT_NULL_ARGUMENT as isize };\n let text = format!(\"{} words on {} lines\", s.words, s.lines);\n if !buf.is_null() && cap > 0 {\n let n = text.len().min(cap - 1);\n // SAFETY: the caller promised `buf` is writable for `cap` bytes and `n + 1 <= cap`.\n unsafe {\n std::ptr::copy_nonoverlapping(text.as_ptr(), buf.cast::<u8>(), n);\n *buf.add(n) = 0;\n }\n }\n text.len() as isize\n}Read the file for the pattern, because the same shape repeats for every Rust library you expose.
\nAn opaque handle. WordStats is an ordinary Rust struct with no #[repr(C)]. C never sees its fields, only a pointer to it. wordstat_new boxes one and returns Box::into_raw, which hands ownership to the caller as a raw pointer. wordstat_free takes that pointer back with Box::from_raw and drops it. The pair is the lifecycle, and there are no other ways in or out. Because the struct is opaque, you can add and reorder fields in the next release without recompiling a single C file.
Data by value. WordStatsSummary is #[repr(C)], which makes its layout match what a C compiler would produce for the same two fields, so it can be returned by value. Use this for small plain-data results. It is the one place where the layout becomes part of your public interface: adding a field later breaks every compiled caller, so such structs deserve more care than opaque handles.
Error codes. The four constants, WORDSTAT_OK and three negative values, are the whole error vocabulary. wordstat_feed checks its pointers for null before touching them, converts the C string with CStr::to_str, which fails for text that is not UTF-8, and returns a distinct code for each case. C callers can switch on the result. Do not return Rust's Result: it has no stable layout.
Panic containment. The body of wordstat_feed runs inside catch_unwind, and a panic becomes WORDSTAT_PANIC instead of unwinding into C. This matters more than it looks. Unwinding through frames compiled by a C compiler is undefined behavior in the older rules, and an abort in current Rust, which kills the host application. A library that crashes its host over a bug in one call is a library no one will embed twice. Wrap every exported function that can panic, which in practice means every function that does more than trivial work.
A string that Rust allocated. wordstat_report builds a String, converts it to a CString and returns into_raw, handing C a char *. The doc comment says what the type system cannot: release it with wordstat_string_free, never with free. The returned pointer was allocated by Rust's allocator, and the matching free must go through the same allocator. On many platforms Rust uses the system malloc, and calling free happens to work, which is the worst possible outcome because it teaches the wrong lesson and then fails when someone switches to a different allocator, or on Windows when the library and the application are linked against different C runtimes.
Writing the header by hand is how the C and Rust sides drift apart. cbindgen reads the crate's source and emits it. A small configuration file sets the language and the include guard:
language = \"C\"\ninclude_guard = \"WORDSTAT_H\"\ncpp_compat = true\ndocumentation = true\ndocumentation_style = \"c\"cbindgen --config cbindgen.toml --crate wordstat -o wordstat.hHere is what it produced, trimmed to the declarations:
\n#ifndef WORDSTAT_H\n#define WORDSTAT_H\n\n#define WORDSTAT_OK 0\n#define WORDSTAT_NULL_ARGUMENT -1\n#define WORDSTAT_INVALID_UTF8 -2\n#define WORDSTAT_PANIC -3\n\n/* Opaque to C: it only ever sees a `WordStats *`. */\ntypedef struct WordStats WordStats;\n\n/* Plain data that C can receive by value. */\ntypedef struct WordStatsSummary {\n uintptr_t words;\n uintptr_t lines;\n} WordStatsSummary;\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif // __cplusplus\n\nstruct WordStats *wordstat_new(void);\nint wordstat_feed(struct WordStats *stats, const char *text);\nstruct WordStatsSummary wordstat_summary(const struct WordStats *stats);\nchar *wordstat_report(const struct WordStats *stats);\nvoid wordstat_string_free(char *s);\nvoid wordstat_free(struct WordStats *stats);\n\n#ifdef __cplusplus\n} // extern \"C\"\n#endif // __cplusplus\n\n#endif /* WORDSTAT_H */The doc comments, including the # Safety sections, were carried across as C comments, so the contract travels with the header. The opaque struct became a forward-declared typedef struct WordStats WordStats, which C code can hold pointers to but cannot instantiate. usize became uintptr_t, and the constants became macros. cpp_compat wrapped everything in extern \"C\" so C++ code can include the same file.
One naming lesson comes from this output. The first version of this library named its types Stats and Summary, and the generated header declared global C types with those names, which collide with any application that has its own Stats. C has no namespaces, so prefix every exported name. The functions already carried wordstat_, and renaming the types to WordStats and WordStatsSummary fixed the types. The [export] prefix option in the configuration can do it automatically.
#include <stdio.h>\n#include <stdlib.h>\n#include \"wordstat.h\"\n\nint main(void) {\n WordStats *stats = wordstat_new();\n\n const char *docs[] = {\"the quick brown fox\", \"jumps over\\nthe lazy dog\"};\n for (int i = 0; i < 2; i++) {\n int rc = wordstat_feed(stats, docs[i]);\n if (rc != WORDSTAT_OK) {\n fprintf(stderr, \"feed failed: %d\\n\", rc);\n return 1;\n }\n }\n\n WordStatsSummary s = wordstat_summary(stats);\n printf(\"summary: %zu words, %zu lines\\n\", s.words, s.lines);\n\n char *report = wordstat_report(stats);\n printf(\"report: %s\\n\", report);\n wordstat_string_free(report); /* not free(): Rust allocated this */\n\n /* Errors are return codes, not crashes. */\n printf(\"feed(NULL text) -> %d\\n\", wordstat_feed(stats, NULL));\n printf(\"feed(bad utf-8) -> %d\\n\", wordstat_feed(stats, \"caf\\xff\"));\n\n wordstat_free(stats);\n wordstat_free(NULL); /* allowed */\n return 0;\n}Build the Rust library in release mode, then compile and link the C program against the static archive. The Rust standard library needs a few system libraries, and rustc will tell you which:
cargo build --release\ncargo rustc --release --crate-type staticlib -- --print native-static-libs\ncc -Wall -Wextra -o demo main.c target/release/libwordstat.a -lSystem -lc -lm\n./demoThe second command prints the line note: native-static-libs: -lSystem -lc -lm on this machine; on Linux it lists -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc. Copy exactly what it prints into your link command. Missing a library produces undefined-symbol errors from std internals that look unrelated to your code.
summary: 9 words, 3 lines\nreport: 9 words on 3 lines\nfeed(NULL text) -> -1\nfeed(bad utf-8) -> -2The first document has four words on one line and the second has five words across two, so nine words and three lines, computed by Rust and printed by C. The next two lines show the border holding. Passing a null text pointer returned -1 where a careless C library would have crashed, and passing the bytes caf\\xff, which end in a byte that is not valid UTF-8, returned -2 rather than corrupting the counter. wordstat_free(NULL) is explicitly allowed, mirroring free(NULL), which makes cleanup code in C simpler to write correctly.
The same build produced both artifacts. The static archive libwordstat.a is 18.7 MB on disk, large because it carries debugging information and all of std, and the linker keeps only what the program uses, so the final executable is much smaller. The dynamic library libwordstat.dylib is 388 KB. Its exported symbols show exactly the six functions in the header and nothing else:
$ nm -gU target/release/libwordstat.dylib | grep wordstat\n0000000000000900 T _wordstat_feed\n0000000000000c2c T _wordstat_free\n0000000000000c40 T _wordstat_new\n0000000000000c78 T _wordstat_report\n0000000000000d6c T _wordstat_string_free\n0000000000000d9c T _wordstat_summaryThat is the stable surface. Anything not marked #[unsafe(no_mangle)] is invisible to C. Check this listing in CI so an accidental export, or a missing one, fails a build. Choose static linking for a single self-contained executable and the simplest deployment; choose the dynamic library when several programs share it, when you want to update the Rust side without relinking, or when the host is a language runtime that loads plugins. With a dynamic library the loader must find it at runtime, via LD_LIBRARY_PATH or DYLD_LIBRARY_PATH while testing, and an install name or rpath in production.
The report function above returns memory that Rust allocated, which forces the two-sided lifecycle: C must remember to call wordstat_string_free. A cleaner border avoids shared ownership entirely by letting the caller supply the memory, the way snprintf does. The caller passes a buffer and its capacity, Rust writes into it, and the return value says how much space the full result needs. Nothing allocated on one side is freed on the other.
#[unsafe(no_mangle)]\npub unsafe extern \"C\" fn wordstat_report_into(stats: *const WordStats, buf: *mut c_char, cap: usize) -> isize {\n let Some(s) = (unsafe { stats.as_ref() }) else { return WORDSTAT_NULL_ARGUMENT as isize };\n let text = format!(\"{} words on {} lines\", s.words, s.lines);\n if !buf.is_null() && cap > 0 {\n let n = text.len().min(cap - 1);\n // SAFETY: the caller promised `buf` is writable for `cap` bytes and `n + 1 <= cap`.\n unsafe {\n std::ptr::copy_nonoverlapping(text.as_ptr(), buf.cast::<u8>(), n);\n *buf.add(n) = 0;\n }\n }\n text.len() as isize\n}(The doc comment, trimmed here, spells out the contract: at most cap bytes are written including the terminator, the return value is the full length excluding it, and a NULL buffer with zero capacity asks only for the length.) The C caller uses the standard two-call pattern, and the second call shows truncation:
ptrdiff_t need = wordstat_report_into(stats, NULL, 0);\nchar *buf = malloc((size_t)need + 1);\nwordstat_report_into(stats, buf, (size_t)need + 1);\nprintf(\"needed %td bytes: \\\"%s\\\"\\n\", need, buf);\nfree(buf); /* correct here: C allocated it, so C frees it */\n\nchar small[8];\nptrdiff_t full = wordstat_report_into(stats, small, sizeof small);\nprintf(\"small buffer holds \\\"%s\\\", full length is %td (truncated: %s)\\n\",\n small, full, (size_t)full >= sizeof small ? \"yes\" : \"no\");needed 18 bytes: \"7 words on 1 lines\"\nsmall buffer holds \"7 words\", full length is 18 (truncated: yes)Here free(buf) is correct, because C allocated the buffer. The eight-byte buffer received \"7 words\", seven characters plus the terminator, and the return value 18 told the caller the output was cut off. The length written is min(len, cap - 1), so the function can never write past the buffer, which is the property a C caller most needs from a library it did not write. This pattern is the better default for any variable-length output, and the library gains a second benefit: there is nothing to leak when the C caller returns early on an error path. Reserve the allocate-and-return style for objects with real lifecycles, like the WordStats handle itself.
A shared library lives longer than any one release of the program that uses it, so the interface needs a versioning plan on day one. Three habits cost almost nothing. Export a function wordstat_version() that returns a number or a string, so a program can check at startup that it linked the library it was compiled against. Never remove or change the meaning of an exported function in a compatible release; add wordstat_feed2 and keep the old name. And keep structs opaque whenever possible. The one by-value struct in this library is also the one place where a future change is a breaking change, so if you expect to add statistics, return them through accessor functions, wordstat_word_count(stats) and so on, and keep the layout private. A change to the header should be a conscious decision with a version bump, which the CI check that regenerates the header and compares it to the committed one is there to enforce.
Symbols deserve the same discipline. On Linux, a version script or -Wl,--exclude-libs keeps Rust's internal symbols from leaking out of the shared object, and on every platform the nm -gU listing above is the audit: the exported set should equal the header, exactly. If it lists more, something has become part of your public interface by accident.
The failures that matter are all things the compiler cannot see, so list them before you ship.
\nFreeing across allocators. Calling free() on the pointer from wordstat_report happened to run without a crash on this platform, because Rust's default allocator here is the system malloc. That is luck. A project that later switches its Rust code to another global allocator, or a Windows build where the DLL and the executable use different C runtimes, will crash or corrupt the heap. The rule is absolute: whoever allocates, frees.
Use after free and double free. C can call wordstat_free twice on the same pointer. The Rust side cannot stop it, because after Box::from_raw it holds no state. If the risk is real, expose handles that are indices into a table, or document it loudly and test with a sanitizer.
Threads. The example is not thread-safe: two C threads feeding one WordStats race. Rust's Send and Sync are meaningless to C. Either document \"one thread per handle\", or put a Mutex inside and make the handle safe to share.
Layout drift. A #[repr(C)] struct returned by value is part of your binary interface. Adding a field changes its size, and old compiled callers read garbage. Version the library, or pass such structs through opaque accessor functions instead.
Strings that are not UTF-8. C strings are arbitrary bytes. The example rejects invalid UTF-8 with an error code; a library that must accept any bytes should take a pointer and a length (*const u8, usize) and work on &[u8], which also removes the NUL-terminator requirement and allows embedded NULs.
Before an exported function is merged, a reviewer should be able to answer yes to each of these, because each one is a way a C caller can hurt the process through a function that looks harmless in Rust.
\ncatch_unwind, so a bug becomes a return value and not an abort of the host?# Safety section naming the lifetime, validity and thread requirements of each pointer?String, Vec, Option, Result, a trait object, a tuple?NULL?A function that passes all seven is boring, and boring is the aim. The adventurous parts of the program, the parsing, the data structures, the concurrency, stay on the Rust side of the line, where the compiler is checking them.
\nTest the Rust logic with ordinary cargo test, and test the border from C. A tiny C test program built in CI and run under valgrind or AddressSanitizer exercises the lifecycle that Rust tests cannot see: create, feed, report, free, and the error paths. Run the same C program against the static and the dynamic build. And run cbindgen in the pipeline with a check that the committed header matches the generated one, so a changed signature in Rust cannot ship without the header changing alongside it.
If the C application can host a different process, an out-of-process service reached over a socket or a pipe removes the whole border, at the cost of a protocol. If the C++ side wants to use rich Rust types directly, the cxx crate generates safe bindings in both directions, instead of flattening everything to C. If the consumer is Python, Node or Java, use the dedicated binding crates (PyO3 and napi-rs), which handle conversions and error translation for you. Use the cbindgen route when the consumer is C, or any language whose FFI speaks C, and when you want one narrow, stable, auditable surface. For the opposite direction, see calling C from Rust with extern and bindgen.
Rust does not need to rewrite the world. Operating systems, databases, codecs, GPU drivers and decades of numerical code are written in C, and a Rust program can call all of it directly, with no runtime and no overhead beyond the call itself. The mechanism is the foreign function interface, and the cost is that every call crosses a boundary the compiler cannot see through. This guide shows how to cross it deliberately: first by hand, so you understand what is being promised, then with bindgen, which writes the declarations for you.
The running example is a small C library, a key-value store with an opaque handle, string arguments, a returned pointer you must not free, and a callback. It is deliberately the same shape as the C libraries you will meet in practice. It was built with cc 1.11.1 and bindgen 0.73.2 on Rust 1.99, edition 2024, on an Apple silicon Mac, and the outputs below are what it printed.
#ifndef KV_H\n#define KV_H\n#include <stddef.h>\n\ntypedef struct kv kv_t;\n\n/* Create an empty store. Returns NULL if memory runs out. */\nkv_t *kv_open(const char *name);\n\n/* Copy key and value into the store. Returns 0 on success, -1 on bad arguments or no memory. */\nint kv_set(kv_t *store, const char *key, const char *value);\n\n/* Returns a pointer owned by the store, valid until the next kv_set of that key or kv_close.\n Returns NULL if the key is absent. */\nconst char *kv_get(const kv_t *store, const char *key);\n\nsize_t kv_len(const kv_t *store);\n\ntypedef void (*kv_visit_fn)(const char *key, const char *value, void *user_data);\n\n/* Call visit once per entry, in insertion order. */\nvoid kv_foreach(const kv_t *store, kv_visit_fn visit, void *user_data);\n\n/* Free the store and everything in it. Passing NULL is allowed. */\nvoid kv_close(kv_t *store);\n\n#endifRead the header the way you will read every C header you bind: for ownership. kv_open returns a handle you must eventually pass to kv_close. kv_set copies its arguments, so the caller keeps ownership of what it passed. kv_get returns a pointer the store owns, valid only until the next kv_set of the same key or until kv_close. kv_foreach takes a function pointer and an opaque void *user_data that it passes back untouched. None of those rules is visible in a C type signature. They live in comments, and your Rust has to honor them.
The implementation, csrc/kv.c, is a hundred lines of linked-list code. It is not reproduced here, because the point of FFI is that you do not need to read it. The build compiles it with the cc crate in a build script:
fn main() {\n cc::Build::new().file(\"csrc/kv.c\").include(\"csrc\").warnings(true).compile(\"kv\");\n println!(\"cargo:rerun-if-changed=csrc/kv.c\");\n println!(\"cargo:rerun-if-changed=csrc/kv.h\");\n}[build-dependencies]\ncc = \"1\"cc finds the platform's C compiler, compiles the file, archives it into a static library named libkv.a, and tells Cargo to link it into your crate. That is all the build setup a vendored C source file needs. For a library already installed on the system, you would instead emit cargo:rustc-link-lib=kv and a search path, usually via the pkg-config crate.
Before using a generator, write the declarations yourself once. It takes five minutes and teaches you what the generator is promising for you.
\nuse std::ffi::{c_char, c_int, c_void, CStr, CString};\n\n// A hand-written declaration of the C API. Each line is a promise that the C side matches.\n#[repr(C)]\npub struct kv_t {\n _private: [u8; 0], // an opaque type: Rust never looks inside\n}\n\nunsafe extern \"C\" {\n fn kv_open(name: *const c_char) -> *mut kv_t;\n fn kv_set(store: *mut kv_t, key: *const c_char, value: *const c_char) -> c_int;\n fn kv_get(store: *const kv_t, key: *const c_char) -> *const c_char;\n fn kv_len(store: *const kv_t) -> usize;\n fn kv_foreach(store: *const kv_t, visit: extern \"C\" fn(*const c_char, *const c_char, *mut c_void), user: *mut c_void);\n fn kv_close(store: *mut kv_t);\n}\n\nextern \"C\" fn print_entry(key: *const c_char, value: *const c_char, _user: *mut c_void) {\n let (k, v) = unsafe { (CStr::from_ptr(key), CStr::from_ptr(value)) };\n println!(\" {} = {}\", k.to_string_lossy(), v.to_string_lossy());\n}\n\nfn main() {\n let name = CString::new(\"settings\").unwrap();\n let store = unsafe { kv_open(name.as_ptr()) };\n assert!(!store.is_null());\n\n for (k, v) in [(\"host\", \"example.test\"), (\"port\", \"8080\"), (\"host\", \"db.example.test\")] {\n let (k, v) = (CString::new(k).unwrap(), CString::new(v).unwrap());\n let rc = unsafe { kv_set(store, k.as_ptr(), v.as_ptr()) };\n assert_eq!(rc, 0);\n }\n\n println!(\"{} entries\", unsafe { kv_len(store) });\n let key = CString::new(\"host\").unwrap();\n let value = unsafe { kv_get(store, key.as_ptr()) };\n println!(\"host -> {}\", unsafe { CStr::from_ptr(value) }.to_str().unwrap());\n\n let missing = CString::new(\"nope\").unwrap();\n println!(\"missing is null: {}\", unsafe { kv_get(store, missing.as_ptr()) }.is_null());\n\n println!(\"all entries:\");\n unsafe { kv_foreach(store, print_entry, std::ptr::null_mut()) };\n\n unsafe { kv_close(store) };\n}2 entries\nhost -> db.example.test\nmissing is null: true\nall entries:\n host = db.example.test\n port = 8080The store ends with two entries even though three kv_set calls ran, because the third set host a second time and the C code replaced the value, so host now maps to db.example.test. The insertion order is preserved, host before port, as the header promised. Every piece of the file is a decision worth understanding.
unsafe extern \"C\" { ... } declares functions that exist elsewhere and follow the C calling convention. In edition 2024 the unsafe before extern is mandatory, and it means something: you assert that every declaration in the block is accurate. The compiler cannot check it. If you declare kv_len as returning u32 while C returns size_t, the program compiles and reads half a register. Omit the keyword and the compiler refuses:
error: extern blocks must be unsafe\n --> src/main.rs:9:1\n |\n 9 | extern \"C\" {\n | ^\n | |\n | _help: needs `unsafe` before the extern keyword: `unsafe`Types map to the std::ffi aliases rather than to Rust's fixed-width types, because C's int and char vary by platform. This table is the one to keep:
| C type | Rust type |
|---|---|
int, unsigned int | c_int, c_uint |
char, signed char, unsigned char | c_char, c_schar, c_uchar |
long, size_t, ssize_t | c_long, usize, isize |
float, double | f32, f64 |
void * | *mut c_void |
const char * | *const c_char |
struct foo * where foo is opaque | *mut Foo with #[repr(C)] struct Foo { _private: [u8; 0] } |
The opaque type deserves its own sentence. kv_t is declared in the header but its fields are private to the C file, so Rust must never see its size or layout. A zero-length array in a #[repr(C)] struct is the idiom: you can name the type and hold pointers to it, but you cannot construct one or take its size. That prevents accidentally copying the handle by value, which would be undefined behavior.
A Rust &str is a pointer and a length, with no terminator. A C string is a pointer to bytes ending in a NUL. They are not interchangeable, and passing \"host\".as_ptr() to C compiles and reads past the end of the string looking for a NUL.
The tool is CString, which allocates a copy with a terminator. Creating one can fail if the text contains a NUL byte of its own, since C would read it as the end of the string. CString::new returns a Result for that reason:
nul byte found in provided data at position: 4\nnul at byte 4That is the output for CString::new(\"sess\\0ion\"). In FFI code, unwrap() on that call is a bug waiting for user input; map the error into your own error type. The reverse direction uses CStr::from_ptr, which measures a C string up to its NUL and borrows it as &CStr. From there to_str() returns a Result because C text need not be UTF-8, and to_string_lossy() always succeeds by substituting replacement characters. Choose the one that matches what you want to do with a malformed byte.
Mind the lifetimes, because the compiler will not. key.as_ptr() is valid only while the CString lives. Writing CString::new(\"k\").unwrap().as_ptr() creates the CString, takes a pointer into it, and drops it at the end of the statement, leaving a dangling pointer. Bind the CString to a variable and keep it alive across the call, as the example does.
kv_get returns a const char * that the store owns. The Rust code reads it immediately inside the same expression and never frees it. Two mistakes follow from getting this wrong. Calling Rust's deallocator on a pointer from C's malloc corrupts the heap, because they can be different allocators. And holding the pointer across a later kv_set of the same key is a use after free, since the C code frees the old value string when it overwrites it. A comment in the header documented that rule; the Rust compiler cannot. The fix in a safe wrapper is to copy the string into an owned String at the call, which is the main thing the wrapper guide does.
kv_foreach takes a function pointer. The Rust side defines extern \"C\" fn print_entry(...), an ordinary function with the C calling convention, and passes it as the argument. It cannot be a closure that captures variables, because a C function pointer has no place to store captured state. The user_data parameter exists for that purpose: pass a pointer to your state, and the callback casts it back. The example passes null_mut() because it needs no state. One more rule: a panic must not unwind out of an extern \"C\" function into C code. Since Rust 1.81, an unwinding panic that reaches an extern \"C\" boundary aborts the process, which is the defined behavior, and the safe fix is to wrap the callback body in std::panic::catch_unwind.
If the build script does not compile the C file, the link fails, and the message is the one to recognize:
\nerror: linking with `cc` failed: exit status: 1\n = note: Undefined symbols for architecture arm64:\n \"_kv_open\", referenced from:\nld: symbol(s) not found for architecture arm64The compiler was happy because the extern block promised that kv_open exists. The linker looked for it and found nothing. On Linux the same error reads undefined reference to 'kv_open'. The causes, in order of likelihood: the build script did not run or did not compile the right file, the library name in cargo:rustc-link-lib is wrong, the library is a C++ one whose symbols are mangled, or a dynamic library is installed but not on the search path.
Hand-written declarations drift. The header changes, the extern block does not, and nobody notices until a crash. bindgen reads the header and generates the block, so the two cannot disagree. Run it once to see what it makes:
bindgen csrc/kv.h --rust-target 1.85 --rust-edition 2024 \\\n --allowlist-function 'kv_.*' --allowlist-type 'kv_.*' --no-layout-tests#[repr(C)]\n#[derive(Debug)]\npub struct kv {\n _unused: [u8; 0],\n}\npub type kv_t = kv;\nunsafe extern \"C\" {\n pub fn kv_open(name: *const ::std::os::raw::c_char) -> *mut kv_t;\n}\nunsafe extern \"C\" {\n pub fn kv_set(\n store: *mut kv_t,\n key: *const ::std::os::raw::c_char,\n value: *const ::std::os::raw::c_char,\n ) -> ::std::os::raw::c_int;\n}\nunsafe extern \"C\" {\n pub fn kv_get(\n store: *const kv_t,\n key: *const ::std::os::raw::c_char,\n ) -> *const ::std::os::raw::c_char;\n}\nunsafe extern \"C\" {\n pub fn kv_len(store: *const kv_t) -> usize;\n}\npub type kv_visit_fn = ::std::option::Option<\n unsafe extern \"C\" fn(\n key: *const ::std::os::raw::c_char,\n value: *const ::std::os::raw::c_char,\n user_data: *mut ::std::os::raw::c_void,\n ),\n>;(The listing is trimmed after the callback type; the real output also declares kv_foreach and kv_close.) Compare it with the hand-written version. It found the opaque struct and generated the same zero-length-array idiom. It mapped size_t to usize. And it made the callback type an Option of an unsafe extern \"C\" fn, because a C function pointer may be NULL. The hand-written version missed that, which means passing a null callback was not expressible; the generated one is more accurate.
The --allowlist-* flags matter. Without them bindgen also emits every type from stddef.h, and from any system header the file includes, which bloats the output and breaks on platform differences. List what you intend to use.
For a real project, generate at build time so the bindings track the header:
\nuse std::{env, path::PathBuf};\n\nfn main() {\n cc::Build::new().file(\"csrc/kv.c\").include(\"csrc\").warnings(true).compile(\"kv\");\n\n let bindings = bindgen::Builder::default()\n .header(\"csrc/kv.h\")\n .allowlist_function(\"kv_.*\")\n .allowlist_type(\"kv_.*\")\n .parse_callbacks(Box::new(bindgen::CargoCallbacks::new()))\n .generate()\n .expect(\"bindgen could not read csrc/kv.h\");\n\n let out = PathBuf::from(env::var_os(\"OUT_DIR\").unwrap());\n bindings.write_to_file(out.join(\"kv_bindings.rs\")).unwrap();\n}#![allow(non_camel_case_types, dead_code)]\n\nmod sys {\n include!(concat!(env!(\"OUT_DIR\"), \"/kv_bindings.rs\"));\n}\n\nuse std::ffi::{CStr, CString};\n\nfn main() {\n let name = CString::new(\"settings\").unwrap();\n let store = unsafe { sys::kv_open(name.as_ptr()) };\n let (k, v) = (CString::new(\"port\").unwrap(), CString::new(\"8080\").unwrap());\n unsafe { sys::kv_set(store, k.as_ptr(), v.as_ptr()) };\n let got = unsafe { CStr::from_ptr(sys::kv_get(store, k.as_ptr())) };\n println!(\"port -> {}\", got.to_str().unwrap());\n unsafe { sys::kv_close(store) };\n}port -> 8080bindgen as a build dependency needs libclang installed on the building machine, which is the cost of this approach: contributors and CI runners must have it. Many -sys crates avoid that by running bindgen once, checking the generated file into the repository, and regenerating only when the header changes. That is a trade of a build dependency for a file to maintain, and it is the usual choice for a crate that other people depend on.
user_dataA bare extern \"C\" fn cannot capture variables, so the printing callback above could only write to the terminal. Real code wants to collect results, and the C API provides the escape hatch it always provides: a void *user_data that comes back untouched. The standard technique is a trampoline, a generic C-callable function that casts user_data back to a pointer to the Rust closure and calls it. Because it is generic over the closure type F, the compiler stamps out one trampoline per closure, and each is an ordinary function pointer that C can store.
use std::ffi::{c_char, c_int, c_void, CStr, CString};\n\n#[repr(C)]\npub struct kv_t {\n _private: [u8; 0],\n}\n\nunsafe extern \"C\" {\n fn kv_open(name: *const c_char) -> *mut kv_t;\n fn kv_set(store: *mut kv_t, key: *const c_char, value: *const c_char) -> c_int;\n fn kv_foreach(\n store: *const kv_t,\n visit: Option<unsafe extern \"C\" fn(*const c_char, *const c_char, *mut c_void)>,\n user: *mut c_void,\n );\n fn kv_close(store: *mut kv_t);\n}\n\n/// One C-callable function that forwards to whatever Rust closure `user` points at.\nunsafe extern \"C\" fn trampoline<F: FnMut(&str, &str)>(key: *const c_char, value: *const c_char, user: *mut c_void) {\n // SAFETY: `user` is the `&mut F` that `for_each` passed in below, and it outlives the C call.\n let closure = unsafe { &mut *(user as *mut F) };\n let (k, v) = unsafe { (CStr::from_ptr(key), CStr::from_ptr(value)) };\n // A panic must not cross into C: swallow it here and let the caller decide what to do.\n let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {\n closure(&k.to_string_lossy(), &v.to_string_lossy());\n }));\n}\n\nfn for_each<F: FnMut(&str, &str)>(store: *const kv_t, mut f: F) {\n unsafe { kv_foreach(store, Some(trampoline::<F>), &mut f as *mut F as *mut c_void) };\n}\n\nfn main() {\n let name = CString::new(\"demo\").unwrap();\n let store = unsafe { kv_open(name.as_ptr()) };\n for (k, v) in [(\"a\", \"1\"), (\"b\", \"22\"), (\"c\", \"333\")] {\n let (k, v) = (CString::new(k).unwrap(), CString::new(v).unwrap());\n unsafe { kv_set(store, k.as_ptr(), v.as_ptr()) };\n }\n\n // The closure captures local state, which a bare `extern \"C\" fn` could not do.\n let mut total_len = 0;\n let mut keys = Vec::new();\n for_each(store, |k, v| {\n total_len += v.len();\n keys.push(k.to_string());\n });\n println!(\"keys {keys:?}, total value length {total_len}\");\n\n // What C does with bad arguments is part of the contract too.\n let rc = unsafe { kv_set(store, std::ptr::null(), std::ptr::null()) };\n println!(\"kv_set with NULL arguments returned {rc}\");\n\n unsafe { kv_close(store) };\n}keys [\"a\", \"b\", \"c\"], total value length 6\nkv_set with NULL arguments returned -1The closure pushed into keys and added to total_len, both locals of main, which proves the state travelled through C and back. Four details make the trampoline sound, and each is something you would otherwise get wrong. &mut f as *mut F as *mut c_void turns the closure into a thin pointer; it is valid only for the duration of kv_foreach, which is exactly how long C may use it, and the comment on the unsafe block says so. The Some(trampoline::<F>) is the Option that bindgen generated for a nullable function pointer. The catch_unwind swallows any panic from the closure so it does not unwind through C frames, which is undefined behavior in older Rust and an abort in current Rust. And AssertUnwindSafe is the honest statement that you accept the closure's captured state may be half-updated after a panic.
The last line of output is a reminder that the C library's behavior on bad input is part of its contract. kv_set with NULL arguments returned -1 rather than crashing, as the header said it would. A library that does not promise that, and many do not, means your wrapper must validate arguments before they reach C. Pass NULL to a function that dereferences it and the result is a segmentation fault at best.
Tests of FFI code are ordinary tests, with one extra tool worth the setup. A Rust test that calls the C library catches wrong declarations, wrong ownership and wrong lifetimes only if the bug causes a visible symptom, and a use after free often does not. Run the test binary under a memory checker. On Linux, valgrind --error-exitcode=1 target/debug/deps/ckv-* reports reads of freed memory and leaks in the C code and in your use of it. On macOS and Linux, building with RUSTFLAGS=\"-Zsanitizer=address\" on nightly, and compiling the C with -fsanitize=address (set through CFLAGS for the cc crate), instruments both sides. Miri cannot run foreign code, so it does not help across the boundary; it remains useful for the pure-Rust parts of your wrapper. Write at least these cases: an empty store, a key that is absent, overwriting a key, a string with a non-ASCII character, and the cleanup path. They cover the ownership rules the header documented only in comments.
If you plan to publish the bindings, the ecosystem has a naming convention worth following. The crate with the raw, generated declarations is named foo-sys, contains no logic, and declares links = \"foo\" in its Cargo.toml so Cargo can ensure that only one version of the native library is linked into a build. A separate crate, foo, builds the safe API on top. The split lets many safe wrappers share one set of bindings, and it keeps the unsafe surface in a place with a small, auditable job. The raw bindings are never meant for application code; they are the foundation for a safe wrapper.
Even with correct declarations, a handful of mistakes remain, each of which compiles.
\nMutex or leave it !Send if in doubt.Drop impl on a wrapper type, not in the code that uses the library.int is 32 bits and a long is 64. They are not on every platform; use the c_* aliases.#[repr(C)], which gives Rust the freedom to reorder fields.assert_eq!(rc, 0) in a demo becomes a real error path in a program.If a Rust crate already does the job, use it: an equivalent written in Rust avoids the entire class of problems above and gets the compiler's checks across the boundary. If the C library is large and you only need a fraction, vendoring that fraction is easier than binding it all. And if the only reason is speed, measure first, because a pure Rust implementation is often within a few percent. When none of that applies, as with a mature library such as SQLite, zlib or libcurl, binding is the right call, and the sequence in this guide, a hand-written trial, then bindgen, then a safe wrapper, is the path. The other direction, exposing Rust to C, is covered in calling Rust from C with cbindgen.
When you face an unfamiliar C library, read its header in this order and note the answer to each question before writing any Rust. Who allocates each pointer and who frees it? Which functions return a status, and which signal failure through a special return value like NULL or -1? Is there global initialization that must run once, and teardown that must run last? Which functions may be called from several threads at the same time? And which pointers returned by the library stay valid only until another call? Write those answers as comments on the extern declarations. They become the # Safety sections of your wrapper, and they are the difference between bindings that merely compile and bindings someone can trust.
Unsafe code in Rust is not meant to be sprinkled through a program. It is meant to live in a small module, behind an interface that cannot be misused, so that the rest of the program can stay in the safe subset and still use the result. That arrangement is called a safe abstraction, and building one is the core skill of unsafe Rust. The standard library is full of them: Vec manages raw memory, Mutex hands out references across threads, Rc counts owners by hand, and callers of all three write no unsafe.
This guide builds one from scratch: a fixed-capacity ring buffer that stores its items inline in an array, with no heap allocation. It is small enough to read in a sitting, it needs real unsafe code, and it has three distinct ways to get wrong that Miri can show you. Everything compiled and ran on Rust 1.99 (edition 2024) and on the nightly toolchain's Miri; the output blocks are what they printed.
\nA ring buffer is a queue with a fixed number of slots. Pushing writes at the tail, popping reads at the head, and both indices wrap around the end of the array. It is the standard structure for a producer and consumer that must never allocate, such as an interrupt handler feeding a main loop, or an audio callback.
\nIn safe Rust, the obvious design is [Option<T>; N]. It works and costs a discriminant per slot, and for a type like String or Box, where Option can reuse the null pointer, the cost is zero. For a type with no spare bit pattern, like u32, every slot grows by a word. More importantly, [Option<T>; N] requires initializing every slot to None up front and cannot move a value out of a slot without take(). The design this guide builds uses MaybeUninit<T>: a slot is raw, uninitialized storage, and the ring itself tracks which ones currently hold values. That tracking is the invariant the unsafe code depends on, and it is what the whole abstraction exists to protect.
Before writing a line, write down what must always be true. This is the step that separates an abstraction from a collection of unsafe blocks.
The ring has three fields: slots, head and len. The invariant is:
head is always less than N, and len is always at most N.len slots starting at head, counting around the end of the array, are initialized.Every method has to preserve it, and every unsafe block is justified by it. If a method breaks it even briefly, even in a state no one can observe, a panic at the wrong moment can expose the broken state. The code below is written so that no method can panic between changing the indices and reading the slot.
\nuse std::mem::MaybeUninit;\n\n/// A fixed-capacity FIFO queue that stores its items inline, with no heap allocation.\n///\n/// Invariant: exactly the slots in the logical range `head .. head + len` (mod N)\n/// are initialized. Every other slot is uninitialized and must never be read or dropped.\npub struct Ring<T, const N: usize> {\n slots: [MaybeUninit<T>; N],\n head: usize,\n len: usize,\n}\n\nimpl<T, const N: usize> Ring<T, N> {\n pub fn new() -> Self {\n Ring { slots: [const { MaybeUninit::uninit() }; N], head: 0, len: 0 }\n }\n\n pub fn len(&self) -> usize {\n self.len\n }\n\n /// Returns the item back if the ring is full.\n pub fn push(&mut self, item: T) -> Result<(), T> {\n if self.len == N {\n return Err(item);\n }\n let tail = (self.head + self.len) % N;\n self.slots[tail].write(item); // writing a MaybeUninit slot is safe\n self.len += 1;\n Ok(())\n }\n\n pub fn pop(&mut self) -> Option<T> {\n if self.len == 0 {\n return None;\n }\n let index = self.head;\n self.head = (self.head + 1) % N;\n self.len -= 1;\n // SAFETY: `index` was inside the initialized range before `len` shrank, and\n // moving `head` forward means no later call can reach this slot again until\n // `push` rewrites it. So the value is read out exactly once.\n Some(unsafe { self.slots[index].assume_init_read() })\n }\n\n pub fn peek(&self) -> Option<&T> {\n if self.len == 0 {\n return None;\n }\n // SAFETY: `head` is inside the initialized range because `len > 0`.\n Some(unsafe { self.slots[self.head].assume_init_ref() })\n }\n}\n\nimpl<T, const N: usize> Drop for Ring<T, N> {\n fn drop(&mut self) {\n // Drop only the initialized items. `MaybeUninit` never drops its contents on its own.\n while self.pop().is_some() {}\n }\n}\n\nstruct Noisy(&'static str);\n\nimpl Drop for Noisy {\n fn drop(&mut self) {\n println!(\"dropping {}\", self.0);\n }\n}\n\nfn main() {\n let mut ring: Ring<String, 3> = Ring::new();\n for word in [\"a\", \"b\", \"c\"] {\n ring.push(word.to_string()).unwrap();\n }\n println!(\"full push rejected: {:?}\", ring.push(\"d\".to_string()));\n println!(\"peek: {:?}\", ring.peek());\n println!(\"pop: {:?}\", ring.pop());\n ring.push(\"e\".to_string()).unwrap(); // wraps around the end of the array\n let drained: Vec<_> = std::iter::from_fn(|| ring.pop()).collect();\n println!(\"drained in order: {drained:?}\");\n\n let mut noisy: Ring<Noisy, 4> = Ring::new();\n noisy.push(Noisy(\"first\")).ok();\n noisy.push(Noisy(\"second\")).ok();\n noisy.push(Noisy(\"third\")).ok();\n let _ = noisy.pop();\n println!(\"leaving main with {} items still queued\", noisy.len());\n}Read the code with the invariant open beside it.
\nnew creates N uninitialized slots with [const { MaybeUninit::uninit() }; N]. The const block is how you repeat a value that is not Copy in an array expression. Creating uninitialized storage is safe, because MaybeUninit makes no promise about its contents.
push checks capacity, computes the tail as (head + len) % N, and writes with MaybeUninit::write. Writing is safe for the same reason: it initializes the slot, it cannot expose anything. If the ring is full, push returns the item back inside Err, which is the idiomatic way to refuse without losing the caller's value. Only after the write does it increase len, which keeps the order of operations safe: the invariant never claims a slot is initialized before it is.
pop is the first place unsafe code appears, and it follows a strict order. It advances head and decrements len first, which removes the slot from the initialized range, and only then reads it with assume_init_read. The comment states the argument: the slot was inside the range, and shrinking the range means no later call can reach it before push rewrites it, so the value is read out exactly once. assume_init_read performs a bitwise copy and leaves the slot logically uninitialized again, which is exactly the move semantics wanted.
peek returns a reference to the front item using assume_init_ref. Its comment cites the invariant: with len > 0, the slot at head is initialized. The reference borrows &self, so the borrow checker prevents using it across a push or pop, which could invalidate it. That is the type system enforcing part of the invariant for free, and the next section shows the compiler doing it.
Drop is the part people forget. A MaybeUninit field never drops its contents, because it does not know whether they are initialized. The ring knows, so its Drop impl pops every remaining item, which runs each item's destructor exactly once, and leaves the uninitialized slots alone. The loop reuses pop instead of repeating the unsafe read, which keeps the number of unsafe blocks at two.
full push rejected: Err(\"d\")\npeek: Some(\"a\")\npop: Some(\"a\")\ndrained in order: [\"b\", \"c\", \"e\"]\ndropping first\nleaving main with 2 items still queued\ndropping second\ndropping thirdWalk through it. The ring has three slots, so the fourth push is refused and the string comes back in Err(\"d\"). Peeking and popping both return \"a\", the oldest item, so the order is first in, first out. After the pop, pushing \"e\" wraps around to the slot \"a\" left behind, and draining returns [\"b\", \"c\", \"e\"] in the right order, which exercises the wraparound arithmetic. The second half uses a type that prints when it is dropped. dropping first appears immediately when let _ = noisy.pop() discards the value. At the end of main two items are still queued, and the ring's Drop impl prints dropping second and dropping third. Every value was dropped exactly once, none twice and none never.
This ran under Miri with no error. That is the standard to hold the abstraction to: Miri passing on tests that cover wraparound, a full ring, an empty ring and drop.
\nThe first draft of main printed the peeked value and the popped value in one statement:
println!(\"peek: {:?}, pop: {:?}\", ring.peek(), ring.pop());error[E0502]: cannot borrow `ring` as mutable because it is also borrowed as immutable\n --> src/bin/ring.rs:76:52\n |\n76 | println!(\"peek: {:?}, pop: {:?}\", ring.peek(), ring.pop());\n | -----------------------------------------------^^^^^^^^^^-\n | | | |\n | | | mutable borrow occurs here\n | | immutable borrow occurs here\n | immutable borrow later used hereThis was an accident, and a good one. peek returns a &T pointing into the slot, and that reference is still alive when pop runs. If the program were allowed to compile, pop would move the value out of the slot while the reference still points at it, and the reference would see memory that no longer holds a valid T. The compiler rejected it because peek takes &self and returns a reference tied to that borrow, while pop takes &mut self. You did not write any rule saying \"do not pop while peeking\"; the signatures encode it. That is what a good safe interface does: it makes a class of misuse a compile error.
Now the failures, because an abstraction is only as good as the bugs it prevents. Each one below was made by changing the working code in one place and running Miri.
\nDelete the Drop impl. The ring now never drops its items. For Copy types that is invisible. For String, every item still queued at the end owns a heap buffer that is never freed:
Some(\"alpha\")\nerror: memory leaked: alloc410 (Rust heap, size: 4, align: 1), allocated here:\n --> .../library/alloc/src/raw_vec/mod.rs:485:41\n |\n485 | AllocInit::Uninitialized => alloc.allocate(layout),Miri reports the leak at exit, with the allocation's size. Nothing crashed and the output looked right. Leaks of this kind pass every ordinary test and surface as slowly growing memory in production, which is why running Miri on unsafe containers is worth the minutes it takes.
\nRemove the two lines in pop that advance head and shrink len. Now popping reads the same slot each time while still claiming it holds a value:
Some(\"alpha\")\nerror: Undefined Behavior: constructing invalid value of type &[u8]: encountered a dangling reference (use-after-free)\n --> .../library/alloc/src/vec/mod.rs:1784:13\n |\n1784 | &*core::intrinsics::aggregate_raw_ptr::<*const [T], _, _>(self.as_ptr(), self.len)\n | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Undefined Behavior occurred hereThe first pop printed Some(\"alpha\") and dropped the returned string. The second pop read the same bytes again, producing a second String whose buffer had already been freed. Miri catches it at the moment the freed buffer is touched. In a normal build this is a double free: the allocator's bookkeeping is corrupted, and the crash happens later, somewhere unrelated. The bug is a single missing statement in a safe-looking method, which is the main hazard of unsafe code: the unsafe block was fine, and the safe code around it broke the invariant it depends on.
The third failure is the subtle one, and it is a bug you cannot demonstrate by deleting a line. If the Drop impl iterated over every slot and called assume_init_drop on each, it would drop uninitialized memory for the empty ones, which is undefined behavior for any T with a destructor. The while self.pop().is_some() {} loop avoids it by construction: pop only touches slots the invariant says are initialized. When you write an unsafe collection, prefer expressing cleanup in terms of your own safe-to-reason-about operations over repeating index arithmetic.
The three failures above were found by hand-editing the code. A proper test suite finds them for you. Four tests cover the cases that matter for this container, and the third one is the drop-count test that would have caught both the missing Drop and the double read.
#[cfg(test)]\nmod tests {\n use super::*;\n use std::cell::Cell;\n use std::rc::Rc;\n\n /// Counts how many times it has been dropped, through a shared cell.\n struct Counted(Rc<Cell<usize>>);\n\n impl Drop for Counted {\n fn drop(&mut self) {\n self.0.set(self.0.get() + 1);\n }\n }\n\n #[test]\n fn fifo_order_survives_wraparound() {\n let mut r: Ring<u32, 3> = Ring::new();\n for round in 0..10 {\n r.push(round).unwrap();\n r.push(round + 100).unwrap();\n assert_eq!(r.pop(), Some(round));\n assert_eq!(r.pop(), Some(round + 100));\n }\n assert_eq!(r.pop(), None);\n }\n\n #[test]\n fn full_ring_returns_the_item() {\n let mut r: Ring<String, 1> = Ring::new();\n r.push(\"kept\".into()).unwrap();\n assert_eq!(r.push(\"rejected\".into()), Err(\"rejected\".to_string()));\n }\n\n #[test]\n fn every_item_is_dropped_exactly_once() {\n let drops = Rc::new(Cell::new(0));\n {\n let mut r: Ring<Counted, 4> = Ring::new();\n for _ in 0..4 {\n r.push(Counted(drops.clone())).ok().unwrap();\n }\n drop(r.pop()); // one dropped by the caller\n assert_eq!(drops.get(), 1);\n r.push(Counted(drops.clone())).ok().unwrap(); // wraps\n } // the ring drops the remaining four\n assert_eq!(drops.get(), 5);\n }\n\n #[test]\n fn zero_sized_items_work() {\n let mut r: Ring<(), 2> = Ring::new();\n r.push(()).unwrap();\n r.push(()).unwrap();\n assert!(r.push(()).is_err());\n assert_eq!(r.pop(), Some(()));\n }\n}Run them under Miri, not just cargo test. A normal test run would pass a ring that leaks or double-drops in many cases, because the damage is silent; Miri turns each into an error.
cargo +nightly miri testrunning 4 tests\n....\ntest result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.46sThe tests earn their keep through what they check. fifo_order_survives_wraparound pushes and pops ten rounds through a three-slot ring, so the head index wraps around the array several times; an off-by-one in the modulo shows up as a wrong value. full_ring_returns_the_item pins the contract that a refused push hands the value back rather than dropping it. every_item_is_dropped_exactly_once counts destructor calls through a shared Rc<Cell<usize>>: one drop from the caller's drop(r.pop()), then four more when the ring goes out of scope, five in total for the five items pushed. If Drop were missing, the final assertion would read 1; if pop read a slot twice, Miri would stop the test first. And zero_sized_items_work covers Ring<(), 2>, a case people forget, since a zero-sized T makes pointer arithmetic degenerate and MaybeUninit<()> take no space at all.
One more property separates a container that is correct from one that is sound under every input. User code runs inside your methods whenever you call T::drop, and that code can panic. In Ring::drop, the loop calls pop, which moves the value out and then lets the caller's Option drop it. If item two's destructor panics, the unwinding begins while items three and four are still queued. What happens to them?
With this implementation, the while self.pop().is_some() {} loop is abandoned mid-way by the panic, and the remaining items are leaked: their destructors do not run. A leak is safe in Rust, so the abstraction is still sound, merely wasteful in a rare case. The alternative, a guard object whose own Drop continues draining even while unwinding, makes the container leak-free at the price of more code and the possibility of a second panic during unwinding, which aborts the process. The standard library's Vec::drain and VecDeque::drop use that guard technique because they are part of the foundation. For a container of your own, the pragmatic rule is to guarantee soundness always and leak-freedom where it is cheap. What you must never do is leave the invariant false when a panic unwinds out of a method: if push incremented len before it wrote the item, and the write could panic (it cannot here, but a clone before the write could), a later pop would read an uninitialized slot. That is why the order inside each method matters as much as the code in it.
A struct made of safe types gets Send and Sync automatically. Ring<T, N> holds [MaybeUninit<T>; N] and two usize, and MaybeUninit<T> is Send when T is and Sync when T is, so the compiler infers that the ring is Send if T: Send and Sync if T: Sync. That inference is correct here, since nothing in the ring is tied to a thread.
You override the automatic rules, with unsafe impl Send, only when the struct contains a raw pointer or other type the compiler refuses to assume is thread-safe, and then the promise is yours. The question to answer is \"if two threads each had access, could they cause a data race or break an invariant?\" For a Ring, a shared &Ring offers only len and peek, which read; push and pop need &mut, which is exclusive. So Sync is sound without any synchronization, and the type system keeps writers exclusive. The moment you want a ring that two threads push into and pop from at the same time, you need atomics for head and len, and the invariant argument becomes a lock-free design problem, a different and much harder one.
unsafe.// SAFETY: comment on every unsafe block that cites the invariant and the specific condition.Drop explicitly, and test it with a type that prints or counts drops, so a leak and a double drop both fail a test.Clone or Drop can panic halfway through your method, ensure the invariant still holds when the panic unwinds.Send and Sync do what you mean, or assert them with a compile-time test.Before shipping a hand-written container, ask whether one already exists with a larger test suite. For this particular structure, heapless::spsc::Queue is the established no-alloc queue for embedded code, arrayvec gives a stack-allocated Vec, and std::collections::VecDeque is a growable ring buffer when allocation is acceptable. Write your own when none fits, or to learn. If you do, the discipline above is what makes it safe to depend on, and the first thing a reviewer should read is the comment that states the invariant. For the language rules the whole exercise rests on, read unsafe Rust explained, and for wrapping C instead of a data structure, safe Rust bindings for a C library.
Keep the finished type small and finish it with documentation a stranger can audit: the invariant list on the struct, the safety comment on each block, and the test names that read as claims. If another person can verify the argument without running anything, the abstraction has done its job.
","date_published":"2026-11-03T09:42:00+00:00","tags":["Systems"]},{"id":"https://rusthacks.dev/library/rust-custom-global-allocator","url":"https://rusthacks.dev/library/rust-custom-global-allocator","title":"Writing a Custom Global Allocator in Rust to Track Memory","summary":"A global allocator is a type implementing GlobalAlloc, registered with one attribute. Wrapping the system allocator with atomic counters gives you live memory numbers.","content_html":"Every Vec::push, String::from and Box::new in a Rust program ends in a call to the same place: the global allocator. Normally you never see it, because the standard library supplies one that forwards to the operating system's malloc. But Rust lets you replace it with a single attribute, and that one hook turns out to be one of the most useful debugging and performance tools in the language. With a few dozen lines you can count every allocation your program makes, assert that a hot path makes none, cap memory in a test, or swap in a completely different allocation strategy.
This guide builds three allocators, each small enough to read in one sitting: one that tracks allocations, one that allocates from a fixed arena and never frees, and a per-thread counter you can use in tests. Everything ran on Rust 1.99 with edition 2024 and only the standard library, in release mode on an Apple silicon Mac. The failures, including a crash with no message, are real and are explained, because writing an allocator is the one place in Rust where a perfectly ordinary line of code, a println!, can bring the whole process down.
The GlobalAlloc trait has two required methods and two optional ones:
pub unsafe trait GlobalAlloc {\n unsafe fn alloc(&self, layout: Layout) -> *mut u8;\n unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout);\n\n // Provided, with default implementations built on the two above:\n unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { /* ... */ }\n unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 { /* ... */ }\n}A Layout carries a size and an alignment. Returning a null pointer from alloc means \"out of memory\". The trait is unsafe because the rest of the program relies on you honoring its contract: the pointer you return must point to at least layout.size() bytes, aligned to layout.align(), not overlapping any live allocation, and valid until dealloc is called with the same layout. Violate it and the symptoms appear anywhere, much later, in code that has nothing to do with your allocator.
You install one with the #[global_allocator] attribute on a static. There can be only one per program, and it applies to everything, including the standard library and every dependency.
The simplest useful allocator wraps the system one and keeps counters. It forwards every call to System and updates four atomics on the way. The measure function runs a closure and prints what it allocated.
use std::alloc::{GlobalAlloc, Layout, System};\nuse std::collections::HashMap;\nuse std::sync::atomic::{AtomicUsize, Ordering::Relaxed};\n\n/// Wraps the system allocator and keeps four counters. It never allocates itself.\nstruct Tracking {\n live: AtomicUsize,\n peak: AtomicUsize,\n allocs: AtomicUsize,\n frees: AtomicUsize,\n reallocs: AtomicUsize,\n}\n\nimpl Tracking {\n const fn new() -> Self {\n Self { live: AtomicUsize::new(0), peak: AtomicUsize::new(0), allocs: AtomicUsize::new(0), frees: AtomicUsize::new(0), reallocs: AtomicUsize::new(0) }\n }\n\n fn snapshot(&self) -> Snapshot {\n Snapshot { live: self.live.load(Relaxed), peak: self.peak.load(Relaxed), allocs: self.allocs.load(Relaxed), frees: self.frees.load(Relaxed), reallocs: self.reallocs.load(Relaxed) }\n }\n}\n\n#[derive(Debug, Clone, Copy)]\nstruct Snapshot {\n live: usize,\n peak: usize,\n allocs: usize,\n frees: usize,\n reallocs: usize,\n}\n\nunsafe impl GlobalAlloc for Tracking {\n unsafe fn alloc(&self, layout: Layout) -> *mut u8 {\n let ptr = unsafe { System.alloc(layout) };\n if !ptr.is_null() {\n self.allocs.fetch_add(1, Relaxed);\n let now = self.live.fetch_add(layout.size(), Relaxed) + layout.size();\n self.peak.fetch_max(now, Relaxed);\n }\n ptr\n }\n\n unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {\n unsafe { System.dealloc(ptr, layout) };\n self.frees.fetch_add(1, Relaxed);\n self.live.fetch_sub(layout.size(), Relaxed);\n }\n\n // Forward realloc so the system allocator can grow in place; the default\n // implementation would allocate, copy and free, which counts as three events.\n unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {\n let new_ptr = unsafe { System.realloc(ptr, layout, new_size) };\n if !new_ptr.is_null() {\n self.reallocs.fetch_add(1, Relaxed);\n if new_size >= layout.size() {\n let now = self.live.fetch_add(new_size - layout.size(), Relaxed) + (new_size - layout.size());\n self.peak.fetch_max(now, Relaxed);\n } else {\n self.live.fetch_sub(layout.size() - new_size, Relaxed);\n }\n }\n new_ptr\n }\n}\n\n#[global_allocator]\nstatic ALLOC: Tracking = Tracking::new();\n\n/// Run `f` and report what it allocated.\nfn measure<R>(label: &str, f: impl FnOnce() -> R) -> R {\n let before = ALLOC.snapshot();\n ALLOC.peak.store(before.live, Relaxed); // peak is measured from this call's start\n let out = f();\n let after = ALLOC.snapshot();\n println!(\n \"{label:<26} allocs {:>5} reallocs {:>3} frees {:>5} net {:>+7} B peak +{:>6} B\",\n after.allocs - before.allocs,\n after.reallocs - before.reallocs,\n after.frees - before.frees,\n after.live as isize - before.live as isize,\n after.peak.saturating_sub(before.live),\n );\n out\n}\n\nfn main() {\n measure(\"Vec::new + 1000 pushes\", || {\n let mut v = Vec::new();\n for i in 0..1000u32 {\n v.push(i);\n }\n v.len()\n });\n\n measure(\"Vec::with_capacity(1000)\", || {\n let mut v = Vec::with_capacity(1000);\n for i in 0..1000u32 {\n v.push(i);\n }\n v.len()\n });\n\n measure(\"1000 format! strings\", || (0..1000).map(|i| format!(\"item-{i}\")).count());\n\n let kept = measure(\"HashMap of 100 Strings\", || {\n let mut m = HashMap::new();\n for i in 0..100 {\n m.insert(i, \"x\".repeat(32));\n }\n m\n });\n println!(\"kept {} entries alive, live bytes now {}\", kept.len(), ALLOC.snapshot().live);\n\n // A leak shows up as live bytes that never come back.\n measure(\"leaking 10 boxes\", || {\n for _ in 0..10 {\n std::mem::forget(Box::new([0u8; 1024]));\n }\n });\n}Running the program produced this:
\nVec::new + 1000 pushes allocs 1 reallocs 8 frees 1 net +0 B peak + 4096 B\nVec::with_capacity(1000) allocs 1 reallocs 0 frees 1 net +0 B peak + 4000 B\n1000 format! strings allocs 1000 reallocs 0 frees 1000 net +0 B peak + 10 B\nHashMap of 100 Strings allocs 106 reallocs 0 frees 5 net +7432 B peak + 8176 B\nkept 100 entries alive, live bytes now 9132\nleaking 10 boxes allocs 10 reallocs 0 frees 0 net +10240 B peak + 10240 BThis table teaches more about Rust's memory behavior than a chapter of explanation, so read it row by row.
\nThe first two rows run the same loop. Pushing 1,000 u32 values into a Vec::new() made one allocation and then eight reallocations, as the capacity doubled from 4 to 8, 16, 32, 64, 128, 256, 512 and 1,024. Vec::with_capacity(1000) made one allocation and zero reallocations, and peaked at 4,000 bytes against 4,096. That is the whole argument for with_capacity when you know the size: the same work with a ninth of the allocator traffic.
The third row creates a thousand short-lived strings. There were 1,000 allocations and 1,000 frees, and the peak was a mere 10 bytes, because each string was dropped before the next was built. Allocation count and memory use are different costs. A program that makes a million small allocations is slow even if it never holds more than a kilobyte.
\nThe fourth row builds a HashMap and keeps it. 106 allocations, 5 frees, and a net 7,432 bytes still live: 100 strings of 32 bytes plus the table itself, which also reallocated as it grew (the 5 frees are the old tables). The line below it confirms the map is still alive and counts total live bytes at 9,132.
The last row is a deliberate leak. std::mem::forget skips the destructor, so ten 1 KiB boxes were allocated and never freed, and net reads +10,240 bytes. A leak is any row where net is positive and nothing holds the data. The tracking allocator will not tell you who owns what, but it will tell you, for any block of code, whether memory came back. Wrap a request handler in measure in a test and a leak announces itself.
realloc is forwardedThe trait provides a default realloc that allocates a new block, copies, and frees the old one. That is correct and wasteful: the system allocator can often grow a block in place without copying. If your wrapper does not forward realloc to System.realloc, you change the program's performance by observing it. The listing forwards it and keeps its own reallocs counter, because without that the Vec row would show a single allocation and hide eight events.
The peak counter is global, so a naive reading includes the peak of everything that ran before. The fix is the line ALLOC.peak.store(before.live, Relaxed) at the start of measure, which resets the high-water mark to the current level. Without it, the third row would have reported the 4,096-byte peak from the first row instead of 10 bytes. This is a common bug in home-made profilers: a counter that never resets reports history, not the thing you measured.
The counters use Ordering::Relaxed, which is correct here because they are independent statistics and nothing else synchronizes through them. Under threads, the numbers are exact in total but the snapshot you read is not a consistent cut across all four; two counters read a few nanoseconds apart may disagree by an in-flight allocation. For profiling that is fine. Do not use these values to make correctness decisions.
Here is the allocator everyone writes first, to see allocations happen:
\nunsafe impl GlobalAlloc for Noisy {\n unsafe fn alloc(&self, layout: Layout) -> *mut u8 {\n eprintln!(\"alloc {} bytes\", layout.size());\n unsafe { System.alloc(layout) }\n }\n unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {\n unsafe { System.dealloc(ptr, layout) }\n }\n}It compiles without a warning. Running a program that uses it printed nothing at all, and the process exited with status 139, a segmentation fault. The cause is recursion. Formatting a message and writing it to a stream needs memory, so eprintln! allocates, which calls alloc, which calls eprintln!, which allocates, until the stack overflows. The stack overflow happens inside the allocator, before any output reaches the screen, so there is no message to read.
The rule is that code inside alloc and dealloc must never allocate and must never take a lock the allocating code might already hold. Writing a number to a file descriptor with the write system call directly is fine. Anything that formats, logs, builds a String or touches a thread_local with lazy initialization is not. Keep the allocator to atomics and plain arithmetic, record events, and do the printing later from outside, as measure does.
At the other end of the design space is the allocator that does almost nothing. A bump allocator owns a block of memory and a cursor. Allocating rounds the cursor up to the requested alignment, moves it forward by the size and returns the old position. Freeing does nothing at all.
\nuse std::alloc::{GlobalAlloc, Layout};\nuse std::cell::UnsafeCell;\nuse std::sync::atomic::{AtomicUsize, Ordering};\n\nconst ARENA_SIZE: usize = 1 << 20; // 1 MiB\n\n/// A bump allocator: allocation moves a pointer forward, freeing does nothing.\n/// Fast and tiny, and the right tool for a short-lived process or a request arena.\nstruct Bump {\n arena: UnsafeCell<[u8; ARENA_SIZE]>,\n next: AtomicUsize,\n}\n\n// SAFETY: the only mutation of `arena` is through pointers handed out once each by `alloc`;\n// `next` is advanced atomically, so two threads never receive overlapping ranges.\nunsafe impl Sync for Bump {}\n\nunsafe impl GlobalAlloc for Bump {\n unsafe fn alloc(&self, layout: Layout) -> *mut u8 {\n let base = self.arena.get() as usize;\n let mut current = self.next.load(Ordering::Relaxed);\n loop {\n // Round the next free address up to the alignment the caller asked for.\n let start = (base + current).next_multiple_of(layout.align()) - base;\n let end = match start.checked_add(layout.size()) {\n Some(end) if end <= ARENA_SIZE => end,\n _ => return std::ptr::null_mut(), // out of arena\n };\n match self.next.compare_exchange_weak(current, end, Ordering::Relaxed, Ordering::Relaxed) {\n Ok(_) => return (base + start) as *mut u8,\n Err(actual) => current = actual,\n }\n }\n }\n\n unsafe fn dealloc(&self, _ptr: *mut u8, _layout: Layout) {}\n}\n\n#[global_allocator]\nstatic ALLOC: Bump = Bump { arena: UnsafeCell::new([0; ARENA_SIZE]), next: AtomicUsize::new(0) };\n\nfn used() -> usize {\n ALLOC.next.load(Ordering::Relaxed)\n}\n\nfn main() {\n let start = used();\n let mut names: Vec<String> = Vec::new();\n for i in 0..1000 {\n names.push(format!(\"user-{i:04}\"));\n }\n println!(\"1000 strings: arena used {} bytes\", used() - start);\n\n let before = used();\n drop(names);\n println!(\"after drop: arena used {} bytes (nothing came back)\", used() - before);\n\n let before = used();\n for _ in 0..3 {\n let scratch: Vec<u8> = vec![0; 100_000];\n std::hint::black_box(&scratch);\n }\n println!(\"3 x 100 kB scratch buffers: arena used {} bytes\", used() - before);\n\n println!(\"arena size {ARENA_SIZE}, used {} so far\", used());\n let big: Vec<u8> = vec![0; 800_000];\n println!(\"an 800 kB allocation succeeded: {}\", big.len());\n}1000 strings: arena used 59062 bytes\nafter drop: arena used 0 bytes (nothing came back)\n3 x 100 kB scratch buffers: arena used 300000 bytes\narena size 1048576, used 360768 so far\nmemory allocation of 800000 bytes failed\nnote: run with `RUST_BACKTRACE=1` environment variable to display a backtraceThe program exited with status 134, an abort. Read the output as the allocator's contract. A thousand strings used 59,062 bytes. Dropping them gave back nothing, as the second line says. Three 100,000-byte scratch buffers used 300,000 bytes, because each one was allocated fresh and none was reclaimed. After that the cursor stood at 360,768 of the 1,048,576 bytes, and an 800,000-byte request did not fit, so alloc returned null and the runtime aborted with \"memory allocation of 800000 bytes failed\".
That behavior is a feature when it is intended. A bump allocator is extremely fast, a few instructions per allocation with no bookkeeping, has zero fragmentation, and is the right tool when everything allocated lives until a known point and then dies together: a compiler pass, a command that runs for a second and exits, one request handled in an arena that is reset afterward. It is the wrong tool for a long-lived server, where a leak-by-design fills memory. For request-scoped arenas inside a normal program, use a local arena type such as bumpalo rather than replacing the global allocator.
Three details in the code are worth studying. The alignment line, (base + current).next_multiple_of(layout.align()) - base, rounds the actual address, not the offset, because the static array itself is only aligned as strictly as its type requires. The checked_add and the comparison guard against both overflow and overrun, returning null instead of handing out memory past the end. And the compare_exchange_weak loop makes the cursor advance atomic, so two threads never receive overlapping ranges. The unsafe impl Sync carries a comment stating why it is sound, which is the convention to follow for every unsafe block you write.
The global counters are awkward in tests, because Rust runs tests on several threads at once, and one test's allocations leak into another's counts. The cure is to count per thread. A thread_local! with a const initializer does not allocate on first use, so it is safe inside the allocator, and try_with avoids a panic during thread teardown.
use std::alloc::{GlobalAlloc, Layout, System};\nuse std::cell::Cell;\n\nthread_local! {\n // `const` initialization: touching this never allocates, which matters inside an allocator.\n static ALLOCS_ON_THIS_THREAD: Cell<u64> = const { Cell::new(0) };\n}\n\nstruct PerThread;\n\nunsafe impl GlobalAlloc for PerThread {\n unsafe fn alloc(&self, layout: Layout) -> *mut u8 {\n let _ = ALLOCS_ON_THIS_THREAD.try_with(|c| c.set(c.get() + 1));\n unsafe { System.alloc(layout) }\n }\n unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {\n unsafe { System.dealloc(ptr, layout) }\n }\n}\n\n#[global_allocator]\nstatic A: PerThread = PerThread;\n\n/// Run `f` and return how many allocations *this thread* made.\nfn count_allocs<R>(f: impl FnOnce() -> R) -> (R, u64) {\n let before = ALLOCS_ON_THIS_THREAD.with(Cell::get);\n let out = f();\n (out, ALLOCS_ON_THIS_THREAD.with(Cell::get) - before)\n}\n\nfn checksum(bytes: &[u8]) -> u32 {\n bytes.iter().fold(0u32, |acc, &b| acc.rotate_left(5) ^ u32::from(b))\n}\n\nfn checksum_hex(bytes: &[u8]) -> String {\n format!(\"{:08x}\", checksum(bytes))\n}\n\nfn main() {\n let data = vec![7u8; 4096];\n\n let (_, n) = count_allocs(|| checksum(&data));\n println!(\"checksum: {n} allocations\");\n assert_eq!(n, 0, \"the hot path must not allocate\");\n\n let (_, n) = count_allocs(|| checksum_hex(&data));\n println!(\"checksum_hex: {n} allocation(s)\");\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn checksum_does_not_allocate() {\n let data = [1u8; 64];\n let (_, n) = count_allocs(|| checksum(&data));\n assert_eq!(n, 0);\n }\n}checksum: 0 allocations\nchecksum_hex: 1 allocation(s)The pair of lines is the point. checksum is a pure computation over a slice and must never allocate, and the program asserts it. checksum_hex wraps the same call in a format!, which costs one allocation. The unit test at the bottom of the file is the pattern to copy: assert that a function on a latency-critical path makes zero allocations, and the build fails the day someone adds a to_string() or a collect::<Vec<_>>() to it. Allocation regressions are among the most common performance bugs and among the least visible in review, because the diff looks innocent.
One caveat applies to this design: the counter sees only allocations made by the thread running the closure. If the function spawns a thread or hands work to a pool, those allocations are counted elsewhere. For code that stays on one thread, which includes most hot-path functions, it is exact.
\nAn obvious worry is that wrapping every allocation in atomic operations slows the program. The cost is easy to measure. The program below makes two million Box allocations and frees, with the counting turned off and on in alternation, and times each pass.
use std::alloc::{GlobalAlloc, Layout, System};\nuse std::sync::atomic::{AtomicUsize, Ordering::Relaxed};\nuse std::time::Instant;\n\nstruct Counting;\nstatic COUNT: AtomicUsize = AtomicUsize::new(0);\nstatic ENABLED: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);\n\nunsafe impl GlobalAlloc for Counting {\n unsafe fn alloc(&self, layout: Layout) -> *mut u8 {\n if ENABLED.load(Relaxed) {\n COUNT.fetch_add(1, Relaxed);\n }\n unsafe { System.alloc(layout) }\n }\n unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {\n unsafe { System.dealloc(ptr, layout) }\n }\n}\n\n#[global_allocator]\nstatic A: Counting = Counting;\n\nfn churn() -> usize {\n let mut total = 0;\n for i in 0..2_000_000u32 {\n let b = Box::new(i);\n total += *std::hint::black_box(b) as usize & 1;\n }\n total\n}\n\nfn main() {\n churn(); // warm up\n for enabled in [false, true, false, true] {\n ENABLED.store(enabled, Relaxed);\n let start = Instant::now();\n churn();\n println!(\"counter {:<8} {:>8.2?} per 2M alloc+free pairs\", if enabled { \"enabled\" } else { \"disabled\" }, start.elapsed());\n }\n}counter disabled 51.99ms per 2M alloc+free pairs\ncounter enabled 52.68ms per 2M alloc+free pairs\ncounter disabled 51.75ms per 2M alloc+free pairs\ncounter enabled 51.78ms per 2M alloc+free pairsThe difference is within noise: about 26 nanoseconds per pair either way, which is the cost of the system allocator itself. An uncontended atomic add is a handful of cycles next to a malloc. That holds for a single thread. With many threads hammering one counter, the cache line holding it bounces between cores and the cost rises sharply; a service that allocates millions of times a second across all cores should shard the counters per thread or per CPU and add them up when read. The per-thread version earlier in this guide has exactly that property.
A custom allocator can enforce a rule rather than just observe. The next one refuses any allocation that would push live bytes past a limit, by returning a null pointer, which is the documented way to signal failure. Normal code then aborts with \"memory allocation failed\", but code that uses the fallible APIs, try_reserve on collections, gets an Err it can handle. That is the right way to treat a length that comes from the network:
use std::alloc::{GlobalAlloc, Layout, System};\nuse std::sync::atomic::{AtomicUsize, Ordering::Relaxed};\n\n/// Refuses any allocation that would push live bytes past a limit.\nstruct Capped {\n live: AtomicUsize,\n limit: AtomicUsize,\n}\n\nunsafe impl GlobalAlloc for Capped {\n unsafe fn alloc(&self, layout: Layout) -> *mut u8 {\n let after = self.live.fetch_add(layout.size(), Relaxed) + layout.size();\n if after > self.limit.load(Relaxed) {\n self.live.fetch_sub(layout.size(), Relaxed);\n return std::ptr::null_mut();\n }\n let ptr = unsafe { System.alloc(layout) };\n if ptr.is_null() {\n self.live.fetch_sub(layout.size(), Relaxed);\n }\n ptr\n }\n\n unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {\n unsafe { System.dealloc(ptr, layout) };\n self.live.fetch_sub(layout.size(), Relaxed);\n }\n}\n\n#[global_allocator]\nstatic A: Capped = Capped { live: AtomicUsize::new(0), limit: AtomicUsize::new(usize::MAX) };\n\n/// Parse a length-prefixed message, refusing to trust the length.\nfn read_message(declared_len: usize) -> Result<Vec<u8>, String> {\n let mut buf = Vec::new();\n buf.try_reserve_exact(declared_len)\n .map_err(|_| format!(\"refusing to allocate {declared_len} bytes for one message\"))?;\n buf.resize(declared_len.min(16), 0xAB);\n Ok(buf)\n}\n\nfn main() {\n // Allow 1 MiB above whatever the runtime already holds.\n A.limit.store(A.live.load(Relaxed) + (1 << 20), Relaxed);\n\n println!(\"small message: {:?}\", read_message(64).map(|m| m.len()));\n println!(\"hostile length: {:?}\", read_message(usize::MAX / 2).map(|m| m.len()));\n println!(\"just over the cap: {:?}\", read_message(2 << 20).map(|m| m.len()));\n}small message: Ok(16)\nhostile length: Err(\"refusing to allocate 9223372036854775807 bytes for one message\")\njust over the cap: Err(\"refusing to allocate 2097152 bytes for one message\")The first message is fine. The second declares a length near the maximum a 64-bit address space allows, the kind of value a malicious or corrupted frame carries. try_reserve_exact reports failure instead of aborting the process, so the parser can reject the frame and move on. The third is a plausible length that exceeds the 1 MiB budget the program set for itself. Without try_reserve, Vec::with_capacity(declared_len) on the hostile value would abort the whole server. This is a security property worth testing: a test that installs a capped allocator and feeds the parser oversized lengths proves that no input can make it allocate more than the cap before validating.
One honest limitation: only code that calls the fallible APIs benefits. Ordinary Vec::push and String growth still abort on failure, because Rust's infallible collection APIs treat out-of-memory as fatal. A cap in the allocator therefore works best as a test tool and as a guard around parsing code written with try_reserve, not as a general mechanism to survive running out of memory.
Two details of the contract trip people who write their own. The alignment in a Layout is a requirement, not a hint, and Layout guarantees it is a power of two. Types such as SIMD vectors or page-sized buffers can need 16, 64 or 4096 bytes, which is why the bump allocator rounds the address and not just the offset. And alloc_zeroed, which backs vec![0; n], has a default implementation that calls alloc and writes zeros. The system allocator can often get zeroed pages straight from the operating system for large requests, with no write at all, so a wrapper that does not forward alloc_zeroed to System.alloc_zeroed turns a lazy, nearly free vec![0u8; 1 << 30] into a gigabyte of real writes. Forward all four methods unless you have a reason not to.
GlobalAlloc is the stable interface, and it has limits: one allocator for the whole program, no way to pass an allocator to a single Vec, and no way to ask for the size actually allocated. The Allocator trait and the Vec::new_in family fix that and are still unstable, so using them requires a nightly toolchain. If you want a per-collection arena on stable today, use a crate that provides its own arena types, such as bumpalo, which hands out references tied to the arena's lifetime.
Most programs never need to write an allocator, but many benefit from swapping one in. The system allocator on some platforms is slow under multithreaded contention or fragments badly in long-running services. Third-party allocators such as mimalloc and tikv-jemallocator install in three lines:
use mimalloc::MiMalloc;\n\n#[global_allocator]\nstatic GLOBAL: MiMalloc = MiMalloc;Whether it helps is an empirical question with a measurable answer. Build once with each, run the same load test at the same request rate, and compare tail latency and resident memory. Allocator changes often move the 99th percentile more than the median, and they trade memory for speed in different ways, so the numbers you read must come from your workload. The tracking wrapper from this guide composes with any of them: wrap MiMalloc instead of System and keep the counters.
Do not replace the global allocator to fix a problem you have not measured. For a leak, a heap profiler such as dhat or the platform's tooling points at the allocating line directly, with no custom unsafe code. For a per-request arena, use bumpalo. For a memory cap in a test, the tracking allocator above is enough, and it takes minutes. On embedded targets with no operating system, the situation inverts: there is no System, and you supply the allocator, usually a small crate such as embedded-alloc or linked-list-allocator, initialized with a region of RAM at startup. Whatever the case, keep to the rules that make allocators safe: no allocation, no locks that can deadlock, no panics, and a comment justifying every unsafe. For the language-level story of what you are bypassing, see Box, Rc and Arc in Rust, and for memory problems in services, memory usage and finding leaks.
A microcontroller with 64 KiB of flash does not have room for a surprise. A firmware that fit last week stops fitting after someone adds a line that prints a temperature, and the linker reports an overflow of a few kilobytes with no hint about which line did it. Shrinking firmware is detective work: measure, find the biggest consumer, replace it with something smaller, measure again. The detective part is easy to get wrong because the intuitive suspects, the optimization level and your own code, are rarely the culprits.
\nThis guide runs that process on a real firmware. It starts at 1,160 bytes, adds the features that real firmware adds, watches the size jump to 32,128 bytes because of one innocent-looking write!, finds the cause with a symbol listing, and brings it back down to 1,368 bytes without losing the feature. Every number below was measured on Rust 1.99 with edition 2024, building for thumbv7em-none-eabihf (Cortex-M4F) with cortex-m-rt 0.7.7, cortex-m 0.7.10 and panic-halt 1.0.0. The code was not run on a board, since size depends on what is compiled, not on running it.
Three numbers get called \"size\", and mixing them up wastes hours. The ELF file that Cargo produces includes debug information, symbol tables and section headers. None of that goes to the chip. The flash footprint is the code, read-only data and initial values of variables: in the llvm-size output, text plus data. The RAM footprint is data plus bss, plus the stack and any heap you chose to add.
rustup component add llvm-tools\n$(rustc --print sysroot)/lib/rustlib/aarch64-apple-darwin/bin/llvm-size target/thumbv7em-none-eabihf/release/fwsizeReplace aarch64-apple-darwin with your host's triple. The rust-size command from the cargo-binutils package is a shorter way to run the same tool. For this guide, the number reported is the first column, text, which on Cortex-M includes the 1,024-byte interrupt vector table that the runtime places at the start of flash. Subtract it if you want to see only your code; it is constant for a given chip.
The test firmware reads an analog sample from a register and writes a pin. Three Cargo features add the things real firmware adds. index smooths the readings with indexed array access. fmt reports a temperature as text with write! and a floating-point conversion, and fmtint does the same with an integer. A feature called manual prints the same temperature using hand-written digit conversion. The build script sets the optimization level through environment variables so the same source can be measured under different profiles.
The core of the formatting variant is ordinary Rust that any developer would write:
\nuse core::fmt::Write;\n\npub struct Uart;\n\nimpl Write for Uart {\n fn write_str(&mut self, s: &str) -> core::fmt::Result {\n for b in s.bytes() {\n unsafe { core::ptr::write_volatile(0x4001_1004 as *mut u32, u32::from(b)) };\n }\n Ok(())\n }\n}\n\npub fn temperature(raw: u32) {\n let celsius = raw as f32 * 0.0806 - 50.0;\n let _ = write!(Uart, \"temp {:.1} C (raw {})\\n\", celsius, raw);\n}That is eighteen lines, and it implements the standard core::fmt::Write trait on a serial port so that write! works. It looks like the kind of code you would write in the first hour of a project.
plain, opt-level 3 1184\nplain, opt-level s 1160\nplain, opt-level z 1168\nindexing, opt-level s 1436\nfmt + float, opt-level s 32128\nfmt + float, opt-level z 32120\nfmt + float, z, lto, cgu=1 17364\ninteger-only write!, opt-level s 4148\nhand-written digits, opt-level s 1368The numbers are bytes of flash. Read them as a sequence of lessons, because each row teaches one thing.
\nThe first three rows differ by 24 bytes. Choosing opt-level = \"s\" over 3 saved 24 bytes, and \"z\", the most aggressive size setting, was larger than \"s\" by 8. On a program with almost no code, there is nothing for the optimizer to shrink. The optimization level matters on large programs with real loops and inlining, and even there it is a tuning step, not the solution. If someone's answer to \"the firmware is too big\" is \"set opt-level = z\", this table is the reply: measure what is big first. The same table also shows why: when a program grew by thirty kilobytes, switching from s to z recovered 8 bytes.
The fourth row adds a smoothing function with three indexed array accesses. The firmware grew by 276 bytes. Each samples[i % 16] can panic if the index is out of range, so the compiler emits a comparison and a branch to a panic routine for each one. The panic routine takes the file and line number as arguments, which means the string constants for the location are stored in flash too. Rust code with many indexing expressions pays that per site. You can remove the check where you can prove it unnecessary: iterate with iter() instead of indexing, use get(i) and handle None, or use chunks_exact. Avoid get_unchecked unless profiling proves it matters, since it trades 20 bytes for undefined behavior if you are wrong.
write! costs 31 kilobytesThe fifth row is the one that ruins projects. Adding a function that formats one floating-point number grows the firmware from 1,160 bytes to 32,128. The code you wrote is a few dozen bytes. The rest is the standard formatting machinery, linked in because {:.1} on a float needs to turn a binary number into decimal digits correctly, and correct float printing is an algorithm, two algorithms in fact, with large lookup tables.
You do not have to guess which parts. The llvm-nm tool, run with --size-sort, lists symbols by size, and the biggest entries tell the story:
$(rustc --print sysroot)/lib/rustlib/aarch64-apple-darwin/bin/llvm-nm --size-sort --demangle --print-size target/thumbv7em-none-eabihf/release/fwsize | tail -808000040 000003c0 R __INTERRUPTS\n08001cdc 00000490 T core::num::imp::flt2dec::strategy::grisu::format_exact_opt\n08001658 000004ea T core::str::count::do_count_chars\n08007410 00000510 R core::num::imp::flt2dec::strategy::grisu::CACHED_POW10\n0800216c 00000862 T core::num::imp::flt2dec::strategy::grisu::format_shortest_opt\n0800538e 00000a78 T core::num::imp::flt2dec::strategy::dragon::mul_pow10\n080029ce 0000146c T core::num::imp::flt2dec::strategy::dragon::format_exact\n08003e3a 00001554 T core::num::imp::flt2dec::strategy::dragon::format_shortestThe second column is the symbol's size in hexadecimal. The two largest are dragon::format_shortest (0x1554, 5,460 bytes) and dragon::format_exact (0x146c, 5,228 bytes), followed by dragon::mul_pow10 (2,680 bytes), the grisu variants (2,146 and 1,168 bytes) and a table of cached powers of ten (1,296 bytes). Grisu is the fast path for float to decimal conversion and Dragon is the exact fallback when Grisu cannot decide. The standard library includes both so that every float prints correctly, and your firmware pays for both whether or not it ever reaches the fallback. This is the single most common cause of an embedded Rust binary that is far larger than its source suggests.
Row six repeats the formatting build with lto = true and codegen-units = 1, and the size falls from 32,120 to 17,364 bytes, a 46 percent drop. With a single compilation unit and whole-program optimization, the compiler can see that parts of the formatting code are unreachable from your use and discard them. It is the cheapest big win available, it costs only build time, and it belongs in every firmware release profile:
[profile.release]\nopt-level = \"s\"\nlto = true\ncodegen-units = 1\npanic = \"abort\"\ndebug = true # stays in the ELF, never reaches flashNote debug = true. Debug information does not increase the flash footprint, as the first section explained, and it lets a debugger show source lines. Do not strip it from the development profile to chase size.
Row seven formats only an integer with write!(Uart, \"raw {}\\n\", raw), and the size falls to 4,148 bytes. The integer path of core::fmt is small compared to the float path but still brings the Formatter, the padding logic and the Arguments machinery: about 3 KiB over the plain program. For many devices that is acceptable, and it keeps your code readable. If it is not, the last row prints the same temperature with no core::fmt at all.
/// Print tenths of a degree as \"-12.3\" with no `core::fmt` and no floats.\npub fn temperature(raw: u32) {\n let tenths = (raw as i32).wrapping_mul(806) / 1000 - 500; // 0.0806 C per count, minus 50 C\n let mut n = tenths.unsigned_abs();\n if tenths < 0 {\n put(b'-');\n }\n let mut digits = [0u8; 10];\n let mut len = 0;\n for slot in digits.iter_mut() {\n *slot = b'0' + (n % 10) as u8;\n len += 1;\n n /= 10;\n if n == 0 && len >= 2 {\n break;\n }\n }\n for (k, d) in digits.iter().take(len).rev().enumerate() {\n if k + 1 == len {\n put(b'.');\n }\n put(*d);\n }\n put(b'\\n');\n}It computes in fixed point: counts times 806 over 1000 gives tenths of a degree as an integer, with no floating point anywhere, and digits are produced by repeated division by ten. The result is 1,368 bytes, an increase of only 208 over the plain firmware, against 30,968 for the version that used {:.1}.
That 1,368 deserves a footnote, because the first attempt at the manual formatter measured 3,920 bytes, almost three times larger, and the reason is the lesson of the \"indexing\" row in disguise. The first draft wrote digits[len] = ... and sliced digits[..len]. Both can panic, and a panic from an out-of-range index is built with core::fmt::Arguments, which references the formatting code for usize. One indexing expression in a function was enough to link in the integer formatting path that the function was written to avoid. Replacing the indexing with iterators, iter_mut() and take(len), which cannot go out of range, removed every panic site, and the size fell to 1,368. The same applies to unwrap, expect, slice ranges and arithmetic when overflow checks are on. If you are fighting for bytes, search the disassembly for calls to core::panicking, or list the symbols that mention fmt, and trace each one back to the line that required it.
Three commands cover most investigations, and you should use them before editing anything.
\nllvm-size -A prints every section with its size, which shows whether code, constant data or RAM is the problem. llvm-nm --size-sort --demangle --print-size, shown above, ranks symbols; the tail is the answer. And cargo bloat --release --crates (from the cargo-bloat crate) groups the same information by crate, which answers \"is it my code, core, or this one dependency?\" in one table. In a typical oversized firmware the crate table shows core or compiler_builtins near the top, and the next step is to ask which of your lines pulled it in. Work from the biggest symbol down, change one thing, and rebuild; the measurement loop takes seconds.
Generic functions are compiled once for every type they are used with, and the usual advice for shrinking a binary is to replace a generic with a trait object so that one copy serves all types. That advice is sometimes right and often untested, so test it. The experiment calls one small function with four element types, u8, u16, u32 and bool, in two forms: a generic weighted<T: Copy + Into<u32>> marked #[inline(never)], so that each copy stays visible, and a version taking &dyn Fn(usize) -> u32 that fetches each element through a closure.
none 1160\ngeneric 1416\ndynamic 1600The generic version added 256 bytes to the baseline, about 64 bytes per instantiation. The trait-object version added 440, nearly twice as much, because each call site now builds a closure and a vtable and the function body has an indirect call inside its loop. The abstraction that was supposed to save space cost more than the duplication it removed. The rule that survives is narrower: a trait object pays off when the function body is large, hundreds of bytes or more, and the per-type difference is small. A 64-byte function copied four times is cheaper than any indirection. Measure with the symbol listing, which shows each monomorphized copy as its own entry (weighted<u8>, weighted<u16> and so on), and only then decide.
Size regressions arrive one innocent commit at a time, and the commit that crosses the line is never the one that caused the problem. A budget turns the slow drift into a failing check. The script below builds the release binary, reads the text column, and fails if it exceeds a limit you set a little above today's number.
#!/bin/sh\nset -eu\n\nLIMIT=${1:?usage: check-size.sh <max-bytes>}\nBIN=target/thumbv7em-none-eabihf/release/fwsize\nSIZE=$(rustc --print sysroot)/lib/rustlib/$(rustc -vV | sed -n 's/host: //p')/bin/llvm-size\n\ncargo build --release --quiet\nused=$(\"$SIZE\" \"$BIN\" | awk 'NR==2 {print $1 + $2}') # text + data = flash footprint\n\necho \"flash used: $used bytes (limit $LIMIT)\"\nif [ \"$used\" -gt \"$LIMIT\" ]; then\n echo \"over budget by $((used - LIMIT)) bytes\" >&2\n exit 1\nfiRun against the plain firmware with a generous and then a tight limit, it behaves like this:
\nflash used: 1184 bytes (limit 2000)\n\nflash used: 1184 bytes (limit 1000)\nover budget by 184 bytesThe second run exits with status 1. Wire it into the pipeline and a change that adds a float format fails loudly with the overage, on the pull request that caused it, instead of surfacing at a linker error on release day. Set the limit about 5 percent above the current size so ordinary work does not trip it, and move the limit down when you win bytes back. Printing the number on every run, passing or not, gives you a free history in the CI logs. For a more detailed view on pull requests, attach the output of cargo bloat --release --crates as a comment, so reviewers see which crate grew.
Compress the work into a fixed order and you waste less time. First turn on lto = true, codegen-units = 1 and panic = \"abort\", which is five minutes of work and, as the formatting rows showed, can remove nearly half of an oversized binary. Second, list the biggest symbols and look at the top ten; if core::fmt or core::num::flt2dec appears, you have found the budget. Third, remove or replace the culprit: drop float formatting, move text to the host with defmt, or hand-write the conversion. Fourth, hunt panic sites, because each one carries a location string and may drag formatting along, as the 3,920-byte formatter showed. Only fifth, tune opt-level, and measure s against z and 3 rather than assuming, since the best choice differs for code that is mostly branches and code that is mostly arithmetic.
defmt for logging. It sends an index and the arguments in binary and leaves the text on your computer, so a log statement costs a few bytes of flash instead of a format string plus the formatting code.&dyn Trait or convert to a common type early.panic-halt or panic-abort for release builds. A panic handler that formats a message with the location pulls in formatting code, and panic-semihosting is a debugging tool, not a shipping one.cargo bloat after adding one.lto = true and codegen-units = 1 and keep them on. They cost nothing at runtime.On nightly there is more to try. -Z build-std rebuilds core with your settings, and panic_immediate_abort removes the panic message machinery entirely. Those options were not measured here, and they require a nightly toolchain, so adopt them only once the stable options are exhausted.
If the firmware fits with room to spare, stop. A 20 percent smaller binary that took a week to achieve and made the formatting code harder to read is a bad trade on a chip with 512 KiB of flash. Optimize size when you hit a limit, when a smaller chip would cut the bill of materials across thousands of units, or when an over-the-air update has to fit a tight radio budget. Optimize speed or power instead when those are the constraints, since the settings do not always agree: opt-level = 3 can be faster and larger, and wfi sleeping saves more battery than any code size win. For the groundwork these numbers rest on, see embedded Rust from blinking an LED to real firmware.
One last caution about the budget number itself. It is only comparable between builds that use the same toolchain, the same dependency versions and the same profile. A compiler upgrade can move the total by a few hundred bytes in either direction, so record the Rust version next to the limit and re-baseline deliberately when it changes. Pinning the toolchain in rust-toolchain.toml makes that a decision you take on purpose, not a failure that appears when a new stable release lands on a Tuesday.
Bevy is a game engine, and the part of it you can use on its own is its entity component system: bevy_ecs, a library that stores your data in a way that makes queries fast and lets independent functions run in parallel without locks. Games are the obvious customer. Simulations, schedulers, editors and anything with thousands of things that each have a different mix of properties are the others. This guide teaches the model with programs that run headless, with no window and no renderer: entities, components, systems and queries; filters, resources and change detection; messages and observers; how the scheduler decides what can run in parallel (with a measurement); the cost of structural changes; and the errors you will meet. Everything compiled and ran on Rust 1.99 with bevy_ecs 0.20.1 (Bevy 0.20.1 is the current release as of this writing). Bevy's API moves between releases, so the version matters: pin it, and read the migration guide before upgrading.
An entity is an id and nothing else. A component is a piece of plain data attached to an entity (a position, a health value, a marker). A system is an ordinary function whose arguments say what data it needs; the scheduler inspects the arguments to learn which components each system reads and writes. A query is how a system asks for \"every entity that has a Position and a Velocity\". Contrast this with object-oriented design, where a Ship class owns its position and a method moves it: in an ECS a ship is just whichever entity happens to have the components that movement, rendering and AI systems look for. Behaviour is added by adding a component, not by subclassing. The storage turns \"all positions\" into a contiguous column, which is what makes iteration over a million entities fast and what lets the scheduler prove that two systems touch different data.
[package]\nname = \"ecs-lab\"\nversion = \"0.1.0\"\nedition = \"2024\"\n\n[dependencies]\nbevy_ecs = \"0.20\"That is the whole dependency. Full Bevy adds windowing, rendering, assets and input as plugins; they all sit on top of this crate. Starting with bevy_ecs alone keeps compile times short (a few seconds for an incremental build) and keeps examples testable in plain cargo test, with no GPU.
use bevy_ecs::prelude::*;\n\n#[derive(Component, Debug)]\nstruct Position { x: f32, y: f32 }\n\n#[derive(Component, Debug)]\nstruct Velocity { x: f32, y: f32 }\n\n#[derive(Component)]\nstruct Name(&'static str);\n\nfn movement(mut query: Query<(&mut Position, &Velocity)>) {\n for (mut pos, vel) in &mut query {\n pos.x += vel.x;\n pos.y += vel.y;\n }\n}\n\nfn report(query: Query<(&Name, &Position)>) {\n for (name, pos) in &query {\n println!(\"{} at ({:.1}, {:.1})\", name.0, pos.x, pos.y);\n }\n}\n\nfn main() {\n let mut world = World::new();\n world.spawn((Name(\"ship\"), Position { x: 0.0, y: 0.0 }, Velocity { x: 1.0, y: 0.5 }));\n world.spawn((Name(\"rock\"), Position { x: 5.0, y: 5.0 }));\n\n let mut schedule = Schedule::default();\n schedule.add_systems((movement, report).chain());\n\n for tick in 1..=2 {\n println!(\"-- tick {tick}\");\n schedule.run(&mut world);\n }\n}\n-- tick 1\nship at (1.0, 0.5)\nrock at (5.0, 5.0)\n-- tick 2\nship at (2.0, 1.0)\nrock at (5.0, 5.0)Everything in an ECS program fits this shape. A World owns all the data. world.spawn(( ... )) creates an entity from a tuple of components, called a bundle: the ship has three components, the rock only two, and no class declares that. The movement system takes Query<(&mut Position, &Velocity)>, so it visits every entity that has both, and the rock, which has no Velocity, is skipped. The report system asks for a name and a position and sees both entities. A Schedule runs the systems; .chain() forces them to run in order, so that the report always sees the moved positions. Without it the scheduler may run them in either order or in parallel, because nothing says one depends on the other.
One practical warning from running this code: the order in which a query returns entities is not specified. When the same program was built against bevy_ecs 0.18 the rock was reported before the ship, and against 0.20 the ship came first. If your output depends on the order, sort explicitly or carry an index component.
Real programs need more than \"every entity with these components\". Filters narrow a query without borrowing the data: With<T> and Without<T> test for presence, Changed<T> matches components modified since the system last ran. Resources are singletons stored in the world, such as a score or a clock. Commands queue structural changes (spawn, despawn, insert, remove) to be applied safely after the system finishes, since changing which entities exist while iterating would invalidate the iterators. This program wires them together: poison drains health, a reaper despawns dead enemies and adds to the score, and a reporter prints the state.
use bevy_ecs::prelude::*;\n\n#[derive(Component, Debug)]\nstruct Health(i32);\n\n#[derive(Component)]\nstruct Enemy;\n\n#[derive(Component)]\nstruct Player;\n\n#[derive(Component)]\nstruct Poisoned;\n\n#[derive(Resource, Default)]\nstruct Score(u32);\n\n#[derive(Resource)]\nstruct Tick(u32);\n\nfn setup(mut commands: Commands) {\n commands.spawn((Player, Health(100)));\n commands.spawn((Enemy, Health(30)));\n commands.spawn((Enemy, Health(50), Poisoned));\n}\n\nfn poison(mut query: Query<&mut Health, With<Poisoned>>) {\n for mut hp in &mut query {\n hp.0 -= 20;\n }\n}\n\nfn reap(mut commands: Commands, mut score: ResMut<Score>, query: Query<(Entity, &Health), (With<Enemy>, Changed<Health>)>) {\n for (entity, hp) in &query {\n if hp.0 <= 0 {\n score.0 += 10;\n commands.entity(entity).despawn();\n }\n }\n}\n\nfn describe(tick: Res<Tick>, score: Res<Score>, players: Query<&Health, With<Player>>, enemies: Query<&Health, (With<Enemy>, Without<Poisoned>)>, poisoned: Query<&Health, With<Poisoned>>) {\n println!(\n \"tick {}: score {}, player {:?}, healthy enemies {}, poisoned {:?}\",\n tick.0,\n score.0,\n players.iter().map(|h| h.0).collect::<Vec<_>>(),\n enemies.iter().count(),\n poisoned.iter().map(|h| h.0).collect::<Vec<_>>()\n );\n}\n\nfn advance(mut tick: ResMut<Tick>) {\n tick.0 += 1;\n}\n\nfn main() {\n let mut world = World::new();\n world.insert_resource(Score::default());\n world.insert_resource(Tick(0));\n\n let mut startup = Schedule::default();\n startup.add_systems(setup);\n startup.run(&mut world);\n\n let mut frame = Schedule::default();\n frame.add_systems((advance, poison, reap, describe).chain());\n for _ in 0..4 {\n frame.run(&mut world);\n }\n}\ntick 1: score 0, player [100], healthy enemies 1, poisoned [30]\ntick 2: score 0, player [100], healthy enemies 1, poisoned [10]\ntick 3: score 10, player [100], healthy enemies 1, poisoned []\ntick 4: score 10, player [100], healthy enemies 1, poisoned []Follow the poisoned enemy, which started with 50 health and lost 20 per tick: 30 after tick 1, 10 after tick 2, and at tick 3 it reached -10, the reaper saw the change, despawned it and added 10 points. The reaper's query uses Changed<Health>, so it only examines enemies whose health was written this frame, which matters when thousands of enemies exist and few are being hurt. The \"healthy enemies\" count stayed at one throughout: the first enemy has no Poisoned marker and is never touched. The marker pattern is the ECS way to express state: instead of a boolean field checked in every system, poisoning is a component, and systems select by presence. Adding or removing it with commands.entity(e).insert(Poisoned) changes which systems apply to that entity from the next frame.
Systems often need to tell each other that something happened. Bevy has two mechanisms, and choosing between them is the main design decision after components. Messages are buffered: a system writes them, and later systems in the same or the next frame read them in order. They suit high-volume, batch-processed signals such as damage events. Observers are triggered immediately: an event is fired and the observer function runs right then, with access to the entity it targets. They suit rare, reactive things such as \"this entity died, play the animation, drop the loot\". The program uses both: the damage message carries an amount to a target, and when health reaches zero the system triggers a Died event on the entity, handled by an observer that despawns it.
use bevy_ecs::prelude::*;\n\n#[derive(Component, Debug)]\nstruct Health(i32);\n\n#[derive(Component)]\nstruct Name(&'static str);\n\n#[derive(Message)]\nstruct Damage {\n target: Entity,\n amount: i32,\n}\n\n#[derive(EntityEvent)]\nstruct Died {\n entity: Entity,\n}\n\nfn attack(mut writer: MessageWriter<Damage>, query: Query<(Entity, &Name)>) {\n for (entity, name) in &query {\n if name.0 == \"goblin\" {\n writer.write(Damage { target: entity, amount: 40 });\n }\n }\n}\n\nfn apply_damage(mut reader: MessageReader<Damage>, mut query: Query<&mut Health>, mut commands: Commands) {\n for dmg in reader.read() {\n if let Ok(mut hp) = query.get_mut(dmg.target) {\n hp.0 -= dmg.amount;\n if hp.0 <= 0 {\n commands.trigger(Died { entity: dmg.target });\n }\n }\n }\n}\n\nfn main() {\n let mut world = World::new();\n world.init_resource::<Messages<Damage>>();\n\n world.add_observer(|died: On<Died>, names: Query<&Name>, mut commands: Commands| {\n println!(\"observer: {} died\", names.get(died.entity).map(|n| n.0).unwrap_or(\"?\"));\n commands.entity(died.entity).despawn();\n });\n\n world.spawn((Name(\"goblin\"), Health(70)));\n world.spawn((Name(\"knight\"), Health(200)));\n\n let mut schedule = Schedule::default();\n schedule.add_systems((attack, apply_damage).chain());\n\n for round in 1..=3 {\n schedule.run(&mut world);\n world.resource_mut::<Messages<Damage>>().update();\n let alive = world.query::<&Name>().iter(&world).map(|n| n.0).collect::<Vec<_>>();\n println!(\"round {round}: alive = {alive:?}\");\n }\n}\nround 1: alive = [\"goblin\", \"knight\"]\nobserver: goblin died\nround 2: alive = [\"knight\"]\nround 3: alive = [\"knight\"]Round one: the goblin took 40 of its 70 health, and both survive. Round two: it took 40 more, health reached -10, the system triggered Died, and the observer printed and despawned it before the round ended. Round three: only the knight is left. The line world.resource_mut::<Messages<Damage>>().update() advances the message buffers, which full Bevy does for you every frame; a standalone world has to do it itself or messages pile up. Note that in earlier releases buffered events were called Event and read with EventReader; if you copy older code, expect to rename those to Message, MessageReader and MessageWriter, and to use On<E> for observers.
Each system declares its data access through its arguments, so the scheduler knows before running anything which components each one reads and writes. Two systems can run at the same time if neither writes something the other touches. You can see it with a deliberate experiment: five arrangements of systems that each sleep for 100 ms. If the scheduler overlaps them, the total is about 100 ms; if it serialises them, about 200 ms.
\nfn run(label: &str, build: impl FnOnce(&mut Schedule)) {\n let mut world = World::new();\n world.spawn((A(0), B(0)));\n let mut schedule = Schedule::default();\n schedule.set_executor(bevy_ecs::schedule::MultiThreadedExecutor::new());\n build(&mut schedule);\n let start = Instant::now();\n schedule.run(&mut world);\n println!(\"{label:<44} {:>4} ms\", start.elapsed().as_millis());\n}\n\nfn main() {\n bevy_tasks::ComputeTaskPool::get_or_init(bevy_tasks::TaskPool::default);\n run(\"slow_a + slow_b (different components)\", |s| {\n s.add_systems((slow_a, slow_b));\n });\n run(\"slow_a + slow_a_again (both write A)\", |s| {\n s.add_systems((slow_a, slow_a_again));\n });\n run(\"read_a + read_a_too (both only read A)\", |s| {\n s.add_systems((read_a, read_a_too));\n });\n run(\"slow_a + read_a (one writes, one reads A)\", |s| {\n s.add_systems((slow_a, read_a));\n });\n run(\"slow_a + slow_b chained\", |s| {\n s.add_systems((slow_a, slow_b).chain());\n });\n}\nslow_a + slow_b (different components) 110 ms\nslow_a + slow_a_again (both write A) 216 ms\nread_a + read_a_too (both only read A) 100 ms\nslow_a + read_a (one writes, one reads A) 200 ms\nslow_a + slow_b chained 213 msThe results follow the rules exactly. Two systems that write different components ran together (109 ms). Two that both write A were serialised (211 ms), as were one that writes and one that reads the same component (216 ms), since a reader must not see a half-written value. Two read-only systems overlapped (109 ms). And .chain() serialises by request (209 ms). One setup detail: a standalone bevy_ecs schedule used the single-threaded executor until it was given MultiThreadedExecutor and the multi_threaded feature was enabled with a task pool created; full Bevy does all of that for you. You never write a lock, and the guarantee comes from the borrow rules applied at the system-parameter level, which is the closest thing to \"Rust's ownership model, but across functions\" in an engine.
A system's arguments are its whole interface, so it pays to know the common ones. Each states what the system needs and, by doing so, what it conflicts with.
\n| Parameter | Gives the system | Notes |
|---|---|---|
Query<D, F> | Every entity matching data D and filter F | The workhorse; &T reads, &mut T writes |
Single<D, F> | Exactly one matching entity | The system is skipped if there are zero or more than one |
Res<T>, ResMut<T> | A resource, read or write | Writers to the same resource cannot overlap |
Commands | A queue of structural changes | Applied after the system ends, before later systems that depend on it |
Local<T> | State private to this system | Persists between runs; no sharing and no conflicts |
MessageReader<T>, MessageWriter<T> | Buffered messages | Each reader tracks its own position |
ParamSet<(A, B)> | Several conflicting parameters, used one at a time | The escape hatch for error B0001 |
This program uses four of them, plus two features of the scheduler that real projects need immediately: system sets, which name a phase so ordering can be stated once for many systems, and run conditions, which skip a system based on a function. The three phases run in order (input, simulate, report) because configure_sets chains them; earn only runs while a not_paused condition holds; Local counts how many times earn has run without a resource; and Single expresses \"the player\" without a loop.
use bevy_ecs::prelude::*;\n\n#[derive(Component)]\nstruct Player;\n\n#[derive(Component)]\nstruct Gold(u32);\n\n#[derive(Resource, Default)]\nstruct Paused(bool);\n\n#[derive(SystemSet, Debug, Clone, PartialEq, Eq, Hash)]\nenum Phase {\n Input,\n Simulate,\n Report,\n}\n\nfn earn(mut gold: Single<&mut Gold, With<Player>>, mut calls: Local<u32>) {\n *calls += 1;\n gold.0 += 5 * *calls;\n}\n\nfn pause_on_three(mut paused: ResMut<Paused>, gold: Single<&Gold, With<Player>>) {\n if gold.0 >= 30 {\n paused.0 = true;\n }\n}\n\nfn report(gold: Single<&Gold, With<Player>>) {\n println!(\"gold = {}\", gold.0);\n}\n\nfn not_paused(paused: Res<Paused>) -> bool {\n !paused.0\n}\n\nfn main() {\n let mut world = World::new();\n world.init_resource::<Paused>();\n world.spawn((Player, Gold(0)));\n\n let mut schedule = Schedule::default();\n schedule.configure_sets((Phase::Input, Phase::Simulate, Phase::Report).chain());\n schedule.add_systems(pause_on_three.in_set(Phase::Input));\n schedule.add_systems(earn.in_set(Phase::Simulate).run_if(not_paused));\n schedule.add_systems(report.in_set(Phase::Report));\n\n for frame in 1..=6 {\n print!(\"frame {frame}: \");\n schedule.run(&mut world);\n }\n}\nframe 1: gold = 5\nframe 2: gold = 15\nframe 3: gold = 30\nframe 4: gold = 30\nframe 5: gold = 30\nframe 6: gold = 30Gold grew by 5, then 10, then 15 on the first three frames (the local call counter makes each award larger), reached 30 on frame 3, and then the input phase set the pause flag at the start of frame 4. The run condition then skipped earn from frame 4 on, while the report still ran every frame. Notice what the sets bought: pause_on_three is declared in a different add_systems call from earn, and the order between them is guaranteed by the phases, not by the order of the calls. In a real game this is how input, physics, AI and rendering are stacked into frames, and it scales to hundreds of systems because each system names its phase and nothing else about the others. A stricter question than \"which order\" is \"which data\": the scheduler orders only by what you state, and overlaps everything else, so a missing ordering between two systems that share data on purpose is a bug that appears as flicker, not as a compile error. State the dependency (before, after, in_set, chain) the moment one system relies on another's results.
A system is a function over a world, so a test builds the world it needs and runs the schedule once. This is the strongest argument for the architecture: no engine, no window, no fixtures beyond a few spawned entities. A test for the reaper might spawn an enemy with health 0, run the frame schedule, and assert that world.query::<&Enemy>().iter(&world).count() == 0 and that the score resource is 10. Because the scheduler is deterministic for systems that are chained, tests that need exact order chain them; tests of independent systems run each one alone, with world.run_system_once(system) (import bevy_ecs::system::RunSystemOnce) as a shorthand when you do not want to build a schedule; it returns a Result, and running heal on a world with one entity of health 5 left it at 15. Keep time out of the systems by passing a Time-like resource that tests control, the same injection idea as in dependency injection.
ECS is often sold as fast, and the honest answer depends on what you compare against. This benchmark updates one million entities' positions from their velocities, in a release build on an Apple M1 Max, best of seven runs, against two hand-written alternatives, then measures adding and removing a component on 100,000 entities:
\n1M entities, one movement update:\n bevy_ecs query 1.07 ms (1.07 ns/entity)\n bevy_ecs, no change tracking 1.10 ms (1.10 ns/entity)\n Vec of 64-byte structs 1.13 ms (1.13 ns/entity)\n two Vecs (struct of arrays) 0.25 ms (0.25 ns/entity)\nstructural change on 100k entities: insert 6.1 ms (61 ns each), remove 4.9 ms (49 ns each)Read the numbers carefully. The query matched a plain Vec of 64-byte structs (1.07 ms, about one nanosecond per entity), which is the layout a straightforward game loop would use, and was about four times slower than a hand-written struct-of-arrays loop over two tight vectors (0.25 ms). Turning off change detection for the writes made no difference, so that bookkeeping was not the gap; I did not profile further, and the likely causes are the generality of a query (it must work for any component set and handles access checks and ticks) and a loop the compiler cannot vectorise. The lesson is not that ECS is slow: 1 ns per entity is about 900 updates per second with a million moving things on one core. The lesson is that ECS is a flexibility tool that is fast enough, not a replacement for a hand-tuned data layout in the one loop that dominates your frame. If a single hot loop is the whole program, write the arrays. If you have dozens of systems over many overlapping groups of entities, the ECS gives you that layout for all of them without writing it.
The second result is the one that bites in practice. Adding a component to an entity moves it to a different archetype (a table holding all entities with exactly that set of components), which costs about 60 ns for an insert and 45 ns for a remove here: a copy of the entity's row. That is cheap for an occasional change and expensive if every entity toggles a flag every frame. For flags that flip often, Bevy offers sparse-set storage, declared on the component: #[component(storage = \"SparseSet\")]. Adds and removes become cheaper, and iteration over entities with that component becomes slower, so use it for transient markers like Stunned and keep ordinary data in tables.
A system that asks for conflicting access to the same data cannot run, and Bevy refuses at the moment it builds the schedule. Here a system holds a mutable query over all Health and a shared query over the players' health at once:
use bevy_ecs::prelude::*;\n\n#[derive(Component)]\nstruct Health(i32);\n\n#[derive(Component)]\nstruct Player;\n\nfn heal_others(mut all: Query<&mut Health>, players: Query<&Health, With<Player>>) {\n let _ = players.iter().count();\n for mut h in &mut all {\n h.0 += 1;\n }\n}\n\nfn main() {\n let mut world = World::new();\n world.spawn((Player, Health(10)));\n let mut schedule = Schedule::default();\n schedule.add_systems(heal_others);\n schedule.run(&mut world);\n println!(\"ran\");\n}\nthread 'main' panicked at bevy_ecs-0.20.1/src/query/state.rs:216:13:\nerror[B0001]: <Enable the debug feature to see the name> in system <Enable the debug feature to see the name> accesses component(s) <Enable the debug feature to see the name> in a way that conflicts with a previous system parameter. Consider using `Without<T>` to create disjoint Queries or merging conflicting Queries into a `ParamSet`. See: https://bevy.org/learn/errors/b0001\nnote: run with `RUST_BACKTRACE=1` environment variable to display a backtraceThis is error B0001. The message hides the names unless you enable the crate's debug feature; do that in development, because \"accesses component(s) Health in a way that conflicts\" is the whole diagnosis. Rust's borrow checker would reject the equivalent code; in an ECS the check happens when the system parameters are validated. There are two fixes. If the queries should select disjoint entities, say so with filters: Query<&mut Health, Without<Player>> and Query<&Health, With<Player>>. If you really need both views of the same data, wrap the queries in a ParamSet and use them one at a time.
The same structure suits any program with many entities and cross-cutting behaviours: a discrete-event simulation (agents with different traits), a build system or scheduler (tasks with dependencies and states as components), an editor (document nodes with optional properties), a network simulator. The tests are easy to write: build a world, spawn entities, run a schedule once, assert on queries. That is one of the strongest reasons to use bevy_ecs directly. Alternatives exist with different trade-offs: hecs is smaller and has no scheduler, specs and shipyard take different approaches to scheduling. The related theory, laying data out for the cache, is in data-oriented design, and the system-and-query machinery in full Bevy is in systems, queries and resources.
Skip it when the program has a handful of distinct objects with fixed behaviour; a few structs and traits are simpler and faster to write. Skip it when the dominant cost is one numeric loop, as the benchmark shows. And be wary of it when your data is a graph or tree with strong parent-child ownership and few entities per type: an ECS can model hierarchy (Bevy has parent and child components), but it is no longer the natural fit. Use it where composition and many entities are the problem, and measure before assuming the layout makes the difference.
","date_published":"2026-11-03T09:36:00+00:00","tags":["Systems"]},{"id":"https://rusthacks.dev/library/rust-linux-daemon-systemd","url":"https://rusthacks.dev/library/rust-linux-daemon-systemd","title":"Writing a Linux Daemon in Rust With Systemd Notify","summary":"Under systemd a daemon does not fork or write pid files. It stays in the foreground, tells systemd when it is ready and exits cleanly on SIGTERM.","content_html":"A Linux service that systemd starts, supervises and stops has to follow a small set of rules, and getting them right is the difference between a daemon that restarts cleanly at three in the morning and one that leaves a half-started process behind. Rust makes the program side easy and leaves the contract with systemd to you. This guide builds the contract piece by piece: how systemd learns that your service is ready (the notify protocol), how it learns that you are still alive (the watchdog), how you reload without dropping connections, how you shut down on SIGTERM, how to take a listening socket from systemd instead of binding one, and which unit-file options turn a normal process into a locked-down one. The notify protocol is a datagram sent to a Unix socket, so it can be tested without systemd; the programs below were run on macOS against a stand-in socket that records the same messages systemd would receive, and compiled with Rust 1.99, tokio 1.54, sd-notify 0.5 and listenfd 1. The unit files are the standard directives from systemd.service(5); check them with systemd-analyze verify on your machine, since this guide could not run systemd itself.
The default service type, Type=simple, treats the service as started the moment the process is forked. That is almost never true: the process is still reading configuration, opening ports and warming caches. Anything ordered After= your service may start while you are not ready, and a deployment script that runs systemctl start returns before the port accepts connections. With Type=notify, systemd waits until the service sends READY=1 over the notification socket. The same channel carries status text, the watchdog heartbeat and reload and stop announcements. The protocol is a short list of KEY=VALUE lines in one datagram sent to the path in the NOTIFY_SOCKET environment variable:
| Message | Meaning | When to send |
|---|---|---|
READY=1 | Startup is finished; the service is serving | After the port is bound and the state loaded |
STATUS=text | Free-form status shown by systemctl status | Whenever the state changes in a way an operator cares about |
RELOADING=1 plus MONOTONIC_USEC=n | A reload is in progress | Immediately when a reload begins; send READY=1 when it ends |
STOPPING=1 | Shutdown has begun | On SIGTERM, before draining |
WATCHDOG=1 | Still alive | At least twice per WatchdogSec |
ERRNO=n, EXTEND_TIMEOUT_USEC=n | Failure code; ask for more time | Slow starts or stops |
A daemon is a loop that waits for one of three things: a termination signal, a reload signal, or time for the next watchdog ping. In async Rust, that is a select! over signal streams and an interval. This is the whole daemon, with the \"work\" left out so the lifecycle is visible:
async fn daemon() {\n let mut term = signal(SignalKind::terminate()).unwrap();\n let mut hup = signal(SignalKind::hangup()).unwrap();\n\n tokio::time::sleep(Duration::from_millis(300)).await;\n sd_notify::notify(&[NotifyState::Ready, NotifyState::Status(\"accepting connections\")]).unwrap();\n\n let mut watchdog = sd_notify::watchdog_enabled().map(|d| d / 2).map(tokio::time::interval);\n loop {\n tokio::select! {\n _ = term.recv() => {\n sd_notify::notify(&[NotifyState::Stopping, NotifyState::Status(\"draining\")]).unwrap();\n tokio::time::sleep(Duration::from_millis(100)).await;\n return;\n }\n _ = hup.recv() => {\n let usec = sd_notify::NotifyState::monotonic_usec_now().unwrap();\n sd_notify::notify(&[NotifyState::Reloading, usec]).unwrap();\n tokio::time::sleep(Duration::from_millis(50)).await;\n sd_notify::notify(&[NotifyState::Ready, NotifyState::Status(\"reloaded\")]).unwrap();\n }\n _ = async { match watchdog.as_mut() { Some(i) => { i.tick().await; } None => std::future::pending().await } } => {\n sd_notify::notify(&[NotifyState::Watchdog]).unwrap();\n }\n }\n }\n}\n\nWalk through what it does. It registers handlers for SIGTERM and SIGHUP first, so a signal arriving during startup is queued rather than killing the default way. The 300 ms sleep stands in for startup work. READY=1 and a status line are sent together in one datagram, which is what the crate's slice argument does. The watchdog interval is half the period systemd asked for, which is the documented rule of thumb: sd_notify::watchdog_enabled() returns Some(duration) only if WATCHDOG_USEC is set (and, with systemd, if WATCHDOG_PID names this process), so the daemon works unchanged outside systemd, where it simply does not ping. On SIGHUP it announces RELOADING=1 with the current monotonic time, which systemd requires for the notify-reload type, does the reload, and announces READY=1 again. On SIGTERM it announces STOPPING=1, drains, and returns, so the process exits with status 0.
To check that these messages are what systemd would see, a test harness plays systemd's part: it binds a datagram socket, starts the daemon as a child with NOTIFY_SOCKET pointing at it and a 400 ms watchdog, sends SIGHUP at 900 ms and SIGTERM at 1,500 ms, and logs everything it receives:
fn harness() {\n let path = std::env::temp_dir().join(format!(\"notify-{}.sock\", std::process::id()));\n let _ = std::fs::remove_file(&path);\n let sock = UnixDatagram::bind(&path).unwrap();\n sock.set_read_timeout(Some(Duration::from_millis(200))).unwrap();\n\n let exe = std::env::current_exe().unwrap();\n let mut child = Command::new(exe)\n .arg(\"daemon\")\n .env(\"NOTIFY_SOCKET\", &path)\n .env(\"WATCHDOG_USEC\", \"400000\")\n .env(\"WATCHDOG_PID\", \"\")\n .stdout(Stdio::null())\n .spawn()\n .unwrap();\n let pid = child.id() as i32;\n let t0 = Instant::now();\n\n let mut log: Vec<(u128, String)> = Vec::new();\n let mut buf = [0u8; 512];\n let mut sent_hup = false;\n let mut sent_term = false;\n loop {\n match sock.recv(&mut buf) {\n Ok(n) => log.push((t0.elapsed().as_millis() / 100 * 100, String::from_utf8_lossy(&buf[..n]).replace('\\n', \" | \"))),\n Err(_) => {}\n }\n let ms = t0.elapsed().as_millis();\n if ms > 900 && !sent_hup {\n unsafe { libc::kill(pid, libc::SIGHUP) };\n sent_hup = true;\n }\n if ms > 1500 && !sent_term {\n unsafe { libc::kill(pid, libc::SIGTERM) };\n sent_term = true;\n }\n if let Some(status) = child.try_wait().unwrap() {\n log.push((t0.elapsed().as_millis() / 100 * 100, format!(\"process exited: {status}\")));\n break;\n }\n }\n let _ = std::fs::remove_file(&path);\n let mut last = String::new();\n for (ms, line) in log {\n if line.starts_with(\"WATCHDOG=1\") && last == line {\n continue;\n }\n println!(\"{ms:>5} ms {line}\");\n last = line;\n }\n}\n\nfn main() {\n if std::env::args().nth(1).as_deref() == Some(\"daemon\") {\n tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap().block_on(daemon());\n } else {\n harness();\n }\n}\n300 ms READY=1 | STATUS=accepting connections | \n 300 ms WATCHDOG=1 | \n 900 ms RELOADING=1 | MONOTONIC_USEC=776077409717 | \n 900 ms READY=1 | STATUS=reloaded | \n 1100 ms WATCHDOG=1 | \n 1500 ms STOPPING=1 | STATUS=draining | \n 1700 ms process exited: exit status: 0The timeline is the lifecycle. At 300 ms the service reports ready with a status, then pings the watchdog (the repeated pings every 200 ms are collapsed in the log). At 900 ms the SIGHUP produced the two-message reload sequence, and the monotonic timestamp is a microsecond count that differs on each run. At 1,500 ms SIGTERM produced STOPPING=1, and 200 ms later the process exited with status 0. A harness like this is worth keeping as an integration test: it asserts the protocol without root, without a VM and without waiting for a real systemd. The one thing it cannot check is systemd's reaction, for instance that a missed watchdog really kills and restarts the service; that belongs in a staging test with systemd-run.
The unit file turns those behaviours on and adds the supervision policy. This one is a production-shaped starting point; the comments say what each line is for:
\n[Unit]\nDescription=My Rust service\nAfter=network-online.target\nWants=network-online.target\n\n[Service]\nType=notify\nNotifyAccess=main\nExecStart=/usr/local/bin/myapp --config /etc/myapp/config.toml\nExecReload=/bin/kill -HUP $MAINPID\n\n# Supervision\nRestart=on-failure\nRestartSec=2\nWatchdogSec=10\nTimeoutStartSec=30\nTimeoutStopSec=30\n\n# Identity and state\nDynamicUser=yes\nStateDirectory=myapp\nConfigurationDirectory=myapp\nLogsDirectory=myapp\n\n# Hardening\nNoNewPrivileges=yes\nProtectSystem=strict\nProtectHome=yes\nPrivateTmp=yes\nPrivateDevices=yes\nProtectKernelTunables=yes\nProtectControlGroups=yes\nRestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX\nCapabilityBoundingSet=\nMemoryMax=512M\n\n[Install]\nWantedBy=multi-user.targetRead it in three groups. Supervision. Type=notify with NotifyAccess=main means only the main process may send notifications, which stops a stray child from claiming readiness. Restart=on-failure restarts after a non-zero exit, a signal death, or a watchdog timeout, but not after a clean stop. WatchdogSec=10 promises systemd a ping at least every 10 seconds (the daemon sends one every five); if none arrives, systemd kills the service and the restart policy applies, which turns a deadlocked process into a restarted one. The two timeouts bound startup and shutdown, after which systemd sends SIGKILL. Identity. DynamicUser=yes allocates a transient unprivileged user for the service, and the three *Directory= options create directories under /var/lib, /etc and /var/log that the user owns, so the service never needs root and never needs a pre-created account. Hardening. NoNewPrivileges blocks privilege escalation via setuid binaries; ProtectSystem=strict makes the whole file system read-only except the directories you declare; PrivateTmp gives the service its own /tmp; CapabilityBoundingSet= with nothing after it drops every Linux capability; RestrictAddressFamilies forbids exotic socket types; MemoryMax turns a leak into an out-of-memory kill of this service only, instead of the host. Check how exposed a unit is with systemd-analyze security myapp.service, which scores each directive. Binding a port below 1024 under these rules needs either socket activation (next section) or AmbientCapabilities=CAP_NET_BIND_SERVICE.
Letting systemd create the listening socket and pass it to your process solves three problems at once. The service can run unprivileged yet listen on port 80, because systemd (root) bound it. Connections that arrive while the service is restarting queue in the kernel's accept backlog instead of being refused, so a restart or deploy drops nothing. And the service can start on demand. The unit pair is a socket unit and a service unit with the same name:
\n[Socket]\nListenStream=0.0.0.0:8080\nAccept=no\n\n[Install]\nWantedBy=sockets.targetOn start, systemd passes the socket as file descriptor 3 and sets LISTEN_FDS=1 and LISTEN_PID to your pid. The listenfd crate reads those and hands you a listener, with a fallback to binding yourself when the process was not started by systemd, so the same binary works in development:
use listenfd::ListenFd;\nuse std::net::TcpListener;\n\nfn listener() -> std::io::Result<TcpListener> {\n let mut fds = ListenFd::from_env();\n match fds.take_tcp_listener(0)? {\n Some(l) => {\n println!(\"using the socket systemd passed in (fd 3)\");\n Ok(l)\n }\n None => {\n println!(\"no inherited socket, binding 127.0.0.1:0 myself\");\n TcpListener::bind(\"127.0.0.1:0\")\n }\n }\n}\n\nfn main() -> std::io::Result<()> {\n let l = listener()?;\n println!(\"listening on {}\", l.local_addr()?.ip());\n Ok(())\n}\nRun outside systemd, it takes the fallback path, as the output shows when it ran here:
\nno inherited socket, binding 127.0.0.1:0 myself\nlistening on 127.0.0.1Under systemd the first branch runs and prints the message about the inherited socket, with the address configured in the socket unit. Convert the listener to Tokio with tokio::net::TcpListener::from_std after calling set_nonblocking(true). Because the socket outlives the process, a systemctl restart myapp never closes the port.
The daemon announces a reload, but the interesting part is what the reload does. Three rules make it safe: parse and validate the new configuration completely before touching the live one; replace the live configuration atomically, so no request ever sees half of the old and half of the new; and keep the old configuration if the new one is bad, reporting the error instead of exiting. A small shared structure delivers all three. Readers take an Arc snapshot and keep using it for the whole request, and the reload swaps the pointer under a write lock held for a few nanoseconds:
use std::sync::{Arc, RwLock};\n\n#[derive(Debug, Clone, PartialEq)]\nstruct Config {\n workers: usize,\n greeting: String,\n}\n\nfn parse(text: &str) -> Result<Config, String> {\n let mut workers = None;\n let mut greeting = None;\n for (n, line) in text.lines().enumerate().filter(|(_, l)| !l.trim().is_empty()) {\n let (key, value) = line.split_once('=').ok_or_else(|| format!(\"line {}: expected key=value\", n + 1))?;\n match key.trim() {\n \"workers\" => {\n let w: usize = value.trim().parse().map_err(|e| format!(\"line {}: workers: {e}\", n + 1))?;\n if !(1..=64).contains(&w) {\n return Err(format!(\"line {}: workers must be 1..=64, got {w}\", n + 1));\n }\n workers = Some(w);\n }\n \"greeting\" => greeting = Some(value.trim().to_string()),\n other => return Err(format!(\"line {}: unknown key {other:?}\", n + 1)),\n }\n }\n Ok(Config { workers: workers.ok_or(\"missing workers\")?, greeting: greeting.ok_or(\"missing greeting\")? })\n}\n\nstruct Shared {\n current: RwLock<Arc<Config>>,\n}\n\nimpl Shared {\n fn snapshot(&self) -> Arc<Config> {\n self.current.read().unwrap().clone()\n }\n\n fn reload(&self, text: &str) -> Result<(), String> {\n let new = parse(text)?;\n *self.current.write().unwrap() = Arc::new(new);\n Ok(())\n }\n}\n\nfn main() {\n let shared = Shared { current: RwLock::new(Arc::new(parse(\"workers = 4\\ngreeting = hello\").unwrap())) };\n\n let in_flight = shared.snapshot();\n println!(\"start: {:?}\", shared.snapshot());\n\n println!(\"reload ok: {:?}\", shared.reload(\"workers = 8\\ngreeting = hello again\"));\n println!(\"now serving: {:?}\", shared.snapshot());\n println!(\"old request still sees: {:?}\", in_flight);\n\n println!(\"reload broken: {:?}\", shared.reload(\"workers = 800\\ngreeting = x\"));\n println!(\"reload typo: {:?}\", shared.reload(\"worker = 2\"));\n println!(\"still serving: {:?}\", shared.snapshot());\n}\nstart: Config { workers: 4, greeting: \"hello\" }\nreload ok: Ok(())\nnow serving: Config { workers: 8, greeting: \"hello again\" }\nold request still sees: Config { workers: 4, greeting: \"hello\" }\nreload broken: Err(\"line 1: workers must be 1..=64, got 800\")\nreload typo: Err(\"line 1: unknown key \\\"worker\\\"\")\nstill serving: Config { workers: 8, greeting: \"hello again\" }Read the output as a sequence of promises. The snapshot taken before the reload (in_flight) still sees four workers after the swap, so a request that started under the old configuration finishes under it; no request ever observes an inconsistent mix. The valid reload took effect for new snapshots. The two bad reloads returned errors that name the line and the problem, and the service kept serving the previous configuration. In the daemon, the SIGHUP branch reads the file, calls reload, and on Err logs the message and sends STATUS=reload failed: ... followed by READY=1, so the operator sees the failure in systemctl status while production traffic continues. The lock is a plain RwLock because the critical section is a pointer copy; for a hot path with millions of reads per second, the arc-swap crate makes reads lock-free, a trade measured in sharing state between tasks.
What a reload cannot change is anything bound to a resource: the listening address, a TLS key already loaded into a handshake context, the number of worker threads in a started runtime. For those, the honest options are to document that they need a restart, or to implement the change by starting new resources, switching traffic and draining the old ones, which is a mini deployment inside the process. Keep the set of reloadable settings small and write it down in the unit's documentation; an operator who sends SIGHUP and discovers that the port did not change has been misled by the signal's name. A related system-level detail: systemd does not run ExecReload when the main process is not active, and treats a non-zero exit from the reload command as a failed reload, which is why the unit above uses kill -HUP (it exits 0 once the signal is delivered) and relies on the notify protocol for the outcome.
A binary that runs both on laptops and under systemd should adapt without flags. Systemd sets a few environment variables you can read. INVOCATION_ID exists for every service started by systemd and is unique per run, so it doubles as a correlation id in logs. JOURNAL_STREAM is set when stdout or stderr is connected to the journal, and its presence is the signal to drop timestamps and colour from your log format. NOTIFY_SOCKET and WATCHDOG_USEC, as above, enable the lifecycle messages, and the sd-notify crate already treats their absence as \"do nothing\", so the same code path is safe everywhere. LISTEN_FDS enables socket activation, handled by listenfd. Checking these is cheaper and more accurate than checking whether the parent process is pid 1.
Environment is also the right place for the deployment's secrets and paths, with EnvironmentFile=/etc/myapp/env in the unit (mode 0600, owned by root), or better LoadCredential=, which hands the service a file in a private directory named by $CREDENTIALS_DIRECTORY that is never visible to other users or to systemctl show. Reading a credential is a plain file read, so the Rust side stays simple, and it keeps keys out of the process environment where /proc/PID/environ and crash reporters can see them.
A service under systemd should not manage log files or rotate them. Write to standard output and standard error; the journal captures both, attaches the unit name, user and timestamps, and rotates by size. Read with journalctl -u myapp -f. Two refinements are worth making. Do not print timestamps yourself, since the journal adds them (configure your tracing subscriber with .without_time() when it detects JOURNAL_STREAM in the environment). And preserve severity: the journal understands a syslog-style prefix, so a line starting with <3> is stored as an error and <6> as info (the SyslogLevelPrefix default is on). The tracing-journald crate sends structured fields directly instead, which makes journalctl -o json output queryable by field. Keep one rule for secrets: nothing the service prints may contain them, because the journal is world-readable to members of the systemd-journal group; see logging without leaking secrets.
When you run systemctl stop, systemd sends SIGTERM to the main process, waits up to TimeoutStopSec, and then sends SIGKILL to everything left. Your job in that window is the drain: stop accepting, finish in-flight work, flush, exit with status 0. The pattern that does it with cancellation tokens and a deadline is in graceful shutdown, and the daemon above is its skeleton. Two details specific to systemd: the stop is announced with STOPPING=1 so that tools showing the state see \"deactivating\" with your status text, and exit status 0 after SIGTERM is what makes Restart=on-failure leave the service stopped. A process that exits non-zero because it caught the signal and returned an error would be restarted by the supervisor, which is rarely what someone typing systemctl stop wanted.
| Symptom | Cause | Fix |
|---|---|---|
systemctl start hangs for 90 seconds then fails with \"timeout\" | Type=notify but the process never sent READY=1 | Send it after startup; check NOTIFY_SOCKET is set and NotifyAccess allows your pid |
Service restarts every WatchdogSec | Pings stopped: a blocked runtime, or the ping interval is longer than half the period | Ping from a task that cannot be starved; send every half period; see blocking the runtime |
\"Failed to set up mount namespacing\" or permission denied under ProtectSystem | The service writes outside its declared directories | Use StateDirectory, RuntimeDirectory or ReadWritePaths= |
| Port below 1024 fails to bind | No capability, no socket activation | Use a .socket unit or AmbientCapabilities=CAP_NET_BIND_SERVICE |
| Reload kills connections | The reload restarts instead of re-reading | Handle SIGHUP in place; keep the listener and swap the config behind a lock or ArcSwap |
| Logs show twice or with double timestamps | The app adds its own timestamp and prefix | Turn them off when running under the journal |
| Service stays \"activating\" forever after SIGHUP | RELOADING=1 sent without READY=1 afterwards | Always pair them |
systemd-analyze verify /etc/systemd/system/myapp.service\nsystemd-analyze security myapp.service\nsystemd-run --user --collect --wait --property=Type=notify ./target/release/myapp\njournalctl -u myapp --since \"5 minutes ago\"The first catches typos and unknown directives, the second scores the sandbox and lists what is still exposed, and the third runs your binary under a transient unit with the same contract, a quick way to try the notify handshake on a laptop that runs systemd. In CI, run the harness from this guide to cover the part that is pure Rust.
\nInside a container, the container runtime is the supervisor: Docker and Kubernetes restart containers, send SIGTERM and read exit codes, but they ignore NOTIFY_SOCKET unless you run systemd inside the container, and their health model is probes, not the watchdog. The shutdown and signal handling carry over directly (see graceful shutdown on Kubernetes), the unit file does not. On macOS, launchd has its own plist format and no equivalent of the notify protocol, and on Windows a service uses the Service Control Manager API. If you target several, put the lifecycle behind a small trait (ready(), status(), stopping()) with a no-op implementation for platforms without it. For the systemd socket and watchdog features in more depth, see systemd sockets and watchdog.