Axum Rate Limiting With Tower Governor and Redis Explained

A rate limit answers a question the load limits in timeouts and body size caps do not: not "how much can the server take at once" but "how much should any one client be allowed". It protects login forms from password guessing, expensive endpoints from scraping and your bill from a runaway client with a retry bug. The implementation is not hard, and what goes wrong is almost always the identity of the client. This post builds a token-bucket limiter as Axum middleware, shows a burst being cut off and the bucket refilling, then covers client keys behind a proxy, the memory a limiter needs, sharing the limit between instances with Redis and the ready-made crate. Compiled and run on axum 0.8.9 and tokio 1 with Rust 1.99.

Three algorithms, one choice

The token bucket is the usual default because it matches how clients behave: a page load fires six requests at once, then nothing for a minute. A fixed-window limit of 60 per minute treats that fine but allows a double burst at a window edge, and a strict per-second limit would reject the page load. A bucket with capacity 20 and a refill of 1 per second allows the burst and holds the long-run rate to 60 a minute. It also needs only two numbers per client, the current token count and the last update time, so memory is constant.

A limiter you can read in one screen

use std::{
    collections::HashMap,
    net::{IpAddr, SocketAddr},
    sync::{Arc, Mutex},
    time::Instant,
};

use axum::{
    Router,
    extract::{ConnectInfo, Request, State},
    http::{HeaderValue, StatusCode, header},
    middleware::{self, Next},
    response::{IntoResponse, Response},
    routing::get,
};

struct Bucket {
    tokens: f64,
    last: Instant,
}

#[derive(Clone)]
struct Limiter {
    buckets: Arc<Mutex<HashMap<IpAddr, Bucket>>>,
    capacity: f64,
    refill_per_sec: f64,
}

impl Limiter {
    fn new(capacity: u32, refill_per_sec: f64) -> Self {
        Limiter { buckets: Arc::default(), capacity: capacity as f64, refill_per_sec }
    }

    /// Ok(()) when allowed, Err(seconds until the next token) when not.
    fn check(&self, ip: IpAddr) -> Result<(), f64> {
        let mut map = self.buckets.lock().unwrap();
        let now = Instant::now();
        let b = map.entry(ip).or_insert(Bucket { tokens: self.capacity, last: now });
        let elapsed = now.duration_since(b.last).as_secs_f64();
        b.tokens = (b.tokens + elapsed * self.refill_per_sec).min(self.capacity);
        b.last = now;
        if b.tokens >= 1.0 {
            b.tokens -= 1.0;
            Ok(())
        } else {
            Err((1.0 - b.tokens) / self.refill_per_sec)
        }
    }
}

async fn rate_limit(
    State(limiter): State<Limiter>,
    ConnectInfo(addr): ConnectInfo<SocketAddr>,
    req: Request,
    next: Next,
) -> Response {
    match limiter.check(addr.ip()) {
        Ok(()) => next.run(req).await,
        Err(wait) => {
            let secs = wait.ceil().max(1.0) as u64;
            let mut res = (StatusCode::TOO_MANY_REQUESTS, "slow down").into_response();
            res.headers_mut().insert(header::RETRY_AFTER, HeaderValue::from(secs));
            res
        }
    }
}

#[tokio::main]
async fn main() {
    let limiter = Limiter::new(5, 1.0);
    let app = Router::new()
        .route("/api", get(|| async { "ok" }))
        .layer(middleware::from_fn_with_state(limiter, rate_limit));
    let listener = tokio::net::TcpListener::bind("127.0.0.1:3000").await.unwrap();
    axum::serve(listener, app.into_make_service_with_connect_info::<SocketAddr>())
        .await
        .unwrap();
}

for i in 1 2 3 4 5 6 7; do curl -s -o /dev/null -w "req$i %{http_code}\n" localhost:3000/api; done
curl -si localhost:3000/api | grep -iE 'HTTP|retry'
sleep 2.1
curl -s -o /dev/null -w "after 2s %{http_code}\n" localhost:3000/api
curl -s -o /dev/null -w "again %{http_code}\n" localhost:3000/api
curl -s -o /dev/null -w "again %{http_code}\n" localhost:3000/api

req1 200
req2 200
req3 200
req4 200
req5 200
req6 429
req7 429
HTTP/1.1 429 Too Many Requests
retry-after: 1
after 2s 200
again 200
again 429

Five tokens, five successes, then refusals. After two seconds two tokens had refilled, so the next two requests passed and the third did not. The refill is computed lazily on each request from the elapsed time, so there is no background task. Two details make it work in Axum. ConnectInfo<SocketAddr> only exists if the server is started with into_make_service_with_connect_info::<SocketAddr>(), as at the bottom of main; forget that and every request fails with a 500 saying the extension is missing. And the middleware takes its state through from_fn_with_state, the pattern from sharing state between handlers. The mutex is a std one held for a few arithmetic operations with no .await inside, the safe pattern from that post.

Who is the client

The limiter is only as good as its key. addr.ip() is the address of the TCP peer, and that is right when clients connect straight to you. Behind a load balancer or reverse proxy, the peer is always the proxy, so every user shares one bucket and the first busy minute locks everyone out. The usual reaction is to read X-Forwarded-For, and that is where a security bug appears: the header is supplied by whoever sent the request, so a client can write any address into it and get a fresh bucket for every request. The rule is to trust a forwarded header only when your own proxy sets it, and to read the right entry. A proxy appends the address it saw to the end of the list, so the entries nearest the client are the forgeable ones; take the address added by your outermost trusted proxy, counting from the right by the number of proxies you run. Better, configure the proxy to overwrite the header (or set X-Real-IP) with the peer it sees and ignore anything the client sent, and have Axum read only that one.

IPv6 changes the arithmetic. A single home connection is commonly given a /64 block with 2^64 addresses, so a limiter keyed on the full address lets one client rotate through effectively unlimited keys. Key IPv6 by its /64 prefix. IPv4 has the opposite problem: carrier-grade NAT can put thousands of mobile users behind one address, so a tight per-address limit punishes innocents. Keep the per-address limit generous and add a tighter one on something that identifies the actor better, once you know it: the authenticated user id, the API key or, for login, the account name being attacked. Limiting login attempts per account, regardless of the source address, is what stops a slow distributed guess against one password, and per-address limits are what stop one machine from trying many accounts. Use both, with the combined design in rate limiting and abuse prevention.

Memory and cleanup

The map above grows by one entry per distinct client forever, which is a memory leak an attacker can drive by spoofing addresses (impossible over TCP, easy through a forwarded header you trusted). Evict entries whose bucket would have refilled completely, since they are indistinguishable from a new client: spawn a task that runs every minute and removes any entry where last is older than capacity / refill_per_sec seconds. Alternatively cap the map size and evict the oldest entries; a bounded cache such as the one in moka does both with a time-to-idle setting, and is the better choice than hand-rolled eviction once the limiter matters. Contention is the other cost: one global mutex serializes every request through the limiter. At a few thousand requests a second it does not show; if it does, shard the map by hashing the key into N independent mutexes, or use the governor crate, which uses atomic operations and no lock.

One process or many

The limiter above lives in one process. Run four instances behind a load balancer and each keeps its own buckets, so a client spread across them gets four times the allowance. For a coarse protection that is often acceptable, because load balancers distribute roughly evenly and the multiplier is bounded. For a limit that must be exact, such as a paid API tier, or one that must hold per account for security, the count has to be shared, and Redis is the usual place. The simplest correct design is a fixed window counter with one atomic script, since INCR then EXPIRE as two commands can leave a key with no expiry if the process dies between them:

local current = redis.call("INCR", KEYS[1])
if current == 1 then
  redis.call("EXPIRE", KEYS[1], ARGV[1])
end
return current

Call it with the key rl:login:[email protected] and a window of 60 seconds; if the returned count exceeds the limit, answer 429 and use TTL on the key for Retry-After. The script runs atomically on the server, which closes the race, and the cost is one round trip per request. That adds latency on every call and makes Redis a dependency of the request path, so decide what happens when it is down. For a login limit, fail closed (refuse the login) is safer; for a general API limit, fail open (allow) keeps the service up. Connection handling for the client is covered in the Redis connection manager guide. For a smoother limit than a fixed window, a token bucket also fits in a Lua script, storing the tokens and timestamp in a hash.

A ready-made layer

The tower_governor crate wraps the governor rate limiter as a Tower layer for Axum: you give it a per-second rate and a burst size, it keys on the peer address by default and provides key extractors for forwarded headers or custom keys, and it returns 429 with Retry-After. It is lock-free, well-tested and the right default for a service that wants limiting this week and not a design project. Check its release notes for the API of the version you pin, since the builder methods and the layer constructor have changed between releases. Use your own middleware, as above, when the key is not an address (a user id taken from a validated token) or when you need to vary the limit per route, say 5 per minute on /login and 100 on the rest, which you do by applying a separate layer to the sensitive sub-router rather than one global rule.

What a well-behaved 429 looks like

Send Retry-After in seconds (as above), and consider the draft RateLimit-Limit, RateLimit-Remaining and RateLimit-Reset headers on every response so clients can pace themselves without hitting the wall. Return a body in your usual error shape (error handling with IntoResponse). Log limit hits with the key and route, since a spike of 429s on /login is an attack signal worth an alert, and place the limiter outside authentication so unauthenticated floods are cheap to refuse, but inside the CORS layer so browsers can read the 429 (CORS with Tower HTTP explains why).

When a rate limit is not the right control

A rate limit does not stop a determined attacker with many addresses, and it does not stop a single expensive request. For the first, put a CDN or web application firewall in front; for the second, bound the work itself with the limits in timeouts and body size caps. For internal services calling each other, prefer client-side limits and backpressure to server-side 429s. And do not rely on a limiter to protect a password: slow hashing (Argon2) makes each guess expensive, so the two controls multiply. The arrangement of these middleware in one application is in an Axum production service.