Axum Integration Tests With a Real Database in Docker

A test suite that mocks the database proves that your mock works. The bugs that reach production live in the gap: a constraint nobody tested, a query that returns the rows in a different order, a migration that fails on a column default. This guide builds a suite for an Axum API that runs every test against a real PostgreSQL in Docker, and measures what each isolation strategy costs so you can choose with numbers. The router and database layer follow Axum and SQLx together; the in-process request technique is from testing Axum with oneshot.

The code below was compiled and run on Rust 1.99, edition 2024, with axum 0.8.9, sqlx 0.9.0, testcontainers 0.27 and testcontainers-modules 0.15 starting postgres:17-alpine.

The application under test

It is deliberately small, a to-do API with three routes, so the testing machinery stays in view. The schema leans on the database for validation, with a CHECK constraint on the title:

CREATE TABLE todos (
    id    BIGSERIAL PRIMARY KEY,
    title TEXT NOT NULL CHECK (length(title) > 0),
    done  BOOLEAN NOT NULL DEFAULT false
);

The router creates a todo, completes it and lists all of them:

use axum::{
    Json, Router,
    extract::{Path, State},
    http::StatusCode,
    routing::{get, post},
};
use serde::{Deserialize, Serialize};
use sqlx::PgPool;

#[derive(Serialize, sqlx::FromRow)]
pub struct Todo {
    id: i64,
    title: String,
    done: bool,
}

#[derive(Deserialize)]
struct NewTodo {
    title: String,
}

async fn create(
    State(pool): State<PgPool>,
    Json(new): Json<NewTodo>,
) -> Result<(StatusCode, Json<Todo>), StatusCode> {
    let todo = sqlx::query_as::<_, Todo>(
        "INSERT INTO todos (title) VALUES ($1) RETURNING id, title, done",
    )
    .bind(new.title)
    .fetch_one(&pool)
    .await
    .map_err(|e| match e {
        // The CHECK constraint is the validation; its failure is the client's mistake.
        sqlx::Error::Database(db) if db.is_check_violation() => StatusCode::UNPROCESSABLE_ENTITY,
        _ => StatusCode::INTERNAL_SERVER_ERROR,
    })?;
    Ok((StatusCode::CREATED, Json(todo)))
}

async fn complete(State(pool): State<PgPool>, Path(id): Path<i64>) -> StatusCode {
    match sqlx::query("UPDATE todos SET done = true WHERE id = $1")
        .bind(id)
        .execute(&pool)
        .await
    {
        Ok(r) if r.rows_affected() == 1 => StatusCode::NO_CONTENT,
        Ok(_) => StatusCode::NOT_FOUND,
        Err(_) => StatusCode::INTERNAL_SERVER_ERROR,
    }
}

async fn list(State(pool): State<PgPool>) -> Result<Json<Vec<Todo>>, StatusCode> {
    sqlx::query_as::<_, Todo>("SELECT id, title, done FROM todos ORDER BY id")
        .fetch_all(&pool)
        .await
        .map(Json)
        .map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)
}

pub fn app(pool: PgPool) -> Router {
    Router::new()
        .route("/todos", get(list).post(create))
        .route("/todos/{id}/done", post(complete))
        .with_state(pool)
}

Two handlers show why a real database matters. create does not check whether the title is empty; the CHECK (length(title) > 0) constraint does, and the handler translates the violation (is_check_violation()) into a 422. A mocked repository would have to reimplement that rule to test it, and then you are testing the copy. complete uses rows_affected() to tell "updated one row" from "no such row", which a mock would also have to fake.

Dependencies for the harness

[dependencies]
axum = "0.8.9"
serde = { version = "1.0.229", features = ["derive"] }
serde_json = "1.0.154"
sqlx = { version = "0.9.0", features = ["runtime-tokio", "postgres", "migrate"] }
tokio = { version = "1.54.1", features = ["full"] }

[dev-dependencies]
dtor = "1.0.6"
http-body-util = "0.1.5"
testcontainers = "0.27"
testcontainers-modules = { version = "0.15.0", features = ["postgres"] }
tower = { version = "0.5.3", features = ["util"] }

testcontainers starts and stops Docker containers from Rust, and testcontainers-modules provides ready-made images, Postgres here. The first thing worth knowing is a version coupling that cargo reports only as a resolution failure: testcontainers-modules 0.15 requires testcontainers 0.27, so adding the newest testcontainers beside it fails with "failed to select a version". Pin testcontainers = "0.27" to match. dtor is for cleanup, explained below.

One container, a database per test

Starting a container is the slow part, so it happens once per test binary. Each test then creates its own database inside that container. The whole harness lives in tests/common/mod.rs:

#![allow(dead_code)] // each test binary uses a different subset of these helpers

use std::sync::{
    OnceLock,
    atomic::{AtomicU32, Ordering},
};

use axum::{Router, body::Body, http::Request};
use http_body_util::BodyExt;
use sqlx::{PgPool, postgres::PgPoolOptions};
use testcontainers::{ContainerAsync, ImageExt, runners::AsyncRunner};
use testcontainers_modules::postgres::Postgres;
use tokio::sync::OnceCell;
use tower::ServiceExt;

/// One container for the whole test binary; each test gets its own database inside it.
struct Server {
    // Held so the container lives as long as the process.
    _container: ContainerAsync<Postgres>,
    host: String,
    port: u16,
}

static SERVER: OnceCell<Server> = OnceCell::const_new();
static CONTAINER_ID: OnceLock<String> = OnceLock::new();

/// A static is never dropped, so the container would outlive the test run.
/// This runs when the test process exits and removes it.
#[dtor::dtor(unsafe)]
fn remove_container() {
    if let Some(id) = CONTAINER_ID.get() {
        let _ = std::process::Command::new("docker")
            .args(["rm", "-f", id])
            .output();
    }
}

async fn server() -> &'static Server {
    SERVER
        .get_or_init(|| async {
            // The module's default tag is an old release; pin the major you run in production.
            let container = Postgres::default()
                .with_tag("17-alpine")
                .start()
                .await
                .expect("start postgres (is Docker running?)");
            let _ = CONTAINER_ID.set(container.id().to_string());
            let host = container.get_host().await.unwrap().to_string();
            let port = container.get_host_port_ipv4(5432).await.unwrap();
            Server {
                _container: container,
                host,
                port,
            }
        })
        .await
}

fn url(srv: &Server, database: &str) -> String {
    format!(
        "postgres://postgres:postgres@{}:{}/{database}",
        srv.host, srv.port
    )
}

/// A freshly migrated database that no other test can see.
pub async fn fresh_pool() -> PgPool {
    create_database(None).await
}

/// The same, copied from a template that was migrated once. Skips running the migrations.
pub async fn fresh_pool_from_template() -> PgPool {
    static TEMPLATE: OnceCell<String> = OnceCell::const_new();
    let template = TEMPLATE
        .get_or_init(|| async {
            let pool = create_database(None).await;
            let name: String = sqlx::query_scalar("SELECT current_database()")
                .fetch_one(&pool)
                .await
                .unwrap();
            // A template cannot be copied while anyone is connected to it.
            pool.close().await;
            name
        })
        .await;
    create_database(Some(template)).await
}

async fn create_database(template: Option<&str>) -> PgPool {
    static NEXT: AtomicU32 = AtomicU32::new(0);

    let srv = server().await;
    let name = format!(
        "t_{}_{}",
        std::process::id(),
        NEXT.fetch_add(1, Ordering::Relaxed)
    );

    let admin = PgPoolOptions::new()
        .max_connections(1)
        .connect(&url(srv, "postgres"))
        .await
        .unwrap();

    // `name` and `template` are built here from a pid and a counter, never from input.
    let statement = match template {
        Some(t) => format!("CREATE DATABASE {name} TEMPLATE {t}"),
        None => format!("CREATE DATABASE {name}"),
    };
    sqlx::query(sqlx::AssertSqlSafe(statement))
        .execute(&admin)
        .await
        .unwrap();

    let pool = PgPoolOptions::new()
        .max_connections(4)
        .connect(&url(srv, &name))
        .await
        .unwrap();
    if template.is_none() {
        sqlx::migrate!("./migrations").run(&pool).await.unwrap();
    }
    pool
}

pub async fn send(app: &Router, req: Request<Body>) -> (u16, String) {
    let res = app.clone().oneshot(req).await.unwrap();
    let status = res.status().as_u16();
    let bytes = res.into_body().collect().await.unwrap().to_bytes();
    (status, String::from_utf8(bytes.to_vec()).unwrap())
}

pub fn json(method: &str, uri: &str, body: &str) -> Request<Body> {
    Request::builder()
        .method(method)
        .uri(uri)
        .header("content-type", "application/json")
        .body(Body::from(body.to_owned()))
        .unwrap()
}

Read it in four pieces.

server() lazily starts the container the first time any test asks for it, using tokio::sync::OnceCell so concurrent tests wait for one startup instead of racing to start many. It stores the host and the mapped port; Docker assigns a random host port, which is what keeps this from colliding with a Postgres already running on your machine.

create_database connects to the maintenance database as the superuser, runs CREATE DATABASE t_<pid>_<n> with a counter for the name, connects a pool to the new database and runs the project's migrations with sqlx::migrate!. Every test therefore starts from the exact schema production gets, built by the exact same files. Because each test has a database of its own, the suite can run on all cores without --test-threads=1 and without any cleanup between tests.

send and json are the small helpers every test needs: build a request, run it through the router with oneshot, return the status and body as strings.

The pinned tag. The line .with_tag("17-alpine") is not decoration. Without it, the Postgres module started postgres:11-alpine: the containers that appeared in docker ps during the first run carried that tag. Version 11 reached end of life in 2023 and behaves differently from current releases in places that matter (generated columns, MERGE, many planner details). A suite that tests against a database you do not run in production gives false confidence in both directions. Pin the major version you deploy.

The failure that shows up with dynamic SQL

The first draft of create_database passed the CREATE DATABASE string straight to sqlx::query. SQLx 0.9 refuses a runtime-built string:

error[E0277]: dynamic SQL strings should be audited for possible injections
   --> tests/common/mod.rs:57:17
    |
 57 |     sqlx::query(&format!("CREATE DATABASE {name}"))
    |     ----------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ dynamic SQL string

CREATE DATABASE cannot take a bind parameter, so a formatted string is the only way, and SQLx wants you to say that you looked. sqlx::AssertSqlSafe(...) is that statement. It is honest here because the name is built from a process id and a counter, never from user input. The comment above the call records why, which is the part a reviewer needs.

Write the tests

With the harness in place the tests read like the specification:

mod common;

use common::{fresh_pool, json, send};

#[tokio::test]
async fn create_complete_list() {
    let app = todos::app(fresh_pool().await);

    let (status, body) = send(&app, json("POST", "/todos", r#"{"title":"write tests"}"#)).await;
    assert_eq!(
        (status, body.as_str()),
        (201, r#"{"id":1,"title":"write tests","done":false}"#)
    );

    assert_eq!(send(&app, json("POST", "/todos/1/done", "")).await.0, 204);

    let (_, body) = send(&app, json("GET", "/todos", "")).await;
    assert_eq!(body, r#"[{"id":1,"title":"write tests","done":true}]"#);
}

#[tokio::test]
async fn empty_title_is_rejected_by_the_database() {
    let app = todos::app(fresh_pool().await);
    let (status, _) = send(&app, json("POST", "/todos", r#"{"title":""}"#)).await;
    assert_eq!(status, 422);
}

#[tokio::test]
async fn completing_a_missing_todo_is_404() {
    let app = todos::app(fresh_pool().await);
    assert_eq!(send(&app, json("POST", "/todos/42/done", "")).await.0, 404);
}

#[tokio::test]
async fn tests_do_not_see_each_others_rows() {
    let a = todos::app(fresh_pool().await);
    let b = todos::app(fresh_pool().await);

    send(&a, json("POST", "/todos", r#"{"title":"only in a"}"#)).await;
    let (_, body) = send(&b, json("GET", "/todos", "")).await;
    assert_eq!(body, "[]");
}

#[tokio::test]
async fn concurrent_creates_get_distinct_ids() {
    let app = todos::app(fresh_pool().await);

    let mut handles = Vec::new();
    for n in 0..20 {
        let app = app.clone();
        handles.push(tokio::spawn(async move {
            let body = format!(r#"{{"title":"task {n}"}}"#);
            send(&app, json("POST", "/todos", &body)).await
        }));
    }
    for handle in handles {
        assert_eq!(handle.await.unwrap().0, 201);
    }

    let (_, body) = send(&app, json("GET", "/todos", "")).await;
    let rows: Vec<serde_json::Value> = serde_json::from_str(&body).unwrap();
    let mut ids: Vec<i64> = rows.iter().map(|r| r["id"].as_i64().unwrap()).collect();
    ids.sort();
    assert_eq!(ids, (1..=20).collect::<Vec<_>>());
}

#[tokio::test]
async fn migrations_produce_the_schema_the_code_expects() {
    let pool = fresh_pool().await;

    let columns: Vec<(String, String, String)> = sqlx::query_as(
        "SELECT column_name, data_type, is_nullable
           FROM information_schema.columns
          WHERE table_name = 'todos'
          ORDER BY ordinal_position",
    )
    .fetch_all(&pool)
    .await
    .unwrap();

    let columns: Vec<(&str, &str, &str)> = columns
        .iter()
        .map(|(a, b, c)| (a.as_str(), b.as_str(), c.as_str()))
        .collect();
    assert_eq!(
        columns,
        [
            ("id", "bigint", "NO"),
            ("title", "text", "NO"),
            ("done", "boolean", "NO")
        ]
    );
}

test empty_title_is_rejected_by_the_database ... ok
test completing_a_missing_todo_is_404 ... ok
test migrations_produce_the_schema_the_code_expects ... ok
test create_complete_list ... ok
test tests_do_not_see_each_others_rows ... ok
test concurrent_creates_get_distinct_ids ... ok
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.51s

create_complete_list walks the happy path through all three routes and asserts exact JSON, including the id 1, which is only deterministic because the database is new. empty_title_is_rejected_by_the_database posts an empty title and expects 422; the proof that the constraint, not Rust code, is doing the work is that the handler never looks at the title. completing_a_missing_todo_is_404 pins the rows_affected branch. concurrent_creates_get_distinct_ids and migrations_produce_the_schema_the_code_expects are explained after the harness notes below. The tests_do_not_see_each_others_rows test is a test of the harness itself: two apps, two databases, and a row written through one must not appear in the other. If someone later "optimises" the harness to share a database, this is the test that goes red.

All six tests finished in 1.51 seconds including the container start, on a machine where the Postgres image was already pulled. The first run on a clean machine is slower by whatever the image pull costs.

Containers outlive tests unless you remove them

This is the part most guides skip. Run the suite twice and look at Docker:

cargo test --test todos
docker ps --format '{{.Image}} {{.Status}}'

Without the cleanup hook, each run left a postgres container running after the test process exited. Over a day of editing and re-running, nine of them accumulated on the machine this was written on, each holding memory. The reason is the static in server(). Rust does not run destructors for statics at process exit, so the ContainerAsync stored in the OnceCell is never dropped, and its Drop impl is what removes the container.

The fix in the harness is remove_container, registered with #[dtor::dtor(unsafe)]. It runs when the process exits, reads the container id saved in a OnceLock<String> and calls docker rm -f on it. After adding it, docker ps shows no container once the tests finish. The unsafe in the attribute is the crate's way of saying the function runs outside the normal Rust runtime guarantees, which is why it only shells out and touches nothing else. The older ctor::dtor spelling no longer exists in ctor 1.0: the destructor macro moved to the separate dtor crate, and using the old path fails with "could not find dtor in ctor".

If your CI runs each job in a fresh VM, the leak never matters there, but it matters on every developer machine and on shared runners.

Which isolation strategy

A database per test is the strongest isolation, and it costs time. Three strategies are in common use, and tests/timing.rs measures them on the same container:

container start + first database :  1160.1 ms
new database + migrate (avg of 20) :   140.2 ms
database from template (avg of 20):   183.0 ms
TRUNCATE ... RESTART IDENTITY     :     4.3 ms
BEGIN, insert, ROLLBACK           :     1.2 ms

(The figures are from the first of three runs; the full set varied from 132 to 240 ms for a new database and 3.5 to 4.3 ms for TRUNCATE, so read them as magnitudes.)

The numbers argue for a decision rule more than a winner. At 200 tests, a new database each costs roughly 30 to 50 seconds of cumulative time, but that spreads across cores because the tests run in parallel, so wall-clock cost is far lower. TRUNCATE is 40 times cheaper per test but forces serial execution, which usually loses overall. Rollback is the fastest and the weakest: code under test that calls commit() or starts its own transaction escapes the wrapper, so the rollback strategy cannot test it.

An obvious optimisation did nothing. CREATE DATABASE ... TEMPLATE migrated_db skips running the migrations by copying a prepared database, and it is supposed to be faster. Measured here it cost 113 to 238 ms, the same as migrating from scratch, because the time is in CREATE DATABASE itself, not in the migration files. With a schema of one table the migration step is trivial, and with a large schema the template might start to win; measure before adopting it. The hook for it is in the harness (fresh_pool_from_template), so you can run the same comparison on your own schema.

Two tests only a real database can write

Real Postgres earns its container on tests like these, which a mock cannot express at all.

concurrent_creates_get_distinct_ids fires twenty POST /todos requests at once from separate Tokio tasks, through one router and one pool of four connections. It then lists the rows and demands ids 1 through 20, exactly once each. The sequence behind BIGSERIAL is what guarantees that, so the test is checking the database and the pool more than the handler, but that is the point: it fails if someone changes the id scheme to something computed in Rust (max(id) + 1), and it would show a pool misconfiguration as timeouts. Twenty tasks against four connections also exercises the queueing path of the pool, which a single sequential request never reaches.

migrations_produce_the_schema_the_code_expects reads information_schema.columns and compares the table to the shape the Rust code assumes. It looks like overkill until the day a migration is edited after being applied somewhere, or a column loses its NOT NULL in a refactor. Remove NOT NULL from the title in the migration and the test names the difference precisely:

thread 'migrations_produce_the_schema_the_code_expects' panicked at tests/todos.rs:85:5:
assertion `left == right` failed
  left: [("id", "bigint", "NO"), ("title", "text", "YES"), ("done", "boolean", "NO")]
 right: [("id", "bigint", "NO"), ("title", "text", "NO"), ("done", "boolean", "NO")]

Note what the failure message is: a tuple that says "title", "text", "YES" where "NO" was expected. You do not have to know SQL to read it. A suite of a dozen such small assertions about the schema is cheap, runs in the time of one request, and catches the migration mistakes that otherwise surface as a NULL in production data.

Debug a failing database test

When a test fails you want to look at the data, and with a database per test that data is gone the moment the container is removed. Two habits help. First, put the database name in the assertion messages or print it, since each name carries the process id and a counter, so a run's output tells you which database belonged to which test. Second, keep the container alive while you investigate: comment out the remove_container call, run the single failing test with cargo test --test todos failing_name -- --nocapture, and connect with psql on the mapped port that docker ps shows. The databases are still there, named as printed. Remove the container by hand afterwards; the cleanup you disabled is the thing that normally does it.

If a test hangs rather than fails, suspect the pool. A test that holds a transaction open while another test's connection waits for the same row will block until the pool's acquire timeout. With a database per test the lock cannot cross tests, so a hang is almost always inside one test: an unfinished transaction, or a pool of size 1 that the test itself exhausted.

When Docker is not there

On a machine without a running Docker daemon the failure is clear, as long as you wrote a clear message. The harness uses .expect("start postgres (is Docker running?)"), and with DOCKER_HOST pointing at a missing socket the test output is:

---- empty_title_is_rejected_by_the_database stdout ----

thread 'empty_title_is_rejected_by_the_database' (18015528) panicked at tests/common/mod.rs:29:18:
start postgres (is Docker running?): Client(Init(SocketNotFoundError("/tmp/nope.sock")))
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Mark the database suite so it is easy to skip: put it in its own test binary (as tests/todos.rs is), and developers without Docker run cargo test --lib while CI runs everything. Alternatively use sqlx::test against a Postgres service the CI job provides, as in the Axum and SQLx guide; it needs DATABASE_URL rather than Docker access from the tests.

Running it in CI

A hosted runner with Docker needs nothing from the project except the image, so the one useful step is to pull it before the tests, where a slow registry shows up as its own line instead of a mysteriously slow test:

docker pull postgres:17-alpine
cargo test --test todos

Two details are worth fixing in the job rather than discovering later. The first is parallelism between test binaries: cargo runs the binaries one after another by default, which means one container at a time, but cargo nextest runs tests from different binaries concurrently and will start several containers. That is correct behaviour and it is faster; check that the runner has the memory for it, since a small Alpine Postgres uses around 40 MB idle and grows with connections. The second is the hook that removes the container: it relies on the test process exiting normally. A job cancelled by the CI system sends a signal, dtor does not run, and the container is left on a runner that is about to be thrown away anyway. On self-hosted runners that are not thrown away, add a final docker ps -q --filter ancestor=postgres:17-alpine | xargs -r docker rm -f step that runs even on cancellation, or you rebuild the leak the hook was meant to fix.

Keep the image tag in one place. The harness pins 17-alpine; the production compose file and the CI service should name the same major version, and when you upgrade the database you change all three together, with the suite as the proof that the upgrade works.

Habits that keep the suite fast and honest

• Keep one test binary per concern. Each file in tests/ is its own binary and therefore its own container. Three binaries mean three startups. Group tests that need the database into one file or use a shared helper binary.
• Assert on exact bodies for the API contract, and on counts or sets where order is not specified. A SELECT without ORDER BY returns rows in storage order, which can differ between runs.
• Seed through the API or through small SQL fixtures, not by sharing a dump. A dump drifts from the migrations.
• Run the migrations from the binary's own sqlx::migrate!. If tests use a different path to build the schema than production, you are testing a different schema.

Where this setup is the wrong choice

If your test machines cannot run Docker (some locked-down CI, many Windows laptops), Testcontainers is the wrong tool and a managed service plus sqlx::test is simpler. If the code under test is a pure function that parses or formats, a database adds nothing; the rules in Rust testing about unit tests still apply. And if you only need isolated state for a prototype, SQLite in memory is a legitimate answer, as long as you accept it will not catch Postgres-specific behaviour. The longer treatment of container based tests is in Postgres with Testcontainers.

Next steps

The order to adopt this in an existing service is to start with the schema test, since it is the cheapest and finds the most embarrassing problems, then move the three or four tests that currently use mocks onto the harness, and only then consider changing the isolation strategy. Add a second test binary and watch the container count; add a fixture that inserts 10,000 rows and see which isolation strategy changes cost; and wire cargo test into CI with a cached image, so the pull is not part of the build.