Ignoring and Filtering Tests in Rust for Slow Suites

A suite grows slow tests, tests that need a database, and tests that only make sense on one platform. Running all of them on every save is what makes people stop running tests. Rust gives you #[ignore] and a set of filters to run exactly the subset you want. This post lists them and the convention that keeps ignored tests alive. It is the last piece of the running-tests advice in the testing in Rust guide.

Rust 1.90.

Ignore with a reason

#[test]
#[ignore = "needs a running PostgreSQL on localhost:5432"]
fn migrates_a_real_database() {
    // ...
}

#[test]
fn parses_a_header() {
    assert!(true);
}

cargo test prints:

test migrates_a_real_database ... ignored, needs a running PostgreSQL on localhost:5432
test parses_a_header ... ok

The reason is part of the report, so a skipped test is never mysterious.

Run them on purpose

cargo test -- --ignored            # only ignored tests
cargo test -- --include-ignored    # everything

The pair is useful in CI: a fast job runs the default set on every push, and a slower job with the database service runs --ignored.

Filter by name

cargo test parser                  # names containing "parser"
cargo test parser::tests::header   # more specific path
cargo test -- --exact tests::parses_a_header
cargo test -- --skip slow --skip network

The filter matches against the full path of the test, module::tests::name, as a substring. Organizing tests into modules (mod parser_tests, mod db_tests) gives you cheap handles to filter on. --skip is repeatable and combines with a filter.

Select by binary:

cargo test --lib
cargo test --test integration
cargo test --doc
cargo test -p core-logic

Conditional ignoring

#[test]
#[cfg_attr(not(feature = "db-tests"), ignore = "enable the db-tests feature")]
fn needs_the_database() {}

A feature can also pull in dependencies that the tests need, such as sqlx. Environment checks inside the test work too, with a visible message:

#[test]
fn needs_network() {
    if std::env::var_os("RUN_NETWORK_TESTS").is_none() {
        eprintln!("skipping: set RUN_NETWORK_TESTS=1");
        return;
    }
}

That test reports ok while doing nothing, which is a quiet lie in the results; prefer #[ignore], which says so.

Stopping ignored tests from rotting

An ignored test that never runs becomes wrong without anyone noticing. Run --ignored on a schedule, or on pull requests that touch the relevant code. Review the list of ignored tests once in a while with cargo test -- --list --ignored. Delete ignored tests that nobody will ever run.

With nextest

Nextest has its own filter language that selects by package, binary and name (cargo nextest). The runner flags are documented in the Rust book on running tests.

Limits

Ignoring is not a fix for a flaky test. If a test fails randomly, find the cause (flaky tests and seven usual causes) before parking it. A suite with many ignored tests is a suite that is lying about its coverage.