Full Text Search in Rust With PostgreSQL or Tantivy

Search is two problems: finding documents that contain the words, and ordering them so the best come first. Both are solved well enough by PostgreSQL for many applications, and by a dedicated library when they are not. This post builds both and compares them. Versions: Rust 1.90 with edition 2024, SQLx 0.8, PostgreSQL 17, Tantivy 0.24.

PostgreSQL: tsvector, GIN and ranking

CREATE TABLE articles (
    id     BIGSERIAL PRIMARY KEY,
    title  TEXT NOT NULL,
    body   TEXT NOT NULL,
    search TSVECTOR GENERATED ALWAYS AS (
        setweight(to_tsvector('english', coalesce(title, '')), 'A') ||
        setweight(to_tsvector('english', coalesce(body, '')),  'B')
    ) STORED
);

CREATE INDEX articles_search_idx ON articles USING GIN (search);

The generated column keeps the vector current on every insert and update. Weights let a title match count more than a body match. The GIN index maps each lexeme to the rows that contain it, so lookups do not scan the table. Use the right text search configuration for your language; 'english' stems "running" to "run" and drops stop words.

use sqlx::PgPool;

#[derive(sqlx::FromRow, Debug)]
struct Hit {
    id: i64,
    title: String,
    rank: f32,
    snippet: String,
}

async fn search(pool: &PgPool, q: &str, limit: i64) -> Result<Vec<Hit>, sqlx::Error> {
    sqlx::query_as::<_, Hit>(
        r#"
        SELECT id, title,
               ts_rank(search, query) AS rank,
               ts_headline('english', body, query, 'MaxWords=20, MinWords=10') AS snippet
        FROM articles, websearch_to_tsquery('english', $1) AS query
        WHERE search @@ query
        ORDER BY rank DESC
        LIMIT $2
        "#,
    )
    .bind(q)
    .bind(limit)
    .fetch_all(pool)
    .await
}

websearch_to_tsquery parses user-style input: quoted phrases, or, and a leading minus to exclude a word, without throwing syntax errors on odd characters (unlike to_tsquery). @@ matches, ts_rank orders and ts_headline builds highlighted snippets. Bind the user's text as a parameter, never into the SQL string (SQL injection and typed queries).

What Postgres does not give you

• Typo tolerance. Add the pg_trgm extension for fuzzy matching on short fields, but it is a separate mechanism.
• Fine relevance control. Ranking is basic (term frequency and weights); no BM25 out of the box.
• Facets and aggregations over search results are possible in SQL but slower than purpose-built engines.
• Multilingual analysis requires choosing a configuration per row or column.

Tantivy: an embedded engine

use tantivy::{
    collector::TopDocs,
    doc,
    query::QueryParser,
    schema::{Schema, STORED, TEXT},
    Index, ReloadPolicy,
};

fn main() -> tantivy::Result<()> {
    let mut builder = Schema::builder();
    let title = builder.add_text_field("title", TEXT | STORED);
    let body = builder.add_text_field("body", TEXT);
    let schema = builder.build();

    let index = Index::create_in_ram(schema);
    let mut writer = index.writer(50_000_000)?; // 50 MB indexing buffer
    writer.add_document(doc!(title => "Ownership in Rust", body => "Each value has one owner."))?;
    writer.add_document(doc!(title => "Async basics", body => "Futures are lazy until awaited."))?;
    writer.commit()?;

    let reader = index.reader_builder().reload_policy(ReloadPolicy::OnCommitWithDelay).try_into()?;
    let searcher = reader.searcher();
    let parser = QueryParser::for_index(&index, vec![title, body]);
    let query = parser.parse_query("owner")?;
    for (score, addr) in searcher.search(&query, &TopDocs::with_limit(5))? {
        let doc: tantivy::TantivyDocument = searcher.doc(addr)?;
        println!("{score:.3} {:?}", doc.get_first(title));
    }
    Ok(())
}

Tantivy scores with BM25, supports phrase and fuzzy queries, per-field analyzers (including stemming and language-specific tokenizers), facets and fast fields for sorting. The index is a directory of segment files (use Index::create_in_dir), and a commit makes new documents visible to readers. A single writer holds a lock on the directory, so run indexing in one process.

The cost of a second store

A separate index is derived data. You must keep it in sync with the database: index after commit, handle failures, and be able to rebuild it from scratch. A reliable way is to write change events in the same transaction as the data and have a worker update the index (the outbox pattern). Back up or be able to regenerate the index, and plan reindexing for schema changes.

Choosing

Whatever you pick, build a small set of real queries with expected results and run them in tests, so analyzer or ranking changes show up as failing cases rather than complaints.