Vector Search From Rust With pgvector and Embeddings

A search box that understands meaning, a "related posts" panel, a retrieval step for a question-answering bot: all of them reduce to the same operation. Turn each document into a list of numbers (an embedding) such that similar documents get nearby lists, store the lists, and when a query arrives, turn it into a list too and ask for the closest stored ones. PostgreSQL does this with the pgvector extension, which adds a vector column type, distance operators and approximate indexes, so the vectors live next to the rest of your data, in transactions, with your backups. This guide builds the whole path from Rust: an extension in a container, a table, inserts, the three distance operators and how they relate, a measured comparison of exact search against an HNSW index (including how much accuracy you give up for speed), the filtered-search trap that returns zero rows, and the operational facts that decide whether it stays fast. Everything ran with Rust 1.99, SQLx 0.9.0, the pgvector crate 0.4.2 and PostgreSQL 17 with pgvector 0.8.7, on an Apple M1 Max with the database in a container. One piece is deliberately a stand-in, the embedding model, and the text says where.

What an embedding search is

An embedding model maps text (or images, or audio) to a vector with a fixed number of dimensions, say 384 or 1,536, so that the geometric distance between two vectors reflects how similar their meanings are. "Rust async runtime" lands close to "tokio executor" and far from "sourdough starter". The database does not know anything about meaning. It stores vectors and answers "which stored vectors are nearest to this one?", and the quality of the answers is entirely the model's. That separation is useful for design: the database layer (this guide) is the same whichever model you pick, and you can swap models by re-computing the vectors.

Nearest means a choice of distance. pgvector offers three operators that you will use:

The negation in <#> exists because Postgres indexes return the smallest value first, so a larger inner product (more similar) has to sort as a smaller number. Follow the model's documentation: most text embedding models are trained for cosine similarity.

Set up the database

Run PostgreSQL in a container, add the extension and create it in your database. On the official Debian-based image the extension is a package from the Postgres apt repository that the image already trusts; there are also ready-made images with it built in:

docker run -d --name pg --shm-size=1g -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=app -p 55433:5432 postgres:17
docker exec pg bash -c "apt-get update -qq && apt-get install -y -qq postgresql-17-pgvector"
docker exec pg psql -U postgres -d app -c "CREATE EXTENSION vector;" -c "SELECT extversion FROM pg_extension WHERE extname = 'vector';"

Two details come from running it. --shm-size=1g is not decoration: Docker gives a container 64 MB of shared memory by default, and building a vector index in parallel needs more. With the default, the index build in this guide failed with could not resize shared memory segment ... No space left on device, an error that mentions disk and is actually about /dev/shm. The other is the extension version, 0.8.7 here; features such as iterative index scans (used below) need pgvector 0.8 or newer, so check extversion before you rely on them.

The Rust side

[dependencies]
sqlx = { version = "0.9", features = ["runtime-tokio", "postgres"] }
pgvector = { version = "0.4", features = ["sqlx"] }
tokio = { version = "1", features = ["full"] }

The pgvector crate provides a Vector type with SQLx Encode and Decode implementations behind its sqlx feature, so you can bind and read vectors like any other column type; it supports SQLx 0.8 and 0.9. The first program creates a table, embeds seven short documents, and queries with each operator:

use pgvector::Vector;
use sqlx::{AssertSqlSafe, PgPool};

const DIMS: usize = 64;

/// A toy embedding: hash each word into one of 64 buckets with a sign, then L2-normalise.
/// A real system would call an embedding model here; everything after this function is unchanged.
fn embed(text: &str) -> Vector {
    let mut v = vec![0f32; DIMS];
    for word in text.split(|c: char| !c.is_alphanumeric()).filter(|w| !w.is_empty()) {
        let mut h: u64 = 0xcbf29ce484222325;
        for b in word.to_lowercase().bytes() {
            h = (h ^ u64::from(b)).wrapping_mul(0x100000001b3);
        }
        let sign = if (h >> 63) & 1 == 0 { 1.0 } else { -1.0 };
        v[(h % DIMS as u64) as usize] += sign;
    }
    let norm = v.iter().map(|x| x * x).sum::<f32>().sqrt().max(1e-9);
    Vector::from(v.into_iter().map(|x| x / norm).collect::<Vec<f32>>())
}

#[tokio::main]
async fn main() -> Result<(), sqlx::Error> {
    let pool = PgPool::connect(&std::env::var("DATABASE_URL").unwrap()).await?;
    sqlx::query("CREATE EXTENSION IF NOT EXISTS vector").execute(&pool).await?;
    sqlx::query("DROP TABLE IF EXISTS docs").execute(&pool).await?;
    sqlx::query("CREATE TABLE docs (id bigserial PRIMARY KEY, body text NOT NULL, embedding vector(64) NOT NULL)").execute(&pool).await?;

    let docs = [
        "tokio is an async runtime for rust network services",
        "async await in rust compiles to state machines",
        "how to bake sourdough bread with a long cold ferment",
        "rust ownership and borrowing explained with examples",
        "postgres index types btree gin gist and hnsw",
        "a recipe for slow roasted tomato soup with basil",
        "vector search finds nearest neighbours by distance",
    ];
    for body in docs {
        sqlx::query("INSERT INTO docs (body, embedding) VALUES ($1, $2)").bind(body).bind(embed(body)).execute(&pool).await?;
    }

    for (label, op) in [("cosine distance  <=>", "<=>"), ("euclidean (L2)  <->", "<->"), ("negative inner  <#>", "<#>")] {
        let q = embed("async rust runtime");
        let sql = format!("SELECT body, embedding {op} $1 AS dist FROM docs ORDER BY embedding {op} $1 LIMIT 3");
        let rows: Vec<(String, f64)> = sqlx::query_as(AssertSqlSafe(sql)).bind(&q).fetch_all(&pool).await?;
        println!("{label}");
        for (body, dist) in rows {
            println!("  {dist:>7.4}  {body}");
        }
    }
    Ok(())
}

cosine distance  <=>
   0.4226  tokio is an async runtime for rust network services
   0.5918  async await in rust compiles to state machines
   0.7818  rust ownership and borrowing explained with examples
euclidean (L2)  <->
   0.9194  tokio is an async runtime for rust network services
   1.0879  async await in rust compiles to state machines
   1.2504  rust ownership and borrowing explained with examples
negative inner  <#>
  -0.5774  tokio is an async runtime for rust network services
  -0.4082  async await in rust compiles to state machines
  -0.2182  rust ownership and borrowing explained with examples

Look at the structure before the numbers. The embed function is the only model-dependent code; it takes text and returns a Vector. Here it is a toy: it hashes each word into one of 64 buckets with a random sign and normalises the result, which makes texts that share words land close together but knows nothing about synonyms. It exists so the example runs with no API key and no model download, and it is not a substitute: in a real system you replace the body with a call to an embedding service or a local model, keep the dimensions in sync with the column (vector(64) becomes vector(1536), say), and everything else in this guide is unchanged. The results are still sensible, because the query "async rust runtime" shares words with the first two documents, and the bread and soup recipes do not appear.

Now the numbers. All three operators ranked the same three documents in the same order, which is expected for unit vectors, and the values relate exactly: the first result has cosine distance 0.4226, so its inner product is 1 - 0.4226 = 0.5774, which is the (negated) value in the third block, and its Euclidean distance is the square root of 2 x 0.4226, which is 0.9194. For normalised vectors the three metrics are the same ranking in different units, so choose by speed and by what your index was built with. A second Rust detail from this build: SQLx 0.9 refuses a runtime-built SQL string at compile time ("dynamic SQL strings should be audited for possible injections"), because query() now only takes literals. The demo builds its query text from a fixed list of operators, so it wraps it in AssertSqlSafe(...), which is you promising that nothing from outside went into the string. Values always go through bind, never through the string.

Exact search is fine until it is not

Without an index, ORDER BY embedding <=> $1 LIMIT 10 compares the query with every row: an exact answer, and a cost that grows with the table. The benchmark loads 100,000 vectors of 128 dimensions, drawn from 100 clusters (real embeddings cluster; uniformly random points in high dimensions are a misleading test), and times 30 queries:

use pgvector::Vector;
use sqlx::{AssertSqlSafe, PgPool, QueryBuilder, Postgres, Row};
use std::time::Instant;

const DIMS: usize = 128;
const N: usize = 100_000;
const CLUSTERS: usize = 100;

struct Rng(u64);
impl Rng {
    fn next(&mut self) -> f32 {
        self.0 = self.0.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
        ((self.0 >> 40) as f32 / (1u64 << 24) as f32) * 2.0 - 1.0
    }
}

fn normalise(mut v: Vec<f32>) -> Vec<f32> {
    let n = v.iter().map(|x| x * x).sum::<f32>().sqrt();
    v.iter_mut().for_each(|x| *x /= n);
    v
}

fn point(rng: &mut Rng, centers: &[Vec<f32>]) -> Vec<f32> {
    let c = &centers[((rng.next() + 1.0) * 0.5 * CLUSTERS as f32) as usize % CLUSTERS];
    normalise(c.iter().map(|x| x + 0.35 * rng.next()).collect())
}

async fn top10(pool: &PgPool, q: &Vector) -> Result<Vec<i64>, sqlx::Error> {
    let rows = sqlx::query("SELECT id FROM items ORDER BY embedding <=> $1 LIMIT 10").bind(q).fetch_all(pool).await?;
    Ok(rows.iter().map(|r| r.get::<i64, _>(0)).collect())
}

#[tokio::main]
async fn main() -> Result<(), sqlx::Error> {
    let pool = PgPool::connect(&std::env::var("DATABASE_URL").unwrap()).await?;
    sqlx::query("CREATE EXTENSION IF NOT EXISTS vector").execute(&pool).await?;
    sqlx::query("DROP TABLE IF EXISTS items").execute(&pool).await?;
    sqlx::query("CREATE TABLE items (id bigserial PRIMARY KEY, embedding vector(128) NOT NULL)").execute(&pool).await?;

    let mut rng = Rng(42);
    let centers: Vec<Vec<f32>> = (0..CLUSTERS).map(|_| normalise((0..DIMS).map(|_| rng.next()).collect())).collect();

    let t = Instant::now();
    for _ in 0..N / 1000 {
        let mut qb: QueryBuilder<Postgres> = QueryBuilder::new("INSERT INTO items (embedding) ");
        qb.push_values(0..1000, |mut b, _| {
            b.push_bind(Vector::from(point(&mut rng, &centers)));
        });
        qb.build().execute(&pool).await?;
    }
    println!("inserted {N} vectors of {DIMS} dimensions in {:.1} s", t.elapsed().as_secs_f64());
    sqlx::query("ANALYZE items").execute(&pool).await?;

    let queries: Vec<Vector> = (0..30).map(|_| Vector::from(point(&mut rng, &centers))).collect();

    let t = Instant::now();
    let mut exact = Vec::new();
    for q in &queries {
        exact.push(top10(&pool, q).await?);
    }
    let seq = t.elapsed().as_secs_f64() * 1000.0 / queries.len() as f64;
    println!("exact (sequential scan): {seq:.1} ms per query");

    let t = Instant::now();
    sqlx::query("CREATE INDEX items_hnsw ON items USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 64)").execute(&pool).await?;
    println!("HNSW index built in {:.1} s", t.elapsed().as_secs_f64());
    let size: i64 = sqlx::query_scalar("SELECT pg_relation_size('items_hnsw')").fetch_one(&pool).await?;
    let table: i64 = sqlx::query_scalar("SELECT pg_relation_size('items')").fetch_one(&pool).await?;
    println!("table {:.0} MB, index {:.0} MB", table as f64 / 1048576.0, size as f64 / 1048576.0);

    for ef in [10, 40, 100, 400] {
        let mut conn = pool.acquire().await?;
        sqlx::query(AssertSqlSafe(format!("SET hnsw.ef_search = {ef}"))).execute(&mut *conn).await?;
        let t = Instant::now();
        let mut hits = 0;
        for (q, truth) in queries.iter().zip(&exact) {
            let rows = sqlx::query("SELECT id FROM items ORDER BY embedding <=> $1 LIMIT 10").bind(q).fetch_all(&mut *conn).await?;
            hits += rows.iter().filter(|r| truth.contains(&r.get::<i64, _>(0))).count();
        }
        let ms = t.elapsed().as_secs_f64() * 1000.0 / queries.len() as f64;
        println!("hnsw.ef_search = {ef:>3}: {ms:>5.2} ms per query, recall@10 = {:.1}%", hits as f64 * 100.0 / (queries.len() * 10) as f64);
    }
    Ok(())
}

inserted 100000 vectors of 128 dimensions in 0.7 s
exact (sequential scan): 12.8 ms per query
HNSW index built in 35.4 s
table 56 MB, index 79 MB
hnsw.ef_search =  10:  0.85 ms per query, recall@10 = 39.3%
hnsw.ef_search =  40:  1.05 ms per query, recall@10 = 67.3%
hnsw.ef_search = 100:  1.58 ms per query, recall@10 = 78.7%
hnsw.ef_search = 400:  4.74 ms per query, recall@10 = 91.7%

Exact search over 100,000 vectors took 12.8 ms per query. That is perfectly acceptable for a small table and does not scale: ten million vectors would be about 1.3 seconds. The remedy is an approximate nearest neighbour index, and the numbers show the cost of approximation honestly, which most introductions leave out. The HNSW index (hierarchical navigable small world, a layered proximity graph) answered in under a millisecond to 4.7 ms depending on one setting, hnsw.ef_search, the size of the candidate list kept during the graph walk. At 10 it was 15 times faster than the scan and found only 39 percent of the true ten nearest neighbours; at 400 it was 2.7 times faster and found 92 percent. This is recall, the fraction of the true top ten that the index returned, measured against the exact answers computed first. No setting reaches 100 percent for free. Choose the recall your product needs (a "related posts" widget tolerates 70 percent; a retrieval step feeding an answer-generation model usually wants 90 or more), measure it on your own data, and set ef_search to the cheapest value that meets it. The index also costs to build and store: 35 seconds and 79 MB here, larger than the 56 MB table, so budget memory for it (a vector index is fast when it fits in cache).

The two build parameters

The index in the benchmark was created with:

CREATE INDEX items_hnsw ON items USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 64);

The operator class (vector_cosine_ops) must match the operator in your queries: an index built for cosine distance is not used for <->. m is the number of connections per node in the graph (16 is the default) and ef_construction is the candidate list size while building (64 is the default); raising either improves recall at a given ef_search and costs build time and index size. They are fixed at build time, while ef_search is a per-session setting you can change per query, which is why tuning starts there. pgvector also offers IVFFlat indexes, which cluster the vectors into lists and search the nearest few lists: they build faster and use less memory, and generally give lower recall at the same speed; they must be built after the table has data, since the clusters come from it. HNSW is the usual default.

The trap: filtered search that returns nothing

Real queries have a filter: nearest documents in this category, for this tenant. With an approximate index, the filter is applied after the index returns its candidates, and that interacts badly with selective filters. On the same table, with a category column that splits the rows into 100 groups of 1,000, a search for the ten nearest items in one category with ef_search = 40:

-- filtered search, default settings
 count
-------
     0

-- same with iterative scan
 count
-------
    10

The default returned zero rows. The index produced its 40 nearest neighbours of the query vector across the whole table, and none of them happened to be in the requested category, so the filter discarded all of them. Nothing failed; the query simply answered "no results". The fix in pgvector 0.8 and newer is iterative index scans: SET hnsw.iterative_scan = relaxed_order; tells the scan to keep walking the graph until it has enough rows that pass the filter (or reaches a limit), and the same query returned 10. The mode name states the trade-off: results are close to, but not strictly in, distance order, and you can re-sort the final ten in SQL. Other remedies when the filter is very selective: a partial index per tenant, a table partition per tenant, or an exact search when the filter leaves few rows (the planner should choose it if statistics are fresh, so run ANALYZE). The worst habit is to test with an unfiltered query and ship a filtered one. Add a test that asserts the filtered query returns the requested number of rows for a selective filter.

Check that the index is used

Limit
  InitPlan 1
    ->  Index Scan using items_pkey on items items_1
          Index Cond: (id = 5000)
  ->  Index Scan using items_hnsw on items
        Order By: (embedding <=> (InitPlan 1).col1)

That is the plan to look for in EXPLAIN: an Index Scan using items_hnsw with an Order By on the distance expression. If you see Seq Scan and a Sort, the index is not used. The usual causes are a mismatched operator class, a missing LIMIT (the index serves ordered top-k queries, not unbounded ones), a cast that changes the type, or stale statistics on a small table where the planner correctly judges the scan cheaper. Reading plans is the subject of PostgreSQL EXPLAIN and indexes.

Putting the model behind a trait

The toy embed function is the seam where the real model goes, and the way to keep that seam clean is a small trait that also enforces the two invariants that matter when you swap models: batch sizes match, and the dimensions equal the column's. A real implementation of this trait would make an HTTP request to an embedding service (or run a local model) in embed; the pipeline around it does not change.

use anyhow::{bail, Result};

trait Embedder {
    fn model(&self) -> &str;
    fn dimensions(&self) -> usize;
    async fn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>>;
}

struct HashEmbedder;

impl Embedder for HashEmbedder {
    fn model(&self) -> &str {
        "toy-hash-64"
    }

    fn dimensions(&self) -> usize {
        64
    }

    async fn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>> {
        Ok(texts
            .iter()
            .map(|t| {
                let mut v = vec![0f32; 64];
                for w in t.split_whitespace() {
                    let h = w.bytes().fold(0xcbf29ce484222325u64, |h, b| (h ^ u64::from(b)).wrapping_mul(0x100000001b3));
                    v[(h % 64) as usize] += 1.0;
                }
                let n = v.iter().map(|x| x * x).sum::<f32>().sqrt().max(1e-9);
                v.iter().map(|x| x / n).collect()
            })
            .collect())
    }
}

struct Row {
    body: String,
    model: String,
    vector: Vec<f32>,
}

async fn embed_all<E: Embedder>(embedder: &E, texts: Vec<String>, batch: usize) -> Result<Vec<Row>> {
    let mut rows = Vec::with_capacity(texts.len());
    for chunk in texts.chunks(batch) {
        let vectors = embedder.embed(chunk).await?;
        if vectors.len() != chunk.len() {
            bail!("model returned {} vectors for {} inputs", vectors.len(), chunk.len());
        }
        for (body, vector) in chunk.iter().zip(vectors) {
            if vector.len() != embedder.dimensions() {
                bail!("expected {} dimensions, got {}", embedder.dimensions(), vector.len());
            }
            rows.push(Row { body: body.clone(), model: embedder.model().to_string(), vector });
        }
    }
    Ok(rows)
}

#[tokio::main]
async fn main() -> Result<()> {
    let texts: Vec<String> = (0..150).map(|i| format!("document number {i} about rust")).collect();
    let rows = embed_all(&HashEmbedder, texts, 64).await?;
    println!("{} rows, model {}, {} dimensions, first body {:?}", rows.len(), rows[0].model, rows[0].vector.len(), rows[0].body);
    Ok(())
}

150 rows, model toy-hash-64, 64 dimensions, first body "document number 0 about rust"

Each row carries the model name, which pays off the first time you change models. The checks in embed_all catch the failures that otherwise surface as a confusing database error ("expected 1536 dimensions, not 768") or, worse, silently mismatched results: a service that returns fewer vectors than inputs when a text is dropped, or a model whose dimension was changed in a new release. Batch the calls (here 64 texts at a time), since embedding services charge and rate-limit per request, and wrap the call with the retry and timeout patterns from async resilience patterns, since a bulk re-embedding job runs for hours and will meet transient failures.

Writes are expensive once the index exists

HNSW makes reads cheap by doing work at write time: each insert has to find its place in the graph. The same 10,000 vectors that loaded in a fraction of a second into the unindexed table took 8.8 seconds to insert into the table that already had the 79 MB index, about 0.88 ms per row against roughly 7 microseconds without it, a factor of 125. Deletes are cheap at the moment and leave dead entries until VACUUM (about 180 ms for this table after deleting 10,000 rows). Three rules follow. For a bulk load, insert first and build the index afterwards (35 seconds for 100,000 rows here, for a table you then query in a millisecond). For a steady trickle of writes, accept the per-row cost, which is small in absolute terms, and batch inserts in a transaction. For a re-embedding migration that touches every row, build a second column or table with the new vectors, index it, and switch queries over, instead of updating in place under a live index.

Operating it

• Store the model name with the vector. Embeddings from two models are not comparable. A column embedding_model text lets you re-embed gradually and query only vectors from the current model.
• Normalise at write time if you use the inner-product operator, and keep the dimension a property of the column, not of the code.
• Chunk long text. Models have input limits and one vector per document averages away the details. A few hundred words per chunk, with the chunk text and its parent id stored beside the vector, is the common arrangement.
• Know the dimension limits. The vector type holds many thousands of dimensions, but indexes are limited to about 2,000; the halfvec type (16-bit floats) halves the storage and indexes up to about 4,000. Check the extension's documentation for the exact figures in your version.
• Give the build memory. Set maintenance_work_mem large enough for the graph during CREATE INDEX, or the build slows sharply once it spills; building the index before loading data is much slower than loading, then indexing.
• Combine with ordinary search. Vector search finds meaning and misses exact terms (names, error codes). Hybrid designs run a full-text query and a vector query, then merge the rankings, which is often better than either alone.

Estimating storage before you commit

A vector(n) value takes 4 bytes per dimension plus an 8 byte header, so 128 dimensions is 520 bytes, and 100,000 of them is 52 million bytes: the benchmark table measured 56 MB once page overhead and the bigserial key are included. Scale it for your case: a million 1,536-dimension vectors is about 6.1 GB of table, and an HNSW index on them can be larger than that again, so the working set that must fit in memory for fast queries is a multiple of the raw data. Do this arithmetic first. If it does not fit, the options are fewer dimensions (some models can be truncated), halfvec storage at half the size, or a different store. An hour of arithmetic here is cheaper than discovering at launch that the index no longer fits in RAM and every query is a disk read.

When pgvector is not the right store

Use it when your data is already in PostgreSQL, the table has up to a few million vectors and you value transactions, joins and one system to operate. Reach for a dedicated vector database when you have hundreds of millions of vectors, very high query rates with tight latency, or need features such as built-in sharding for vectors. Skip vectors entirely when keyword search already answers the question; PostgreSQL full-text search and trigram indexes are cheaper to run and easier to explain (see full-text search from Rust). The SQLx patterns used here, pooling, transactions and the compile-time macros, are covered in SQLx explained, and the broader cache-and-index thinking behind choosing a store is in PostgreSQL versus SQLite.