Cache Invalidation Strategies for Rust Services Compared

Caching is easy to add and hard to keep correct. The hard part is deciding when an entry stops being true. This post compares the strategies and shows the two that need code: coalescing and versioning. Everything compiles on Rust 1.90 with edition 2024 and tokio 1; the in-process cache libraries are described in in-process caches.

The strategies

Most services start with TTL, add delete on write for data users edit, and keep the TTL as a safety net.

TTL plus delete on write

use std::collections::HashMap;
use std::sync::Mutex;
use std::time::{Duration, Instant};

pub struct Cache<V> {
    ttl: Duration,
    map: Mutex<HashMap<String, (Instant, V)>>,
}

impl<V: Clone> Cache<V> {
    pub fn new(ttl: Duration) -> Self {
        Self { ttl, map: Mutex::new(HashMap::new()) }
    }

    pub fn get(&self, key: &str) -> Option<V> {
        let map = self.map.lock().unwrap();
        map.get(key)
            .filter(|(at, _)| at.elapsed() < self.ttl)
            .map(|(_, v)| v.clone())
    }

    pub fn put(&self, key: &str, value: V) {
        self.map.lock().unwrap().insert(key.to_string(), (Instant::now(), value));
    }

    pub fn invalidate(&self, key: &str) {
        self.map.lock().unwrap().remove(key);
    }
}

Reads honor the TTL; the write path calls invalidate. Invalidate after the database commit, not before: invalidating first lets a concurrent reader refill the cache with the old value before the write lands.

Versioned keys

use std::sync::atomic::{AtomicU64, Ordering};

pub struct Versioned {
    version: AtomicU64,
}

impl Versioned {
    pub fn key(&self, base: &str) -> String {
        format!("{base}:v{}", self.version.load(Ordering::Relaxed))
    }

    pub fn bump(&self) {
        self.version.fetch_add(1, Ordering::Relaxed);
    }
}

Bumping the version after a write makes every old key unreachable at once, with no scan or delete; the old entries age out by TTL or eviction. In a multi-instance deployment the version lives in a shared store such as Redis (Redis) or in the database row (updated_at), so all instances agree.

Coalescing recomputation

use std::{collections::HashMap, sync::Arc};
use tokio::sync::{Mutex, OnceCell};

pub struct Loader {
    in_flight: Mutex<HashMap<String, Arc<OnceCell<String>>>>,
}

impl Loader {
    pub async fn load(&self, key: &str) -> String {
        let cell = {
            let mut map = self.in_flight.lock().await;
            map.entry(key.to_string()).or_default().clone()
        };
        let value = cell
            .get_or_init(|| async { slow_lookup(key).await })
            .await
            .clone();
        self.in_flight.lock().await.remove(key);
        value
    }
}

async fn slow_lookup(key: &str) -> String {
    format!("value for {key}")
}

Concurrent callers for the same key share one OnceCell; only the first runs slow_lookup, the rest await its result. Crates such as moka provide this as get_with, along with eviction and TTL; use one rather than writing your own for production.

Stale while revalidate

Return the expired value immediately and refresh in the background. Users see no latency spike; the cost is serving data older than the TTL for one refresh interval. It suits data like exchange rates or a product catalog where slightly stale beats slow.

Decisions to write down

• What staleness can each cache tolerate? (Seconds for a feed, zero for a balance.)
• Which operations invalidate which keys?
• What happens if invalidation fails? (The TTL.)
• How is the cache bounded? An unbounded cache is a memory leak with a hit rate (memory growth).

A caching layer that cannot answer these is a source of bugs the first time two writers race. The simple case is in memoization and caching.