In Process Caches in Rust Moka Quick Cache and Cached

Compare moka, quick_cache and cached for in-memory caching in Rust: size limits, eviction, time to live, concurrency, async support and a single-flight loader.

3 min read

Summarize in AI

Quick Summary

For a shared in-memory cache in a Rust service, use moka when you need async support, time to live or time to idle, and size-based eviction. Use quick_cache for a smaller, faster sync cache, and cached for simple function memoization. Bound every cache by entry count or weight, and decide invalidation before you ship it.

Key takeaways

A cache without a size bound is a memory leak waiting for enough distinct keys.
moka supports TTL, TTL since last access, weighted capacity and get_with, which coalesces concurrent loads of one key.
Eviction policy matters for hit rate, and TinyLFU-based caches like moka and quick_cache beat plain LRU on many workloads.
Cache values should be cheap to clone, so store Arc<T> for large values.
Decide staleness: a short TTL is often a simpler invalidation strategy than explicit deletes.

A cache in the process is the fastest cache you can have: a hash lookup with no network. It is also easy to get wrong, in three ways. It has no size limit, it never expires, or a hundred requests all miss the same key at once and stampede the database. The mature crates solve each of those, and the choice among them is about features. This post compares three and shows the moka pattern for async services. It continues from memoization.

Rust 1.90, moka 0.12.

What to compare

	`moka`	`quick_cache`	`cached`
Sync and async API	both	sync	sync, with async feature
Eviction	TinyLFU + LRU	clock-based	LRU, TTL, size
TTL / TTI	yes / yes	no	TTL
Weighted capacity	yes	yes	no
Single-flight loading	yes (`get_with`)	yes (guard API)	no
Best for	services	small hot caches	memoizing functions

An async cache with a loader

Cargo.tomlTOML

[dependencies]
moka = { version = "0.12", features = ["future"] }
tokio = { version = "1", features = ["full"] }

rust

use std::{sync::Arc, time::Duration};
use moka::future::Cache;

#[derive(Debug)]
struct User { id: u64, name: String }

async fn load_user(id: u64) -> Arc<User> {
    // imagine a database query
    Arc::new(User { id, name: format!("user{id}") })
}

#[tokio::main]
async fn main() {
    let cache: Cache<u64, Arc<User>> = Cache::builder()
        .max_capacity(10_000)
        .time_to_live(Duration::from_secs(300))
        .build();

    let a = cache.get_with(7, load_user(7)).await;
    let b = cache.get_with(7, load_user(7)).await;
    println!("{} {}", a.name, Arc::ptr_eq(&a, &b));
}

It prints user7 true. The second get_with finds the cached Arc. When several tasks ask for the same missing key concurrently, only one runs the loader and the rest wait for its result, which stops the stampede. The Arc makes returning a value cheap, since moka clones values on read.

Choose limits deliberately

max_capacity(10_000) counts entries by default. If values vary in size, use weigher to count bytes:

rust

Cache::builder()
    .max_capacity(64 * 1024 * 1024)
    .weigher(|_k, v: &Vec<u8>| v.len() as u32)
    .build::<String, Vec<u8>>();

Pick the number from a memory budget, not a guess, and expose hit and miss counts so you can tune it.

Staleness and invalidation

time_to_live bounds how old an entry can be, and time_to_idle evicts entries unused for a while. A five minute TTL tolerates five minutes of stale data and needs no code to invalidate. If correctness needs immediate freshness, call cache.invalidate(&key) after the write, and note that with several instances each has its own copy: they will not hear about it. That is when a shared cache or a pub/sub invalidation is required.

Where to put it

In an Axum service, put the cache in AppState (sharing state) so handlers share it. Do not cache errors for long, do not cache user specific data under a shared key, and keep personal data out of long-lived caches.

When not to cache

Cheap queries, rarely repeated keys and data that must be strongly consistent. A cache adds a second source of truth and a class of bugs; add one when a measurement shows the underlying call is too slow or too costly. The moka docs describe the algorithms at docs.rs/moka.

Also inWeb

Taggedmoka