Tokio Uring and Linux io uring From Rust a Cautious Look

The first time we ran a tokio-uring program in a stock Docker container it died before doing any I/O, with Operation not permitted from a line that only builds the runtime. The crate was fine. The container's default seccomp profile had blocked the io_uring_setup system call, and a growing number of hosts do the same on purpose. That one failure says a lot about where io_uring fits: it is a real speedup for a narrow set of programs, and it comes with an operational price. Everything below ran on Linux kernel 7.0 inside a Docker Desktop VM, with Rust 1.99, edition 2024 and tokio-uring 0.5.

What io_uring changes

With epoll, the kernel tells you a socket is ready, and then you make a system call to move the data. With io_uring, you write a description of the operation (read 4096 bytes at offset 0 from this file descriptor) into a submission queue that is shared memory between your process and the kernel. The kernel performs it and writes the result into a completion queue. You can queue many operations and enter the kernel once, or with polling modes not at all.

Two consequences matter for Rust. First, regular files get a real asynchronous path, which epoll never offered, so the thread hop that tokio::fs pays is gone (why tokio::fs is not what you expect). Second, the model is completion-based: the kernel owns your buffer from the moment you submit until the result comes back, and that does not fit the borrowing API of AsyncRead.

Ownership moves into the call

tokio-uring deals with that by taking the buffer by value and handing it back with the result. Here is a complete program that writes a file and reads it back.

[package]
name = "uring-demo"
version = "0.1.0"
edition = "2024"

[dependencies]
tokio-uring = "0.5"

fn main() -> std::io::Result<()> {
    tokio_uring::start(async {
        let file = tokio_uring::fs::File::create("hello.txt").await?;
        let (res, _buf) = file.write_at(&b"hello from io_uring\n"[..], 0).submit().await;
        let n = res?;
        file.sync_all().await?;
        file.close().await?;
        println!("wrote {n} bytes");

        let file = tokio_uring::fs::File::open("hello.txt").await?;
        let buf = vec![0u8; 64];
        let (res, buf) = file.read_at(buf, 0).await;
        let n = res?;
        println!("read {n} bytes: {:?}", String::from_utf8_lossy(&buf[..n]));
        Ok(())
    })
}

wrote 20 bytes
read 20 bytes: "hello from io_uring\n"

Read the tuple (res, buf) closely. read_at(buf, 0) consumes the Vec and returns it alongside the io::Result, so a failure still gives you the buffer back to reuse. You slice it with the count the kernel reports, since the vector's length is still 64. This shape is the main thing that makes code written for tokio-uring look different from ordinary Tokio code, and it is why the two do not share an I/O trait.

The first failure: Operation not permitted

Run the same binary in a default Docker container and it panics immediately.

thread 'main' panicked at tokio-uring-0.5.0/src/lib.rs:148:48:
called `Result::unwrap()` on an `Err` value: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }

Docker's default seccomp profile does not allow io_uring_setup, a choice made because several kernel vulnerabilities came through this interface. Kubernetes RuntimeDefault profiles, some cloud sandboxes and hardened distributions behave the same way, and recent kernels add a kernel.io_uring_disabled sysctl to switch it off host-wide. For the experiment we passed --security-opt seccomp=unconfined. In production, check that your platform permits it before you design around it, because a service that needs io_uring to start is a service that fails to schedule on the wrong node pool.

What it bought on a file-read loop

We repeated the small-file read loop from the file I/O page: open a 4 KiB file, read it, close it, 20,000 times, once through tokio::fs and once through tokio-uring, in a release build.

use std::time::Instant;
use tokio::io::AsyncReadExt;

const N: usize = 20_000;

fn main() -> std::io::Result<()> {
    std::fs::write("/tmp/small.txt", vec![b'x'; 4096])?;

    let rt = tokio::runtime::Builder::new_current_thread().enable_all().build()?;
    rt.block_on(async {
        let start = Instant::now();
        for _ in 0..N {
            let mut f = tokio::fs::File::open("/tmp/small.txt").await?;
            let mut buf = Vec::with_capacity(4096);
            f.read_to_end(&mut buf).await?;
        }
        println!("tokio::fs:   {:>8.1?} for {N} reads", start.elapsed());
        Ok::<_, std::io::Error>(())
    })?;

    tokio_uring::start(async {
        let start = Instant::now();
        for _ in 0..N {
            let f = tokio_uring::fs::File::open("/tmp/small.txt").await?;
            let (res, _buf) = f.read_at(vec![0u8; 4096], 0).await;
            res?;
            f.close().await?;
        }
        println!("tokio-uring: {:>8.1?} for {N} reads", start.elapsed());
        Ok::<_, std::io::Error>(())
    })?;
    Ok(())
}

tokio::fs:       2.4s for 20000 reads
tokio-uring:     1.5s for 20000 reads

That is 1.6 times faster, at 120 µs and 75 µs per iteration. Do not read the absolute numbers as typical: a Docker Desktop VM on a laptop makes system calls far more expensive than bare metal, which inflates both columns. The ratio is the portable part, and even it is a ceiling for this workload, since every iteration is dominated by the open and the close. A loop that keeps one file open and issues many reads would show a different gap, and a sequential read of one big file would show almost none.

Why this is not a drop-in replacement

• tokio_uring::start builds its own single-threaded runtime. You cannot use tokio_uring::fs::File inside a normal #[tokio::main], and its tasks are not Send. A multi-core server means one such runtime per core, with the connections spread across them (non-Send futures and LocalSet).
• Most crates you rely on speak AsyncRead and AsyncWrite. A hyper or axum server does not run on tokio-uring as it stands, so you are choosing a smaller ecosystem.
• Cancelling a future does not cancel the kernel's operation. The buffer stays alive until the completion arrives, which is safe but means a dropped read still finishes.
• Linux only, with a kernel recent enough to have the opcodes you use. On macOS and Windows the crate does not compile.

Alternatives exist with the same model: monoio and glommio are thread-per-core runtimes built around io_uring, and they make the same trade between speed and compatibility. For many teams the practical comparison is not with tokio::fs, it is with a spawn_blocking batch, which gets most of the way for far less change.

A cheaper first step on the same machine

Before adopting a second runtime, try the variant that stays inside ordinary Tokio: one spawn_blocking closure that does the whole batch with std::fs. On the macOS run on the file I/O page that approach took 223 ms for the 20,000 reads, against 609 ms for tokio::fs, a 2.7x gain from changing three lines and nothing about the architecture. It does not remove the thread, but it removes the repeated hops, which were the avoidable part. Keep the io_uring option for the point where that stops being enough, and measure on the kernel and the container policy you will actually deploy.

When to spend the effort

Consider it when a profile shows file or network system calls as the dominant cost, you control the kernel and the container policy, and the work is a thread-per-core design: a storage engine, a proxy, a log shipper moving gigabytes. Profile first (flamegraph profiling) and check that the cost is the hop and not the disk. If the answer is a web service doing a few thousand requests per second, the standard runtime is the better place to be (how the runtime works).

Also be clear about what we did not run: a network workload, a bare-metal host, a kernel with SQPOLL enabled, or any comparison with monoio. Treat claims about those as documentation, not measurements.