Rust Error Too Many Open Files and Raising the Limit

The message appears in many shapes: Too many open files (os error 24) from File::open, from TcpListener::accept or from a DNS lookup. All mean the same thing: the process has as many file descriptors open as the kernel lets it. This post shows how to see the limit and the usage, how to tell a leak from legitimate load, and the fixes. Commands are for Linux; code compiles on Rust 1.90 with edition 2024 and Tokio 1.47.

Step 1: the limit and the usage

ulimit -n                                   # soft limit for this shell
ulimit -Hn                                  # hard limit
grep "open files" /proc/$(pgrep myservice)/limits
ls /proc/$(pgrep myservice)/fd | wc -l      # descriptors open right now

Compare the count with the soft limit. Sample it every few seconds while the service runs under load: a count that climbs and never falls is a leak, one that rises with traffic and falls with it is load. To see what kind of descriptors dominate, list the link targets: ls -l /proc/PID/fd shows socket:[...], pipe:[...] and file paths. A thousand sockets to the same upstream host point at a missing connection pool; a thousand open log files point at a loop that opens a file per item.

Step 2: the usual causes in Rust

Ownership closes a File or TcpStream when it is dropped, so classic "forgot to close" leaks are rare. The real causes are different:

• Unbounded concurrency: spawning one task per incoming item, each opening a socket, with no limit.
• A new client per request: building a reqwest::Client inside the handler instead of reusing one, so each request opens a fresh connection pool.
• Handles kept alive: stored in a cache, a long-lived task or a channel, so they never drop (tasks that never finish).
• mem::forget or a reference cycle that prevents Drop from running.
• Many watchers or inotify instances in file-watching code.
• Connections stuck in CLOSE_WAIT: the peer closed but your code never dropped its end.

// Bug: a new client (and connection pool) for every call
async fn fetch(url: &str) -> reqwest::Result<String> {
    reqwest::Client::new().get(url).send().await?.text().await
}

// Fix: one client, cloned cheaply, reused
use std::sync::LazyLock;
static CLIENT: LazyLock<reqwest::Client> = LazyLock::new(reqwest::Client::new);

async fn fetch_ok(url: &str) -> reqwest::Result<String> {
    CLIENT.get(url).send().await?.text().await
}

Step 3: bound what you open

use std::sync::Arc;
use tokio::sync::Semaphore;

async fn process_all(urls: Vec<String>) {
    let limit = Arc::new(Semaphore::new(200));        // at most 200 in flight
    let mut tasks = Vec::new();
    for url in urls {
        let permit = limit.clone().acquire_owned().await.unwrap();
        tasks.push(tokio::spawn(async move {
            let _permit = permit;                      // released when the task ends
            let _ = fetch_ok(&url).await;
        }));
    }
    for t in tasks {
        let _ = t.await;
    }
}

A semaphore turns "as many as the input has" into a fixed ceiling well below the descriptor limit (limiting concurrency with a semaphore). Do the same for file operations in directory walkers: read entries in a stream with a buffer limit rather than opening everything at once.

Step 4: raise the limit where it belongs

# current shell only
ulimit -n 65536

# /etc/systemd/system/myservice.service
[Service]
LimitNOFILE=65536

docker run --ulimit nofile=65536:65536 myimage

# Docker Compose
services:
  api:
    ulimits:
      nofile: { soft: 65536, hard: 65536 }

A limit set in your shell does not apply to a service started by systemd or a container runtime, which have their own defaults. Raise it in the place that starts the process. In Kubernetes the limit comes from the container runtime's configuration on the node, so you may need to change the runtime or use an init step. A value in the tens of thousands is normal for servers; raising it does not fix a leak, it only delays the failure.

Step 5: raise it from the program

use rustix::process::{getrlimit, setrlimit, Resource};

fn raise_nofile() -> rustix::io::Result<()> {
    let mut lim = getrlimit(Resource::Nofile);
    if let Some(max) = lim.maximum {
        lim.current = Some(max);          // soft limit up to the hard limit
        setrlimit(Resource::Nofile, lim)?;
    }
    Ok(())
}

A process can raise its own soft limit up to the hard limit at startup, which is what many servers do (Nginx and others have a worker setting for this). The rustix or rlimit crates wrap setrlimit safely (system calls with rustix). On macOS the kernel also caps by kern.maxfilesperproc.

Step 6: handle accept errors gracefully

use tokio::net::TcpListener;

async fn serve(listener: TcpListener) {
    loop {
        match listener.accept().await {
            Ok((_sock, _addr)) => { /* spawn handler */ }
            Err(e) => {
                eprintln!("accept failed: {e}");
                tokio::time::sleep(std::time::Duration::from_millis(100)).await; // do not spin on EMFILE
            }
        }
    }
}

When the limit is hit, accept fails repeatedly and immediately; a loop that just retries burns a core. A short sleep keeps the process alive and lets existing connections finish and free descriptors. Monitor the descriptor count as a metric and alert at about 80 percent of the limit (reading /proc metrics). A similar-looking failure with a different cause is connection refused.