Debugging a Stack Overflow in Rust and Recursion Limits

The program died with a one-line message and no backtrace. That is how Rust reports a stack overflow, because by the time it happens there is no room left to unwind. The line is short, and the cause is usually one of two things: a function that calls itself with no end, or a local variable bigger than the stack. This post tells them apart and gives four fixes. It is the error piece in the debugging cluster.

Rust 1.90, 64 bit Linux.

The symptom

fn depth(n: u64) -> u64 {
    if n == 0 { 0 } else { 1 + depth(n - 1) }
}

fn main() {
    println!("{}", depth(10_000_000));
}

thread 'main' has overflowed its stack
fatal runtime error: stack overflow
Aborted (core dumped)

Each call uses a stack frame, often tens of bytes in release builds and more in debug. Ten million frames of 16 to 48 bytes is hundreds of megabytes, far past 8 MiB. Release mode may optimize this specific example into a loop; debug mode will not. The exact depth at which a program fails depends on the frame size and build mode, so do not rely on a number.

Find the culprit

1. Run under a debugger: rust-lldb target/debug/demo or gdb. When it stops at the overflow, bt 20 shows the repeating frames. The function that appears over and over is the recursion.
2. If no recursion is obvious, look for big locals. The compiler does not warn about them. Run cargo clippy with clippy::large_stack_arrays and clippy::large_types_passed_by_value.
3. For deep data structures, check Drop: dropping a long linked list or deeply nested Box tree recurses once per node, and a list of a million nodes overflows when it is dropped, not when it is built.

Fix 1: iterate with an explicit stack

fn depth_iter(mut n: u64) -> u64 {
    let mut count = 0;
    while n > 0 {
        count += 1;
        n -= 1;
    }
    count
}

For tree traversal, replace recursion with a Vec of work items. The heap grows as needed and is limited only by memory.

Fix 2: heap allocate big values

let buf = vec![0u8; 10_000_000];        // heap
let boxed: Box<[u8; 1_000_000]> = vec![0u8; 1_000_000].into_boxed_slice().try_into().unwrap();

A plain Box::new([0u8; 1_000_000]) may still build the array on the stack first in debug builds. A Vec is the reliable choice. See stack and heap in Rust.

Fix 3: a bigger stack for the work

let handle = std::thread::Builder::new()
    .stack_size(256 * 1024 * 1024)
    .spawn(|| depth(1_000_000))
    .unwrap();
let result = handle.join().unwrap();

This only moves the limit and is a stopgap for code that you cannot restructure. The memory is reserved virtually, so a large number costs little until used.

Fix 4: limit the depth

For recursive parsers handling outside input, count depth and return an error past a bound. serde_json stops at a recursion limit of 128 by default for this reason. Unbounded recursion on untrusted data is a denial-of-service bug, discussed in DoS through parsers.

Async and embedded

Futures are state machines on the heap or in the task, so deep async recursion needs Box::pin. On microcontrollers the stack may be a few kilobytes, and overflows corrupt memory silently unless a guard region exists. The platform behavior is described in the standard library docs for thread::Builder.