Debugging Rust From Printing to Breakpoints and Backtraces

Debugging Rust is mostly a matter of choosing the cheapest tool that can answer your question. The compiler answers most questions before the program runs. A panic message answers the next batch, a backtrace the batch after that, and only then do you need to print values, add structured logs, attach a debugger or run a sanitizer. Teams lose hours by starting at the wrong end of that list, for example opening a debugger to find a typo that the compiler already reported. This guide walks the list in order, on one small program with one real bug: an order parser that panics on bad input. Each tool is run against it, with the output we actually got. One section is different: the interactive debugger could not be run on the machine used here, and the guide says so where it happens and shows the commands, not a fabricated session. Versions: rustc 1.99.0, nightly 1.101 for the sanitizer, tracing 0.1.44 and tracing-subscriber 0.3.23, on an Apple M1 Max.

The program and the bug

The input is four lines of comma-separated orders: id, sku and quantity. The program parses each line, totals the quantities per sku and prints the totals. The third line has a typo (x where a quantity should be) and the fourth is missing a field. This is how it first looks, written the quick way:

use std::collections::HashMap;

#[derive(Debug)]
struct Order {
    id: u32,
    sku: String,
    qty: u32,
}

fn parse_order(line: &str) -> Order {
    let parts: Vec<&str> = line.split(',').collect();
    Order {
        id: parts[0].parse().unwrap(),
        sku: parts[1].to_string(),
        qty: parts[2].parse().unwrap(),
    }
}

fn total_by_sku(orders: &[Order]) -> HashMap<String, u32> {
    let mut totals = HashMap::new();
    for o in orders {
        *totals.entry(o.sku.clone()).or_insert(0) += o.qty;
    }
    totals
}

fn main() {
    let input = "1,apple,3\n2,pear,2\n3,apple,x\n4,fig";
    let orders: Vec<Order> = input.lines().map(parse_order).collect();
    println!("{:?}", total_by_sku(&orders));
}

Step one: read what the compiler already said

Before the program ran, the build printed a warning. It is easy to scroll past, and it is the first clue that a field is unused:

warning: field `id` is never read
 --> src/main.rs:5:5
  |
4 | struct Order {
  |        ----- field in this struct
5 |     id: u32,
  |     ^^
  |
  = note: `Order` has a derived impl for the trait `Debug`, but this is intentionally ignored during dead code analysis
  = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default

This warning does not explain the crash, and that is the instructive part: compiler warnings find a class of bugs (unused values, unreachable code, a let that shadows what you meant) cheaply, but they are silent about runtime data. Treat a warning-free build as a precondition for debugging anything else, because a warning you ignored has a habit of being the actual bug an hour later. In CI, RUSTFLAGS="-D warnings" makes this impossible to ignore. When the compiler does reject code, read the whole message, including the help: lines: for a large share of errors the fix is printed there.

Step two: the panic message is a location, not a diagnosis

Running the program stops at the third line:

thread 'main' panicked at src/main.rs:15:31:
called `Result::unwrap()` on an `Err` value: ParseIntError { kind: InvalidDigit }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Read it as three facts. The panic happened at src/main.rs:15:31, which is the parse().unwrap() for the quantity; the error was a ParseIntError of kind InvalidDigit, so the text was not a number; and there is a backtrace available. What the message does not say is which line of input failed or what the bad text was. That is the signature of unwrap(): it reports where the failure was observed and discards everything the surrounding code knew. Whenever a panic message leaves you asking "which input?", the missing information is the real bug, and the later steps are about recovering it.

Step three: the backtrace, read from the right frame

Set RUST_BACKTRACE=1 and run again. The panic location is the same, now with the call stack that led to it:

stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/b940084d7eb6a299eb4bfeb8e34901bc051e7ac4/library/std/src/panicking.rs:679:5
   1: core::panicking::panic_fmt
             at /rustc/b940084d7eb6a299eb4bfeb8e34901bc051e7ac4/library/core/src/panicking.rs:80:14
   2: core::result::unwrap_failed
             at /rustc/b940084d7eb6a299eb4bfeb8e34901bc051e7ac4/library/core/src/result.rs:1871:5
   3: <core::result::Result<u32, core::num::error::ParseIntError>>::unwrap
             at /rustc/b940084d7eb6a299eb4bfeb8e34901bc051e7ac4/library/core/src/result.rs:1232:23
   4: dbg::parse_order
             at ./src/main.rs:15:31
   5: <dbg::parse_order as core::ops::function::FnMut<(&str,)>>::call_mut
             at /rustc/b940084d7eb6a299eb4bfeb8e34901bc051e7ac4/library/core/src/ops/function.rs:166:5

Frames are numbered from the point of the panic outward. Frames 0 to 3 are the standard library's panic machinery and are never where the bug lives. The first frame with your crate's name is frame 4, dbg::parse_order at ./src/main.rs:15:31, and that is where to start reading. Frames above it that belong to core, such as FnMut::call_mut here, are the iterator adapters that called your function; they tell you the call came through a map, which is useful when a bug depends on which element was being processed. Two settings change what you see. RUST_BACKTRACE=full stops hiding the runtime's internal frames. And the file and line information appears only because the build included debug info: a release build with debug = false shows function names but no lines, and a stripped binary shows addresses. Keep debug = "line-tables-only" in release profiles if you want useful production backtraces at a modest size cost.

Step four: dbg! to look at the data

The backtrace shows where, not what. To see the failing input, the fastest tool is the dbg! macro. It prints the file, line and the expression text with its value to standard error, and returns the value, so it can wrap an expression in place:

fn parse_order(line: &str) -> Order {
    let parts: Vec<&str> = line.split(',').collect();
    dbg!(&parts);
    Order {
        id: parts[0].parse().unwrap(),
        sku: parts[1].to_string(),
        qty: parts[2].parse().unwrap(),
    }
}

[src/main.rs:12:5] &parts = [
    "1",
    "apple",
    "3",
]
[src/main.rs:12:5] &parts = [
    "2",
    "pear",
    "2",
]
[src/main.rs:12:5] &parts = [
    "3",
    "apple",
    "x",
]

thread 'main' panicked at src/main.rs:16:31:
called `Result::unwrap()` on an `Err` value: ParseIntError { kind: InvalidDigit }

Now the cause is on screen: the third record is ["3", "apple", "x"]. Notice two details of the output. The panic location changed from line 15 to line 16, because adding the dbg! line shifted everything below it; do not be surprised when a location moves after you add a print. And dbg! writes to standard error, so it does not corrupt output you are piping from standard out. Its limits are worth knowing. It takes its argument by value and returns it, so dbg!(parts) without the & would move the vector and break the next line. It needs the type to implement Debug. And it is a temporary tool: clippy has a dbg_macro lint, off by default, that you can turn on in CI so none reaches a release.

Step five: make the error say what it knew

Printing found the cause. The durable fix is to stop throwing the information away. The function should return a Result whose error carries the offending line, the field and the value, and the caller decides whether to skip, retry or stop. This version uses a small error enum and implements Display so the message is readable:

#[derive(Debug, PartialEq)]
enum ParseError {
    MissingField { line: String, index: usize },
    BadNumber { line: String, field: &'static str, value: String },
}

impl fmt::Display for ParseError {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        match self {
            ParseError::MissingField { line, index } => write!(f, "line {line:?} has no field {index}"),
            ParseError::BadNumber { line, field, value } => {
                write!(f, "line {line:?}: {field} {value:?} is not a number")
            }
        }
    }
}

fn parse_order(line: &str) -> Result<Order, ParseError> {
    let parts: Vec<&str> = line.split(',').collect();
    let field = |i: usize| {
        parts.get(i).copied().ok_or_else(|| ParseError::MissingField { line: line.to_string(), index: i })
    };
    let number = |name: &'static str, i: usize| -> Result<u32, ParseError> {
        let v = field(i)?;
        v.parse().map_err(|_| ParseError::BadNumber { line: line.to_string(), field: name, value: v.to_string() })
    };
    Ok(Order { id: number("id", 0)?, sku: field(1)?.to_string(), qty: number("qty", 2)? })
}

The two closures keep the function readable: field(i) turns a missing index into MissingField, and number(name, i) turns a parse failure into BadNumber with the text it saw. The ? operator propagates either one. Because the error is a value, it is also testable, and the bug is now pinned by two tests that name the failure the program originally crashed on:

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn rejects_a_non_numeric_quantity() {
        let err = parse_order("3,apple,x").unwrap_err();
        assert_eq!(err, ParseError::BadNumber { line: "3,apple,x".into(), field: "qty", value: "x".into() });
    }

    #[test]
    fn reports_a_missing_field() {
        assert!(matches!(parse_order("4,fig"), Err(ParseError::MissingField { index: 2, .. })));
    }
}

running 2 tests
test tests::rejects_a_non_numeric_quantity ... ok
test tests::reports_a_missing_field ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

A test is the best debugger output to keep: it runs in milliseconds, it points at the exact function, and it fails the moment the bug returns. When you find a bug by any means in this guide, the last step is to write the test that would have caught it. The missing-field case, line 4, also turned up here: the original code would have panicked on parts[2] with "index out of bounds" the moment the first bug was fixed, and nobody had written that case down.

A hand-written error enum is the right choice for a library, where callers need to match on the cases. In an application that only reports failures, the same effect takes less code with the anyhow crate: parts[2].parse::<u32>().with_context(|| format!("bad quantity in line {line:?}"))? attaches the sentence to the error and prints the chain when it reaches main. Either way the principle is the one that matters for debugging: put the context at the place that knows it. The parser knows the line; main does not.

Step six: structured logs with tracing

Printing works when you are sitting at the program. For a service, or for a bug that only happens in a long run, you want records you can turn on and off without recompiling. The tracing crate provides events (point-in-time facts, like a log line) and spans (a region of time with fields attached), and a subscriber decides what to show. The setup is two lines in main and one attribute on the function:

use tracing::{debug, error, info, instrument};

#[instrument(level = "debug")]
fn parse_order(line: &str) -> Result<Order, ParseError> {
    // ... as above, then before returning:
    // debug!(?order, "parsed");
    // Ok(order)
}

fn main() {
    tracing_subscriber::fmt()
        .with_env_filter(tracing_subscriber::EnvFilter::from_default_env())
        .without_time()
        .with_ansi(false)
        .init();
    let input = "1,apple,3\n2,pear,2\n3,apple,x\n4,fig";
    let mut orders = Vec::new();
    for line in input.lines() {
        match parse_order(line) {
            Ok(o) => orders.push(o),
            Err(e) => error!(%e, "skipping bad line"),
        }
    }
    info!(count = orders.len(), "parsed orders");
}

The behaviour is chosen at run time by the RUST_LOG environment variable, with no rebuild. With the variable unset, the default filter shows only errors. With RUST_LOG=debug the spans and the debug events appear:

DEBUG parse_order{line="1,apple,3"}: dbg: parsed order=Order { id: 1, sku: "apple", qty: 3 }
DEBUG parse_order{line="2,pear,2"}: dbg: parsed order=Order { id: 2, sku: "pear", qty: 2 }
ERROR dbg: skipping bad line e=line "3,apple,x": qty "x" is not a number
ERROR dbg: skipping bad line e=line "4,fig" has no field 2
 INFO dbg: parsed orders count=2
[("apple", 3), ("pear", 2)]

Read the first line closely. parse_order{line="1,apple,3"} is the span: #[instrument] recorded the function's arguments as fields, so every event emitted inside it carries the input that produced it, without you writing the field into each message. The two error lines came from the caller, which now logs the error and keeps going. The program finished with the two good orders totalled, which is the behaviour the original crash denied us. At RUST_LOG=info the same run prints the two errors and the count line without the debug records, so the same binary is quiet in production and chatty when you ask. Two practical notes. Filters can target a module (RUST_LOG=dbg=debug,hyper=warn), which is how you read your own debug records without a wall of library output. And the tracing_subscriber crate needs its env-filter feature turned on for EnvFilter; I forgot it on the first build, and the compiler's note, "the item is gated behind the env-filter feature", named the fix.

Step seven: an interactive debugger

A debugger lets you stop the program, look at any variable and walk the stack without editing code. Rust ships wrapper scripts (rust-lldb and rust-gdb) that load pretty-printers so a Vec or a String displays as values and not as raw pointers. Break on the panic machinery, run, and inspect the frame that is yours:

cargo build
rust-lldb target/debug/dbg
(lldb) breakpoint set -n rust_panic
(lldb) run
(lldb) bt
(lldb) frame select 6
(lldb) frame variable

I could not run this on the machine used for this guide. Developer Mode is switched off on it, and lldb set the breakpoint (it resolved to rust_panic in the standard library's panicking.rs) and then stalled at run and had to be killed. So the block above is the standard recipe, not a transcript, and the frame number in it depends on your build. The debugger earns its keep when the question is "what is the whole state right now", for example a large struct or a deep call stack, and it is slower than dbg! for a question about one value. It also needs a debug build: in an optimised build variables are in registers or gone, and the debugger will report them as optimised out.

Step eight: memory bugs and the sanitizer

Safe Rust cannot read past the end of a buffer, so a panic in safe code is a logic bug. Inside unsafe that guarantee is yours to keep, and a violation does not panic. It reads whatever is there. Here is the smallest version of that failure, an off-by-one past a three-byte vector:

fn main() {
    let v = vec![10u8, 20, 30];
    let p = v.as_ptr();
    // Off by one: reads the byte after the allocation.
    let x = unsafe { *p.add(3) };
    println!("{x}");
}

Run normally, it printed 0 and exited successfully. That is the nasty part: nothing crashed, and the value was garbage that happened to look plausible. AddressSanitizer instruments every memory access to catch exactly this. It needs a nightly compiler and a flag:

RUSTFLAGS="-Zsanitizer=address" cargo +nightly run --target aarch64-apple-darwin

==94209==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6020000000b3 at pc 0x0001046551e8 bp 0x00016b7a9df0 sp 0x00016b7a9de8
READ of size 1 at 0x6020000000b3 thread T0
...
0x6020000000b3 is located 0 bytes after 3-byte region [0x6020000000b0,0x6020000000b3)
allocated by thread T0 here:

The report names the bug class (heap-buffer-overflow), the operation (a read of one byte) and the geometry: the address is "0 bytes after 3-byte region", which is the off-by-one stated exactly. On this machine the stack frames came out as raw addresses, because the sanitizer's symboliser (atos) also stalled with Developer Mode off; setting ASAN_OPTIONS=symbolize=0 let the report print, and I trimmed the address-only frames above. On Linux, with llvm-symbolizer installed, the same report names the function and line. Two cautions about sanitizers: they need nightly, and they slow the program several times, so they belong in a dedicated CI job and in the hunt for a specific failure, not in every build. For a crate with unsafe code, Miri, the interpreter for Rust's memory model, catches a wider range of undefined behaviour than ASan. I did not run it for this guide.

Step nine: debug builds catch what release builds hide

One more category sits between logic bugs and memory bugs: arithmetic. In a debug build, integer overflow panics. In a release build it wraps silently, by default. A counter that climbs past its type's maximum shows the difference:

fn main() {
    let mut level: u8 = 250;
    for _ in 0..10 {
        level += 1;
    }
    println!("level = {level}");
}

debug build:
thread 'main' panicked at src/main.rs:4:9:
attempt to add with overflow

release build:
level = 4

The debug build stopped at the exact line. The release build printed level = 4: 250 plus ten is 260, and 260 modulo 256 is 4. Nothing warned. This is a bug that exists only in the build you ship. Three tools address it. Use checked_add, saturating_add or wrapping_add when you know which behaviour you mean, so the intent is in the code. Set overflow-checks = true in [profile.release] if a panic is better than a wrong number for your program, at a small speed cost. And run your tests in both profiles with cargo test --release, since a test suite that only runs in debug will never see the wrapped value.

Which tool for which question

The order matters more than any single row. In this guide the first three steps took seconds and located the cause; the fix, with its tests, took minutes. The debugger and the sanitizer were never needed for the parser, because a parser that returns errors has no hidden state to inspect. They earned their place only on the problems that the cheaper tools cannot see, such as the silent 0 from the unsafe read.

Where this advice stops

• Printing can change the bug. Adding a dbg! or a log call shifts timing and memory layout. For a race condition or a memory error, the symptom may vanish when you observe it, which is itself evidence. Use a sanitizer or the thread sanitizer instead of more prints.
• Async code is harder to read in a backtrace. A panic inside a future shows the executor's frames, not the logical chain of awaits. Spans from tracing are the better tool there, because they follow the task and not the thread.
• Do not leave unwrap() in code you cannot test. It is a fine tool in a test and in a throwaway, and the cause of the unreadable panic that started this guide. In production paths, return an error with context.
• The debugger section was not verified here. The commands are the standard ones, but I did not capture a session, so check the frame numbers against your own backtrace before copying them.
• The sanitizer section is one platform. The run was on Apple silicon with nightly, with unsymbolised frames. Linux gives fuller reports, and Windows support for sanitizers differs.
• A debugger is not always the end of a search. If the program works under the debugger and fails outside it, the cause is usually timing, uninitialised data or a difference between debug and release, and the answer is more tests, not more stepping.

The measured outcome is short. A parser that panicked with "InvalidDigit" and no context now reports line "3,apple,x": qty "x" is not a number, skips the bad lines, totals the good ones to [("apple", 3), ("pear", 2)], and has two passing tests that say why. Every step before the debugger found or fixed something, and the debugger step is the one this guide could not demonstrate.