Reading Standard Input in Rust Lines Bytes and Pipes

Read stdin as lines, bytes or all at once, handle empty input, detect pipes versus terminals, and avoid loading huge streams into memory.

2 min read

Summarize in AI

Unix tools compose because they read from stdin and write to stdout, and a Rust tool joins the family with about ten lines. The decisions are how much to hold in memory and what to do when nobody is piping anything in. This post covers both, with a counting example. It is part of the command line tool series.

Rust 1.90, edition 2024.

Line by line

rust

use std::io::{self, BufRead};

fn main() -> io::Result<()> {
    let stdin = io::stdin();
    let mut lines = 0usize;
    let mut words = 0usize;
    for line in stdin.lock().lines() {
        let line = line?;
        lines += 1;
        words += line.split_whitespace().count();
    }
    println!("{lines} {words}");
    Ok(())
}

With two lines piped in (a b and c) it prints 2 3. This streams: memory use is one line at a time, so a 20 GB log is fine. lock() takes the stdin lock once for the whole loop.

Everything at once

rust

let mut input = String::new();
io::stdin().read_to_string(&mut input)?;

Convenient for configuration and small documents. It needs use std::io::Read. It also errors on invalid UTF-8; use read_to_end into a Vec<u8> for arbitrary bytes.

Interactive prompts

rust

use std::io::{self, Write};

print!("name: ");
io::stdout().flush()?;
let mut name = String::new();
io::stdin().read_line(&mut name)?;
let name = name.trim();

print! does not flush on its own, so the prompt may not appear until after the read; the explicit flush fixes it. read_line keeps the trailing newline, hence trim. The dialoguer and inquire crates add richer prompts.

Pipe or terminal

rust

use std::io::IsTerminal;

if io::stdin().is_terminal() {
    eprintln!("usage: tool < input.txt   (or pipe data in)");
    std::process::exit(2);
}

Without this check, a user who types the command name alone sees the program hang. Combine with the files argument pattern: read the files given, else stdin.

Binary input

rust

let mut buf = Vec::new();
io::stdin().lock().read_until(b'\n', &mut buf)?;

read_until works on bytes and does not validate UTF-8, which suits logs with mixed encodings. Use String::from_utf8_lossy to display them.

Errors and EOF

read_line returns Ok(0) at end of input, and loops that wait for a sentinel should stop on that. A closed pipe in the middle of output is a different problem, discussed in broken pipe when writing to stdout.

Where this breaks down

Very long lines allocate the whole line, so a hostile input with a 4 GB line without newlines exhausts memory. Read with a limit using take. The BufRead trait is documented in the std::io::BufRead page.

Taggedstdin