Build a Todo CLI in Rust With Serde and a JSON File

A todo list sounds like a toy, and it is the smallest program that has the whole lifecycle most command line tools need: arguments, a data file, a data format, every kind of failure and, if you look closely, concurrency. This guide builds one properly. It uses clap for arguments, serde for the file format and the standard library for locking, and each claim about crashes, ids and races is backed by a run you can repeat.

Everything compiled and passed on Rust 1.99, edition 2024, with clap 4.6, serde 1, serde_json 1, anyhow 1, dirs 7 and, for the tests, assert_cmd 2.2, predicates 3 and tempfile 3. The finished tool supports add, list (with a --json flag), done, rm and clear. If you have not met clap before, clap subcommands and flags goes through the derive attributes in more depth, and serde_json basics covers the file format.

Plan the data before the commands

A command line tool that stores data has one design decision that is expensive to change later: what the file looks like. The todo list needs three facts about each item (an id, a title, whether it is done), plus one fact about the list as a whole: the next id to hand out.

That last field is the one beginners skip, and it is the reason this guide exists. The tempting scheme is to use an item's position in the list as its id. It breaks the first time you remove something: with three items, done 3 completes the third, you remove item 1, and now done 3 refers to nothing because the list has two items. Worse, if you add an item after removing, ids shift under the user's hands. The user typed 3 after reading 3 on their screen; they meant the item they saw. An id has to stay attached to its item for its whole life, which means it must be stored, and the counter that hands the next one out must be stored with it so it never goes backwards.

The file ends up like this after a few commands:

{
  "next_id": 2,
  "items": [
    { "id": 2, "title": "write the post", "done": false }
  ]
}

Item 1 was completed and removed; the counter still says 2, so the next item is 3 and the id 1 is never reissued. That property is what the ids_are_not_reused_after_remove test checks later.

Project setup

[package]
name = "todo"
version = "0.1.0"
edition = "2024"

[dependencies]
anyhow = "1.0.104"
clap = { version = "4.6.7", features = ["derive"] }
dirs = "7.0.0"
serde = { version = "1.0.229", features = ["derive"] }
serde_json = "1.0.154"

[dev-dependencies]
assert_cmd = "2.2.2"
predicates = "3.1.4"
tempfile = "3.27.0"

clap with the derive feature gives the argument parser from annotated types. anyhow carries errors with context to main; dirs finds the platform's data directory (~/Library/Application Support on macOS, ~/.local/share on Linux). Run cargo new todo and add them with cargo add to get the same manifest.

The store: loading, saving, and what each failure means

The file handling goes in its own module so that the commands stay small:

use std::{
    fs::{self, File, OpenOptions},
    io::{self, Write},
    path::{Path, PathBuf},
};

use anyhow::{Context, Result};
use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize, Clone, PartialEq)]
pub struct Todo {
    pub id: u32,
    pub title: String,
    pub done: bool,
}

#[derive(Debug, Serialize, Deserialize, Default)]
pub struct Store {
    pub next_id: u32,
    pub items: Vec<Todo>,
}

pub fn data_path() -> PathBuf {
    if let Some(p) = std::env::var_os("TODO_FILE") {
        return PathBuf::from(p);
    }
    dirs::data_dir()
        .unwrap_or_else(|| PathBuf::from("."))
        .join("todo")
        .join("todos.json")
}

impl Store {
    pub fn load(path: &Path) -> Result<Store> {
        match fs::read_to_string(path) {
            Ok(text) => serde_json::from_str(&text)
                .with_context(|| format!("{} is not valid JSON", path.display())),
            Err(e) if e.kind() == io::ErrorKind::NotFound => Ok(Store::default()),
            Err(e) => Err(e).with_context(|| format!("reading {}", path.display())),
        }
    }

    pub fn save(&self, path: &Path) -> Result<()> {
        if let Some(dir) = path.parent() {
            fs::create_dir_all(dir)?;
        }
        // The temp name carries the pid so two writers never share a temp file.
        let tmp = path.with_extension(format!("json.{}.tmp", std::process::id()));
        let mut file = File::create(&tmp)?;
        file.write_all(serde_json::to_string_pretty(self)?.as_bytes())?;
        // Without this the rename can reach the disk before the data does.
        file.sync_all()?;
        fs::rename(&tmp, path)?;
        Ok(())
    }

    pub fn add(&mut self, title: String) -> u32 {
        self.next_id += 1;
        self.items.push(Todo {
            id: self.next_id,
            title,
            done: false,
        });
        self.next_id
    }

    pub fn complete(&mut self, id: u32) -> Option<&Todo> {
        let item = self.items.iter_mut().find(|t| t.id == id)?;
        item.done = true;
        Some(item)
    }

    pub fn remove(&mut self, id: u32) -> bool {
        let before = self.items.len();
        self.items.retain(|t| t.id != id);
        self.items.len() != before
    }
}

/// Load, change and save while holding an exclusive lock on a sidecar file.
/// Without the lock two processes can read the same state and the later write wins.
pub fn update<T>(path: &Path, change: impl FnOnce(&mut Store) -> Result<T>) -> Result<T> {
    if let Some(dir) = path.parent() {
        fs::create_dir_all(dir)?;
    }
    let lock: File = OpenOptions::new()
        .create(true)
        .write(true)
        .truncate(false)
        .open(path.with_extension("lock"))?;
    lock.lock()?; // released when `lock` is dropped, even if `change` fails

    let mut store = Store::load(path)?;
    let out = change(&mut store)?;
    store.save(path)?;
    Ok(out)
}

Go through it from the top, because every function encodes a decision.

data_path checks TODO_FILE before anything else. That one environment variable is what makes the tool testable: the tests point it at a temp directory, and so do the experiments below, so nothing ever touches your real list. Only when the variable is absent does it fall back to the platform data directory.

Store::load distinguishes three outcomes of reading the file. A missing file is a new user, so it returns an empty Store::default(). A file that exists but does not parse is an error naming the file; a file that cannot be read for another reason (permissions) is an error naming the file too. The line to notice is that a corrupt file is never replaced by an empty list. If the code ran unwrap_or_default() it would be shorter, and the next add would overwrite the damaged file with a clean one, quietly deleting whatever the user could have recovered by hand. The corrupt_file_is_an_error_not_a_reset test checks both halves of that: the command fails with is not valid JSON, and the broken file is byte for byte unchanged afterwards.

Store::save writes to a temporary file and renames it over the real one. Rename replaces a file in one step on the same filesystem, so a reader sees either the whole old file or the whole new one, never half of each. The naive version, fs::write(path, ...), truncates the file first and then writes, so a crash or a full disk in between leaves a half-written file, and the next load reports it as corrupt. Two details beyond the textbook version matter. The temp file's name includes the process id, so two processes saving at once do not write the same temp file and interleave their bytes. And file.sync_all() runs before the rename: without it the operating system may persist the rename before it persists the contents, and a power cut can leave a correctly named but empty file. sync_all costs time, which the end of this guide measures.

add, complete and remove are plain methods on Store. add increments next_id first and uses the new value, so ids start at 1. complete returns Option<&Todo> and remove returns a bool, which lets the caller decide what "not found" means rather than burying a message in the data layer.

The commands

main.rs is argument parsing and glue:

mod store;

use anyhow::{Result, bail};
use clap::{Parser, Subcommand};
use store::{Store, data_path, update};

#[derive(Parser)]
#[command(version, about = "A tiny todo list")]
struct Cli {
    #[command(subcommand)]
    command: Cmd,
}

#[derive(Subcommand)]
enum Cmd {
    /// Add an item
    Add { title: Vec<String> },
    /// List items
    List {
        /// Include completed items
        #[arg(short, long)]
        all: bool,
        /// Print JSON instead of a table, for other programs to read
        #[arg(long)]
        json: bool,
    },
    /// Mark an item done
    Done { id: u32 },
    /// Remove an item
    Rm { id: u32 },
    /// Remove every completed item
    Clear,
}

fn main() -> Result<()> {
    let path = data_path();

    match Cli::parse().command {
        Cmd::Add { title } => {
            if title.is_empty() {
                bail!("give the item a title");
            }
            let id = update(&path, |s| Ok(s.add(title.join(" "))))?;
            println!("added {id}");
        }
        Cmd::List { all, json } => {
            let store = Store::load(&path)?;
            let shown: Vec<_> = store.items.iter().filter(|t| all || !t.done).collect();
            if json {
                println!("{}", serde_json::to_string(&shown)?);
            } else {
                for t in shown {
                    println!(
                        "{:>3} [{}] {}",
                        t.id,
                        if t.done { "x" } else { " " },
                        t.title
                    );
                }
            }
        }
        Cmd::Clear => {
            let removed = update(&path, |s| {
                let before = s.items.len();
                s.items.retain(|t| !t.done);
                Ok(before - s.items.len())
            })?;
            println!("removed {removed}");
        }
        Cmd::Done { id } => update(&path, |s| match s.complete(id) {
            Some(_) => Ok(()),
            None => bail!("no item with id {id}"),
        })?,
        Cmd::Rm { id } => update(&path, |s| {
            if !s.remove(id) {
                bail!("no item with id {id}");
            }
            Ok(())
        })?,
    }
    Ok(())
}

#[derive(Parser)] turns Cli into the parser and #[derive(Subcommand)] turns the enum into the commands. The /// doc comments on the variants become the help text, which you can see with todo --help. Add { title: Vec<String> } collects every remaining word, so todo add buy milk works without quotes; the program joins them back with spaces. List has two boolean flags, --all/-a and --json, both off by default.

list is the one command that only reads, so it does not call update. It takes no lock and cannot block behind a writer. It prints a right aligned id in a width of three, a checkbox and the title, or the same items as one line of JSON when --json is given.

Try the commands. This session ran against a fresh file:

$ todo add buy milk
added 1
$ todo add write the post
added 2
$ todo done 1
$ todo list
  2 [ ] write the post
$ todo list --all
  1 [x] buy milk
  2 [ ] write the post
$ todo done 9
Error: no item with id 9
$ todo list
Error: /tmp/t.json is not valid JSON

Caused by:
    key must be a string at line 1 column 3
(file still holds: { nope)

Note the behaviour at the end. With a corrupt file, list stops with an error that names the file and the parse problem, and afterwards the file still contains exactly what it did. And done 9 fails with a plain message and a non-zero exit status, which scripts can test for.

The bug a single user never sees

The tool above works perfectly when you run one command at a time. People do not use tools that way. A shell script runs todo add in a loop; two terminals are open; a cron job and a human collide. Every command here is a read, a change and a write, and nothing stops two of them from interleaving. Process A reads the file with 3 items, process B reads the same 3 items, A writes 4, B writes 4: B's version has no trace of A's item. This is a lost update.

To see it, remove the lock and start 40 todo add processes at the same time against one empty file:

(40 `todo add` processes started at once against one empty file)
without the lock: 1 of 40 items survived
without the lock: 2 of 40 items survived
without the lock: 3 of 40 items survived
with the lock   : 40 of 40 items survived
with the lock   : 40 of 40 items survived
with the lock   : 40 of 40 items survived

Out of 40 adds, one to three survived. That number is not a typo. Each process finishes its sync_all and rename, and the last one to rename wins the file; the others' items are overwritten. The fsync, ironically, made it worse: the longer the write takes, the longer the window in which another process can read stale data. Without the sync the same experiment left 17 to 26 of 40 on this machine, which is also a disaster, just a less obvious one.

The fix is in update, the function every mutating command calls. It opens a sidecar file (todos.lock beside todos.json) and takes an exclusive lock on it with File::lock, which has been in the standard library since Rust 1.89. The lock is held for the whole load, change and save sequence and is released when the File is dropped, which also happens if the closure returns an error or panics. Other processes calling lock block until it is free, so they run one at a time. With the lock back in place, all three runs of the same 40-process experiment kept 40 of 40.

Why a sidecar file rather than locking the data file itself? Because save replaces the data file with a rename. A lock belongs to the file *object*, not to the name, so after the rename the next process would be locking a different file than the one the previous holder held, and two writers could be inside the critical section together. The lock file is never renamed, so it is a stable thing to lock.

One more consequence: the lock only protects processes that use it. A text editor that opens todos.json and saves it is not taking the lock. For a personal tool that is the correct boundary; if other programs write the file, switch to SQLite, which solves this properly.

Testing the real binary

The tests run the compiled program with a temp directory, which is how a user meets it:

use assert_cmd::Command;
use predicates::str::contains;

fn todo(file: &std::path::Path) -> Command {
    let mut cmd = Command::cargo_bin("todo").unwrap();
    cmd.env("TODO_FILE", file);
    cmd
}

#[test]
fn add_done_list() {
    let dir = tempfile::tempdir().unwrap();
    let file = dir.path().join("t.json");

    todo(&file)
        .args(["add", "buy", "milk"])
        .assert()
        .success()
        .stdout("added 1\n");
    todo(&file)
        .args(["add", "write", "post"])
        .assert()
        .success()
        .stdout("added 2\n");
    todo(&file).args(["done", "1"]).assert().success();

    todo(&file)
        .arg("list")
        .assert()
        .success()
        .stdout("  2 [ ] write post\n");
    todo(&file)
        .args(["list", "--all"])
        .assert()
        .success()
        .stdout(contains("[x] buy milk"));
}

#[test]
fn ids_are_not_reused_after_remove() {
    let dir = tempfile::tempdir().unwrap();
    let file = dir.path().join("t.json");

    todo(&file).args(["add", "a"]).assert().success();
    todo(&file).args(["rm", "1"]).assert().success();
    todo(&file)
        .args(["add", "b"])
        .assert()
        .success()
        .stdout("added 2\n");
}

#[test]
fn corrupt_file_is_an_error_not_a_reset() {
    let dir = tempfile::tempdir().unwrap();
    let file = dir.path().join("t.json");
    std::fs::write(&file, "{ this is not json").unwrap();

    todo(&file)
        .arg("list")
        .assert()
        .failure()
        .stderr(contains("is not valid JSON"));
    assert_eq!(
        std::fs::read_to_string(&file).unwrap(),
        "{ this is not json"
    );
}

#[test]
fn unknown_id_fails_with_a_message() {
    let dir = tempfile::tempdir().unwrap();
    let file = dir.path().join("t.json");
    todo(&file)
        .args(["done", "9"])
        .assert()
        .failure()
        .stderr(contains("no item with id 9"));
}

#[test]
fn json_output_and_clear() {
    let dir = tempfile::tempdir().unwrap();
    let file = dir.path().join("t.json");

    todo(&file).args(["add", "one"]).assert().success();
    todo(&file).args(["add", "two"]).assert().success();
    todo(&file).args(["done", "1"]).assert().success();

    todo(&file)
        .args(["list", "--all", "--json"])
        .assert()
        .success()
        .stdout("[{\"id\":1,\"title\":\"one\",\"done\":true},{\"id\":2,\"title\":\"two\",\"done\":false}]\n");

    todo(&file)
        .arg("clear")
        .assert()
        .success()
        .stdout("removed 1\n");
    todo(&file)
        .args(["list", "--all"])
        .assert()
        .success()
        .stdout("  2 [ ] two\n");
    // Ids stay unique even though the completed item is gone.
    todo(&file)
        .args(["add", "three"])
        .assert()
        .success()
        .stdout("added 3\n");
}

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test unknown_id_fails_with_a_message ... ok
test corrupt_file_is_an_error_not_a_reset ... ok
test ids_are_not_reused_after_remove ... ok
test add_done_list ... ok
test json_output_and_clear ... ok
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.36s

assert_cmd::Command::cargo_bin("todo") builds and runs the real binary, and .env("TODO_FILE", ...) is the seam described earlier. The first test walks a normal session and asserts the exact output, including the numbering. The second proves the counter survives a removal. The third and fourth pin the failure behaviour: corrupt data is an error and unknown ids exit non-zero with a message. The fifth covers --json and clear together. These tests are about behaviour a user can see, not about functions, so they survive a rewrite of the internals. Testing CLIs with assert_cmd goes through the rest of the API.

Adding a flag the compiler checks for you

--json and clear were added after the first version worked, and the way the compiler handled the change is worth showing because it is the reason enum-based subcommands are pleasant. Adding json: bool to the List variant without touching the match produces this:

error[E0027]: pattern does not mention field `json`
  --> src/main.rs:46:9
   |
46 |         Cmd::List { all } => {
   |         ^^^^^^^^^^^^^^^^^ missing field `json`
   |
help: include the missing field in the pattern

The error is E0027, "pattern does not mention field", and it points at the exact arm that needs updating. A hand-written argument parser built on args[1] == "list" would have compiled, ignored the flag and shipped. With the derive, every place that handles List is forced to decide what to do with the new field, and adding the Clear variant forces a new arm in the same match the same way. Once the match handles both, the commands work end to end:

$ todo add buy milk
added 1
$ todo add write the post
added 2
$ todo add ship it
added 3
$ todo done 1
$ todo list --all --json
[{"id":1,"title":"buy milk","done":true},{"id":2,"title":"write the post","done":false},{"id":3,"title":"ship it","done":false}]
$ todo clear
removed 1
$ todo list --all
  2 [ ] write the post
  3 [ ] ship it

list --all --json prints one line, which a script can pass to jq or parse with any JSON library, and the ids and the done flag are the same values the table shows. Separating the data (shown, a filtered Vec of references) from its presentation (a table or a JSON line) is why the new flag was five lines. The same split makes it easy to add --format csv later.

clear goes through update, so it takes the lock like every other write, and it reports how many items it removed. Notice what it does not do: it never resets next_id. The test json_output_and_clear ends by adding a fourth item and expecting id 3, which proves that clearing the completed items does not reuse ids either.

Help text and exit codes are part of the interface

The /// comments on the variants and arguments are not just documentation for the source. They are the --help output a user reads before anything else:

A tiny todo list

Usage: todo <COMMAND>

Commands:
  add    Add an item
  list   List items
  done   Mark an item done
  rm     Remove an item
  clear  Remove every completed item
  help   Print this message or the help of the given subcommand(s)

Options:
  -h, --help     Print help
  -V, --version  Print version

Three habits make the tool pleasant to script. Errors go to standard error and the exit status is non-zero, which anyhow and a main that returns Result give you for free: a failing done 9 exits with status 1 and prints Error: no item with id 9. Successful commands print a result on standard output that is stable and easy to parse (added 3, removed 1). And reading commands never write: list leaves the file untouched, so you can run it from a prompt or a status bar on every keystroke without disturbing a concurrent add.

Where the file lives

A tool that keeps data in the current directory loses it the moment the user changes directory, and one that writes next to the executable fails the moment the binary is installed somewhere read only. The dirs crate asks the operating system for the conventional per-user data directory, so the same code stores the list under ~/Library/Application Support/todo/ on macOS, ~/.local/share/todo/ on Linux and %APPDATA% on Windows. Respecting that convention is what lets users find, back up and delete the data without reading your source.

Creating the directory is the part people forget. A first run has no todo folder, so save and update both call create_dir_all on the parent before they write, which is a no-op when the folder exists. If you skip it, the first command a new user runs fails with "No such file or directory" and the error does not mention which path was missing, which is the worst possible first impression of a tool.

What sync_all costs

Durability is not free. A hundred sequential todo add x runs against one file took 0.92 seconds in total on this machine, about 9 milliseconds each, and that includes starting the process, taking the lock, parsing a file that grows to a hundred items, and the fsync. For a todo list that is imperceptible. For a tool that writes thousands of records per second it would be a reason to batch writes or use a database. The honest summary is that atomic rename answers "can a crash leave half a file", sync_all answers "can a crash leave an empty file", and the lock answers "can two runs lose each other's work". Each costs a few lines and guards a different failure.

Where this design stops

A JSON file read and rewritten in full on every command is fine up to a few thousand items. At a hundred thousand it takes noticeable time, because each add parses and serialises everything. That is the point to move to SQLite through rusqlite, which also gives you real concurrency control for free. The lock here is advisory, so it works for cooperating copies of this program on one machine; it does not work across a network filesystem reliably, and it does nothing about two machines syncing the file with a cloud drive. If you need that, you are building a sync tool, and a different one.

Platform behaviour also differs: File::lock uses flock on Unix and LockFileEx on Windows, and rename over an existing file works on both, but on Windows it can fail if another process has the target open. That is acceptable here because the target is only ever opened briefly by the same program, under the lock.

Ideas for next steps

Add edit to change a title, a --due flag that parses a date, and a clear --done command that removes completed items in one pass. Print the list as JSON with --json so other tools can consume it, which is a good exercise in separating data from presentation. Then package the tool: clap to a shipped release walks through publishing it, and cross-compiling covers building for other platforms.