disktree --du: GNU du as a headless mode of disktree, with a persistent index

AGENTS.md

@@ -119,6 +119,7 @@ and `cargo build --release` directly; CI runs the gate on both systems.
 | what macOS lists and measures in bulk | `crates/disktree-core/src/macos.rs` — `unsafe` for `getattrlistbulk` |
 | what Windows lists, measures and compares differently | `crates/disktree-core/src/windows.rs` — `unsafe` for Win32 |
 | reading a whole NTFS drive from its file table | `crates/disktree-core/src/mft.rs` |
+| GNU du arguments, walk, output and index | `crates/disktree-core/src/du/` |
 | a key, a screen transition, a mark | `crates/disktree-app/src/state.rs` |
 | spacing, type and size | `crates/disktree-app/src/ui.rs` — tokens only, no `px` in layout |
 | the mosaic's painting or labels | `crates/disktree-app/src/treemap_view.rs` |

Cargo.toml

@@ -1,6 +1,10 @@
 [workspace]
 resolver = "3"
-members = ["crates/disktree-app", "crates/disktree-core", "xtask"]
+members = [
+    "crates/disktree-app",
+    "crates/disktree-core",
+    "xtask",
+]
 
 [workspace.package]
 version = "0.11.0"

README.md

@@ -372,6 +372,85 @@ tested:
   fsmonitor, hooks and pager off, and a checkout that defines its own filter
   drivers is not asked for its status at all ("changes unknown").
 
+## As `du`
+
+`disktree --du` is GNU du: the same options, output, messages and exit
+status. It runs headless from the existing `disktree` binary, so it needs no
+display and adds no second binary to install. Everything after `--du` is a
+GNU du argument.
+
+```sh
+disktree --du -sh ~/src
+ln -s "$(command -v disktree)" ~/.local/bin/du  # early on PATH
+du -sh ~/src
+```
+
+When invoked through a link whose basename is `du`, the same binary enters
+this mode automatically. Existing scripts and coding agents can keep calling
+`du` without learning a new command.
+
+It follows coreutils 9.11. `crates/disktree-app/tests/gnu.rs` runs 78 argument
+sets against the GNU du it finds (`gdu` from Homebrew's coreutils, or a `du`
+that is GNU's) and compares output, messages and exit status. GNU du 9.4 and
+later match, apart from two corner cases GNU changed since then:
+`--max-depth` below zero, and the wording of a bad `--time-style`.
+
+It is faster three ways:
+
+- The walk runs on every core. Each directory is still visited in the order
+  GNU's `fts` visits it, so hard links, `-x`, `--exclude`, depths and
+  thresholds come out the same.
+- On macOS, `getattrlistbulk` reads a directory's entries and their sizes in
+  one system call instead of one per file.
+- Every walk is kept as a snapshot, in `~/Library/Caches/disktree/du` on
+  macOS and `~/.cache/disktree/du` elsewhere (`DISKTREE_DU_INDEX_DIR` moves
+  it). With `--max-age=AGE`, a snapshot younger than `AGE` is brought up to
+  date instead of walked again. On macOS the FSEvents journal names every
+  directory that changed since, and only those are read. Elsewhere every
+  directory is stat'ed and read again if it changed.
+
+On a 670 GB tree of 6.65 million entries (Apple M5 Max, APFS, warm cache):
+
+| command | time |
+| --- | --- |
+| GNU du 9.11, `du -s` | 148 s |
+| `disktree --du -s` | 25 s |
+| `disktree --du -s --max-age=1h`, through the journal | 2 to 3 s |
+| `disktree --du -s --max-age=1h`, by directory | 3.4 s |
+
+Without `--max-age` every answer is a full walk, so it is what GNU du would
+say. With it, two things can be missed. A file still open for writing is not
+in the journal until it is closed, so files modified within an hour of the
+snapshot are stat'ed again every time, which covers the usual case, a log or
+a build still being written. And by directory, a file that grew in place
+without its directory changing is only caught if it was one of those recent
+files. Either way `AGE` counts from the last full walk, and a catch-up does
+not reset it, so nothing can go unseen for longer than `AGE`. `--fresh` walks
+everything again.
+
+Known differences from GNU du, all rare:
+
+- `du >&-` exits 0: Rust reopens a closed standard output on `/dev/null`
+  before `du` starts, so the write error GNU reports never happens.
+- Hard links to one file in the same directory of more than 10,000 entries
+  can be listed in another order, since GNU sorts those with the C library's
+  unstable `qsort`.
+- Numbers use the C locale's `.` and no digit grouping, and `--time` formats
+  go through chrono rather than the C library's `strftime`: `full-iso`,
+  `long-iso` and `iso` match, but `%Z` prints an offset and `%x` a two-digit
+  year.
+- In a UTF-8 locale, `--exclude` wildcards match bytes rather than
+  characters, and diagnostics do not escape control characters inside
+  `‘...’` quotes.
+
+`--json` prints the same entries as one document, each with its size, its
+kind, why it can be had back, the command that gives the space back (`cargo
+clean`, `pnpm store prune`, …) and when it was last written, plus `as_of`
+and `source` (`walk`, `journal` or `directories`) for the whole answer.
+Behind a `du` symlink the same switches come from the environment:
+`DISKTREE_DU_JSON=1`, `DISKTREE_DU_MAX_AGE=1h`, `DISKTREE_DU_FRESH=1`, and
+`DISKTREE_DU_INDEX=0` to keep no snapshots.
+
 ## On Hyprland
 
 Hyprland tiles new windows, so disktree opens into whatever tile it is given.
@@ -406,6 +485,8 @@ gone while their neighbours are not.
 | `crates/disktree-app/src/treemap_view.rs` | painting the mosaic and its labels |
 | `crates/disktree-app/src/ui.rs` | the spacing, type and size scale, in `rem` |
 | `crates/disktree-app/src/tests.rs` | end-to-end tests through a real window |
+| `crates/disktree-core/src/du` | GNU du on a parallel walk and its snapshots |
+| `crates/disktree-app/tests/gnu.rs` | comparison against GNU du |
 | `packaging/`, `assets/`, `Makefile` | the desktop entry, the icon, and install |
 
 The interface follows the

crates/disktree-app/Cargo.toml

@@ -31,3 +31,6 @@ gpui-kit = { version = "=0.6.6", default-features = false, features = [
     "assets",
     "test-support",
 ] }
+
+[target.'cfg(unix)'.dev-dependencies]
+rustix = { version = "1.1", features = ["fs"] }

crates/disktree-app/src/main.rs

@@ -25,7 +25,9 @@ mod ui;
 mod views;
 mod widgets;
 
-use std::path::PathBuf;
+use std::ffi::{OsStr, OsString};
+use std::path::{Path, PathBuf};
+use std::process::ExitCode;
 #[cfg(target_os = "macos")]
 use std::{
     io::IsTerminal as _,
@@ -81,15 +83,42 @@ options:
   -h, --help            show this help
 ";
 
-fn main() -> Result<()> {
+fn main() -> Result<ExitCode> {
     #[cfg(windows)]
     console::attach();
-    let outcome = run();
+    let mut argv = std::env::args_os();
+    let argv0 = argv.next().unwrap_or_else(|| OsString::from("disktree"));
+    let outcome = match du_invocation(&argv0, argv) {
+        Some((program, arguments)) => {
+            Ok(disktree_core::du::run(&program, &arguments))
+        }
+        None => run().map(|()| ExitCode::SUCCESS),
+    };
     #[cfg(windows)]
     console::detach();
     outcome
 }
 
+/// Select the `du` personality before loading settings or starting GPUI.
+///
+/// Keeping headless personalities at this boundary also leaves one obvious
+/// dispatch point for a future text-tree mode such as issue #74 proposes.
+fn du_invocation(
+    argv0: &OsStr,
+    mut arguments: impl Iterator<Item = OsString>,
+) -> Option<(Vec<u8>, Vec<OsString>)> {
+    if Path::new(argv0).file_name() == Some(OsStr::new("du")) {
+        return Some((
+            argv0.to_owned().into_encoded_bytes(),
+            arguments.collect(),
+        ));
+    }
+    if arguments.next().as_deref() == Some(OsStr::new("--du")) {
+        return Some((b"du".to_vec(), arguments.collect()));
+    }
+    None
+}
+
 fn run() -> Result<()> {
     let saved = power::settings_path()
         .map_or_else(

crates/disktree-app/src/tests.rs

@@ -5,6 +5,7 @@
 //! that panics while painting, a binding that never fires, a removal that
 //! reports success without removing anything.
 
+use std::ffi::{OsStr, OsString};
 use std::path::{Path, PathBuf};
 
 use disktree_core::removal::RemovalMode;
@@ -48,6 +49,29 @@ fn options() -> ScanOptions {
     }
 }
 
+#[test]
+fn du_mode_is_selected_only_at_the_headless_dispatch_point() {
+    let args = [OsString::from("--du"), OsString::from("-sh")];
+    let (program, rest) =
+        super::du_invocation(OsStr::new("/usr/bin/disktree"), args.into_iter())
+            .expect("--du mode");
+    assert_eq!(program, b"du");
+    assert_eq!(rest, [OsString::from("-sh")]);
+
+    let args = [OsString::from("-sh")];
+    let (program, rest) =
+        super::du_invocation(OsStr::new("/tmp/du"), args.into_iter())
+            .expect("du multicall");
+    assert_eq!(program, b"/tmp/du");
+    assert_eq!(rest, [OsString::from("-sh")]);
+
+    let args = [OsString::from("--help")];
+    assert!(
+        super::du_invocation(OsStr::new("/usr/bin/disktree"), args.into_iter())
+            .is_none()
+    );
+}
+
 type Window = VisualTestContext;
 
 /// Open a window over a real scan of `root`, without the background walk.