feat: delta runs, signed and proven rounds, one honest progress bar, gentle by default

CHANGELOG.md

@@ -10,6 +10,9 @@ and the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.
 Lessons from the 2026 agent-driven cloud attacks: Microsoft's Storm-3168,
 Sysdig's JADEPUFFER and Sygnia's AI-assisted intrusion.
 
+Repeat runs are a true delta, every round is signed and proven before the
+next is added, and a run never chokes the machine.
+
 ### Added
 - **Cloud audit logs as a second witness.** `--endpoint` (on `analyze`,
   `run` and `correlate`) now reads AWS CloudTrail, Azure Activity Log and
@@ -46,6 +49,17 @@ Sysdig's JADEPUFFER and Sygnia's AI-assisted intrusion.
   pen-tester writes exactly that.
 - Secret formats `ALIBABA_ACCESS_KEY` (`LTAI…`) and `TENCENT_SECRET_ID`
   (`AKID…`).
+- **Every round is signed** with a per-machine key (`--sign` for another
+  key, `--no-sign` to opt out), and its digest is recorded in an anchor
+  log outside the case and printed. Before a round is added, the previous
+  signature, the anchor and the sealed files are verified
+  (`--verify-prior full` also re-hashes every blob); a failure is recorded
+  in the round (`prior_integrity: FAILED`) and the run exits 4.
+- Each round archives the previous signature to `seals/SEAL.sig.<n>` and
+  records `prev_seal_sha256`, `signer` and `prior_integrity`.
+- The reused analysis overlay is integrity-checked: segment and event
+  hashes, and a MAC over its state under the machine key. Anything that
+  does not match is rebuilt from the sealed evidence.
 
 ### Changed
 - `CLOUD_CREDENTIAL_EXPORT` also covers listing storage account keys and
@@ -54,10 +68,61 @@ Sysdig's JADEPUFFER and Sygnia's AI-assisted intrusion.
 - The Protect tab's steps for a leaked secret say that deleting it from an
   issue, pull request or commit does not remove it from edit history,
   forks or caches.
+- **The content scans remember each artifact's result** (rule packs over
+  raw transcripts and configs; credential, injection and invisible-character
+  scans), keyed on its content address, its record, the analysis code and
+  the rules, and authenticated with the machine key. An unchanged
+  transcript is not scanned again. Rule-pack regexes are also prefiltered on
+  the literals every match must contain (case-folded exactly as `(?i)`
+  folds), and each event's match subject is prepared once instead of once
+  per rule. On a real case (359,000 events) a repeat analysis went from
+  111 s to 42 s, and a repeat `run` from about 5 minutes to under 1.
+- **Repeat `run` recomputes only what changed.** Cached parses and stored
+  results are keyed on a fingerprint of the parsing and analysis code, not
+  on the release number, so a release that changes no parser or rule no
+  longer re-parses and re-analyzes every case. Staleness is decided on the
+  content of the evidence, not the manifest's modification time: a round
+  that only carried files forward rebuilds nothing, and when neither the
+  evidence nor the code changed, `run` reuses the stored results outright
+  (`--reanalyze` forces it).
+  The first run after upgrading re-parses once: caches written by earlier
+  versions carry no fingerprint and no segment hashes, so they are not
+  trusted.
+- The host-witness step at collection time refreshes the per-artifact
+  overlay instead of fully parsing every transcript; analysis then finds
+  the overlay current. Analysis decodes the events once instead of seven
+  times, and writes annotated events once.
+- **One progress display for the whole run**: an overall bar, elapsed time,
+  and a time remaining predicted from this machine's own history
+  (`perf.jsonl` in the home), corrected by today's pace and counting down
+  steadily. It never reads 0:00 while work remains.
+- **Gentle by default** (`--priority gentle|background|normal`): lowered
+  CPU priority, at most half the CPUs, a soft heap limit
+  (`--max-memory-mb`), reads capped at 200 MB/s (`--max-read-mbps`), a
+  pause while the machine is under load (`--no-governor`), and a free-disk
+  floor it never crosses (`--min-free-gb`; exit 5 when a round cannot fit).
 
 ### Fixed
 - Re-running analysis with the same endpoint log appended the same
   corroboration note to an event again on every run.
+- **Every other repeat run re-read the whole profile.** A carried-forward
+  record dropped the inode and change time it was judged unchanged on, so
+  the next round had nothing to compare and re-read those files — 2.7 GB on
+  a real machine, on alternate runs.
+- **Every analysis re-parsed the whole case** once any `node_modules`/`.git`
+  records existed: retiring them from the scan set counted every excluded
+  record (and each round's own policy placeholders) as newly retired and
+  forced a full re-parse. It now counts only newly retired content, and
+  retiring is an incremental rebuild.
+- **An interrupted round no longer damages the case.** A round that never
+  sealed (an error, Ctrl+C, a crash) left records in both hash chains that
+  no seal covered, so the case stopped verifying. Rounds are now
+  transactional: an unsealed round is rolled back to the last seal and the
+  next round records it as `round_aborted`.
+- `run --sign` signed before sealing, so its signature covered the previous
+  round's `SHA256SUMS` and never verified.
+- The collect-step time remaining collapsed to about zero on repeat runs:
+  carried-forward files counted as bytes read.
 
 ## [3.1.2] — 2026-09-29
 

README.md

@@ -247,6 +247,9 @@ that case instead of producing another multi-gigabyte copy. A second round:
 - carries forward files that are unchanged by **size, inode and ctime** — never mtime alone, which any writer can set — recorded as `carried_forward` with the round that actually read them, so carried evidence is never presented as a fresh acquisition
 - stores only the **new tail** of a transcript that grew, after proving the earlier bytes still hash to what was preserved
 - continues both hash chains from their previous last line (a chain that is already broken is refused, not extended) and archives the seal that closed the previous round
+- **proves the earlier rounds first**: every round is signed (a per-machine key, or `--sign <key>`), its digest is recorded in an anchor log outside the case and printed, and the next run checks the signature, the anchor and the sealed files before adding anything. A failure is recorded in the new round for good and the run exits 4
+- **re-analyzes only what changed**: only new or grown transcripts are parsed, and when neither the evidence nor the parsing and rule code changed, the stored results are reused outright. A new release that doesn't touch parsers or rules doesn't trigger a re-analysis
+- **stays out of the way**: lowered priority, half the CPUs, a soft memory cap, paced reads, a pause while the machine is busy, and a free-disk floor it won't cross (`--priority normal` for full speed). One progress bar covers the whole run, with elapsed time and a time remaining learned from this machine's earlier runs
 
 On the same machine as above, a second run re-read 10 files, carried 8,269
 forward, and added **157 KB** to disk.

SECURITY.md

@@ -40,6 +40,38 @@ passed through a session. Treat it as you would any evidence store.
   other case holds its own real link to the bytes it needs.
 - `--no-share` keeps a case's bytes entirely inside its own directory.
 
+### Signed rounds and what they prove
+
+Every round is signed with this machine's key (`keys/seal.ed25519` in the
+home, `0600`, created on first use) unless `--sign` names another key or
+`--no-sign` is given, and each sealed round's digest is appended to
+`anchors.jsonl` in the home and printed at the end of the run. Before a
+new round is added, the previous signature, the anchor and the sealed
+files are checked; a failure is recorded in the new round permanently and
+the run exits 4.
+
+Limits, stated plainly: the key and the anchor log sit on the same machine,
+under the same account, as the evidence. They stop anyone who can only
+reach the case directory (a share, a copy, a backup restore), and they
+catch accidental corruption, but someone who controls the account can
+rewrite a case, re-sign it and edit the anchor log. The digest printed at
+the end of each run — pasted into a ticket or case notes somewhere else —
+is what that attacker cannot reach. Sign with an external key (`--sign`)
+when that matters.
+
+The analysis overlay is reused across rounds. It is authenticated with a
+key derived from the machine key and every cached file is hashed; anything
+that does not match is rebuilt from the sealed evidence, never trusted.
+
+### Resource use
+
+`agentdfir run` is gentle by default: lowered CPU priority, at most half the
+CPUs (4 workers), a soft heap limit, acquisition reads capped at 200 MB/s,
+a pause while the machine is under load, and a free-disk floor (2 GiB, or
+1% of the volume up to 5 GiB) it will not cross — it refuses to start, or
+stops cleanly, instead. `--priority background` also lowers I/O priority;
+`--priority normal` restores full speed.
+
 A collect→seal cycle holds an exclusive lock on the package. If a run is
 killed, the next run reports and reclaims the stale lock; a lock held by a
 live process is never stolen.

docs/adfir-spec.md

@@ -20,23 +20,32 @@ case.adfir/
 ├── chain-of-custody.jsonl  # sealed: hash-chained custody log
 ├── case.json               # sealed: case / operator / clock metadata, rounds
 ├── seals/SHA256SUMS.<n>    # sealed: the seal each earlier round was closed with
+├── seals/SEAL.sig.<n>      # sealed: the signature over that seal, when it was signed
 ├── SHA256SUMS              # sealed: covers the sealed zone exactly
-├── SEAL.sig                # optional: ed25519 detached signature
+├── SEAL.sig                # ed25519 signature over SHA256SUMS (every round, by default)
 ├── .lock                   # transient: held during a collect→seal cycle; not sealed
 ├── normalized/             # overlay: events / entities / relationships (JSONL)
 │   ├── events.jsonl        #   uncompressed: index/ addresses it by byte offset
 │   ├── events/<parser>/    #   per-artifact segments (.jsonl.gz), the parse cache
-│   └── state.json          #   which artifact each segment holds, and at what offset
+│   ├── state.json          #   which artifact each segment holds, at what offset, with what hash
+│   └── state.mac           #   HMAC of state.json under the machine's cache key, when it has one
 ├── detections/             # overlay: findings.json
 ├── index/                  # overlay: events.idx, the explorer's offset index
 ├── reports/                # overlay: HTML/JSON/CSV/STIX/OTel
 └── redaction-manifest.json # present only in derived support packages
 ```
 
 `SHA256SUMS` covers exactly: `case.json`, the manifest, `collection.jsonl`,
-`chain-of-custody.jsonl`, every `seals/SHA256SUMS.<n>`, and every file in
+`chain-of-custody.jsonl`, every file in `seals/`, and every file in
 `raw/`. Regenerating the overlay never changes the seal.
 
+The overlay is derived and rebuildable, but it is reused across rounds, so
+it is not trusted blindly: `state.json` records the sha256 of every cached
+segment and of `events.jsonl`, and the code fingerprint and evidence digest
+it was built from. A segment or `events.jsonl` that no longer matches, or a
+`state.json` whose `state.mac` does not verify, is discarded and rebuilt
+from the sealed zone.
+
 Everything in the overlay is gzipped except `normalized/events.jsonl`,
 which stays plaintext because `index/events.idx` records a byte offset per
 event and a gzip stream cannot be seeked. The segments under
@@ -121,11 +130,26 @@ A package may be collected into more than once. Each collection is a
   verify the whole existing chain before appending: extending a broken
   chain would hide the break behind valid-looking records.
 - Before writing a new `SHA256SUMS`, the previous one is copied to
-  `seals/SHA256SUMS.<n>` where `n` is the round it closed. Earlier sealed
-  states stay provable.
-- `case.json` gains a `rounds` array summarizing each round.
+  `seals/SHA256SUMS.<n>` where `n` is the round it closed, and the previous
+  `SEAL.sig`, if any, to `seals/SEAL.sig.<n>`. Both are covered by the new
+  `SHA256SUMS`, so every seal commits to all earlier seals and signatures.
+- `case.json` gains a `rounds` array summarizing each round. A round records
+  `prev_seal_sha256` (the sha256 of the `SHA256SUMS` it replaced), `signer`
+  (the hex ed25519 public key it was sealed for) and `prior_integrity`:
+  what checking the earlier rounds found before this round was added —
+  `verified`, `unsigned`, or `FAILED` with `prior_integrity_problems`.
 - A producer MUST hold an exclusive lock (`.lock`) for a collect→seal
   cycle. `.lock` is not evidence and is not covered by `SHA256SUMS`.
+- Rounds are transactional. Before a round writes anything, the producer
+  records the sealed state it starts from (the lengths of the manifest and
+  both chains, and `case.json`) in `.round-pending.json`. A round that is
+  not sealed — abandoned or crashed — is rolled back to exactly that state,
+  by the producer on exit or by the next round before it checks or extends
+  the package; the next round records each one as a `round_aborted` custody
+  event with what was discarded. A round is sealed once its new
+  `SHA256SUMS` is in place (written atomically, last); it is never rolled
+  back after that. `.round-pending.json` and `.round-aborted.json` are not
+  evidence and are not covered by `SHA256SUMS`.
 
 A file a later round did not re-read is recorded with
 `collection_method: "carried_forward"` and the `acquired_in_round` that
@@ -134,6 +158,33 @@ acquired. A producer MUST decide "unchanged" on properties an unprivileged
 writer cannot forge (inode plus change time); modification time alone is
 not sufficient.
 
+## Round integrity
+
+Before a producer adds a round to an existing package, and after taking
+the lock but before writing anything, it SHOULD check the earlier rounds:
+
+1. `SEAL.sig` verifies over the current `SHA256SUMS`, with the public key
+   the last signed round recorded as its `signer`;
+2. the current `SHA256SUMS` matches the producer's last recorded anchor for
+   this case (below);
+3. a quick verification (steps 1, 2, 4, 5 of the next section) passes.
+
+A failure MUST NOT stop the new round, and MUST NOT be repaired: the round
+is sealed with `prior_integrity: "FAILED"` and the problems listed, so the
+failure is part of the record from then on. A `SEAL.sig` that does not
+match when no round recorded a `signer` predates signed rounds (earlier
+`run --sign` signed before sealing); it is reported and not trusted, and it
+is not evidence of tampering.
+
+A producer SHOULD sign every round (AgentDFIR uses a per-machine key under
+its home, `keys/seal.ed25519`, unless `--sign` names another or `--no-sign`
+is given) and SHOULD record each sealed round's `SHA256SUMS` digest in an
+anchor log outside the package (AgentDFIR: `anchors.jsonl` in its home, a
+hash chain of its own) and print it. A signature proves a round was sealed
+by the holder of the key; the anchor, and the printed digest kept somewhere
+else, are what show a package was not rewritten and re-signed, or rolled
+back to an earlier copy, by someone who controls the machine.
+
 ## Compatibility
 
 Readers MUST accept both manifest forms: `manifest.jsonl` (0.2) and a