Skip to content

feat: delta runs, signed and proven rounds, one honest progress bar, gentle by default - #68

Merged
efij merged 1 commit into
mainfrom
feat/delta-integrity
Sep 30, 2026
Merged

efij merged 1 commit into
mainfrom
feat/delta-integrity

Conversation

@efij

@efij efij commented Sep 30, 2026

Copy link
Copy Markdown
Owner

Repeat agentdfir run now recomputes only what changed. Every round is signed and proven before the next one is added, one progress display covers the whole run, and the run is gentle on the machine by default.

Measured on a copy of a real case (12.5k artifacts, 3.4 GB evidence, 360k events):

Before After
Repeat run ~4:44 ~0:59
Collect, repeat run 2:22 on every other run 1 s
Repeat analysis 111 s 42 s

Delta

  • Cached parses and stored results are keyed on a generated fingerprint of the parsing and analysis code (internal/fingerprint), not the release number. TestCurrent fails if the fingerprint is stale; regenerate it with go generate ./internal/fingerprint.
  • Staleness is decided by the content of the evidence, not by the manifest's mtime. When nothing changed, run reuses the stored results.
  • The collect-time host witness refreshes the overlay per artifact instead of parsing every transcript again, and analysis decodes events once instead of seven times.
  • The results of per-artifact content scans are memoized and authenticated with the machine key (internal/artmemo).
  • Rule-pack regexes are prefiltered on literals with exact case-folding. With and without the prefilter, findings are identical on 360,521 real events, and 150k fuzz executions found no divergence.

Integrity

  • Every round is signed with a per-machine key, anchored in an anchors.jsonl file outside the case, and the seal digest is printed.
  • Before the next round starts, the previous signature, the anchor and a quick verify are checked (casepkg.ReopenChecked, after the lock is taken and before anything is written). A failure is recorded in the round and the run exits with code 4.
  • SEAL.sig is archived for every round.
  • Rounds are transactional. An unsealed round (error, Ctrl+C, crash) is rolled back to the last seal and logged as round_aborted.
  • The reused overlay is checked with segment and event hashes plus a MAC over its state.

Progress and resource use

  • internal/progress: one bar for the whole run, with elapsed time and an ETA taken from perf.jsonl history that counts down steadily.
  • internal/gentle: lowered priority, capped workers and heap, paced reads, a pause under load, and a free-disk floor (exit 5).

Fixes

  • run --sign signed before sealing.
  • Carried-forward records dropped inode/ctime, so every other run re-read the whole profile.
  • Retiring excluded records forced a full re-parse on every analysis.
  • An interrupted round broke the case's verification.
  • Carried bytes made the collect ETA drop to zero.

Verification

  • gofmt, go vet and go test -race ./... are green on macOS.
  • All six release targets cross-compile and vet.
  • The multi-round delta-vs-full equivalence test passes with and without the memo.
  • Tamper tests cover the signature, anchor, rollback, segment, events, state MAC and memo.

Generated with Claude Code

…gentle by default

Repeat runs recompute only what changed:
- cached parses and stored results are keyed on a generated fingerprint of
  the parsing and analysis code, not the release number
- staleness is decided on the evidence's content, not manifest mtime
- the collect-time host witness refreshes the per-artifact overlay instead
  of fully parsing every transcript; analysis decodes events once
- per-artifact content scans are memoized (authenticated with the machine
  key); rule-pack regexes get an exact case-folding literal prefilter and
  event subjects are prepared once per scope

Rounds are signed with a per-machine key, anchored outside the case and
proven (signature, anchor, quick verify) before the next round is added;
failures are recorded in the round and exit 4. Rounds are transactional:
an unsealed round is rolled back to the last seal and logged as
round_aborted. The reused overlay is integrity-checked (segment and event
hashes, MAC over its state).

One progress display for the whole run with elapsed time and an ETA from
this machine's timing history that counts down steadily. Gentle by
default: lowered priority, capped workers and heap, paced reads, a pause
under load and a free-disk floor.

Fixes: run --sign signed before sealing; carried-forward records dropped
their inode/ctime so alternate runs re-read everything; retiring excluded
records forced a full re-parse on every analysis; carried-forward bytes
made the collect ETA collapse to zero.

Co-Authored-By: Claude <noreply@anthropic.com>
@efij
efij merged commit 6cd2a16 into main Sep 30, 2026
6 checks passed
@efij
efij deleted the feat/delta-integrity branch September 30, 2026 09:55
@efij efij mentioned this pull request Oct 1, 2026
efij added a commit that referenced this pull request Oct 1, 2026
Cloud audit logs as a second witness (#67), delta runs with signed and
proven rounds (#68), and the Cmd/Ctrl+K command palette (#69).

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant