Skip to content

feat(stories): delivery-record checks — the record against itself, and against GitHub - #308

Open
ophiocus wants to merge 1 commit into
theam:mainfrom
ophiocus:feat/delivery-record-checks
Open

feat(stories): delivery-record checks — the record against itself, and against GitHub#308
ophiocus wants to merge 1 commit into
theam:mainfrom
ophiocus:feat/delivery-record-checks

Conversation

@ophiocus

@ophiocus ophiocus commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

TL;DR — Read-only checks that the delivery record agrees with itself (the turn_git_evidence row against its two story-evidence events, at the start and at settlement) and with GitHub (the head SHAs the mirror stored as a witness). One module that returns findings; it surfaces nothing yet, gates nothing, and needs no migration. 16 tests, each "break the promise → the report names it". Where findings should show up is a product choice, proposed separately so it can be discussed on its own.

Follow-on to #264, taking @javiertoledo's "up to you" as the brief. #289 retired the receipt machinery and, with it, the only verifier the tree had — deliberately, and I think rightly: the mediation layer was where the hard defects lived (#195, #233, #226 were all bugs in the machinery of distrust, not in the work it guarded). What 0.12 kept instead is a delivery record that Insights, the story timeline, and turn attribution all stand on — and nothing can notice when that record stops agreeing with itself. This PR adds the noticing, as a read, using only what 0.12 already writes.

Credit where it starts: @Julian-Genuario's #226, @javiertoledo's re-verification and diagnosis on #264, and @adrian-lorenzo's #289, whose double-write is the reason this needs no digest of its own.

The class, in one sentence

Forward-only verification can't see deletion. — #264

A verifier that enumerates from the ledger it is verifying can only confirm what survived; absence has no row to iterate. The old fix was a reverse pass in one file. That file is gone, but the new tree writes each turn's facts in more than one place from more than one code path, so the reverse pass becomes cross-ledger reads — every check enumerates from a ledger other than the one it verifies, and reports over that denominator:

check enumerates from verifies
completeness turns (settled) turn_git_evidence
coherence the evidence row the turn:{id}:context and turn:{id}:git story-evidence events
attribution GitHub's reported head SHAs both internal tellings

What 0.12 already writes (nothing new is written to detect)

  1. Two tellings of every turn, at both ends. TurnGitEvidenceService.start() writes the row's initial branch/SHA and a turn:{id}:context event with the same starting point; complete() writes the final SHA, commits, changed files and dirty state into turn_git_evidence and a turn:{id}:git event with the same facts — or, on the failure path, a captureError on the row and a git.capture_failed event under the same key. Two ledgers, one writer; the comparer was the only thing missing.
  2. A third telling on a channel the database does not write. The mirror stores the head SHAs GitHub reports — branches, pull heads, CI heads — arriving over the HMAC-verified webhook and the authenticated mirror scan. The mirror already uses finalSha == headSha to link a pull request to the exact turn that produced it. Read the other way, those heads are a witness: content-addressed, and not ours to rewrite.

The checks — each one a promise 0.12 makes, made checkable

Completeness"Final SHA, commits, changed files, and dirty state are captured when it settles, including failure and cancellation paths." Every settled turn carries either a completed capture or an explicit captureError. A settled turn with neither is missing-evidence or unsettled-capture.

Coherence — the row and its events agree. At the start: the context event exists (missing-context) and matches the row on initialSha, initialBranch, workspaceId, engineSessionId. At settlement: a completed capture is paired with git.changes_recorded and agrees on initialSha, finalSha, the commit sequence and dirty; a capture error is paired with git.capture_failed. A missing pair is missing-event; a wrong or disagreeing one is divergent. Insights and the timeline read from this substrate, so divergence means the surfaces are wrong, not merely inconsistent.

Attribution"GitHub facts are linked to an exact Facility turn when their head SHA matches that turn's recorded final SHA." A settled turn whose recorded final SHA GitHub has reported is witnessed (witnessedTurns makes coverage visible). When the two internal tellings disagree about the final SHA and GitHub has seen exactly one of them, the finding becomes witness-disagrees and names the side that drifted. Both seen, or neither: it stays a coherence finding — the witness never guesses. An unwitnessed turn is not a finding; it simply hasn't travelled that far yet.

Proof — every shape is "break the promise, watch the report"

services/api/test/delivery-record.integration.test.ts (16 tests, scoped fixture org):

mutate the report says
clean lifecycle (row, context event, git event all agree) 0 findings
honest capture error paired with its git.capture_failed event 0 findings
delete the evidence row of a settled turn completeness: missing-evidence
null finalSha and erase captureError completeness: unsettled-capture
delete the turn:{id}:context event coherence: missing-context
context event names a different starting SHA coherence: divergent — "row and context event disagree on initialSha"
capture error on the row, its git.capture_failed event never landed coherence: missing-event
completed capture, but the event on record says the capture failed coherence: divergent
delete the turn:{id}:git event coherence: missing-event
change the event's finalSha coherence: divergent
…then GitHub reports the row's SHA attribution: witness-disagrees — "the evidence event drifted"
GitHub reports the event's SHA instead "the evidence row drifted"
GitHub has seen neither stays coherence: divergent
walk the record with limit: 4 and the cursor same findings, same witnessedTurns, nothing double-counted
another org's project never inspected

Suite: services/api 24 files / 155 tests green (1 pre-existing opt-in skip); tsc and biome clean.

Bounded by construction

Read-only; keyset-paginated over turns on the existing (orgId, projectId) scope; every companion query — evidence rows, events, and the GitHub witness lookup — is an IN over the page. Memory does not grow with org history (#295 absorbed rather than inherited). Findings are counted against checkedTurns — all settled turns — never against survivors.

Limits, stated rather than papered over

  • The witness corroborates SHAs, and only for work that reached GitHub. Commit lists and dirty state have no external telling in 0.12; they are checked ledger-to-ledger only.
  • The settled turn is the unit. A turn deleted whole is visible only from outside the unit — a GitHub head no turn claims, which reconciliation already keeps as an honest, unattributed category.
  • This observes accidental corruption, control-plane bugs, reconciliation drift and silent data loss. It is not a defence against a principal with write access to both internal ledgers; only the external witness constrains that, and only where it overlaps.

Non-goals — and what is deliberately not in this PR

No receipts, no approvals, no leases, no per-agent profiles, no gating of any operation, ever. No migration.

And no surface: this PR adds no attention items, changes no health, schedules no job. The checks are facts about the record; where those facts should appear — and whether a finding should ever change what a person sees on a story — is a product call I would rather discuss than assume. Those come as separate proposals.

Files

  • services/api/src/stories/delivery-record.tsverifyDeliveryRecord (new)
  • services/api/test/delivery-record.integration.test.ts

stories/evidence.ts and turns/git-evidence.ts — the two writers — are untouched.

Verify locally

docker compose -f docker-compose.dev.yml up -d postgres
pnpm --filter @facility/api exec vitest run test/delivery-record.integration.test.ts
pnpm --filter @facility/api test
pnpm typecheck && pnpm lint

🤖 Generated with Claude Code

…d against GitHub

Read-only checks over what 0.12 already writes, returning findings; nothing
here gates, and nothing here is surfaced yet.

- completeness: every settled turn (failure and cancellation paths included)
  carries a completed capture or an explicit captureError.
- coherence: the turn_git_evidence row agrees with its two story-evidence
  events — turn:{id}:context at the start (initialSha, initialBranch,
  workspaceId, engineSessionId) and turn:{id}:git at settlement
  (initialSha, finalSha, commit sequence, dirty) — and each end is paired
  with the right event type: git.changes_recorded for a completed capture,
  git.capture_failed for an explicit failure.
- attribution: the head SHAs GitHub reported (branches, pull heads, CI
  heads, via the signed webhook or the mirror scan) form a witness. A turn
  whose final SHA GitHub has seen is witnessed; when the two internal
  tellings disagree on the final SHA and GitHub has seen exactly one, the
  finding names the drifted side. Neither or both seen: it stays a
  coherence finding. Unwitnessed is not a finding.

Every check enumerates from a ledger other than the one it verifies, so a
deleted row is visible from the ledger that still expects it. Keyset
paginated over turns; every companion query, the GitHub lookup included,
is an IN over the page.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ophiocus
ophiocus force-pushed the feat/delivery-record-checks branch from f0c9593 to 5d38906 Compare September 7, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant