Skip to content

Fix Bronze duplicate detection for CRLF records - #18

Merged
patschmittdev merged 2 commits into
mainfrom
patschmittdev-bronze-crlf-ingestion
Sep 12, 2026
Merged

patschmittdev merged 2 commits into
mainfrom
patschmittdev-bronze-crlf-ingestion

Conversation

@patschmittdev

Copy link
Copy Markdown
Owner

Summary

The Bronze store required literal LF frontmatter delimiters while corpus collection and live evidence reading already normalized CRLF to LF. Converting a valid stored Bronze record to CRLF therefore made ingestion skip its hash and potentially create duplicate evidence. Atomic no-overwrite remained intact; this is a robustness fix, not an admission bypass.

Remove the divergent splitBronzeFile parser and reuse parseCorpusDocument through parseBronzeFile for store reads, hash collection, and verification. Public function/result shapes remain unchanged. CRLF normalization remains CRLF-only: lone carriage returns and their stored body hashes are preserved. Invalid frontmatter still throws on direct reads, malformed files retain the existing deduplication skip policy, and body-hash corruption still blocks ingestion. Parser failures now use the shared parser's safe diagnostics rather than raw YAML/schema error details.

Adds test/bronze-line-endings.test.ts and clarifies duplicate and line-ending behavior in the ingest guide. Based on audited main a62686c3ab6d58c67f7b1c58529436099372b95d.

Trust-boundary impact

Bronze duplicate detection and hash verification now agree with existing corpus/live reading for LF and CRLF files. No changes to generic corpus normalization, authorization canonicalization, receipts, schemas, model capabilities, Silver staging, Gold admission, profile isolation, or instruction authority. Atomic no-overwrite and inbox retention on duplicate/refusal remain unchanged. Broader skipped-file diagnostics are outside this PR's scope.

This PR is for independent human review and existing CI. AI review is not human approval; no merge or bypass is requested.

Validation

  • Regression before implementation: four CRLF cases failed while LF and lone-CR baselines passed.
  • New real-file tests: 10 passed. LF/CRLF x ordinary/lone-CR fixtures cover store parsing, verification, corpus collection, live reading, expected hashes, distinct-inbox-filename duplicate status, no extra Bronze record, inbox preservation, stored-byte preservation, corrupted-body refusal, invalid YAML, duplicate keys, and strict schema rejection.
  • Focused Bronze/corpus suite: 33 passed, 2 existing permission tests skipped because this Windows filesystem does not enforce chmod read restrictions.
  • npm run check: 705 passed, 2 skipped; 11 additional script tests passed; documentation links and style passed.
  • node dist/src/cli/main.js check --root . --audit-clean-room: PASS, no findings in the actual worktree.
  • git diff --check and staged diff check: passed.
  • Local platform: Windows, Node 24.16.0, npm 12.0.2. Cross-platform Node 22/24 and docs-site checks are delegated to repository CI.

Checklist

  • Behavior changes include tests that use real temporary files.
  • Model pathways still cannot write Bronze, knowledge, trust, receipts, reviewed metadata, or indexes.
  • No signer, apply, approve, promote command, or --promote flag was added.
  • Retrieved content remains reference-only with instruction_authority: none.
  • Documentation and compatibility notes match the implementation.
  • npm run check passes.
  • node dist/src/cli/main.js check --root . --audit-clean-room passes.
  • No private key, credential, personal path, or private vault content is included.

Reuse the canonical corpus parser for Bronze reads, hash collection, and verification while preserving lone-CR hashes and corruption refusal.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
patschmittdev added a commit that referenced this pull request Sep 12, 2026
The sole maintainer reviewed the changes and explicitly authorized a one-time review-only exception for PRs #18, #19, #20, and #21. Required CI checks and all other branch protections remained enforced. This is not a recorded independent GitHub approval.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@patschmittdev
patschmittdev merged commit d574f0d into main Sep 12, 2026
7 checks passed
@patschmittdev
patschmittdev deleted the patschmittdev-bronze-crlf-ingestion branch September 12, 2026 18:29
patschmittdev added a commit that referenced this pull request Sep 12, 2026
The sole maintainer reviewed the changes and explicitly authorized a one-time review-only exception for PRs #18, #19, #20, and #21. Required CI checks and all other branch protections remained enforced. This is not a recorded independent GitHub approval.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
patschmittdev added a commit that referenced this pull request Sep 12, 2026
The sole maintainer reviewed the changes and explicitly authorized a one-time review-only exception for PRs #18, #19, #20, and #21. Required CI checks and all other branch protections remained enforced. This is not a recorded independent GitHub approval.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
patschmittdev added a commit that referenced this pull request Sep 12, 2026
## Summary

Astro removed newline/indentation whitespace immediately before three
inline anchors in `site/src/pages/index.astro`. The production DOM
contained `Clone therepository`, `then theattack-control mapping`, and
`and theguarantees and residual risks`. Add explicit `{' '}` at those
three boundaries, preserving commas, the final period, navigation/flex
spacing, layout, and palette. No CSS changes.

Audit all six homepage prose anchors with DOM Range assertions before
and after each link, including the already-correct provenance, overview,
and project-status links. The regression runs in the existing visual
suite and catches joined text regardless of CSS spacing or axe results.
Screenshots now use Playwright's output directory so artifacts can
remain outside the checkout. A validated decimal `PORT` override selects
the same loopback port as the static server; occupied ports fail instead
of silently reusing another worktree's server.

Qualify one consequential claim: a missing signer is not OS-enforced
private-key isolation. The homepage now explicitly makes custody outside
the vault and access restrictions for the model/Ziggurat process an
operator responsibility. AdmissionFigure uses the corresponding
imperative rather than implying automatic isolation. Other wording and
historical measurement caveats are unchanged.

## Trust-boundary impact

None to runtime behavior, contracts, signing bytes, admission, or
retrieval. This is a presentation correction and test isolation change.

Focused claim trace uses final integrated main
**`a8e3606d1820ee90ae8e6aea0a993ccb5b29eaef`**, including #19, #18, #20,
and #21:

- Strict model draft v1, host materialization/staging of Silver v2, and
live evidence checks remain in `src/contracts/refinement-draft.ts` and
`src/refine/`.
- Bronze remains CRLF-only canonical text; shared parsing now aligns
duplicate detection and verification. Gold uses signed canonical line
endings, while conditional `source_body_sha256` retains live staleness
detection without becoming receipt authority.
- Gold still requires the external Ed25519 receipt plus every
eligibility check. Three physically separate profiles, Gold-only
read-only MCP, and reference/none output labels remain unchanged.
- Human CLI display escaping does not change decoded JSON values.
Loopback adapter limits and lack of a shipped signer remain unchanged.

The custody qualification was the only substantive homepage mismatch
identified in this scoped trace. This is not a comprehensive
correctness/security certification, proof of human attention or semantic
truth, or production-readiness claim. Source-only/pre-1.0 status remains
explicit. Existing measurements are bounded historical development
evidence; this PR does not re-benchmark the updated runtime.

## Validation

- Confirmed the new DOM regression fails on the unfixed production build
for all three missing-space locations.
- `npm run site:check`: passes; zero Astro diagnostics, 7 plugin tests,
24 built pages and link/asset validation.
- `npm run visual -- --grep 'homepage inline prose link boundaries| /
visual and accessibility' --output <external-artifact-directory>` from
`site`, with `PORT=4339`: **12 passed**, desktop/tablet/mobile in
light/dark. Includes text boundaries, custody wording, layout
thresholds, and axe.
- Inspected bounded desktop/mobile light/dark screenshots and readable
evaluation/custody crops. This is a focused local inspection, not
independent human visual signoff.
- Port probes: 7 invalid values rejected, 3 valid values discover all
six text cases, occupied port explicitly refused.
- `npm run check`: **728 passed, 3 platform skips** in the 731-test
runtime suite; 11 script tests passed; documentation links and style
passed.
- `node dist/src/cli/main.js check --root . --audit-clean-room`: PASS,
no findings.
- `git diff --check`: passes. Screenshots/logs are outside the checkout;
no dependency, runtime, or CSS changes.

This PR requires the normal independent review and required CI. It is
not covered by the one-time exception for the four preceding PRs and
must not be merged under that exception.

## Checklist

- [x] Behavior changes include tests that use real temporary files.
- [x] Model pathways still cannot write Bronze, knowledge, trust,
receipts, reviewed metadata, or indexes.
- [x] No signer, apply, approve, promote command, or `--promote` flag
was added.
- [x] Retrieved content remains reference-only with
`instruction_authority: none`.
- [x] Documentation and compatibility notes match the implementation.
- [x] `npm run check` passes.
- [x] `node dist/src/cli/main.js check --root . --audit-clean-room`
passes.
- [x] No private key, credential, personal path, or private vault
content is included.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant