Skip to content

fix(validate): count every level-4 header as a scenario in the loss guard - #1521

Open
clay-good wants to merge 5 commits into
mainfrom
fix/scenario-loss-parity
Open

fix(validate): count every level-4 header as a scenario in the loss guard#1521
clay-good wants to merge 5 commits into
mainfrom
fix/scenario-loss-parity

Conversation

@clay-good

@clay-good clay-good commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Status

LGTM — surgical fix for a silent data-loss bug found by an adversarial swarm on post-v1.8.0 main. Not a release blocker for the published v1.8.0 artifact (latent bug, no install/runtime breakage); good fast-follow for the next release.

Underlying issue

Completes the scenario-loss guard from #1477 (fixed by #1482). #1482 closed the reported case — a MODIFIED block omitting a #### Scenario:-labeled child — but tightened only the delta path, leaving the two scenario counters out of parity. This PR closes that remaining gap so #1477's guarantee holds for every level-4 child, labeled or not. (No separate issue was filed; the parity gap was caught by review before it could bite a user.)

What was wrong

The scenario-loss guard added in #1482 recognized a scenario only when its header was written exactly as #### Scenario:. But the spec path — SCENARIO_HEADER / countScenarios in requirement-text.ts — counts every non-fenced #### child of a requirement as a scenario, and its comment explicitly warns: "Don't tighten this to Scenario: without changing both paths together." The delta path (parseScenarioBlocks) was tightened; the spec path was not.

Consequence: a MODIFIED requirement that drops a differently-labeled level-4 child (e.g. #### Edge case) passes openspec validate and is then permanently deleted by openspec archive with no error or warning — exactly the loss #1477/#1482 exist to prevent. Both the validate guard (validator.ts:539) and the archive guard (specs-apply.ts:463) route through the same findMissingCurrentScenariosparseScenarioBlocks, so they under-count identically and the drop is invisible to both.

How it was fixed

parseScenarioBlocks now treats any non-fenced #### header as a scenario boundary (matching SCENARIO_HEADER exactly), deriving the name from the header text with an optional Scenario: prefix and a CommonMark closing # run stripped. The code-fence mask is unchanged, so fenced samples are still ignored, consistent with countScenarios. Both current and incoming blocks run through the same function, so the multiplicity-aware comparison stays internally consistent, and messages for conventional #### Scenario: headers are unchanged.

Replication / proof

Before: a main spec with #### Scenario: Happy path + #### Edge case, and a MODIFIED block keeping only Happy path, validated clean and archive silently deleted Edge case.

After (live CLI on this branch):

✗ [ERROR] widget/spec.md: MODIFIED "Widget display" omits scenario(s) the current spec still has: "Edge case". ...

Regression tests in validation.scenario-loss.test.ts assert both validate and archive name the dropped Edge case (parity), including the reverse (drop the labeled sibling, keep the unlabeled one). Unit tests in requirement-blocks.test.ts pin the boundaries: level-5 ##### excluded, fenced #### excluded on both the current and incoming sides, Scenario:/lowercase-scenario: label normalization, ATX-closed header folding, and multiplicity. Full parser + validation + archive suites pass (243 tests in those suites; whole suite green).

Adversarial review

Two rounds of independent reviewers ran against the diff (correctness / test-coverage / blast-radius / integration):

  • Correctness: no input where validate and archive disagree, no realistic silent-loss — they share one fence-aware, any-#### parser. Two over-strict nits were found and fixed here: an ATX-closed header (#### Foo ####) now folds to its open form, and the closing-#-run strip matches only a space/tab before the run (CommonMark's rule), so an exotic space (e.g. NBSP) can't over-strip and fold two distinct names.
  • Test-coverage: the fix is genuinely guarded and parity is real; added the suggested gaps (incoming-side fence symmetry, lowercase label, ATX fold).
  • Blast-radius / integration: no regressions — exporting SCENARIO_HEADER introduces no cycle, .raw slicing feeds no consumer, labeled-scenario names and message text are unchanged, and both surfaces route through the one shared findMissingCurrentScenarios so they cannot drift.

Notes

Companion PR (#1522) fixes an unrelated init/update .agents target interaction found in the same swarm.

🤖 Generated with Claude Code

…uard

The scenario-loss guard (#1482) recognized only `#### Scenario:` headers, but
the spec path (SCENARIO_HEADER / countScenarios) counts every `#### ` child of
a requirement as a scenario. A MODIFIED block that dropped a differently-labeled
level-4 child (e.g. `#### Edge case`) therefore passed validate and was silently
deleted by archive. Align parseScenarioBlocks with the spec path so both agree.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@clay-good
clay-good requested a review from a team as a code owner August 5, 2026 21:43
@clay-good
clay-good requested review from TabishB and removed request for a team August 5, 2026 21:43
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The parser now treats every non-fenced level-4 requirement header as a scenario. Validation and archive processing use this behavior. Tests cover unlabeled scenarios, labeled scenarios, nested headers, fenced content, and duplicate names. A patch changeset documents the fix.

Changes

Scenario loss parity

Layer / File(s) Summary
Broaden scenario header parsing
src/core/parsers/requirement-text.ts, src/core/parsers/requirement-blocks.ts, test/core/parsers/requirement-blocks.test.ts
The shared header pattern is exported. Parser helpers recognize non-fenced level-4 headers and normalize optional Scenario: prefixes. Tests cover nested headers, fenced content, equivalent labels, and duplicate scenarios.
Validate and document scenario loss
test/core/validation.scenario-loss.test.ts, .changeset/fix-scenario-loss-parity.md
Tests verify matching validation and archive errors for omitted labeled and unlabeled scenarios. The changeset documents the patch.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested reviewers: tabishb, alfred-openspec

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: counting every level-4 header as a scenario in the validation loss guard.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/scenario-loss-parity

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.changeset/fix-scenario-loss-parity.md:
- Line 7: Update the changelog entry in fix-scenario-loss-parity.md to remove
the trailing space from the inline code span, while describing the required
whitespace outside the span so the markdown remains accurate.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6cb32b98-7682-49f6-a062-99250ba4ee77

📥 Commits

Reviewing files that changed from the base of the PR and between d578896 and ab6f840.

📒 Files selected for processing (3)
  • .changeset/fix-scenario-loss-parity.md
  • src/core/parsers/requirement-blocks.ts
  • test/core/validation.scenario-loss.test.ts

Comment thread .changeset/fix-scenario-loss-parity.md Outdated
alfred-openspec
alfred-openspec previously approved these changes Aug 5, 2026

@alfred-openspec alfred-openspec left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at exact head ab6f840. The parser now uses the same non-fenced level-4 boundary as countScenarios, so validate and archive cannot miss an unlabeled scenario the spec path counts. Fresh frozen install, build, lint, and 259 focused validation/archive/parser tests pass; the hosted matrix is green.

Harden the scenario-loss parity fix after a multi-agent review:
- Export SCENARIO_HEADER from requirement-text.ts and reuse it in the delta
  path (scenarioHeaderAt/scenarioNameAt) so parity is guaranteed by
  construction, not two matching literals plus a comment.
- Add boundary tests for the widened matcher: a level-5 (#####) header must
  not count, an unlabeled #### inside a fence must not count, an optional
  Scenario: label normalizes (relabel is not a loss), and unlabeled scenarios
  are counted by multiplicity. Plus an integration case: a dropped labeled
  scenario is caught even when an unlabeled sibling is kept (validate/archive
  parity, both directions).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adversarial review of the scenario-loss guard surfaced one over-strict nit
and one untested symmetry:

- scenarioNameAt now also strips a CommonMark closing `#` run, so `#### Foo`
  and `#### Foo ####` fold to the same scenario name. Without this, relabeling
  a scenario's header on one side (ATX-open vs ATX-closed) read as a dropped
  scenario — a false-abort. Safe direction only: a genuine drop still lowers a
  folded name's count and is caught.
- Add unit tests for the untested incoming-side fence mask (a fenced `####` in
  the MODIFIED block must not satisfy a real scenario), lowercase `scenario:`
  label normalization, and the ATX-closed header fold.

Behavior for conventional `#### Scenario:` headers is unchanged; parser,
validation, and archive suites stay green (269 tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 6, 2026

Copy link
Copy Markdown

Deploying openspec-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 9cb9ab7
Status: ✅  Deploy successful!
Preview URL: https://2355ce43.openspec-docs.pages.dev
Branch Preview URL: https://fix-scenario-loss-parity.openspec-docs.pages.dev

View logs

@clay-good
clay-good force-pushed the fix/scenario-loss-parity branch from 2167681 to 9cb9ab7 Compare August 6, 2026 19:56
clay-good and others added 2 commits August 7, 2026 08:19
…set nit

Second adversarial-review round follow-ups:

- scenarioNameAt's ATX-closing-sequence strip now matches only a space/tab
  before the trailing `#` run (`[ \t]` not `\s`), exactly as CommonMark defines
  a closing sequence. A looser `\s` could strip a `#` run after an exotic space
  (e.g. NBSP) that CommonMark keeps rendered, folding two distinct scenario
  names into one and masking a real loss. Correct-direction hardening for a
  data-loss guard; no behavior change for real space/tab-authored headers.
- Changeset: describe the header whitespace outside the code span to satisfy
  markdownlint MD038 (no trailing space inside `#### `). Resolves CodeRabbit.

Parser/validation/archive suites green (243 tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@alfred-openspec alfred-openspec left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at exact head 22296bd. The shared non-fenced level-4 matcher, symmetric fence handling, multiplicity, and CommonMark header folding preserve the scenario-loss guard across validate and archive; fresh frozen install, build, lint, and 324 focused parser/validation/archive tests pass, and the hosted matrix is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants