fix(validate): count every level-4 header as a scenario in the loss guard - #1521
fix(validate): count every level-4 header as a scenario in the loss guard#1521clay-good wants to merge 5 commits into
Conversation
…uard The scenario-loss guard (#1482) recognized only `#### Scenario:` headers, but the spec path (SCENARIO_HEADER / countScenarios) counts every `#### ` child of a requirement as a scenario. A MODIFIED block that dropped a differently-labeled level-4 child (e.g. `#### Edge case`) therefore passed validate and was silently deleted by archive. Align parseScenarioBlocks with the spec path so both agree. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
📝 WalkthroughWalkthroughThe parser now treats every non-fenced level-4 requirement header as a scenario. Validation and archive processing use this behavior. Tests cover unlabeled scenarios, labeled scenarios, nested headers, fenced content, and duplicate names. A patch changeset documents the fix. ChangesScenario loss parity
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.changeset/fix-scenario-loss-parity.md:
- Line 7: Update the changelog entry in fix-scenario-loss-parity.md to remove
the trailing space from the inline code span, while describing the required
whitespace outside the span so the markdown remains accurate.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 6cb32b98-7682-49f6-a062-99250ba4ee77
📒 Files selected for processing (3)
.changeset/fix-scenario-loss-parity.mdsrc/core/parsers/requirement-blocks.tstest/core/validation.scenario-loss.test.ts
alfred-openspec
left a comment
There was a problem hiding this comment.
Approved at exact head ab6f840. The parser now uses the same non-fenced level-4 boundary as countScenarios, so validate and archive cannot miss an unlabeled scenario the spec path counts. Fresh frozen install, build, lint, and 259 focused validation/archive/parser tests pass; the hosted matrix is green.
Harden the scenario-loss parity fix after a multi-agent review: - Export SCENARIO_HEADER from requirement-text.ts and reuse it in the delta path (scenarioHeaderAt/scenarioNameAt) so parity is guaranteed by construction, not two matching literals plus a comment. - Add boundary tests for the widened matcher: a level-5 (#####) header must not count, an unlabeled #### inside a fence must not count, an optional Scenario: label normalizes (relabel is not a loss), and unlabeled scenarios are counted by multiplicity. Plus an integration case: a dropped labeled scenario is caught even when an unlabeled sibling is kept (validate/archive parity, both directions). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adversarial review of the scenario-loss guard surfaced one over-strict nit and one untested symmetry: - scenarioNameAt now also strips a CommonMark closing `#` run, so `#### Foo` and `#### Foo ####` fold to the same scenario name. Without this, relabeling a scenario's header on one side (ATX-open vs ATX-closed) read as a dropped scenario — a false-abort. Safe direction only: a genuine drop still lowers a folded name's count and is caught. - Add unit tests for the untested incoming-side fence mask (a fenced `####` in the MODIFIED block must not satisfy a real scenario), lowercase `scenario:` label normalization, and the ATX-closed header fold. Behavior for conventional `#### Scenario:` headers is unchanged; parser, validation, and archive suites stay green (269 tests). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Deploying openspec-docs with
|
| Latest commit: |
9cb9ab7
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://2355ce43.openspec-docs.pages.dev |
| Branch Preview URL: | https://fix-scenario-loss-parity.openspec-docs.pages.dev |
2167681 to
9cb9ab7
Compare
…set nit Second adversarial-review round follow-ups: - scenarioNameAt's ATX-closing-sequence strip now matches only a space/tab before the trailing `#` run (`[ \t]` not `\s`), exactly as CommonMark defines a closing sequence. A looser `\s` could strip a `#` run after an exotic space (e.g. NBSP) that CommonMark keeps rendered, folding two distinct scenario names into one and masking a real loss. Correct-direction hardening for a data-loss guard; no behavior change for real space/tab-authored headers. - Changeset: describe the header whitespace outside the code span to satisfy markdownlint MD038 (no trailing space inside `#### `). Resolves CodeRabbit. Parser/validation/archive suites green (243 tests). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
alfred-openspec
left a comment
There was a problem hiding this comment.
Approved at exact head 22296bd. The shared non-fenced level-4 matcher, symmetric fence handling, multiplicity, and CommonMark header folding preserve the scenario-loss guard across validate and archive; fresh frozen install, build, lint, and 324 focused parser/validation/archive tests pass, and the hosted matrix is green.
Status
LGTM — surgical fix for a silent data-loss bug found by an adversarial swarm on post-v1.8.0 main. Not a release blocker for the published v1.8.0 artifact (latent bug, no install/runtime breakage); good fast-follow for the next release.
Underlying issue
Completes the scenario-loss guard from #1477 (fixed by #1482). #1482 closed the reported case — a
MODIFIEDblock omitting a#### Scenario:-labeled child — but tightened only the delta path, leaving the two scenario counters out of parity. This PR closes that remaining gap so #1477's guarantee holds for every level-4 child, labeled or not. (No separate issue was filed; the parity gap was caught by review before it could bite a user.)What was wrong
The scenario-loss guard added in #1482 recognized a scenario only when its header was written exactly as
#### Scenario:. But the spec path —SCENARIO_HEADER/countScenariosinrequirement-text.ts— counts every non-fenced####child of a requirement as a scenario, and its comment explicitly warns: "Don't tighten this toScenario:without changing both paths together." The delta path (parseScenarioBlocks) was tightened; the spec path was not.Consequence: a
MODIFIEDrequirement that drops a differently-labeled level-4 child (e.g.#### Edge case) passesopenspec validateand is then permanently deleted byopenspec archivewith no error or warning — exactly the loss #1477/#1482 exist to prevent. Both the validate guard (validator.ts:539) and the archive guard (specs-apply.ts:463) route through the samefindMissingCurrentScenarios→parseScenarioBlocks, so they under-count identically and the drop is invisible to both.How it was fixed
parseScenarioBlocksnow treats any non-fenced####header as a scenario boundary (matchingSCENARIO_HEADERexactly), deriving the name from the header text with an optionalScenario:prefix and a CommonMark closing#run stripped. The code-fence mask is unchanged, so fenced samples are still ignored, consistent withcountScenarios. Both current and incoming blocks run through the same function, so the multiplicity-aware comparison stays internally consistent, and messages for conventional#### Scenario:headers are unchanged.Replication / proof
Before: a main spec with
#### Scenario: Happy path+#### Edge case, and a MODIFIED block keeping onlyHappy path, validated clean and archive silently deletedEdge case.After (live CLI on this branch):
Regression tests in
validation.scenario-loss.test.tsassert both validate and archive name the droppedEdge case(parity), including the reverse (drop the labeled sibling, keep the unlabeled one). Unit tests inrequirement-blocks.test.tspin the boundaries: level-5#####excluded, fenced####excluded on both the current and incoming sides,Scenario:/lowercase-scenario:label normalization, ATX-closed header folding, and multiplicity. Full parser + validation + archive suites pass (243 tests in those suites; whole suite green).Adversarial review
Two rounds of independent reviewers ran against the diff (correctness / test-coverage / blast-radius / integration):
####parser. Two over-strict nits were found and fixed here: an ATX-closed header (#### Foo ####) now folds to its open form, and the closing-#-run strip matches only a space/tab before the run (CommonMark's rule), so an exotic space (e.g. NBSP) can't over-strip and fold two distinct names.SCENARIO_HEADERintroduces no cycle,.rawslicing feeds no consumer, labeled-scenario names and message text are unchanged, and both surfaces route through the one sharedfindMissingCurrentScenariosso they cannot drift.Notes
Companion PR (#1522) fixes an unrelated init/update
.agentstarget interaction found in the same swarm.🤖 Generated with Claude Code