Skip to content

feat(validate): add --archived to lint task completion of archived changes - #1604

Open
clay-good wants to merge 4 commits into
mainfrom
claude/openspec-backlog-triage-4e8c61
Open

feat(validate): add --archived to lint task completion of archived changes#1604
clay-good wants to merge 4 commits into
mainfrom
claude/openspec-backlog-triage-4e8c61

Conversation

@clay-good

@clay-good clay-good commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Status

LGTM — ready for review. New behavior is fully opt-in; no existing validate invocation changes.

What was missing

openspec validate only ever looks at active changes and specs — active-change discovery explicitly skips changes/archive/. So there was no way to check that changes were finished when they were archived. A change can be archived with unchecked tasks (e.g. via archive --yes past the incomplete-task warning), and nothing surfaces it afterward. #205 asks for exactly this: a lint that fails in a pre-commit hook when archived changes still have open tasks.

What it does

Adds an opt-in --archived flag to openspec validate:

# Fail if any archived change still has unchecked tasks
openspec validate --archived
  • Scans every change directory under changes/archive/ and checks its tasks.md checkboxes.
  • Exits 1 if any archived change has incomplete tasks; prints per-change results and a totals line. Supports --json with the same shape as validate --all.
  • Changes with zero tasks pass (nothing to complete).
  • A missing archive directory (ENOENT) prints a friendly message and exits 0 — but a real archive-read failure (permissions, I/O, or an archive path that is a file) throws and exits 1 rather than being silently treated as "no archived changes", which would let a pre-commit lint pass without inspecting anything.
  • An archived tasks.md that exists but cannot be read is reported as an ERROR (naming the file) and fails, instead of being silently counted as zero tasks.

It is a standalone scope: the --archived branch returns before any existing bulk path, so validate, --all, --changes, --specs, and <item> are byte-for-byte unchanged. It intentionally does not re-validate spec deltas of archived changes (their operations were already applied at archive time) — it only checks task completion, which is the concrete ask in #205.

Why it's safe

  • Opt-in, isolated: guarded solely by options.archived; no default behavior changes.
  • No forked logic: the archived path uses getTaskProgressDetailForChange — the same task counter status, list, and archive use (now wrapped by getTaskProgressForChange, which drops the extra "unreadable" detail so those three callers get byte-identical totals). "What counts as a task" never forks; --archived only additionally surfaces unreadable files.
  • Store-aware: reads root.archiveDir, so it respects a selected --store like the rest of the command.
  • Reviewed hard: five adversarial reviews across two rounds (correctness/safety, design/scope, refactor behavior-preservation, error-handling/edge-cases, docs/linkage) plus CodeRabbit — no blockers survived. CodeRabbit's two "Major" findings (swallowing all archive-read errors; counting unreadable task files as complete) are fixed and covered by new tests.

Proof it works

test/cli-e2e/validate-archived-tasks.test.ts (5 cases):

  • archived change with an unchecked nested sub-task → exit 1, active changes are not scanned;
  • all archived changes complete → exit 0;
  • no archive directory → exit 0 with a friendly message;
  • archive path is a file → exit 1 (not a silent pass);
  • archived tasks.md unreadable → exit 1 with a "could not read" error.

Plus unit tests in test/utils/task-progress.test.ts for the new detail variant (normal / unreadable / missing).

$ openspec validate --archived
✗ change/one-left
  ✗ 1 incomplete task (1/2 completed)
Totals: 0 passed, 1 failed (1 items)   # exit 1

Full suite green locally (3867 passing; two unrelated failures — artifact-workflow "creates skills for Cursor tool" and config-profile "confirmed project apply…" — are pre-existing on main and pass in CI).

Notes / scope

  • --archived is a standalone scope and does not compose with --all/--strict; a "lint everything including archives" gate is two commands: openspec validate --all --strict && openspec validate --archived. Kept standalone deliberately to keep the change surgical and side-effect-free.
  • Complementary to (not overlapping) the in-flight feat(validation): Add tasks.md validation with checkbox and content checks (#354) #576, which validates active-change tasks.md structure; this validates archived task completion.
  • Output matches bulk validate: rows print change/<id> (mapping to the JSON type), issue path values follow the tasks.md / root-relative-file convention, and schema resolution is memoized across archived changes so a large archive isn't re-parsed per change.
  • Docs updated (docs/cli.md, regenerated website reference, and a docs/troubleshooting.md one-liner) and a minor changeset added.

Closes #205

Summary by CodeRabbit

  • New Features

    • Added the opt-in openspec validate --archived command.
    • Checks archived changes for incomplete tasks, including nested subtasks, while excluding active changes.
    • Supports human-readable and JSON results with appropriate exit codes.
    • Reports unreadable task files and validation errors; missing archives are treated as empty.
  • Documentation

    • Added CLI usage examples and troubleshooting guidance.
  • Tests

    • Added coverage for complete, incomplete, nested, missing, invalid, and unreadable archived changes.

…anges

`openspec validate --archived` scans every change under changes/archive/
and fails (exit 1) if any has unchecked tasks in tasks.md. This catches
changes archived with unfinished work — which the normal validate flow
never sees, since it only looks at active changes — and is meant for a
pre-commit or CI hook.

It is a standalone, opt-in scope: it returns before any existing bulk
path, so no current `validate` invocation changes behavior, and it does
not re-validate already-applied spec deltas. Reuses getTaskProgressForChange
(the same counter status/list/archive use) so task counting never forks,
and reads root.archiveDir so it is store-aware.

Closes #205

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@clay-good
clay-good requested a review from a team as a code owner August 7, 2026 14:17
@clay-good
clay-good requested review from TabishB and removed request for a team August 7, 2026 14:17
@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The openspec validate --archived option checks task completion in archived changes. It supports JSON and human-readable output, reports unreadable task files, returns failure for incomplete tasks, and leaves existing validation behavior unchanged.

Changes

Archived task validation

Layer / File(s) Summary
CLI option and validation contract
.changeset/..., docs/cli.md, docs/troubleshooting.md, src/cli/index.ts, src/core/completions/command-registry.ts
Adds the --archived option, documentation, completion metadata, and changeset entry.
Detailed task-progress reporting
src/utils/task-progress.ts, test/utils/task-progress.test.ts
Adds schema-glob caching and unreadable task-file reporting while preserving the existing aggregate progress contract.
Archived discovery and task validation
src/commands/validate.ts
Discovers archived changes, checks task progress, reports results, handles archive errors, and sets the exit status.
Archived validation end-to-end coverage
test/cli-e2e/validate-archived-tasks.test.ts
Tests incomplete and nested tasks, active-change exclusion, completed archives, missing archives, invalid archive paths, and unreadable task files.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant ValidateCommand
  participant ArchiveDirectory
  participant TaskProgress
  participant Output
  CLI->>ValidateCommand: execute with archived=true
  ValidateCommand->>ArchiveDirectory: discover archived change IDs
  ValidateCommand->>TaskProgress: evaluate archived tasks
  TaskProgress-->>ValidateCommand: completion and unreadable-file details
  ValidateCommand->>Output: emit validation summary
  ValidateCommand-->>CLI: set exit status
Loading

Possibly related PRs

Suggested reviewers: tabishb, alfred-openspec

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The implementation satisfies issue #205 by validating unchecked tasks in archived changes with reporting, exit codes, and pre-commit support.
Out of Scope Changes check ✅ Passed The code, tests, documentation, and changeset directly support archived task validation and its required behavior.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding the --archived option to validate archived task completion.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/openspec-backlog-triage-4e8c61

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 7, 2026

Copy link
Copy Markdown

Deploying openspec-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 6a1ca84
Status: ✅  Deploy successful!
Preview URL: https://6ee59831.openspec-docs.pages.dev
Branch Preview URL: https://claude-openspec-backlog-tria-w0o2.openspec-docs.pages.dev

View logs

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/commands/validate.ts`:
- Around line 447-458: Update getTaskProgressForChange and the validation flow
around it so task-file read failures are exposed and converted into an ERROR
issue instead of being treated as complete. Preserve the documented successful
0/0 result only when no task files exist, while retaining the existing
incomplete-task reporting for successfully read files.
- Around line 411-420: Update listArchivedChangeIds to return an empty list only
when fs.readdir fails with an ENOENT error; propagate permission, I/O, and
invalid-path errors instead of treating them as an empty archive. Add a
regression test covering an existing archive path that is not a directory.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 2b07756e-1290-4c1a-a738-0a88b969f13b

📥 Commits

Reviewing files that changed from the base of the PR and between e50bd09 and accd1ce.

📒 Files selected for processing (6)
  • .changeset/validate-archived-tasks.md
  • docs/cli.md
  • src/cli/index.ts
  • src/commands/validate.ts
  • src/core/completions/command-registry.ts
  • test/cli-e2e/validate-archived-tasks.test.ts

Comment thread src/commands/validate.ts
Comment thread src/commands/validate.ts Outdated
… files

Address adversarial + CodeRabbit review of `validate --archived`:

- listArchivedChangeIds now returns [] only for ENOENT (missing archive
  dir) and rethrows permission/I/O/ENOTDIR errors, so a real archive-read
  failure exits 1 instead of silently reading as "no archived changes".
- Add getTaskProgressDetailForChange, which reports task files that exist
  but cannot be read; --archived turns those into an ERROR (naming the
  file) rather than silently counting them as zero tasks. The shared
  getTaskProgressForChange now wraps it and drops the detail, so
  status/list/archive totals are byte-identical.
- Start the spinner after listing so a thrown listing error never leaves
  a spinner running.

Adds regression tests (archive path is a file; archived tasks.md is
unreadable) and unit tests for the new detail variant. Docs: align the
--archived table verb and add a troubleshooting one-liner.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/cli-e2e/validate-archived-tasks.test.ts`:
- Around line 129-132: Update the path construction in the archived-task fixture
to pass “openspec”, “changes”, “archive”, “unreadable-change”, and “tasks.md” as
separate arguments to path.join, preserving the same filesystem location with
platform-native separators.

In `@test/utils/task-progress.test.ts`:
- Around line 338-347: Update the unreadable-file assertions in
getTaskProgressDetailForChange’s test to verify the reported path, not just
unreadable count. Compare the actual path with the created bad/tasks.md path
after canonicalizing both via fs.realpathSync.native().
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8909cf11-d5e6-4431-8c59-4bd3ef0dfaa2

📥 Commits

Reviewing files that changed from the base of the PR and between accd1ce and 6a1ca84.

📒 Files selected for processing (6)
  • docs/cli.md
  • docs/troubleshooting.md
  • src/commands/validate.ts
  • src/utils/task-progress.ts
  • test/cli-e2e/validate-archived-tasks.test.ts
  • test/utils/task-progress.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • docs/cli.md
  • src/commands/validate.ts

Comment thread test/cli-e2e/validate-archived-tasks.test.ts
Comment thread test/utils/task-progress.test.ts
clay-good and others added 2 commits August 7, 2026 10:04
- Build the unreadable-fixture path from separate path.join components
  instead of a hard-coded Unix-separator string.
- Assert the reported unreadable path (canonicalized with
  realpathSync.native), not just the count, so a wrong path can't pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
From three fresh adversarial reviews (scale/perf, flag/output-shape,
filesystem/security):

- perf: memoize schema→glob resolution across archived changes via a
  run-scoped SchemaGlobCache, so the same schema.yaml isn't re-parsed
  once per change (the archive is append-only and can hold thousands).
  Threaded as an optional arg; existing callers are unchanged. Loop stays
  sequential by design (per-change work is synchronous) — now documented.
- output shape: issue `path` now follows validate's convention —
  'tasks.md' for incomplete tasks, and the POSIX root-relative file path
  for an unreadable file (one issue per file) instead of the bare 'tasks'.
- plain output: print `change/<id>` (matching the JSON `type` and bulk
  validation) instead of `archived/<id>`.
- docs: correct the "Never throws" docstrings (glob resolution can throw
  on a malformed/unsafe schema; the caller guards it) and note the
  load-bearing projectRoot override for the archive path depth.

Store-mode resolution confirmed correct by review. Tests updated + a memo
regression test added.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/utils/task-progress.ts (1)

106-110: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Normalize schema names before caching.

resolveSchema in src/core/artifact-graph/resolver.ts accepts schema names with or without .yaml or .yml. This cache uses the raw name, so equivalent names create separate entries and re-parse the same schema. Normalize the key with the same rule as resolveSchema, or share a common normalizer.

Proposed fix
     const schemaName = resolveSchemaForChange(changeDir, undefined, projectRoot);
-    if (schemaGlobCache?.has(schemaName)) return schemaGlobCache.get(schemaName);
+    const cacheKey = schemaName.replace(/\.ya?ml$/, '');
+    if (schemaGlobCache?.has(cacheKey)) return schemaGlobCache.get(cacheKey);
     const schema = resolveSchema(schemaName, projectRoot);
     const generates = findTrackedTasksArtifact(schema)?.generates;
-    schemaGlobCache?.set(schemaName, generates);
+    schemaGlobCache?.set(cacheKey, generates);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/utils/task-progress.ts` around lines 106 - 110, Normalize schemaName
using the same rule as resolveSchema before checking or updating
schemaGlobCache, so names with and without .yaml or .yml share one cache entry.
Update the cache access in the surrounding task-progress flow and reuse a shared
normalizer if one already exists rather than introducing divergent normalization
logic.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/utils/task-progress.ts`:
- Around line 106-110: Normalize schemaName using the same rule as resolveSchema
before checking or updating schemaGlobCache, so names with and without .yaml or
.yml share one cache entry. Update the cache access in the surrounding
task-progress flow and reuse a shared normalizer if one already exists rather
than introducing divergent normalization logic.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 3a6e658c-bcbf-47d4-8041-9d29e87f66cb

📥 Commits

Reviewing files that changed from the base of the PR and between 92c5b57 and 581b780.

📒 Files selected for processing (4)
  • src/commands/validate.ts
  • src/utils/task-progress.ts
  • test/cli-e2e/validate-archived-tasks.test.ts
  • test/utils/task-progress.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • test/utils/task-progress.test.ts
  • src/commands/validate.ts

@alfred-openspec alfred-openspec left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 581b780. The archived scope fails closed on archive and task read errors, respects the resolved Store and each schema's tracked task files, and leaves existing validation modes unchanged. Local build, lint, 281 focused tests, and the hosted Linux, macOS, Windows, security, CodeQL, and docs checks pass. The cache-key normalization comment is only an optional efficiency cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Strenghten linting support: validate unchecked tasks in the archive directory

2 participants