Turn the verification template into the Agent Skill Verifier CLI product (v0.1.0)#3
Merged
Merged
Conversation
… eval over real CLI tools)
…able exit codes - Reusable verification core (verifySkill) with explicit skill/cases paths - verify / validate / replay / report commands (non-interactive, CI-safe) - Versioned canonical result schema (1.0.0), JUnit XML, events.jsonl, metrics.json, and per-run replay artifacts - Documented exit-code contract (0-6), NO_COLOR support, pure-JSON mode - YAML and JSON evaluation cases plus skill-verification.(yaml|json) config - fixtures/valid-skill + fixtures/evals.yaml smoke fixtures - ESLint flat config; 99 tests passing
…ests, checksums - build-cli: single-file CJS bundle with build-time version injection, no source maps, npm deps inlined (SEA-compatible) - build-standalone: official Node SEA flow (blob + postject + macOS codesign) with a --version self-check from outside the repo - package-release: platform zip/tar.gz + portable node.zip, each with LICENSE, QUICKSTART.md, and a release-manifest.json (per-file sha256) - generate-checksums: deterministic SHA256SUMS.txt/.json (+ --verify) - release-check: extracts each archive; validates manifests, hashes, forbidden files, version/tag consistency, portable --version - smoke-test: runs the packaged CLI from a temp dir outside the repo; 16 checks covering commands, artifacts, and exit codes
…blication - ci.yml: lint + typecheck + test + build + packaged smoke test, contents:read, concurrency cancellation, pinned Node 22.13.1 - release.yml: verify job (tag/version consistency), native-runner matrix (windows-x64, linux-x64, macos-x64 on macos-15-intel, macos-arm64), portable bundle job, then a draft-first release job that validates manifests/checksums/expected assets before publishing via gh - workflow_dispatch on release.yml is a build-only dry run (never tags or publishes); only the release job has contents:write - release-check: --tag-only and --require-assets modes - release-notes generator with install table and checksum guidance - 17 workflow-contract tests (permissions, triggers, publication guards)
- README rewritten for the Agent Skill Verifier CLI: download/install per platform, commands, configuration, exit codes, report formats, pinned- version CI example, security, reproducibility limitations - original template/tutorial README preserved as docs/reference-implementation.md - CHANGELOG.md with the 0.1.0 release section - docs/threat-model.md covering skill/eval-file/path/release threats - docs/release-process.md (pinned Node 22.13.1, SEA, dry-run, draft-first) - examples/skill-verification.yaml project configuration
Final release commit for Agent Skill Verifier v0.1.0: removes the temporary branch dry-run trigger after three successful cross-platform dry runs (windows-x64, linux-x64, macos-x64, macos-arm64, portable).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
This branch turns the repository's verification harness into a production, cross-platform CLI product — agent-skill-verifier — and adds the release automation that already published Release v0.1.0 from this branch's tag.
src/cli/):verify,validate,replay,report,--help/--version. Non-interactive, CI-safe (NO_COLOR, pure-JSON mode, CWD-relative paths, output confined to the workspace).src/core/verification-service.ts): the existing eval runner extracted behindverifySkill()— no verification logic rewritten; legacy npm scripts and tests unchanged.summary.json) plus JUnit XML, self-contained HTML,events.jsonl,metrics.json, and per-run replay artifacts; escaping of hostile content is regression-tested.scripts/): esbuild single-file bundle → official Node SEA standalone binaries (pinned Node 22.13.1), per-archiverelease-manifest.json,SHA256SUMS.txt/.json, archive content validation, packaged-binary smoke tests (run from outside the repo).ci.yml(contents:read; lint/typecheck/test/build/smoke) andrelease.yml(native-runner matrix for windows-x64/linux-x64/macos-x64/macos-arm64 + portable; draft-first publication;contents:writeon the release job only;workflow_dispatch= build-only dry run).docs/reference-implementation.md.yamladded (runtime), esbuild/postject pinned (build), ESLint added, vitest upgraded 2→4 (clears all npm audit findings; 0 vulnerabilities).Why
The repo demonstrated solid verification internals but was only usable via repo-local npm scripts. This makes the same engine consumable as a real product: downloadable binaries, stable exit codes and schemas for CI, and an auditable, draft-first release pipeline.
Reviewer notes
main(README badge/CI registration follow the merge).src/core/paths.ts: workspace root is now pinned to CWD by the CLI (previously module-relative walk-up) — required for bundled/standalone operation; all existing in-repo invocations behave identically.actions/*@v4→@v5to silence Node-24 runner deprecation annotations.