Skip to content

feat(bench-kit): zone-aware update + agent-tool profile in init - #31

Merged
psmyrdek merged 1 commit into
feat/bench-kit-workflows-copyfrom
feat/bench-kit-update-and-tool
Aug 14, 2026
Merged

feat(bench-kit): zone-aware update + agent-tool profile in init#31
psmyrdek merged 1 commit into
feat/bench-kit-workflows-copyfrom
feat/bench-kit-update-and-tool

Conversation

@psmyrdek

Copy link
Copy Markdown
Collaborator

Implements the two remaining bench-kit CLI pieces (points 1–2 of the current plan; design contract: DESIGN.md in 10x-benchmark-webinar / template README zone contracts).

Stacked on #30 (init copies workflows) — merge that first, then retarget/merge this into master.

init: agent-tool recognition

  • Resolution chain: --tool flag → manifest tool on repair → interactive @clack pick pre-filled by lib/tool-detect.ts marker detection → detection result → claude-code (also the --yes path).
  • Template skills are materialized under the chosen profile's skill root (.claude/skills/, .agents/skills/, .cursor/skills/, …) instead of hard-coding the template's location; no empty .claude/ shell is left behind.
  • The chosen tool lands in .bench-kit/instance.json so update knows where to propose skill diffs. Repair preserves the manifest (tool, initializedAt, detected base repo).

update: zone-aware template upgrade

Mirrors the manual adoption procedure done four times (0.3.0, 0.4.0, 0.4.1, 0.4.2):

Zone Behavior
.bench-kit/ replaced wholesale via staged swap; manifest survives, version-bumped (updatedAt)
.github/workflows/ synced from the new template
skills (per manifest tool) synced into the working tree as an uncommitted proposal — company reviews git diff; company-only skills never deleted
tasks/, evaluation-pool/, bench.config.yaml untouched
  • Clean-worktree gate (dirty_worktree) so the proposal diff stays reviewable; up to date no-op when versions match; not_an_instance guard.
  • Closing hint points at bench validate, which owns schema-compatibility checking per the design.
  • Template skill source auto-detects the planned .agents/skills/ migration (falls back to .claude/skills/).

Testing

  • 22 unit tests in tests/bench-kit-command.test.ts (init tool placement, repair preservation, all update zones + guards); full suite 503 green, typecheck + lint clean.
  • Also fixed a latent race in the test helper captureStreams (restore ran after the next capture's patch, leaking output to real stdout).
  • E2E against the real template: init --template-version v0.3.0 --tool codex → skills under .agents/skills/; customized a template skill + added a company skill → update to 0.4.2: runtime replaced (new leaderboard files in), skill customization shown as reviewable diff, company skill preserved, manifest bumped, dirty guard verified.

🤖 Generated with Claude Code

- init resolves the agent tool (--tool flag > manifest on repair >
  interactive pick pre-filled by marker detection > claude-code) and
  materializes template skills under the profile's skill root
  (.claude/skills, .agents/skills, ...); the chosen tool is recorded in
  instance.json so update knows where to propose skill diffs
- update works zone by zone: .bench-kit/ replaced wholesale (staged swap,
  manifest survives version-bumped), workflows and skills synced into the
  working tree as an uncommitted proposal reviewed via git diff, company
  content never touched; clean-worktree gate keeps the proposal reviewable
- template skill source auto-detects the planned .agents/skills migration

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@psmyrdek
psmyrdek merged commit 0756049 into feat/bench-kit-workflows-copy Aug 14, 2026
7 of 8 checks passed
psmyrdek added a commit that referenced this pull request Aug 14, 2026
…#32)

- init resolves the agent tool (--tool flag > manifest on repair >
  interactive pick pre-filled by marker detection > claude-code) and
  materializes template skills under the profile's skill root
  (.claude/skills, .agents/skills, ...); the chosen tool is recorded in
  instance.json so update knows where to propose skill diffs
- update works zone by zone: .bench-kit/ replaced wholesale (staged swap,
  manifest survives version-bumped), workflows and skills synced into the
  working tree as an uncommitted proposal reviewed via git diff, company
  content never touched; clean-worktree gate keeps the proposal reviewable
- template skill source auto-detects the planned .agents/skills migration

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@psmyrdek
psmyrdek deleted the feat/bench-kit-update-and-tool branch August 14, 2026 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant