Skip to content

fixes from the fifth adversarial review (command-set contracts) - #16

Merged
mnkprs merged 2 commits into
mainfrom
codex5-fixes
Aug 30, 2026
Merged

fixes from the fifth adversarial review (command-set contracts)#16
mnkprs merged 2 commits into
mainfrom
codex5-fixes

Conversation

@mnkprs

@mnkprs mnkprs commented Aug 30, 2026

Copy link
Copy Markdown
Owner

Why

An adversarial review of the new command set found eight issues: plugin installs expose namespaced command names the README did not document; the release moved without a version bump; five skills could not locate the engine from a fresh standalone install; the baseline still cloned the task repository and never printed its promised unverified-evidence trailer; the documented no-argument score invocation exited with usage; two skill descriptions promised "offline / clones nothing" while requiring ~1.2 GB of clones; trigger descriptions overlapped between paid and free operations; and the README still said "two skills" and omitted the baseline's authentication requirement.

What changed

  • src/lib/run.ts + src/screen.ts: a variant with no tools and no subagents (the baseline) now runs without cloning or validating the task repository — the guard is attached only for repo-using variants — and the verdict trailer prints "evidence not machine-verified — this variant has no verifier" when no verifier ran.
  • src/score.ts: no run ids defaults to baseline v3-verify (covered by a new test; 29 total).
  • Version 0.1.1 in package.json, both plugin manifests and the marketplace entry; validate:manifests now enforces version consistency across all four and checks the engine pins match v<version>; the two screening skills pin --branch v0.1.1.
  • All five read-only skills and fairtask-eval share one engine-location step: current dir → $CLAUDE_PLUGIN_ROOT$FAIRTASK_HOME~/.fairtask → clone the pinned release (announced).
  • Honest contracts: fairtask-report and fairtask-eval disclose the ~1.2 GB workspace clones in their descriptions and announce before cloning; fairtask-baseline triggers only when a specific task is supplied and points published-number comparisons at report/score.
  • README §2b: "seven agent skills, one command per operation"; the plugin row shows the namespaced forms; both screening commands' authentication stated; the report row discloses the workspace clone.

Manos Kaparos and others added 2 commits August 31, 2026 02:07
…re default, v0.1.1, honest skill contracts

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mnkprs
mnkprs merged commit 8abc7f0 into main Aug 30, 2026
2 checks passed
@mnkprs
mnkprs deleted the codex5-fixes branch August 30, 2026 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant