feat(yield): add evaluation evidence surface

.github/workflows/yield-lab.yml

@@ -22,6 +22,11 @@ jobs:
       - uses: actions/setup-node@v4
         with:
           node-version: "24"
+      - name: Validate evaluation cases and published summaries
+        working-directory: labs/22-yield/yield/evals
+        run: |
+          npm ci
+          npm run validate
       - name: Build and smoke-test the packed TypeScript SDK
         working-directory: labs/22-yield/yield/sdk/typescript
         run: |
@@ -80,6 +85,11 @@ jobs:
       - uses: actions/setup-node@v4
         with:
           node-version: "24"
+      - name: Validate projected evaluation cases and summaries
+        working-directory: ${{ runner.temp }}/yield-projected/evals
+        run: |
+          npm ci
+          npm run validate
       - name: Build and smoke-test the projected TypeScript package
         working-directory: ${{ runner.temp }}/yield-projected/sdk/typescript
         run: |

labs/22-yield/distribution/release-notes/2026-08-01-evaluation-surface.md

@@ -0,0 +1,9 @@
+# Evaluation cases and evidence boundary
+
+- Add six provenance-bound skill-conversion cases with their thin model-facing
+  skills and Yield workflow programs.
+- Add a reproducible source-size harness and fail-closed summary validator.
+- Keep raw transcripts, temporary repositories, and large run artifacts out of
+  Git; published behavior summaries must bind an immutable artifact digest.
+- Validate the same evaluation surface in both the source tree and projected
+  public repository.

labs/22-yield/public-readme/README.md

@@ -81,6 +81,8 @@ use the documentation by job:
 - [tutorials](docs/tutorials/README.md) — review, approval, environment
   repair, bounded debugging, and migration;
 - [examples](docs/examples.md) — working programs in all four languages;
+- [evaluations](evals/README.md) — pinned conversion cases, reproducible
+  measurements, and the raw-evidence publication boundary;
 - [convert an existing skill](docs/convert-existing-skill.md) — move
   control flow into code without claiming that fixture execution proves
   every reading of the original prose;

labs/22-yield/publish.config.json

@@ -31,6 +31,9 @@
         "dist",
         "build",
         "runs",
+        "raw",
+        "artifacts",
+        ".worktrees",
         "target"
       ]
     },

labs/22-yield/yield/docs/examples.md

@@ -60,3 +60,15 @@ YSKILL=/tmp/yskill /tmp/yskill test examples/convert-skill
 When adapting an example, change the repository-specific commands and model
 instructions. Keep stable operation IDs for existing steps so saved runs can
 replay them.
+
+## Conversion cases used by evaluations
+
+The [evaluation cases](../evals/cases/) are different from the example library.
+Each one pins a public third-party skill to an exact commit and digest, then
+shows the smaller model-facing `SKILL.md` beside the TypeScript workflow that
+owns its order, commands, gates, and completion.
+
+The source-size harness and early summaries live in [`evals/`](../evals/).
+Large transcripts and temporary repositories are external artifacts, never
+committed source. A behavioral result is publishable only when its summary
+binds the exact artifact URI and SHA-256.

labs/22-yield/yield/evals/.gitignore

@@ -0,0 +1,5 @@
+runs/
+raw/
+artifacts/
+.worktrees/
+node_modules/

labs/22-yield/yield/evals/README.md

@@ -0,0 +1,39 @@
+# Yield evaluations
+
+This directory contains the public, reviewable part of Yield's evaluation
+system: case definitions, pinned source identities, conversion programs,
+measurement code, validation rules, and small result summaries.
+
+Raw agent transcripts, temporary repositories, command logs, and model
+responses do not belong in Git. A full campaign uploads those files as one
+immutable artifact bundle and records its URI and SHA-256 in the published
+summary. Until that bundle exists, the summary must say `unpublished`.
+
+## Layout
+
+- `cases/` — pinned public source identities plus the thin skill and Yield
+  program used for each conversion.
+- `scripts/measure-source.mjs` — reproduces the source-size comparison from
+  pinned upstream files.
+- `scripts/validate.mjs` — fail-closed validation for cases and summaries.
+- `results/latest.json` — small website-safe summary. It is not raw evidence.
+- `runs/` — local or CI output; ignored by Git and projected releases.
+
+## Run
+
+```bash
+npm install
+npm run validate
+npm run measure
+```
+
+`npm run measure` writes a fresh summary under `runs/`. Publishing that result
+requires a separate promotion step that binds the raw artifact digest, the
+exact Yield commit, model identity, harness version, and case-set digest.
+
+## Claim boundary
+
+Source-size measurements show how much model-facing text and workflow source
+the prototypes contain. They do not prove behavioral equivalence. Behavioral
+claims require executable fixtures, held-out oracles, repeated model runs, and
+the immutable raw artifact named by the result summary.

labs/22-yield/yield/evals/cases/README.md

@@ -0,0 +1,15 @@
+# Conversion cases
+
+These are measured rewrites, not automatic equivalence claims. Each case keeps
+the model-facing judgment in a short `SKILL.md` and moves repeatable control
+flow into a TypeScript Yield program. The pinned original remains in its source
+repository and is identified by commit plus SHA-256 in `index.json`.
+
+| Case | Thin skill | Yield program | Pinned original |
+|---|---|---|---|
+| GStack review | [SKILL.md](gstack-review/SKILL.md) | [workflow.ts](gstack-review/workflow.ts) | [source](https://github.com/garrytan/gstack/blob/a3259400a366593e0c909dd9ac3e59752efd2488/review/SKILL.md) |
+| Anthropic doc co-authoring | [SKILL.md](doc-coauthoring/SKILL.md) | [workflow.ts](doc-coauthoring/workflow.ts) | [source](https://github.com/anthropics/skills/blob/b29e7cf65e5cb78a5ac33d582270551bc74a14eb/skills/doc-coauthoring/SKILL.md) |
+| Superpowers systematic debugging | [SKILL.md](systematic-debugging/SKILL.md) | [workflow.ts](systematic-debugging/workflow.ts) | [source](https://github.com/obra/superpowers/blob/44c9b2d6e889982ac18c27d05a19fefe335194e1/skills/systematic-debugging/SKILL.md) |
+| Vercel deploy | [SKILL.md](vercel-deploy/SKILL.md) | [workflow.ts](vercel-deploy/workflow.ts) | [source](https://github.com/vercel-labs/agent-skills/blob/7c180d9044c9ae2b442b567aad4e42a28dd5ed62/skills/deploy-to-vercel/SKILL.md) |
+| Microsoft MCP builder | [SKILL.md](mcp-builder/SKILL.md) | [workflow.ts](mcp-builder/workflow.ts) | [source](https://github.com/microsoft/skills/blob/4a2873faffc1b101a33a0b59c24713d4ed78142f/.github/skills/mcp-builder/SKILL.md) |
+| Trail of Bits actions auditor | [SKILL.md](actions-auditor/SKILL.md) | [workflow.ts](actions-auditor/workflow.ts) | [source](https://github.com/trailofbits/skills/blob/1256982d4d925a0acfe11e26c2253c32052c6247/plugins/agentic-actions-auditor/skills/agentic-actions-auditor/SKILL.md) |

labs/22-yield/yield/evals/cases/actions-auditor/SKILL.md

@@ -0,0 +1,17 @@
+---
+name: agentic-actions-auditor
+description: Audit AI-enabled CI workflows for concrete security risks.
+---
+
+# Agentic actions auditor
+
+Inspect the supplied workflow files and repository context. Trace untrusted input
+to agent prompts, tools, credentials, write permissions, network access, and
+mutable dependencies. Report only findings with a concrete attack path.
+
+Each finding must include severity, workflow and line, source, capability reached,
+impact, evidence, and remediation. Distinguish exploitable paths from hardening
+advice. State coverage gaps explicitly.
+
+Yield owns file discovery, scope, evidence capture, required fields, report
+generation, and completion.

labs/22-yield/yield/evals/cases/actions-auditor/workflow.ts

@@ -0,0 +1,13 @@
+export default defineSkill((ctx) => {
+  const files = ctx.runCommand("discover", "find .github/workflows -type f -name '*.yml' -o -name '*.yaml'", 30)
+  ctx.require(files.exit_code === 0 && files.stdout.length > 0, "workflow files found", files)
+
+  const context = ctx.runCommand("context", "git ls-files && git status --short", 30)
+  const audit = ctx.agentTask("audit", prompts.audit, { files: files.stdout, context })
+  ctx.require(audit.coverage.reviewed === audit.coverage.discovered, "all workflows reviewed", audit.coverage)
+  ctx.require(audit.findings.every((finding) => finding.evidence), "every finding has evidence", audit)
+
+  const report = ctx.runCommand("report", "node scripts/render-audit.mjs", 60, { stdin: audit })
+  ctx.require(report.exit_code === 0, "report generated", report)
+  return ctx.complete({ findings: audit.findings, report: report.stdout })
+})

labs/22-yield/yield/evals/cases/doc-coauthoring/SKILL.md

@@ -0,0 +1,17 @@
+---
+name: doc-coauthoring
+description: Help a user turn context into a clear document for a named reader.
+---
+
+# Doc co-authoring
+
+At each model step, ask only the questions needed for the current stage. Keep
+the author's voice. Make claims concrete, expose missing evidence, and organize
+the document around what its intended reader must understand or decide.
+
+For reader testing, act as a fresh reader with no hidden context. List unclear
+terms, unanswered questions, and assumptions the document makes. Suggest the
+smallest edits that resolve them.
+
+Yield owns stage order, saved answers, iteration limits, user choices, and the
+definition of done.

labs/22-yield/yield/evals/cases/doc-coauthoring/workflow.ts

@@ -0,0 +1,15 @@
+export default defineSkill((ctx) => {
+  const audience = ctx.askUser("audience", "Who will read this document?")
+  const outcome = ctx.askUser("outcome", "What should the reader know or decide?")
+  const context = ctx.askUser("context", "Paste the source context.")
+
+  const outline = ctx.agentTask("outline", prompts.outline, { audience, outcome, context })
+  const chosen = ctx.askUser("outline-approval", "Use this outline?", ["use", "revise"])
+  ctx.require(chosen === "use", "outline approved", outline)
+
+  const draft = ctx.agentTask("draft", prompts.draft, { outline, context })
+  const test = ctx.agentTask("reader-test", prompts.readerTest, { audience, draft })
+  const final = ctx.agentTask("revise", prompts.revise, { draft, test })
+  ctx.require(test.blocking_questions.length === 0, "reader has no blocking questions", test)
+  return ctx.complete(final)
+})

labs/22-yield/yield/evals/cases/gstack-review/SKILL.md

@@ -0,0 +1,18 @@
+---
+name: review
+description: Review the current branch for important defects before shipping.
+---
+
+# Review
+
+Inspect the supplied diff and evidence. Look for failures tests may miss:
+trust-boundary mistakes, unsafe side effects, incomplete error handling,
+concurrency problems, and behavior that contradicts the surrounding code.
+
+Return structured findings with `severity`, `confidence`, `category`, `file`,
+`line`, `problem`, and `fix`. Use only `critical` or `informational`. Count only
+defects caused by this branch. Prefer a small number of specific findings over
+general advice.
+
+Yield owns repository checks, ordering, policy validation, saved state, and
+completion. Do not reproduce those rules in prose.

labs/22-yield/yield/evals/cases/gstack-review/workflow.ts

@@ -0,0 +1,12 @@
+export default defineSkill((ctx) => {
+  const diff = ctx.runCommand("diff", "git diff --merge-base origin/main HEAD", 60)
+  ctx.require(diff.exit_code === 0 && diff.stdout.length > 0, "branch has a readable diff", diff)
+
+  const checks = ctx.runCommand("checks", "npm test && npm run typecheck", 600)
+  ctx.require(checks.exit_code === 0, "tests and types pass", checks)
+
+  const review = ctx.agentTask("review", prompts.review, { diff: diff.stdout, checks })
+  const critical = review.findings.some((finding) => finding.severity === "critical")
+  ctx.require(!critical, "no unresolved critical findings", review)
+  return ctx.complete(review)
+})

labs/22-yield/yield/evals/cases/index.json

@@ -0,0 +1,84 @@
+{
+  "schema_version": 1,
+  "methodology_version": "0.1",
+  "cases": [
+    {
+      "id": "gstack-review",
+      "label": "GStack review",
+      "source": {
+        "repo": "garrytan/gstack",
+        "commit": "a3259400a366593e0c909dd9ac3e59752efd2488",
+        "path": "review/SKILL.md",
+        "license": "MIT",
+        "sha256": "92ee16af71d5e0088326869b0a211c50f94b9261eeae75656bc21f9bcfae2031"
+      },
+      "thin_skill": "gstack-review/SKILL.md",
+      "workflow": "gstack-review/workflow.ts"
+    },
+    {
+      "id": "doc-coauthoring",
+      "label": "Anthropic doc co-authoring",
+      "source": {
+        "repo": "anthropics/skills",
+        "commit": "b29e7cf65e5cb78a5ac33d582270551bc74a14eb",
+        "path": "skills/doc-coauthoring/SKILL.md",
+        "license": "See source repository",
+        "sha256": "2e47d78846faeea4a56e9809c52700087a15a2155a3f293a3efbaded81398ef4"
+      },
+      "thin_skill": "doc-coauthoring/SKILL.md",
+      "workflow": "doc-coauthoring/workflow.ts"
+    },
+    {
+      "id": "systematic-debugging",
+      "label": "Superpowers systematic debugging",
+      "source": {
+        "repo": "obra/superpowers",
+        "commit": "44c9b2d6e889982ac18c27d05a19fefe335194e1",
+        "path": "skills/systematic-debugging/SKILL.md",
+        "license": "MIT",
+        "sha256": "808fc5717aa88ad65efff312b11c186294d3e6ee301afb584e2f86599b137787"
+      },
+      "thin_skill": "systematic-debugging/SKILL.md",
+      "workflow": "systematic-debugging/workflow.ts"
+    },
+    {
+      "id": "vercel-deploy",
+      "label": "Vercel deploy",
+      "source": {
+        "repo": "vercel-labs/agent-skills",
+        "commit": "7c180d9044c9ae2b442b567aad4e42a28dd5ed62",
+        "path": "skills/deploy-to-vercel/SKILL.md",
+        "license": "See source repository",
+        "sha256": "cfcc3dd479ab2e0ae721ddf39b8af84d977321487672f1487c8d6855f576927b"
+      },
+      "thin_skill": "vercel-deploy/SKILL.md",
+      "workflow": "vercel-deploy/workflow.ts"
+    },
+    {
+      "id": "mcp-builder",
+      "label": "Microsoft MCP builder",
+      "source": {
+        "repo": "microsoft/skills",
+        "commit": "4a2873faffc1b101a33a0b59c24713d4ed78142f",
+        "path": ".github/skills/mcp-builder/SKILL.md",
+        "license": "MIT",
+        "sha256": "621e771c22224140752ddf933923467b2d1148580194ad59d3f1f21f9f27bdc9"
+      },
+      "thin_skill": "mcp-builder/SKILL.md",
+      "workflow": "mcp-builder/workflow.ts"
+    },
+    {
+      "id": "actions-auditor",
+      "label": "Trail of Bits actions auditor",
+      "source": {
+        "repo": "trailofbits/skills",
+        "commit": "1256982d4d925a0acfe11e26c2253c32052c6247",
+        "path": "plugins/agentic-actions-auditor/skills/agentic-actions-auditor/SKILL.md",
+        "license": "CC-BY-SA-4.0",
+        "sha256": "80e36ab06e3ee667ac45bab036ed395fc425c4d54fa72c4b4981a5b982a389aa"
+      },
+      "thin_skill": "actions-auditor/SKILL.md",
+      "workflow": "actions-auditor/workflow.ts"
+    }
+  ]
+}

labs/22-yield/yield/evals/cases/mcp-builder/SKILL.md

@@ -0,0 +1,17 @@
+---
+name: mcp-builder
+description: Design and review a small MCP server around a defined use case.
+---
+
+# MCP builder
+
+Turn the supplied use case into a minimal tool surface. For each tool, define a
+specific name, typed inputs, bounded output, error behavior, and one realistic
+example. Prefer fewer composable tools. Keep secrets out of arguments and make
+destructive effects explicit.
+
+During review, check schema clarity, transport errors, authentication boundaries,
+pagination, idempotency, and whether evaluations cover success and failure paths.
+
+Yield owns research approval, scaffold and test commands, evaluation thresholds,
+saved artifacts, and completion.

labs/22-yield/yield/evals/cases/mcp-builder/workflow.ts

@@ -0,0 +1,19 @@
+export default defineSkill((ctx) => {
+  const useCase = ctx.askUser("use-case", "What must this MCP server enable?")
+  const constraints = ctx.askUser("constraints", "Which APIs, auth, and runtime apply?")
+  const design = ctx.agentTask("design", prompts.design, { useCase, constraints })
+  ctx.require(design.tools.length <= 8, "tool surface stays small", design)
+
+  const approval = ctx.askUser("design-approval", "Build this tool surface?", ["build", "revise"])
+  if (approval !== "build") ctx.blocked("design needs revision")
+  const scaffold = ctx.runCommand("scaffold", "npm run scaffold:mcp", 120)
+  ctx.require(scaffold.exit_code === 0, "server scaffolds", scaffold)
+  const tests = ctx.runCommand("tests", "npm test", 600)
+  ctx.require(tests.exit_code === 0, "tests pass", tests)
+
+  const review = ctx.agentTask("review", prompts.review, { design, tests })
+  ctx.require(review.blockers.length === 0, "review has no blockers", review)
+  const evals = ctx.runCommand("evals", "npm run evals", 900)
+  ctx.require(evals.exit_code === 0, "evaluations pass", evals)
+  return ctx.complete({ design, review, evals })
+})

labs/22-yield/yield/evals/cases/systematic-debugging/SKILL.md

@@ -0,0 +1,18 @@
+---
+name: systematic-debugging
+description: Diagnose a reproducible failure from evidence before proposing a fix.
+---
+
+# Systematic debugging
+
+Use the supplied failure output and code context to identify one falsifiable
+root-cause hypothesis. Separate observations from inference. Name the mechanism,
+the evidence that supports it, and the smallest experiment that could disprove
+it. Do not recommend a fix until the hypothesis survives that experiment.
+
+When reviewing a candidate fix, check that it addresses the mechanism rather
+than hiding the symptom and that the original failure now passes without a new
+regression.
+
+Yield owns phase order, experiment bounds, commands, saved evidence, and exit
+conditions.

labs/22-yield/yield/evals/cases/systematic-debugging/workflow.ts

@@ -0,0 +1,16 @@
+export default defineSkill((ctx) => {
+  const failure = ctx.runCommand("reproduce", "npm test -- --runInBand", 300)
+  ctx.require(failure.exit_code !== 0, "failure reproduces", failure)
+
+  const hypothesis = ctx.agentTask("hypothesis", prompts.hypothesis, { failure })
+  ctx.require(Boolean(hypothesis.experiment), "hypothesis is falsifiable", hypothesis)
+  const experiment = ctx.runCommand("experiment", hypothesis.experiment, 300)
+  const verdict = ctx.agentTask("evaluate", prompts.evaluate, { hypothesis, experiment })
+  ctx.require(verdict.supported, "root cause supported by evidence", verdict)
+
+  const fix = ctx.askUser("fix", "Apply the proposed fix?", ["apply", "stop"])
+  if (fix !== "apply") ctx.blocked("fix not approved")
+  const verification = ctx.runCommand("verify", "npm test", 600)
+  ctx.require(verification.exit_code === 0, "full test suite passes", verification)
+  return ctx.complete({ hypothesis, verification })
+})