Skip to content

feat(skills): summarize-delivery — the delivery-side closure leg (0.17.0)#63

Merged
OriNachum merged 16 commits into
mainfrom
feat/summarize-delivery-skill
Jul 9, 2026
Merged

feat(skills): summarize-delivery — the delivery-side closure leg (0.17.0)#63
OriNachum merged 16 commits into
mainfrom
feat/summarize-delivery-skill

Conversation

@OriNachum

@OriNachum OriNachum commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

What

Adds summarize-delivery — the fifth origin skill and the delivery-side
closure leg of the devague method. The flow now runs end to end:

scope → think → spec-to-plan → assign-to-workforce → summarize-delivery

think, spec-to-plan, and assign-to-workforce move from vague intent toward
executable work. This skill closes the loop from executable work back to
accountable reality: it records where execution obeyed the plan, where it
changed, and what is actually safe to claim as delivered. The plan becomes a
contract, not a fiction.

It is not a polite progress report. A delivery claim without evidence is
marked unverified, never asserted as done; partial and failed runs are valid
inputs, reported faithfully.

Contents

  • .claude/skills/summarize-delivery/SKILL.md — method-only (no script, no
    CLI verb, mirroring /scope at birth). Eight-section template: Intent,
    Planned Work, Actual Delivery, Mid-work Decisions, Drift From Plan, Evidence,
    Delivery Claims, Remaining Work / Follow-up. Delivery-claim rows require a
    confidence level plus a resolvable evidence pointer; drift entries require
    exactly one of acceptable / risky / needs-follow-up.
  • docs/skill-sources.md — registered as the fifth origin skill.
  • Flow docsCLAUDE.md, README.md, docs/skills.md, plus an
    "after the final PR" handoff section in assign-to-workforce.
  • docs/deliveries/ — two real artifacts, not examples:
    • 2026-07-09-sharper-end-to-end-method.md — the #53 run. All 14 tasks
      accounted for; 3 classified drift entries; one claim marked unverified
      because task t10's summary named cli/_status.py but its binding
      acceptance criteria never required it, and it was never implemented.
    • 2026-07-09-summarize-delivery-skill.md — this run, reported by the skill
      it ships.

The devague CLI gains no new verbs, no LLM calls, and no orchestration
(#20). Release 0.17.0. It is not purely skills-only: PR review surfaced a
real data-loss bug in the frame/plan stores, fixed here — see below.

Method

Built with devague itself: /think converged a 21-claim, 15-honesty-condition
frame (c41800b); /spec-to-plan converged a 5-task plan covering all 30
targets (9166f73); /assign-to-workforce fanned it out across 3 waves to a
mixed-backend workforce in isolated worktrees, TDD-gated on every merge
(tests + lint before and after).

Review round — one real bug fixed, one pushback

Qodo found two things. I was initially wrong about this: I hand-rolled a
regex over its comment, scraped only <summary> tags, and reported "no bugs."
Both findings live in the collapsed <details> bodies. agex pr read surfaced
them on the first call.

1. Frame schema_version wrong — FIX (833ab78). Real, and the root cause
was in the CLI, not the artifact. store.py::save() and plan_store.py::save()
re-emitted the schema_version loaded from disk instead of the one the running
binary writes. This frame was created via think.sh, which resolves an
installed devague (then 0.14.1, SCHEMA_VERSION = 1) ahead of the checkout;
every later move ran on 0.16.0 and added v2-only payload while save() kept
re-stamping 1. An old binary passes the 1 > 1 guard, loads the file, silently
drops all 5 scope_entries and 15 instruction fields, and rewrites — data loss.
Fixed with upgrade-on-write in both stores plus a regression test each; I verified
both tests fail against the pre-fix source before accepting them. An old
binary now correctly refuses the file. Suite 418 → 420.

2. LLM tasks stored as confirmed — PUSHBACK. plan.py:109 is
status = "proposed" if origin == "llm" else "confirmed". LLM tasks land
proposed and were confirmed by the user at the review gate; confirm flips
status and leaves origin as immutable provenance. origin=llm, status=confirmed is the expected terminal state after a human gate. Qodo rates
it ⭐ Low and notes the same suggestion was rejected in #15. The legitimate
residue — the state records no who/when of confirmation, so a static reader
can't distinguish user- from auto-confirmed — is recorded as follow-up.

Honest notes for the reviewer

Both delivery artifacts record their own drift. The ones that matter here:

  • The workforce was mixed-backend, and one backend mostly failed. Tasks
    were assigned to the colleague CLI (local vLLM Qwen3.6-27B). It delivered
    t2 correctly. It failed t3, t5, and the post-review schema fix — twice
    each — by emitting malformed tool calls as prose at step 0, changing zero
    files, and exiting 0 every time: 6 silent failures in 7 attempts.
    Silent failure is indistinguishable from success to any caller checking $?.
    The operator reassigned each to opus after two failures; task contracts and
    briefs were unchanged. Recorded as drift (acceptable).
  • assign-to-workforce's worked example is known-wrong. It prescribes
    agent/<task-id> branches, but plan-local task ids collide in the
    repo-global namespace — unmerged agent/t1t5 from #53 were still
    present. This run used agent/sd-*. Recorded as drift
    (needs-follow-up); the example should be fixed upstream.
  • t5 exceeded its stated scope limit by committing uv.lock (a one-line
    version self-pin). It flagged this rather than hiding it. Recorded as drift
    (acceptable).
  • The release grew past "skills-only" after review. Fixing the schema bug
    touches devague/. CHANGELOG.md gained a ### Fixed section and the
    delivery summary gained a release-scope drift entry, rather than letting the
    "skills-only" claim quietly go stale.
  • This PR's own summary is an unplanned sixth deliverable. The plan only
    called for the #53 artifact (t4). Recorded as drift (acceptable).
  • Three claims are deliberately not asserted: the final PR gate
    (unverified — it is this PR), guildmaster's re-broadcast (unverified,
    outside this repo), and the skill's generality (low — one successful
    application of it to a run it did not execute is weak evidence).

Verification

Closes the open end of the method described in the issue.


  • devague (Claude)

OriNachum and others added 13 commits July 9, 2026 18:18
Converged Announcement Frame + exported spec for the fifth origin skill:
the delivery-side closure leg (planned vs actual, mid-work decisions,
drift, evidence-backed delivery claims, remaining work).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Converged build plan for the fifth origin skill: t1 SKILL.md + template,
t2 skill-sources registration, t3 flow docs + workforce handoff,
t4 dogfood delivery summary for the sharper-method run, t5 release hygiene.
Waves: [t1 t2 t3] -> [t4] -> [t5].

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add summarize-delivery row to the origin-skills table and update all
surrounding prose enumerations (intro, think/scope note, vendoring-policy
bullet) to include the new skill alongside scope, think, spec-to-plan,
and assign-to-workforce.
…wn run

The skill's second use and first self-application: the run that shipped
summarize-delivery, reported by summarize-delivery. 5/5 plan tasks
accounted for; 5 classified drift entries (2 model reassignments after
colleague's silent failures, 1 scope-limit overrun, this unplanned
artifact, and the branch-namespace deviation); 3 claims held at
unverified/low rather than asserted.

Unplanned deliverable — recorded as drift (acceptable) in the artifact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

feat(skills): add summarize-delivery closure skill (0.17.0)

✨ Enhancement 📝 Documentation ⚙️ Configuration changes 🕐 40+ Minutes

Grey Divider

AI Description

• Add /summarize-delivery as the fifth origin skill to close the workflow loop.
• Document the new five-leg flow and register provenance across skill/docs sources.
• Dogfood with committed delivery artifacts and cut a skills-only 0.17.0 release entry.
Diagram

graph TD
  A["/assign-to-workforce"] --> B[("Plan baseline\n(devague plan waves --json)")] --> C["/summarize-delivery"] --> D[("Committed artifact\ndocs/deliveries/*.md")]
  C --> E["Evidence inputs\n(git/PR/tests)"]
  F["Flow + provenance docs\n(CLAUDE/README/skills/skill-sources)"] --> G["Release notes\nCHANGELOG + version bump"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Add a deterministic devague CLI command to generate delivery summaries
  • ➕ More consistent output shape (machine-generated skeleton + checks)
  • ➕ Could automatically validate evidence pointers / task accounting
  • ➖ Expands the CLI surface and risks creeping orchestration/state mutation concerns (Expose plan dependency waves as deterministic metadata, not orchestration #20 boundary)
  • ➖ Harder to keep claims “honest” (tools may encourage over-assertion)
2. Introduce a first-class delivery state store in .devague/
  • ➕ Enables per-task accounting, drift classification, and evidence tracking as structured data
  • ➕ Supports later automation and reporting
  • ➖ Adds schema/versioning burden and new state semantics
  • ➖ Premature without more dogfooding evidence that machine state is necessary
3. Fold summarize-delivery into assign-to-workforce as a required final gate
  • ➕ Ensures every run produces an artifact
  • ➕ Reduces “forgot to summarize” risk
  • ➖ Effectively becomes a fourth gate; contradicts the intended “wrap-up, not a gate” design
  • ➖ Blurs responsibilities between execution and post-hoc accountability

Recommendation: The PR’s method-only approach is a good v1: it closes the loop without expanding devague’s deterministic CLI or introducing new mutable state. The deferred “delivery engine” is a reasonable follow-up once multiple runs show that structured machine state would materially improve verifiability beyond what a committed artifact + resolvable evidence pointers already provides.

Files changed (15) +1657 / -15

Enhancement (1) +379 / -0
SKILL.mdIntroduce summarize-delivery skill spec + eight-section template +379/-0

Introduce summarize-delivery skill spec + eight-section template

• Adds the new origin skill 'summarize-delivery' as a method-only SKILL.md (no script/CLI verb). Defines the eight required sections, hard rules (no overclaiming; partial/failed runs valid; read-only verification; no devague state mutation), and contracts for drift entries and delivery-claim rows, plus a worked example.

.claude/skills/summarize-delivery/SKILL.md

Documentation (12) +1276 / -13
SKILL.mdAdd post-merge handoff to /summarize-delivery +23/-0

Add post-merge handoff to /summarize-delivery

• Adds an "After the final PR" section instructing operators to run the new sibling '/summarize-delivery' skill after the final PR merges. Clarifies this is a wrap-up artifact step (method-only, read-only) rather than an additional gate.

.claude/skills/assign-to-workforce/SKILL.md

summarize-delivery-skill.jsonAdd exported devague frame for summarize-delivery-skill +399/-0

Add exported devague frame for summarize-delivery-skill

• Adds the exported frame JSON capturing claims, honesty conditions, scope exploration entries, and decisions used to derive the skill and its boundaries. Serves as provenance/working-state for the run that produced the skill artifacts.

.devague/frames/summarize-delivery-skill.json

summarize-delivery-skill.jsonAdd exported devague plan JSON (tasks/waves/acceptance) +298/-0

Add exported devague plan JSON (tasks/waves/acceptance)

• Adds the exported plan JSON describing the five tasks (t1–t5), their acceptance criteria, coverage targets, and wave structure. This is the plan baseline referenced by the delivery artifact and supports auditing planned vs actual.

.devague/plans/summarize-delivery-skill.json

CHANGELOG.mdAdd 0.17.0 skills-only release entry +33/-0

Add 0.17.0 skills-only release entry

• Introduces the 0.17.0 changelog section describing the new 'summarize-delivery' skill, its registration as an origin skill, the new dogfood delivery artifact, and the flow-doc updates. Explicitly notes the CLI is unchanged.

CHANGELOG.md

CLAUDE.mdUpdate operator-skill family list to include summarize-delivery +5/-4

Update operator-skill family list to include summarize-delivery

• Extends the described operator-skill flow to include '/summarize-delivery' as the execution→accountability closure leg. Updates the origin-skill provenance paragraph to include the new skill in the devague-origin set.

CLAUDE.md

README.mdDocument five-leg operator-skill flow in the repo overview +9/-5

Document five-leg operator-skill flow in the repo overview

• Replaces the older two-skill description with the full end-to-end operator-skill family, ending in '/summarize-delivery'. Preserves the distinction that '/think' and '/spec-to-plan' are the CLI-driving core while situating them in the larger workflow.

README.md

2026-07-09-sharper-end-to-end-method.mdAdd delivery summary artifact for the #53 sharper-method run +140/-0

Add delivery summary artifact for the #53 sharper-method run

• Adds a committed delivery summary produced using the new template, accounting for all 14 plan tasks and explicitly classifying drift entries. Includes resolvable evidence pointers and one deliberately 'unverified' claim to avoid overclaiming.

docs/deliveries/2026-07-09-sharper-end-to-end-method.md

2026-07-09-summarize-delivery-skill.mdAdd delivery summary artifact for this summarize-delivery run +186/-0

Add delivery summary artifact for this summarize-delivery run

• Adds a committed delivery summary for the run that shipped 'summarize-delivery', including task-by-task actual delivery, mid-work decisions (including mixed-backend execution issues), drift classification, and evidence-backed claims. Explicitly marks out-of-repo or not-yet-observed claims as 'unverified'/'low'.

docs/deliveries/2026-07-09-summarize-delivery-skill.md

2026-07-09-summarize-delivery-skill.mdAdd exported build plan markdown for summarize-delivery +61/-0

Add exported build plan markdown for summarize-delivery

• Adds the exported build plan markdown listing tasks t1–t5 with instructions, coverage, acceptance criteria, dependencies, and risks. Provides a durable plan artifact parallel to the JSON plan state.

docs/plans/2026-07-09-summarize-delivery-skill.md

skill-sources.mdRegister summarize-delivery as the fifth origin skill +5/-4

Register summarize-delivery as the fifth origin skill

• Updates origin-skill prose and adds a new table row for 'summarize-delivery', including provenance and re-vendoring guidance. Updates the vendoring-policy bullet to include the new devague-origin skill in the non-revendor set.

docs/skill-sources.md

skills.mdAdd summarize-delivery to the skills catalog and flow table +17/-0

Add summarize-delivery to the skills catalog and flow table

• Extends the skills flow table with the execution→accountability leg and adds a dedicated section describing summarize-delivery’s purpose, read-only evidence model, and method-only v1 constraints. Includes a source link to the new skill.

docs/skills.md

2026-07-09-summarize-delivery-skill.mdAdd spec for summarize-delivery (requirements/boundaries) +100/-0

Add spec for summarize-delivery (requirements/boundaries)

• Adds the spec describing the motivation (closing the loop), required template shape, honesty conditions, and strict boundaries (no orchestration, no devague state mutation). Serves as the upstream requirements document for the shipped SKILL.md.

docs/specs/2026-07-09-summarize-delivery-skill.md

Other (2) +2 / -2
current_planUpdate current plan pointer to summarize-delivery-skill +1/-1

Update current plan pointer to summarize-delivery-skill

• Switches the tracked current plan slug to 'summarize-delivery-skill', reflecting the plan being executed/exported in this branch.

.devague/current_plan

pyproject.tomlBump project version to 0.17.0 +1/-1

Bump project version to 0.17.0

• Updates the package version from 0.16.0 to 0.17.0 to match the skills-only release described in the changelog.

pyproject.toml

@qodo-code-review

qodo-code-review Bot commented Jul 9, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (1) 📎 Requirement gaps (0) 📜 Skill insights (0)

Context used
✅ Compliance rules (platform): 66 rules
✅ Skills: cicd, communicate

Grey Divider


Action required

1. Frame schema_version wrong ✓ Resolved 🐞 Bug ≡ Correctness
Description
.devague/frames/summarize-delivery-skill.json declares schema_version: 1 despite containing
v2-added fields (scope_entries and per-item instruction). This undermines the repo’s
schema-versioning contract (fail-closed on unknown/newer schemas) and creates a real
compatibility/data-loss risk if an older devague loads/round-trips the file.
Code

.devague/frames/summarize-delivery-skill.json[R1-8]

+{
+  "slug": "summarize-delivery-skill",
+  "title": "summarize-delivery skill",
+  "schema_version": 1,
+  "status": "exported",
+  "created": "2026-07-09T15:12:14Z",
+  "updated": "2026-07-09T15:17:54Z",
+  "claims": [
Relevance

⭐⭐ Medium

No prior accepted reviews on bumping schema_version in committed .devague frames; schema-version
strictness suggestions often rejected.

PR-#16
PR-#58

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The frame file is marked schema v1 but includes v2-added fields. The code documents those fields as
v2 additions and uses schema_version for compatibility gating (reject only when the persisted
schema is newer than supported), so the on-disk artifact should be labeled v2 to preserve the
fail-closed contract.

.devague/frames/summarize-delivery-skill.json[1-26]
.devague/frames/summarize-delivery-skill.json[346-398]
devague/frame.py[12-16]
devague/store.py[127-143]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The committed frame `.devague/frames/summarize-delivery-skill.json` is labeled `schema_version: 1` even though it includes fields introduced in frame schema v2 (`scope_entries`, and `instruction` fields on claims/honesty conditions). This breaks the intended meaning of `schema_version` and weakens fail-closed behavior for older devague versions.

## Issue Context
- `devague/frame.py` documents that v2 adds `Frame.scope_entries` and `Claim/HonestyCondition.instruction`, and sets `SCHEMA_VERSION = 2`.
- `devague/store.py` only rejects frames when `frame.schema_version > SCHEMA_VERSION`, so leaving this at `1` increases the chance older tooling will accept a file that carries v2-era data.

## Fix Focus Areas
- .devague/frames/summarize-delivery-skill.json[1-8]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Informational

2. LLM tasks stored as confirmed 📘 Rule violation ≡ Correctness
Description
Newly added plan/frame artifacts store LLM-originated tasks/claims (including associated honesty
conditions) with status: "confirmed", violating the requirement that all origin: "llm" items
must be introduced as proposed and not auto-confirmed without explicit human review. This breaks
the intended human-gate and auditability guarantees for LLM-generated work items.
Code

.devague/plans/summarize-delivery-skill.json[R161-167]

+  "tasks": [
+    {
+      "id": "t1",
+      "summary": "author .claude/skills/summarize-delivery/SKILL.md - the method-only skill with the eight-section delivery-summary template and the no-overclaiming hard rules",
+      "origin": "llm",
+      "status": "confirmed",
+      "acceptance_criteria": [
Relevance

⭐ Low

Same suggestion (LLM-origin tasks must start proposed, not confirmed) was explicitly rejected in PR
#15.

PR-#15

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
PR Compliance IDs 738290 and 738294 require that any LLM-originated Claim/Task records—and
honesty conditions attached to LLM-originated claims—must start in proposed status rather than
being introduced as confirmed. In the added .devague artifacts, items explicitly labeled
origin: "llm" are stored with status: "confirmed" (e.g., task t1 and claim c12 in the
plan/frame data), and the frame file further shows an honesty condition (e.g., h11 under
LLM-originated claim c13) also set to status: "confirmed", directly demonstrating the rule
violation.

Rule 738290: LLM-originated claims and tasks must start in proposed status and never be auto-confirmed
Rule 738294: Assign LLM-originated honesty conditions proposed status
.devague/plans/summarize-delivery-skill.json[161-167]
.devague/frames/summarize-delivery-skill.json[200-206]
.devague/frames/summarize-delivery-skill.json[211-222]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
New `Task`/`Claim` records marked with `origin: "llm"` (and honesty conditions associated with LLM-originated claims) are being committed with `status: "confirmed"`, but the compliance rules require all LLM-originated items to be introduced as `proposed` and not be auto-confirmed without explicit human review.

## Issue Context
This PR adds new state artifacts under `.devague/plans/...` and `.devague/frames/...` for `summarize-delivery-skill`. In these artifacts, entries explicitly marked `origin: "llm"` are represented as already `confirmed`, including an honesty condition under an LLM-originated claim (e.g., claim `c13` with honesty condition `h11`) that is also set to `confirmed`.

## Fix Focus Areas
- .devague/plans/summarize-delivery-skill.json[161-167]
- .devague/frames/summarize-delivery-skill.json[200-206]
- .devague/frames/summarize-delivery-skill.json[211-222]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Comment thread .devague/frames/summarize-delivery-skill.json
OriNachum and others added 3 commits July 9, 2026 19:15
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013gVnMHmeZiw2EcjSnJDGZq
Qodo (PR #63) surfaced a real data-loss bug in store.py/plan_store.py.
Fixing it makes 0.17.0 "skills + one correctness fix", not skills-only —
CHANGELOG gains a Fixed section, and the delivery summary gains the
release-scope drift entry, the corrected claims (420 tests; CLI gains no
verb but does change), and the retraction of the operator's wrong
"Qodo found no bugs" reading.

Addresses review comment 3553078018.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@sonarqubecloud

sonarqubecloud Bot commented Jul 9, 2026

Copy link
Copy Markdown

@OriNachum

Copy link
Copy Markdown
Contributor Author

Re: finding 2 — LLM tasks stored as confirmed

PUSHBACK. The rule is satisfied; the artifact shows the terminal state after the human gate, not an auto-confirmation.

devague/plan.py:109 is the gate:

status = "proposed" if origin == "llm" else "confirmed"

Every --origin llm task lands proposed. The run's own CLI output shows added t1 (proposed)added t5 (proposed). They were then surfaced to the user for review, and confirmed by an explicit devague plan confirm t1 … t5 — a user-only move. confirm transitions status and deliberately leaves origin untouched, because origin is immutable provenance ("an LLM proposed this") while status is the gate ("a human accepted it"). So origin: "llm", status: "confirmed" is precisely what a correctly-gated LLM proposal looks like once the human has signed off. Any other terminal state would mean the gate never ran.

The same holds for c12 and honesty condition h11 in the frame: proposed by the LLM, confirmed by the user at the review gate (devague reviewdevague confirm).

Noting that the tool itself rates this ⭐ Low and records that the identical suggestion was explicitly rejected in #15.

That said, the finding gestures at something real, and I don't want to dismiss it wholesale. The persisted state records what an item's origin was and what its terminal status is — but not who confirmed it or when. A static reader of the JSON genuinely cannot distinguish a user-confirmed item from a hypothetically auto-confirmed one; only the CLI's behavior enforces the difference. That is a real auditability gap for a project whose whole thesis is the anti-fabrication contract. I've recorded it as follow-up in docs/deliveries/2026-07-09-summarize-delivery-skill.md (Remaining Work: "Confirmation provenance is not persisted"), rather than change confirmed-state semantics inside a PR that has already converged.

  • devague (Claude)

@OriNachum OriNachum merged commit 0b4f8e2 into main Jul 9, 2026
8 checks passed
@OriNachum OriNachum deleted the feat/summarize-delivery-skill branch July 9, 2026 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant