feat(skill): add context-intelligence-hill-climbing skill for audit-trail tracking#77
Merged
colombod merged 1 commit intoJul 17, 2026
Conversation
…rail tracking Add a thin, shared skill that instructs agents to use the todo tool as a hill-climb log during multi-step investigations. Wire both graph-analyst and session-navigator agents to load this skill when facing provenance tracing, cross-session synthesis, or other multi-step investigation tasks. The skill governs HOW an investigation is TRACKED (seed open questions as todos, write each finding + its citation into items, mark dead leads) but deliberately does NOT re-teach navigation mechanics (those remain in graph-query / session-navigation / navigation-budget-discipline). Previous evaluation showed both agents already navigate safely and correctly (bounded probes, no context flooding) but leave inconsistent audit trails. This skill adds observability to that discipline: turning the todo tool into a structured climb-log so a later reader can see how an answer was reached. 🤖 Generated with Amplifier Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a thin, shared
context-intelligence-hill-climbingskill (todo-as-climb-log) and wire bothgraph-analystandsession-navigatoragents to load it on multi-step investigations. An earlier evaluation found both agents already navigate safely and correctly (bounded probes, no context flooding) but leave an inconsistent audit trail — baseline session-navigator used 0 todo calls while still answering. This skill's purpose is observability/consistency, NOT correctness or efficiency.Scope / guardrails
agents/graph-analyst.md(added "Track multi-step investigations in a todo climb-log" guidance block)agents/session-navigator.md(identical wiring block)skills/context-intelligence-hill-climbing/SKILL.md(thin skill, ~2 KB, focused on using todo as climb-log)logging_handler.py,_DestinationDispatcher,fanout.py), navigation discipline mechanics (those stay ingraph-query,session-navigation,navigation-budget-discipline.md), bundle structureThe skill governs HOW an investigation is tracked (seed open questions, write findings + citations, mark dead leads) — it deliberately does NOT re-teach navigation mechanics.
Verification
modules/tool-context-intelligence-query: 132 passedtests/: 18 passedruff check+ruff format --checkclean — 0 errorspyrightclean — 0 errorsscripts/validate-full.sh→validation_mode: fullThe lone mode-advertising ERROR is a documented FALSE POSITIVE (name collision between bundle name, storage path, skill name, and internal mode) — see AGENTS.md. Validator confirmed overall verdict PASS.
Real evidence on seams (not mock-only)
This change crosses a seam: agent wiring (skill loading). Per AGENTS.md, seam changes require real DTU run + evaluation harness with captured evidence.
Lightweight A/B evidence (delegate-based, n=1 per cell, skill injected as inline guidance because not yet installed):
Both agents were asked the same real RCA/provenance question (session
1ade4f66…, which session produced the auto-recovery design first?) on both surfaces (graph-analyst, session-navigator), cold baseline vs. +skill treatment.The +skill runs produced a structured, cited todo climb-log (open questions seeded up front; findings written into items with node-id/line citations). Correctness held at baseline and remained correct with the skill. Cost rose ~50–80% (expected overhead for an observability feature).
Session-navigator's final climb-log from the +skill run read:
NOTE: This repo's AGENTS.md requires agent/skill/mode edits to pass "a real DTU run + the evaluation harness, with captured evidence" — not just lightweight delegate tests. That full DTU-isolated, multi-trial run has NOT been executed. RECOMMEND running the DTU + evaluation-harness gate before merge (see
.amplifier/digital-twin-universe/profiles/for available profiles).Docs & diagrams
bundle.dot/bundle.pngregenerated if bundle structure changed —N/A— no bundle structure change (skill addition is additive, no new seams in bundle.md or behaviors layout)N/A— no new convention learnedNotes / follow-ups
Gate not fully met: This PR provides lightweight evidence (delegate-based, n=1 per cell) that the skill produces structured audit trails without breaking correctness. Full DTU + evaluation harness validation remains as a pre-merge recommendation per AGENTS.md § Testing & what "done" looks like. The DTU profiles exist in
.amplifier/digital-twin-universe/profiles/(e.g.,context-intelligence-bundle-smoke-test.yaml) — running one of them with this branch's agents will give the full proof required before shipping.