Phase 1: read Claude Code transcripts into the neutral representation - #40
Merged
Conversation
…tation
The first real implementation of the capture port, and the first thing to put
the neutral representation under load. Two corrections to it fell out of
building this, both measured rather than argued:
- no transcript record in any sampled version carries a price. 401 files and
135,001 records, zero occurrences. Even the reference harness cannot supply
money from a transcript, so the golden session's cost profile cannot be
read from the thing being blessed.
- a tool result records whether the call errored and never the code it
returned, so ExitCode becomes a pointer. Filling it with zero would have
reported success beside an outcome saying the call failed, and zero is
itself a meaningful code.
The capability declaration is measured, not assumed. Cost, the workspace patch,
file mutations and their content are not claimed. Model identity is partial
because a transcript records what served a message and never the alias asked
for. Tool results are partial because large ones are shortened in place.
Delegation is partial, and that was nearly an overclaim: it read correctly from
the synthetic fixture and produced nothing at all from 300 real transcripts. A
scan of 601 project-root files found no inline delegation — every one keeps it
in sidecar files this adapter does not read. Declared because the inline form
is handled, marked partial because on the real store the answer is always
empty, so an assertion resolves to an error rather than to a false negative.
Verified against the real store: 300 of 300 transcripts parsed, no refusals,
15,022 turns, and 5,112 tool calls matched to 5,112 results — the exact pairing
the earlier survey measured. Every committed fixture is synthetic; no
transcript content is committed, and the redaction pass that would make an
export safe is not trustworthy yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ew64uotJqtvVDN3twMAVEQ
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first implementation of the capture port, and the first thing to put the
neutral representation under real load. Claude Code is the reference adapter,
not the substrate: nothing outside this package learns a harness name, and the
architecture test that enforces that stays green.
Two corrections to the representation fell out of building it, both measured
rather than argued:
135,001 records, zero occurrences. Even the reference harness cannot supply
money from a transcript, so
CanSeeCostis not claimed.so
ExitCodebecame a pointer. Filling it with zero would have reportedsuccess beside an outcome saying the call failed, and zero is itself a
meaningful code.
The capability declaration is measured. Cost, the workspace patch, file
mutations and their content are not claimed. Model identity is partial because a
transcript records what served a message and never the alias asked for. Tool
results are partial because large ones are shortened in place. Delegation is
partial because a scan of 601 project-root transcripts found no inline
delegation at all — every one keeps it in sidecar files this adapter does not
read, so an assertion about delegated work resolves to an error rather than to a
false negative.
Against the real store: 300 of 300 transcripts parsed, no refusals, 15,022
turns, 5,112 tool calls matched to 5,112 results — the same pairing the earlier
survey measured. Every committed fixture is synthetic; no transcript content is
committed.
Known defects, filed before merge
An adversarial review after this commit landed found fifteen issues, six of
which are severe enough to name here. They are filed rather than fixed in this
branch, so that the review's evidence is on the record before any of it is
touched:
CanSeeTokensdeclared whole while cache token fields go unreadOrderCompletestays true on a compacted session, so the flag is deadDiscoverwith aDirquery silently matches almost nothingModel.Provideris filled with a value no transcript statesCanSeeWorkspacedeclared whole whileRevisionandOriginare unsetThe rest — #31 through #39 — cover wall-clock duration, inline delegation,
sentinel selection, version joining, empty transcripts, two hardening items, a
group of small parser gaps, and one structural gap in the conformance kit
itself: it catches an adapter claiming too little and not one claiming too much.
Four of these are the same class of mistake, and it is worth naming: a
capability declared whole while the fields it governs go unfilled. The
declaration is what decides whether a missing value reads as
ERRORor asFAIL, so an overclaim here manufactures exactly the regression therepresentation exists to prevent. #39 is why the kit did not catch them.
The distiller reads this output, so #25 and #27 want fixing before anything is
blessed from it.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Ew64uotJqtvVDN3twMAVEQ