fix(codex): carry the token total and stop counting synthesized spans as tool calls

[drewstone]

Stacked on #110 (fix/codex-command-and-turn-facts), which is still open. Base this PR's review on the diff against that branch; retarget to main after #110 merges. It merges cleanly with both #110's head and main today.

What this implements

Wave-2 changes 2, 5 and 6 of the Codex adapter work, from a measured audit of twelve private Codex sessions. On those sessions a mechanical extraction from the OTLP spans this adapter already writes scores a deterministic mean of 0.858, against 0.389 for the traces analyst arm and 0.906 / 0.917 for an Opus subagent fleet. The two adapter gaps below are what keeps that extraction from 0.983.

1. The cumulative token total (+0.125 measured on the deterministic mean). Codex reports token_count.info.total_token_usage beside the per-turn last_token_usage delta. The adapter read that object and used it only as a de-duplication signature, so no span in the corpus held the session total: an extractor could only sum per-turn deltas (9,492,255 against a truth of 17,025,686 on one session) or sum cumulative snapshots (5,896,355,271 against 78,761,456 on another). The last reported snapshot equals ground truth on 13 of 13 sessions, so it is carried verbatim onto the root span — nothing is summed or derived:

• traces.session.total_tokens, plus total_input_tokens / total_output_tokens / total_reasoning_tokens / total_cached_input_tokens from that same snapshot, and traces.session.total_tokens_source.
• It is read before the per-turn delta gate, so a token_count event that advances the counter without reporting a delta still contributes the total.
• A session that reported no total gets no attribute. Unknown stays unknown; it never becomes zero.

2. The synthesized subagent span (required for any tool-call count to be right). ensureSubagentSpan built a child thread's lifecycle as kind: TOOL, name: tool.Agent, tool.name: Agent. It records that a subagent started or finished — not a call the main agent issued — so every span-kind tool count ran high by exactly the number of child threads. On the twelve sessions, count(TOOL) - count(tool.Agent) reproduced ground truth on 12 of 12.

3. The pre-fork prefix and compacted records (unmeasured, n = 1 judged question). The task-scope walk stopped at the fork boundary and compacted records were never parsed at all, so a forked child's human context reached no span: on one session the human's words exist in 2 user records and 37 compacted records, and in 0 spans. Both are now kept, marked, deduplicated on the turn text, and bounded.

Schema change (for downstream miners)

Before	After
Codex subagent lifecycle: kind: TOOL, name: tool.Agent, tool.name: Agent	kind: AGENT, name: subagent.lifecycle, no tool.name, traces.span.synthesized = true, traces.span.synthesized_from = codex.sub_agent_activity, traces.codex.subagent_type
no session token total on any span	root span carries traces.session.total_tokens (+ the rest of the same snapshot and traces.session.total_tokens_source)
pre-fork prefix and compacted records: no spans	user.prompt spans and one session.compacted span per record, each traces.session.inherited = true with traces.session.inherited_source (pre-task-prefix | compacted); root carries traces.session.inherited_span_count and traces.session.inherited_spans_omitted

A miner that counted kind == TOOL and subtracted tool.Agent by name now gets the model's tool calls from the kind alone. A miner that wants the lifecycle selects traces.span.synthesized. A count of what this scope did must exclude traces.session.inherited; a reader that wants the human's context selects it. isSynthesizedSpan, isInheritedSpan and the attribute keys are exported from the SDK (src/adapters/provenance.ts).

Counts that changed in this repo

• src/evidence.ts — metrics.toolCallCount, erroredToolCallCount, signals.toolErrorRate and the metrics.tools histogram now count model-issued calls only (no Agent row). metrics.firstSpanAt / lastSpanAt exclude inherited spans, so a fork's session window does not stretch back over its parent's work (assembleSessionBundle joins external evidence by exactly that window).
• src/session-index.ts — reads record.metrics.toolCallCount, so the index and its toolCalls total follow the evidence record with no code change.
• src/live.ts — toolCallCount / erroredToolCallCount and the high-tool-error-rate analyst; inherited spans are not treated as assistant prose.
• src/pipelines.ts → src/report.ts — the agent-eval tool-use views key on kind == TOOL, so the report's tool-use line follows the kind change.
• src/run-span-tree.ts — a node's toolCalls.
• src/adoption.ts — subagent-spawn counts still recognize the Codex lifecycle span, now by its synthesized marker instead of tool.name == Agent, so adoption numbers are unchanged.
• src/report.ts / src/reactions.ts — the report subject and the human-reaction pairing use in-scope turns, never an inherited one.

Tests

tests/codex-token-and-provenance.test.ts (8 cases, synthetic inline rollouts; no recorded session content):

• Token counter fixture — a duplicate cumulative snapshot, and a final token_count that advances the total with no per-turn delta. Asserts the root total is the last reported value (4,100), that it differs from the delta sum (3,050), and that the breakdown comes from that same snapshot. A second case asserts a session with no reported total gets no attribute.
• Synthesized spans, before and after on one rollout — 3 real tool calls and 2 subagent threads. countedBefore (TOOL-kind or synthesized) minus countedAfter (TOOL-kind) equals the number of synthesized spans exactly, and the remaining count is 3. The same rollout is asserted through buildPolicyEvidenceRecord (toolCallCount === 3, no Agent row), analyzeLiveBatch and runPipelines.
• Fork fixture — a parent prefix with two typed turns, two compacted records repeating them, and a parent tool call, then the child's own turn. Asserts the inherited quotes appear once each with actor == human and a source citation, that the compaction summaries and window ids are kept, and that the child's own prompt count, tool count, evidence window and report subject are untouched.
• Compaction inside the parsed scope, and the inherited cap (200 per session, with the 60 dropped records counted on the root).

pnpm check:source, typecheck, test (831 passing), build and check:package are green locally.

What this does not establish

Every lift figure above is a re-score of existing artifacts under the battery's own scorers, not a new benchmark run. Change 3 (the inherited prefix) is unmeasured: it rests on one judged question with one judge. Neither transfer session was a Claude Code session, so nothing here is evidence about that adapter.

🤖 Generated with Claude Code

[drewstone]

Adversarial review — wave 2, adapter track 2

Verdict: the three intended changes are implemented correctly and the measured facts hold, but two consumers that read the new inherited spans were not updated, and one new count is wrong. All three are small fixes in files this PR already touches.

Checked out 94c257d in a fresh worktree. Local check set is green (check:source, typecheck, test 831 passing, build, check:package), CI on both Node 22.13.0 and 24.18.0 is green, and the branch merges cleanly with both #110's current head and main. I also merged #110's head (bbf1ba5, which this branch does not yet contain) locally: typecheck and 832 tests pass, so the memory-bound work in that commit and the inherited-span walk do not conflict.

What I verified rather than took on faith:

• 6 of the 8 new tests fail on the base commit (7bfc37d) with provenance.ts copied in; the 2 that pass are the negative case (no total reported) and the "this scope is untouched" regression guard, which is the right shape for both.
• Token attribute carries the harness value unchanged: 4100 verbatim, distinct from the delta sum (3050); a token_count with no delta still advances it; absent stays absent; 0 is carried as 0; a string or a negative total is refused (attribute absent, not zero).
• Adoption is genuinely unchanged: on a rollout with one tool.spawn_agent call and one lifecycle span, base and branch both report {parser_audit: 1}, total 1. The canonical-vs-fallback branch keeps working because parseInput reads input.value, not the span kind.
• Privacy: no path, session id, repo name, or recorded content from any real session appears anywhere in the diff. Fixtures are inline synthetic rollouts.
• Empty session (session_meta only): parses to one span, no token attribute, no inherited count. No crash.

Blocking

1. src/run-span-tree.ts:299 — a fork's run elapsed now stretches back over its parent. You fixed exactly this in src/evidence.ts:151 for firstSpanAt, but the node roll-up in the same PR still takes Math.min over every span including inherited ones. Measured on a synthetic fork whose prefix starts at t=2 and whose own scope starts at t=10:

BASE   nodes 1  start 00:00:10  end 00:00:15
BRANCH nodes 2  start 00:00:02  end 00:00:15

snapshot.durationMs and the elapsed … line at run-span-tree.ts:424 are computed from that start, so traces run reports a session that ran for 13 s as one that ran for 8 s longer than it existed. On a real fork the prefix can be hours old. Fix: skip inherited spans in the host.startMs/host.endMs update the same way timeBounds does.

The second half of that output is a separate, unstated consequence: kind: AGENT makes the lifecycle span a node boundary (isNodeBoundary, run-span-tree.ts:169), so the run tree now gains one node per child thread, labeled with the enclosing agent name (codex) and zero tool calls. That is arguably the right tree, but it is a visible change the PR body's "counts that changed" table does not mention, and the node would read better with the subagent path as its agent.name.

2. src/hodoscope.ts:305 — exported trajectories now replay user turns that already happened. writeHodoscopeInput turns every user.prompt span into a role: user message, ordered by step, with no inherited filter. Any Codex session that compacted (not only a fork) now exports a conversation that repeats earlier turns in the middle. Measured on a synthetic session with three asks and one compaction after the second:

[["user","FIRST ASK"],["assistant","working on it"],["user","SECOND ASK"],
 ["user","FIRST ASK"],["user","SECOND ASK"],["user","THIRD ASK"]]

That is training/eval input describing a conversation that did not happen. session.compacted is correctly dropped there; the inherited prompts need the same treatment (skip them, or emit them once as a prefix block before the first in-scope turn).

3. src/adapters/codex.ts:958 — traces.session.inherited_spans_omitted counts occurrences, not records. A turn refused by the cap is never added to inheritedTurnKeys (line 994 runs only after claimInheritedSpan() succeeds), so every later compacted record that repeats it increments the counter again. Measured with 210 distinct prefix turns and 3 later compactions repeating all of them: the root reports inherited_spans_omitted = 43 where 13 distinct records (10 turns + 3 summaries) were actually dropped. The analysis corpus has sessions with 37 compacted records, where the same shape would report roughly ten times what was lost. inherited_span_count is a count of distinct spans, so the two numbers are in different units and cannot be read together. Fix: inheritedTurnKeys.add(dedupKey) before the cap check (or count refused keys in a set). The test at "counts the inherited records its per-session cap dropped" passes today only because its 260 turns each appear once.

Non-blocking

• codex.ts:991 — the bounded dedup key drops distinct turns. ${key.length}:${key.slice(0, 256)} collides for two turns of equal length sharing a 256-character prefix. Probed with two 304-character turns differing only in their last four characters: one is kept, the other is silently gone — and it is gone from the one place the human's words were supposed to be recoverable. createHash('sha256').update(key).digest('hex') is bounded too and cannot collide.
• Inherited turns in a child session skip the child-actor rule. recordSubmittedTurn forces actor = 'agent' when sessionRole === 'child'; recordInheritedTurn calls codexActor unconditionally. A compacted record inside a child session therefore can label an agent-issued brief tangle.actor = 'human'. Counts are safe (they filter inherited), but a text answer to "what did the human ask?" is not.
• src/live.ts:531 — sessionEvent.startedAt is ordered[0].start_time, which now includes inherited spans. Same concept as firstSpanAt, two answers.
• src/adoption.ts:123 — hasMaterializedSkillEvidence matches on tangle.actor === 'injected' plus content, and an inherited prefix turn carrying the parent's <codex_internal_context> block can now flip a session into sessionsWithMaterializedSkills.
• The token attribute is scoped to the parsed task, and its name does not say so. With taskScope: 'turn' the walk breaks at the next boundary, so a two-task session reports traces.session.total_tokens = 1000 for turn 1 and 5000 for the whole file. That is the harness counter as of the end of the scope, which is defensible, but a miner reading a turn-scoped export will read it as the session total. One sentence in the attribute comment, or an …_as_of timestamp, closes it.
• reportedCount accepts a non-integer. A total_tokens of 4100.7 is carried through. Harmless today; Number.isInteger is the same guard for free.
• The provenance keys and predicates are exported from src/index.ts; the six traces.session.total_* keys the PR body advertises to downstream miners are not. Export them from the same place.
• The branch is one commit behind its base (#110 gained bbf1ba5 after this branched). Rebase before merge — I ran the merged tree and it is green, so this is bookkeeping, not a risk.

Not a defect, recorded

The last-snapshot-wins rule means a counter that ever decreased would be reported at its lower final value. That is exactly the rule the analysis measured against ground truth (13 of 13), so carrying it is right; a max would be a derivation, which the change deliberately refuses.

An in-scope turn that a compaction repeats is emitted twice — once in scope, once inherited (the "compaction inside the parsed scope" test asserts this). Counts are unaffected because they filter inherited, and the prefix claims cap slots before in-scope compactions do, so parent context is not crowded out. Worth a line in the schema table so a miner unions the two sets instead of concatenating them.

[drewstone]

Landed on main in d96f933a — the merge of #115, which brought the whole audit-facts stack across on top of main's reviewed #110. This PR was auto-closed when its base branch was deleted, so it never showed as merged; its commit 94c257d (token total and synthesized-span markers) is an ancestor of origin/main.

Both of its properties survive the merge alongside main's read-time span construction: the session token totals are written next to placeInnerSpans, and isSynthesizedSpan/isInheritedSpan still gate the tool and text classifiers in src/live.ts.