feat(facts): compute a deterministic session-facts sheet from the spans

[drewstone]

Stacked on #108 (feat/ask-questions), because this wires the sheet into the ask path that PR introduces. Base it on main once #108 merges.

What it implements

Ranked change 1 of analysis.md, the largest measured item: a deterministic session-facts sheet computed from the spans, with no model call.

Measured on twelve private audit sessions, over the OTLP files those runs already wrote:

Question	traces analyst arm	deterministic extraction from the same spans
tool calls	0.167	1.000
subagents	0.357	1.000
files edited	0.571	0.910
user messages	0.222	0.972
final message	0.417	1.000
log start/end	0.833	1.000
deterministic mean	0.389	—

Expected lift for this change alone: +0.414 deterministic mean (0.389 → 0.803), against an Opus subagent fleet at 0.906 dev / 0.917 holdout.

Nothing was missing from the spans. No tool returned a whole-session fact — viewTrace degrades to a 20-entry name histogram above its 150,000-byte ceiling, countTraces counts traces rather than spans, viewSpans needs span ids the reader does not have, and searchTrace stops at 500 hits — so a model asked for a tool-call total added up a capped histogram and decided by eye which names counted.

The benchmark harness that produced those numbers is private and lives outside this repo. No session content from it appears here: every fixture is synthetic inline JSONL.

What is in the change

• src/session-facts.ts. computeSessionFacts(spans) returns, per trace: tool calls excluding synthesized spans (with the excluded count kept separately, and a by-name histogram), every spawn_agent with its task name, human user.prompt turns in order with actor and timestamp (plus every turn by actor), the last message of the session's own agent and of each subagent task kept apart, changed paths from patch headers and file-editing tool arguments, first and last record times, unread records from the session integrity receipt, and the harness token total when a span carries traces.session.total_tokens.
• traces facts. JSON by default, --format text for the short readable form, --out to write it, for one session or many. No model call, no engine, no budget: $0.
• Prepared context. The same sheet reaches the built-in analyst kinds and every ask question through TraceAnalystDefinition.prepareContext, bounded at 30,000 bytes — a fifth of the documented perCallByteCeiling of 150,000 — with the shed reported in omitted_fields rather than applied silently.

Two rules hold for every field: it names the span ids it came from, so a reader can open them and check the number; and a fact the spans cannot support is null with a stated reason, never guessed and never a silent zero. The sheet is not a span and cannot be cited — trace:// citations still resolve against the raw spans, which is why each fact carries its span ids.

facts exits non-zero when a selected session produced no record spans, rather than printing a sheet of zeros for a session it could not read.

Schema and behavior changes

• New exported constant SESSION_TOKEN_TOTAL_ATTR = 'traces.session.total_tokens', the span attribute the sheet reads for the harness token total. No adapter emits it yet, so tokenTotal is null with its reason; ranked change 2 emits it, and that is worth a further +0.125 on its own.
• New exported constant SPAN_SYNTHESIZED_ATTR = 'traces.codex.span_synthesized', honored when present. Until ranked change 5 sets it, a synthesized subagent span is identified by the traces.codex.subagent_thread_id + traces.codex.subagent_path pair only that span carries.
• New report kinds traces.session_facts and traces.session_facts_report, both schemaVersion: 1.
• An analyst definition that receives the sheet has +session-facts.1 appended to its version. createTraceAnalyst records prepare_context in the exact-run identity, so a changed prepared context must not hide behind an unchanged version. analyzeSpans({ sessionFactsContext: false }) runs an analyst without it.
• ask's rules text now names the sheet and stays inside the 500-character DSPy preview head; --otlp is now a reading flag on facts, and --current / --workflow are accepted.
• No change to agent-eval.

Tests

17 new tests in tests/session-facts.test.ts over a synthetic Codex rollout (tests/session-facts-fixture.ts) whose gold is written by hand:

• every field exact against the hand-written gold, and every span id the sheet names exists in the trace;
• a synthesized tool.Agent span does not inflate the tool count — 5 TOOL spans, 4 tool calls, 1 excluded;
• a session whose serialized spans exceed the 150,000-byte trace-tool ceiling still yields exact facts (404 tool calls, both human turns, the right final message, the right changed paths);
• tokenTotal is null with its reason, and reports the value once a span carries the attribute;
• prepared context stays inside its bound at 30,000, 8,000, 2,000, 900, 400 and 100 bytes, never grows as the ceiling tightens, reports every shed, and keeps the tool-call count until last; a definition's own prepared context is preserved;
• analyzeSpans hands the sheet to the built-in kinds and omits it under sessionFactsContext: false;
• traces facts prints both forms, keeps the facts and names the unread records for a partly corrupt session, and exits non-zero for a session that cannot be read and for an unknown --format.

pnpm check:source, typecheck, test (845 passed), build and check:package are green locally.

Documented in README.md (Session facts) and docs/trace-analysts.md (Session facts, including that the sheet is deterministic and free).

🤖 Generated with Claude Code