feat(ask): answer free-form questions concurrently under one budget

README.md

@@ -297,9 +297,10 @@ See [Replay verification](./docs/replay-verify.md) for setup, semantics, and hon
 | `--since <t>` | `upload`: window, `30m`/`2h`/`7d` or ISO (default 24h); `analyze`: ISO cutoff |
 | `--out <path>` | Write the report to a file |
 | `--dir <path>` | `improve`: write the full artifact pack to this directory; `ask`: write `answers.json` + `report.md` there |
-| `--otlp <file\|dir>` | **READ** OTLP-JSONL from any system, skipping the adapters; a directory reads the OTLP files under it (only `otlp/` when the producer made one) and names the JSONL that is not OTLP. `validate`, `analyze`, `investigate`, `improve`, `stream` |
+| `--otlp <file\|dir>` | **READ** OTLP-JSONL from any system, skipping the adapters; a directory reads the OTLP files under it (only `otlp/` when the producer made one) and names the JSONL that is not OTLP. `validate`, `analyze`, `investigate`, `improve`, `ask`, `stream` |
 | `--otlp-out <path>` | **WRITE** the OTLP artifact here (also evidence provenance / dry-run upload preview) |
 | `--format <kind>` | File `analyze`, `export`, or `stream`: `auto`, `policy-evidence`, `sandbox-events`, `openinference`, `intelligence-spans`, or `chat-trajectory` |
+| `--source-bundle <dir>` | `analyze` / `investigate` / `improve` / `ask`: read a retained full bundle and explicitly grant source-field reads |
 | `--llm` / `--budget <usd>` | Enable agentic analysts (needs `TANGLE_API_KEY` + Python with `agent-eval-rpc[dspy]`) / cap their spend |
 | `--question <text>` | `ask`: one question, repeatable. Kept short so the engine sees it whole |
 | `--questions <file>` | `ask`: JSON array of questions — strings, or `{ id?, question, instructions?, answerSchema? }` |
@@ -499,7 +500,8 @@ What it checks, and what it costs:
 - **Citations.** Every `trace://<trace_id>/span/<span_id>` URI in an answer is looked up in the trace. An answer that cites a span the trace does not hold fails.
 - **Budget.** `--budget` is one ceiling shared by every question. `--question-budget` bounds one question. The run refuses to start when the budget cannot cover a single model call, and warns when the budget admits fewer concurrent calls than `--concurrency`.
 - **Cost.** Each cost carries its provenance: `observed` from a provider receipt, `estimated` from token counts, or `uncaptured`. An uncaptured cost stays null; it never becomes zero.
-- **Exit code.** `ask` writes both artifacts first, then exits 1 when any question failed, returned no answer, broke its schema, or cited a span that does not exist. A failed question never costs the other answers.
+- **Exit code.** `ask` writes both artifacts first, then exits 1 when any question failed, returned no answer, broke its schema, or cited a span that does not exist. A failed question never costs the other answers, and neither does Ctrl-C: the answers already bought are recorded, the questions the run never reached are recorded as `aborted`, and both artifacts are still written.
+- **Wall time.** `totals.wallTimeMs` covers the whole run, including writing and indexing the trace file; `totals.setupTimeMs` says how much of it that setup was. `totals.peakConcurrency` is how many questions actually overlapped, which is at most `min(--concurrency, questions)`.
 
 `ask` uses the same engine and credentials as `--llm`, so it needs `TANGLE_API_KEY` and a Python interpreter with `agent-eval-rpc[dspy]`.
 Do not pass `--llm`; the command is model-backed by definition.

docs/trace-analysts.md

@@ -110,16 +110,20 @@ Read many questions from a file:
 
 An `answerSchema` makes the answer one JSON value a scorer can compare field by field.
 The supported keywords are `type`, `properties`, `required`, `additionalProperties`, `items`, `enum`, `const`, `title`, and `description`.
+A keyword that constrains one JSON type must declare it: `required`, `properties`, and `additionalProperties` need `"type": "object"`, and `items` needs `"type": "array"`, or the constraint would be skipped for an answer of another shape.
 Any other keyword is rejected when the run starts.
 This package carries no JSON Schema library, and a constraint that is quietly ignored would let a wrong answer pass as checked.
 
 ### What the run guarantees
 
 - Every question receives the deterministic [session-facts sheet](#session-facts) as prepared context, before its first model call. It costs nothing and answers the counting questions the bounded trace tools cannot.
 - Every `trace://<trace_id>/span/<span_id>` URI in an answer is resolved against the store. An unresolvable citation fails that question.
+  A citation the model wrapped in Markdown emphasis (`**...**`, `_..._`, `~~...~~`) or ended a sentence with resolves like a bare one: the delimiters are prose, not part of the span ID.
 - Findings the answer submits still pass the same evidence gate as the built-in kinds. Refused findings are counted by reason in both artifacts.
 - A failed question never stops the others. Its failure is recorded on its own answer, and the remaining answers are written.
+- Ctrl-C keeps what the run already bought. The signal reaches the engine, not the checks that follow it: an answer that came back is kept with its citations resolved, and each question the run never reached is recorded as `aborted`.
 - The artifacts are written before the exit code is decided. `ask` exits 1 when any question failed, returned no answer, broke its schema, or cited a missing span.
+- `totals.wallTimeMs` covers the whole run, including writing and indexing the trace file; `totals.setupTimeMs` names that part. `result.effectiveConcurrency` is the number of workers the run created, `min(--concurrency, questions)`, and `totals.peakConcurrency` is how many actually overlapped.
 
 ### Budget under concurrency
 
@@ -131,6 +135,11 @@ Two consequences follow.
 - A budget that admits fewer concurrent reservations than `--concurrency` still runs, and the report carries a warning naming how many concurrent calls it covers.
 
 A question the ledger refuses is reported as `budget-refused`, and the answers already produced are kept.
+That kind is decided from the accounting, not only from the error text.
+The refusal happens inside the model proxy, behind the DSPy bridge, whose HTTP error handling can replace the Node error with its own message.
+So a failed question is reported as `budget-refused` whenever the shared ledger's settled spend left less than one model call's reservation at the time it failed and nothing else in the failure names its cause.
+A failure whose own text names its cause keeps that cause: a bridge version mismatch stays `error`, so the reinstall hint still prints, and an empty answer from the bridge stays `no-answer`.
+The message itself is kept verbatim on the answer.
 
 ### SDK
 
@@ -250,7 +259,7 @@ Without the reason, a report showing "0 findings" reads as "the model found noth
 
 `analyze`, `investigate`, `improve`, and `ask` now carry those refusals:
 
-- the CLI log prints the reason and the offending URI on each `finding rejected` line;
+- the CLI log adds whatever the event carries beyond its own text to each `finding rejected` line: the cause (`reason` for the gate's rejections and the bridge-row rejection, `issues` for a schema failure), the offending URI, the citation counts, and the subject;
 - the analyst table's Detail cell names the reasons and their counts;
 - `result.findingRejections` (investigation and improvement) and `answers.json` (`ask`) hold the counts per analyst and reason.
 
@@ -259,9 +268,13 @@ The common reasons are an excerpt the cited span does not contain, a span the tr
 ## External analyzer failures exit non-zero
 
 `--analyzer halo|hodoscope|prime|<command>` promises that engine's output.
-An analyzer that fails now writes its error into the report as before, and then `analyze` exits 1 naming every analyzer that failed.
+An analyzer that fails now writes its error into the report as before, and then the command exits 1 naming every analyzer that failed.
 Scripts that treated exit 0 as "the analyzer ran" were reading a report that said otherwise.
 
+The check runs on `analyze`, `investigate`, and `improve`, and it covers every analyzer the run requested, not only the ones named on the command line.
+`investigate` and `improve` load the default traces config file, so an analyzer declared in `externalAnalyzers` there is a requested analyzer too, and a flaky one now turns those two commands red.
+`analyze` does not load a default config, so only its own `--analyzer` flags reach the check.
+
 ## Codex tool outcomes
 
 Both function and custom tool outputs use the same status parser.

skills/build-trace-analyst/SKILL.md

@@ -17,6 +17,9 @@ Check live analysts separately only for `traces stream`.
 Extend an existing analyst when it already emits the target with usable evidence.
 Do not inspect bundled `dist` or `node_modules`.
 
+Answer a one-off question with `traces ask --question "<text>"` instead.
+Build an analyst when the question repeats and its answer must be scored, not read.
+
 ## Implement
 
 - Use `Analyst`, `AnalystRegistry`, `TraceAnalysisStore`, and `makeFinding` from `@tangle-network/traces`.

skills/inspect-agent-traces/SKILL.md

@@ -5,13 +5,12 @@ description: Inspect real agent workflows with the published Traces CLI and expo
 
 # Inspect agent traces
 
-Use the deterministic CLI first.
-Keep inspection local and read-only.
+Use the deterministic CLI first; keep inspection local and read-only.
 
 ## Choose the way in
 
-If the trace is already OTLP (a system that emits `@tangle-network/agent-trace-contract`
-spans, or any conforming exporter), read it directly. No adapter is involved.
+If the trace is already OTLP (any `@tangle-network/agent-trace-contract` emitter or other
+conforming exporter), read it directly. No adapter is involved.
 
 ```bash
 traces validate spans.otlp.jsonl          # what can this trace answer? exit 1 on error findings
@@ -22,14 +21,13 @@ traces analyze --otlp results/sessions --out .traces/all.md   # a directory of e
 Use `--harness` only for coding agents whose on-disk format we do not control.
 Those adapters are the legacy edge, not the way to integrate a system you own.
 
-A run directory holds the span export beside raw event, stream and SDK logs that are
-also `*.jsonl`. Only the OTLP files are read; the rest are listed with what they hold.
-An `otlp/` subdirectory, when present, is read on its own.
+In a run directory only the OTLP files are read; other `*.jsonl` logs are listed with
+what they hold. An `otlp/` subdirectory, when present, is read on its own.
 
-Any section headed `inputs incomplete`, or carrying an `Inputs incomplete` line above
-its table, is computed from a field the trace does not carry everywhere. Report those
-numbers as uncaptured, never as zero spend. The `trace conformance` section at the top
-names every such capability once; the markers repeat it where the number actually is.
+A section headed `inputs incomplete`, or carrying an `Inputs incomplete` line above its
+table, is computed from a field the trace does not carry everywhere. Report those numbers
+as uncaptured, never as zero spend. The `trace conformance` section names each such
+capability once; the markers repeat it at the number.
 
 For a loop trace, read `round-over-round convergence` (did round N+1 improve on N) and
 `steering chain` (which verdict caused which retry) before drawing any conclusion about
@@ -48,7 +46,7 @@ traces analyze --harness codex --current --latest-turn --workflow \
 - Use `--session <id-or-path>` to pin a listed session.
 - Use `--latest-turn` for the current task in a resumed Codex or Claude Code session.
 - Use `--workflow` to include workers linked by stable parent and child IDs.
-- Use `--max-workflow-sessions <n>` only when the default 100-file bound is insufficient.
+- Use `--max-workflow-sessions <n>` only when the default 100-file bound is too small.
 - For Claude Code, use `--harness claude-code --session <path> --latest-turn`; nested subagents are included.
 
 Never join agents by display name or timestamp when Traces reports missing or conflicting IDs.
@@ -70,7 +68,7 @@ traces improve --harness codex --current --latest-turn --workflow \
 ```
 
 `improve` writes findings, evidence, a report, and spans.
-It does not edit an agent, repository, memory store, or knowledge base.
+It edits no agent, repository, memory store, or knowledge base.
 
 Write one session's durable evidence directory for a later reader:
 
@@ -80,12 +78,11 @@ traces bundle --harness claude-code --session <id-or-path> --out .traces/bundle
 
 `bundle` copies the transcript, the derived report and spans, the `.evolve` ledger rows inside the session window, and a `manifest.json` with a SHA-256 per file.
 It spends no model call.
-A missing transcript stops the assembly.
-An absent optional input is recorded in `manifest.absent` with the probed path.
+A missing transcript stops the assembly; an absent optional input is recorded in `manifest.absent` with the probed path.
 
 ## Pick the view for the reader
 
-That bundle is the FULL view (`manifest.view: "full"`), and it holds the whole session transcript.
+That bundle is the FULL view (`manifest.view: "full"`) and holds the whole session transcript.
 Never give it to a writer that must not see an earlier conclusion: the transcript holds the text of every file the session wrote, so a check for the earlier report FILE passes while its CONTENT is still readable.
 
 Project the writer's copy instead:
@@ -94,17 +91,22 @@ Project the writer's copy instead:
 traces bundle-view .traces/bundle --view evidence-only --out .traces/writer
 ```
 
-That view carries `derived/session-index.json`, `derived/evidence.jsonl`, and the structured `ledger/` records.
-It excludes the transcripts, the report, the spans, and every prose ledger file by name, with rules and hashes in `manifest.excluded`.
-Before writing, it compares each carried file against each excluded file for shared 8-word runs of prose, and drops any file that repeats one.
+It carries `derived/session-index.json`, `derived/evidence.jsonl`, and the structured `ledger/` records.
+It excludes the transcripts, report, spans, and every prose ledger file by name, with the rules and hashes in `manifest.excluded`.
+It also drops any carried file that repeats an 8-word run of prose from an excluded one.
 `manifest.view` names which copy you hold.
 
+## Ask your own question
+
+`traces ask --last 1 --question "<text>" --budget 2 --dir .traces/ask` keeps the engine's
+prose answer, resolves every `trace://` citation in it, and spends model calls to do it.
+
 ## Report
 
 - State the source (`--otlp <path>` or the harness), selected task boundary, session and span counts, and integrity warnings.
 - State which capabilities the trace could not support, and name the analyses that reported nothing because of it.
 - Cite each finding with its exact `trace://` reference.
-- Mark missing outcome, cost, token, skill, or relationship data as unknown.
+- Mark missing outcome, cost, token, skill, or relationship data unknown.
 - Do not infer task success from a completion message.
 - Do not infer skill use from reading a `SKILL.md`.
 - Do not upload traces unless explicitly requested.

src/answer-schema.ts

@@ -9,7 +9,9 @@
  *
  * Supported keywords: `type`, `properties`, `required`, `additionalProperties`
  * (boolean), `items` (one schema), `enum`, `const`, and the annotations
- * `title` and `description`.
+ * `title` and `description`. The four keywords that constrain one JSON type
+ * must declare that type: `required` without `"type": "object"`, or `items`
+ * without `"type": "array"`, checks nothing against an answer of another shape.
  */
 
 export type AnswerSchema = Readonly<Record<string, unknown>>
@@ -51,6 +53,22 @@ export function assertAnswerSchema(schema: unknown, path = 'answerSchema', depth
       throw new TypeError(`${path}.type must be one of ${[...JSON_TYPES].join(', ')}, or an array of them`)
     }
   }
+  // `properties`, `required`, `additionalProperties` and `items` constrain one
+  // JSON type each and are skipped for every other type. A schema that carries
+  // one without declaring that type checks nothing at all against an answer of
+  // the wrong shape — `{ required: ['a'] }` would accept the answer `5` — so it
+  // is rejected here rather than passing a wrong answer as checked.
+  const declared = schema.type === undefined
+    ? undefined
+    : (Array.isArray(schema.type) ? schema.type : [schema.type]) as string[]
+  for (const keyword of ['properties', 'required', 'additionalProperties'] as const) {
+    if (schema[keyword] !== undefined && !declared?.includes('object')) {
+      throw new TypeError(`${path}: "${keyword}" is checked only for an object; declare "type": "object" alongside it`)
+    }
+  }
+  if (schema.items !== undefined && !declared?.includes('array')) {
+    throw new TypeError(`${path}: "items" is checked only for an array; declare "type": "array" alongside it`)
+  }
   if (schema.properties !== undefined) {
     if (!isRecord(schema.properties)) throw new TypeError(`${path}.properties must be an object`)
     for (const [name, child] of Object.entries(schema.properties)) {