PR5: agent run IDs and trace correlation - #61
Conversation
…steps/usage/history)
🤖 CodeAnt AI — Review Status
|
Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
|
ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing |
codeant-ai
Bot
added
the
size:L
There was a problem hiding this comment.
Pull request overview
This PR implements PR5 from the master plan by introducing an agent run ID concept and threading it through agent-mode output, step recording, usage logging, execution history, and execution backend recording to enable end-to-end trace correlation for a single logical agent run.
Changes:
- Added
--run-id(agent mode) pluscreateRunId()auto-generation, and propagated the run ID through agent results and step records. - Recorded
run_idin usage log entries and execution history records when calls are agent-triggered. - Added test coverage for run-id propagation in agent mode and CLI integration.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/cli-integration.test.mjs | Adds CLI-level integration tests for --run-id, generated IDs, usage/history correlation, and invalid inputs. |
| test/agent-mode.test.mjs | Adds unit tests validating generated/explicit run IDs and per-step run ID consistency. |
| src/types.ts | Extends CLI options with runId. |
| src/logging/usage.ts | Adds run_id field to usage log entry schema. |
| src/history/types.ts | Extends execution history record schema with optional run_id. |
| src/execution/backend.ts | Persists run_id on execution records when options.runId is present. |
| src/engine/fanout.ts | Threads runId into provider calls and stamps usage log entries with run_id. |
| src/cli.ts | Adds --run-id parsing and includes run_id in agent-mode CLI JSON output. |
| src/agent/context.ts | Stores run ID in research context and stamps steps with run_id. |
| src/agent/agent.ts | Generates/validates run IDs and passes them into backend searches for correlation. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| case "--run-id": | ||
| i++; | ||
| const runId = args[i]; | ||
| // Empty/whitespace-only explicit run IDs fail early: a generated ID | ||
| // is always non-empty, so a blank explicit value is a caller bug. A |
| case "--run-id": | ||
| i++; | ||
| const runId = args[i]; | ||
| // Empty/whitespace-only explicit run IDs fail early: a generated ID | ||
| // is always non-empty, so a blank explicit value is a caller bug. A | ||
| // flag-looking token means the value was omitted (like --config). | ||
| if (runId === undefined || runId.startsWith("--") || runId.trim() === "") { | ||
| throw new Error( | ||
| `Invalid --run-id: ${runId ?? "(none)"}. Use a non-empty run ID, e.g. --run-id run_20260622T173012Z_7f3a9c.` | ||
| ); | ||
| } | ||
| options.runId = runId; | ||
| break; |
| const explicitRunId = options.runId ?? this.runId; | ||
| if (explicitRunId !== undefined && explicitRunId.trim() === "") { | ||
| throw new Error("Explicit run ID must not be empty or whitespace-only"); | ||
| } | ||
| const runId = explicitRunId ?? createRunId(); |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d4293f0ae1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| * without it. `run_id` never replaces `id` — every execution keeps its own | ||
| * PR 2 execution ID. | ||
| */ | ||
| run_id?: string; |
There was a problem hiding this comment.
When an operator starts with the run_id returned by agent mode, history recent drops this field in summarizeExecution, while searchHistory never matches it, so coldsearch history search <run_id> returns no executions and there is no CLI-supported way to discover the correlated execution IDs. Include run_id in history summaries and index it as a history-search predicate so the newly persisted correlation can actually be followed.
Useful? React with 👍 / 👎.
| case "--run-id": | ||
| i++; | ||
| const runId = args[i]; | ||
| // Empty/whitespace-only explicit run IDs fail early: a generated ID | ||
| // is always non-empty, so a blank explicit value is a caller bug. A | ||
| // flag-looking token means the value was omitted (like --config). | ||
| if (runId === undefined || runId.startsWith("--") || runId.trim() === "") { | ||
| throw new Error( | ||
| `Invalid --run-id: ${runId ?? "(none)"}. Use a non-empty run ID, e.g. --run-id run_20260622T173012Z_7f3a9c.` | ||
| ); | ||
| } | ||
| options.runId = runId; |
There was a problem hiding this comment.
When --run-id is supplied to a non-agent command such as coldsearch search --run-id run_x query, parsing succeeds, but the normal search/extract/crawl paths never pass options.runId to the backend, so the command silently produces history and usage entries without the requested ID. Since this option is documented as agent-only, reject it unless --agent is active rather than accepting a correlation identifier that is then discarded.
Useful? React with 👍 / 👎.
| type, | ||
| description, | ||
| timestamp: new Date(), | ||
| ...(this.runId ? { run_id: this.runId } : {}), |
There was a problem hiding this comment.
When an agent uses fetch or refine, or returns a final answer without searching, this in-memory ResearchContext is the only place where the run's steps exist: runAgentMode emits only steps.length, and backend history is written only for provider searches. Once the process exits, those actions, their ordering, timing, and outcomes cannot be inspected by run_id, so the change still cannot reconstruct the durable agent/tool flow it is intended to correlate; persist structured step events keyed by the run ID.
AGENTS.md reference: AGENTS.md:L16-L16
Useful? React with 👍 / 👎.
Address PR #61 review findings: - Trim surrounding whitespace from explicit --run-id in the CLI parser and the agent boundary so the stored ID always matches output, steps, usage logs, and history records. - Reject --run-id unless --agent is active: non-agent commands never thread the ID into backend calls, so accepting it silently produced history and usage entries without the requested correlation ID.
Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
codeant-ai
Bot
added
size:L
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f793e3a850
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // Run IDs are an agent-mode feature: non-agent commands never pass the ID | ||
| // into backend calls, so accepting it there would silently produce history | ||
| // and usage entries without the requested correlation ID. | ||
| if (options.runId !== undefined && !options.agent) { |
There was a problem hiding this comment.
Fresh evidence in the current tree shows this check can still be bypassed: commands such as coldsearch history recent --agent --run-id run_x pass because options.agent is true, but main() dispatches history, tool, cache, config, batch, and status commands before reaching agent mode, so the run ID is silently ignored. Validate that no higher-priority command mode is selected, rather than checking the flag alone.
Useful? React with 👍 / 👎.
| } | ||
| // Trim surrounding whitespace so the canonical ID is stable for | ||
| // correlation; a padded value would mismatch across logs and output. | ||
| const runId = explicitRunId?.trim() ?? createRunId(); |
There was a problem hiding this comment.
When --run-id is omitted and a later uncaught LLM request fails—such as the second completion after a successful search—this ID already tags the provider history and usage records, but research() rejects before returning it and the CLI error path prints only the exception. The operator therefore receives no generated ID with which to correlate the surviving records from the failed run; generate the ID at a boundary that can include it in failure output or attach it to propagated errors.
Useful? React with 👍 / 👎.
Code Review SummaryStatus: 4 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (10 files)
Fix these issues in Kilo Cloud Reviewed by laguna-s-2.1:free · Input: 531.2K · Output: 68.3K · Cached: 2.1M |
User description
Summary
Implements PR5 from the remaining implementation master plan: agent run IDs and trace correlation.
Adds:
--run-idCLI option to stamp each invocation with an explicit run id (auto-generated when omitted).run_idthreaded through normalized search/extract/crawl output, agent steps, usage records, and history entries so a single logical run can be traced end-to-end.runId.run_idper usage event.Validation
npm run typecheck— passnpm run test:docs— passnpm test— 239/239 passCloses the PR5 item of the master plan. Reviewers: this PR is based on current main; PR4 config/status UX (#59) and its late-review follow-up (#60) are separate.
CodeAnt-AI Description
Correlate each agent research run across output, history, and usage records
What Changed
--run-id--run-idon non-agent commands now fail with clear errorsImpact
✅ Traceable agent research runs✅ Correlated usage and execution history✅ Clearer invalid run ID errors💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.