Skip to content

feat(gooddata-eval): capture conversation_id/response_id through the agentic-CLI path - #1713

Open
Tomkess wants to merge 1 commit into
feat/chat-client-reasoning-stepsfrom
feat/agentic-conversation-response-ids
Open

feat(gooddata-eval): capture conversation_id/response_id through the agentic-CLI path#1713
Tomkess wants to merge 1 commit into
feat/chat-client-reasoning-stepsfrom
feat/agentic-conversation-response-ids

Conversation

@Tomkess

@Tomkess Tomkess commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Stacked on feat(gooddata-eval): capture agent reasoning steps in ChatResult #1708 — targets feat/chat-client-reasoning-steps, not master.
  • Same gap feat(gooddata-eval): capture agent reasoning steps in ChatResult #1708 fixed for reasoning_steps, applied to conversation_id/response_id: cli/agentic_runner.py -> evaluate_agentic_* never surfaced them onto ItemReport for the three agentic kinds (agentic_alert_skill/agentic_metric_skill/agentic_conversation), even though each ChatResult already carries both.
  • Widens each evaluate_agentic_* return from list[str] to (reasoning_steps, conversation_id, response_id); attaches all three to the raised exception on failure (mirrors the existing conversation_id-on-exception idiom in ChatClient.ask()); run_agentic_items unpacks either form.
  • response_id is new at the RunResult layer for all three kinds — captured as the last non-null value across a run's turns, same accumulation pattern already used for reasoning_steps.
  • general_question/guardrail/search_tool/visualization untouched — already populated via the single-turn runner.py path, unaffected by this gap.

Test plan

  • uv run pytest tests/test_agentic_alert_skill.py tests/test_agentic_conversation.py tests/test_agentic_metric_skill.py tests/test_agentic_runner.py tests/test_alert_skill_evaluator.py tests/test_metric_skill_evaluator.py — 65 passed
  • Full package suite (uv run pytest) — same 9 pre-existing failures with or without this patch (missing openai in local env), zero new failures
  • ruff check clean on all touched files
  • ty check — same 4 pre-existing openai-unresolved-import diagnostics with or without this patch, zero new diagnostics

🤖 Generated with Claude Code

…agentic-CLI path

8010bd4 wired reasoning_steps through cli/agentic_runner.py -> evaluate_agentic_*,
but conversation_id/response_id stayed unset on ItemReport for every agentic kind
(agentic_alert_skill/agentic_metric_skill/agentic_conversation) -- each ChatResult
already carries both, and conversation_id was already threaded up to the
Alert/Metric/ConversationRunResult layer, but neither ever reached the top-level
evaluate_agentic_* return value or its failure exception, so run_agentic_items had
nothing to read.

Mirrors the reasoning_steps idiom exactly: widens each evaluate_agentic_*'s return
from list[str] to (reasoning_steps, conversation_id, response_id), attaches all
three to the raised exception on failure, and has run_agentic_items unpack either
form (tuple or the untouched kinds' bare list/None) onto ItemReport.conversation_id
/response_id. response_id is new at the RunResult layer for all three kinds --
captured as the last non-null value across a run's turns, same pattern already
used for reasoning_steps accumulation.

general_question/guardrail/search_tool/visualization untouched (already populated
via the single-turn runner.py path, not this one).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Tomkess
Tomkess requested review from hkad98, lupko and pcerny as code owners August 5, 2026 12:06
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fdbe6b55-616f-44a9-bac4-afc7d63a58cd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant