Skip to content

feat(desktop): improve tool output rendering and interaction - #5030

Open
colaforniaw wants to merge 1 commit into
apache:mainfrom
colaforniaw:feat/tool-output-rendering
Open

feat(desktop): improve tool output rendering and interaction#5030
colaforniaw wants to merge 1 commit into
apache:mainfrom
colaforniaw:feat/tool-output-rendering

Conversation

@colaforniaw

@colaforniaw colaforniaw commented Sep 8, 2026

Copy link
Copy Markdown

Summary

A long turn renders as a wall of tool rows, and opening one rarely helps: every tool shares the same generic detail — raw invocation lines, unstructured dumps, and for results the runtime has archived, a bare [archived_tool_result] tag with nothing behind it.

This PR rebuilds the desktop tool transcript around reading: collapsed rows state the action, target, and result count; large results render as bounded, structured previews; and whatever the preview omits stays reachable through an explicit, sha256-verified "Open full output" in the Files pane.

Current state

  • Every tool renders through the same generic detail, so expanding a row yields invocation noise regardless of what the tool did.
  • Results too large to keep inline are archived by the runtime; the desktop renders the placeholder as a bare [archived_tool_result] tag, so a growing share of tool output has no reader-facing path at all.
  • Large text, JSON, and diff results dominate the conversation, and WebFetch lands as page-sized text instead of a source reference.
  • During streaming, terminal output follows the tail and drags the viewport back when the reader scrolls away.

Implementation approach

  • Add semantic tool summaries for file operations, search results, and success receipts so collapsed rows communicate the action and outcome directly.
  • Use bounded, redacted previews for text, JSON, diffs, and shell output. Preserve complete retained results behind an explicit “View saved output” action in the Files panel.
  • Render successful WebFetch results as compact citation cards with the title, source link, byte count, line count, and truncation state; keep failed fetches readable as text errors.
  • Introduce a reusable output scroller that pauses tail-following when the user scrolls away and provides a “Jump to bottom” action to resume it.
  • Transfer archived output through validated, chunked Host reads with bounded caching, while preserving existing storage and permission behavior.

Before / After

1. Collapsed tool rows: invocation lines to semantic targets

Before After
image image

Collapsed rows lead with the file, target, or result count instead of the raw invocation line, and surface result counts next to the row.

2. WebFetch: page-sized text to a citation card

Before After
image image

A compact citation replaces the page body. Users visit the source through its external link instead of expanding the body in the conversation.

3. Retained results: unreachable to inspectable

Before After
image image

Results omitted from the preview — including archived ones, which previously rendered as an invocation echo with no path to the body — can now be inspected and copied from the Files panel.

4. A full tool turn, expanded: dumps to bounded previews

Before After
image image

Expanding several completed tools stacks page-sized text walls and raw dumps. Each detail becomes a bounded, structured preview, with retained output reachable on demand.

Close #4863

Verification

  • Targeted Core, UI interaction, Desktop IPC/client, and Runtime Host checks passed.
  • Desktop builds and workspace typechecking passed.
  • Staged Biome, ASF header, protocol epoch, and whitespace checks passed.

Full repository lint, format, build, test, and Desktop/UI Knip checks are not confirmed by the recorded validation. Real-window visual verification remains pending.

AI use

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: OpenAI Codex contributed implementation, regression tests, review-feedback analysis, local verification, and this PR description. The commit includes Generated-by: OpenAI Codex; retain it in the squash commit.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@github-actions github-actions Bot added the effort/XL Under 2500 readable lines label Sep 8, 2026
@colaforniaw
colaforniaw force-pushed the feat/tool-output-rendering branch from f8d4cca to 59b1c16 Compare September 8, 2026 14:43
@colaforniaw colaforniaw changed the title feat(desktop): readable tool rows, WebFetch citations, and on-demand full retained output feat(desktop): improve tool output rendering and interaction Sep 8, 2026
@colaforniaw
colaforniaw marked this pull request as ready for review September 8, 2026 16:48
@colaforniaw
colaforniaw force-pushed the feat/tool-output-rendering branch 2 times, most recently from 86d29d8 to 4338172 Compare September 8, 2026 17:09

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for improving the tool-output reading experience! We reviewed 4338172 and left one small functional issue inline, plus a non-blocking request to measure the long-text rendering path. The Write issue was checked against the base version; the text-rendering note is a measurement suggestion, not a confirmed performance regression.

Separately, the current CI test job reports three unused exports in the Desktop story barrel; the Windows packaging check has passed.

This review was assisted by OpenAI Codex, including independent reviewer passes. The maintainer has confirmed the visual direction; the code findings and checks above are AI-assisted verification.

import { formatBytes, readResultText, webFetchReference } from './preview-utils.js';

function isSuccessReceipt(result: ToolActivityItem['result']): boolean {
if (result?.kind === 'file_write') return true;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Keep the full destination path inspectable for Write results

Could we retain a detail view for file_write results? This is a normal Write outcome when the runtime does not return a diff (for example, a 70,000-byte write). The collapsed row now shows only the basename, and treating this result as a status-only receipt disables its disclosure, so a write to /repo/packages/backend/config.json leaves no way to inspect the full destination from the tool row. This makes same-named files difficult to distinguish. I compared the same result against the base: the full path and disclosure were present there, and both are absent here. Keeping the compact basename while preserving the full path in the existing detail view would address it.

// No heading: the collapsed row above already carries the invocation,
// and it can be an archive ref — repeating it here wraps a raw URI
// across the block's title slot.
text={formatUserVisibleToolText(redactSecrets(content.text), locale)}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: could we measure the long-text path?

This now passes the complete text through ToolTextPreview to CodeBlock, whereas the previous path capped it at 500 lines. The 208px height limit bounds the visible area, but the complete text still reaches the renderer. I have not reproduced a slowdown, so this is a request for validation rather than a confirmed regression. Could we compare base and head with a representative large retained text result (including many short lines), checking expansion time and responsiveness while scrolling? If the cost is material, we could keep a bounded inline preview and use the existing saved-output viewer for the full text; if the measurements are healthy, recording them would be sufficient.

@colaforniaw
colaforniaw force-pushed the feat/tool-output-rendering branch from 4338172 to 2723fed Compare September 9, 2026 04:56
Refine WebFetch citation cards, bounded text previews, shell tail-following, archived output access, and tool invocation labels across the Desktop transcript surface.

Generated-by: OpenAI Codex
@colaforniaw
colaforniaw force-pushed the feat/tool-output-rendering branch from 2723fed to 9cf26ab Compare September 9, 2026 06:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/XL Under 2500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(desktop): tailor tool result rendering to the content

2 participants