Skip to content

Gate browser tool requests on server support#64

Draft
neubig wants to merge 4 commits intomainfrom
fix/browser-tool-availability-gating
Draft

Gate browser tool requests on server support#64
neubig wants to merge 4 commits intomainfrom
fix/browser-tool-availability-gating

Conversation

@neubig
Copy link
Copy Markdown
Collaborator

@neubig neubig commented May 1, 2026

Summary

  • cache /server_info tool metadata during the compatibility probe and gate browser-tool requests on usable_tools
  • only request browser_tool_set when the env flag allows it and the connected server advertises browser support
  • update mocks and tests for the new browser gating behavior, then merge the latest main

Verification

  • npm test
  • npm run typecheck
  • npm run build

Depends on OpenHands/software-agent-sdk#3028 for the /server_info.usable_tools metadata. When the server does not advertise tool metadata, the frontend keeps the previous allow-all behavior.

This PR was created by an AI agent (OpenHands) on behalf of the user.

Co-authored-by: openhands <openhands@all-hands.dev>
@vercel
Copy link
Copy Markdown

vercel Bot commented May 1, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agent-server-gui Ready Ready Preview, Comment May 1, 2026 7:30pm

Request Review

Copy link
Copy Markdown

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Good taste - Clean feature detection pattern with proper backward compatibility.

[IMPROVEMENT OPPORTUNITIES] (Optional)

Consider adding a test case for backward compatibility when available_tools is missing:

it("allows all tools when server does not provide available_tools metadata", async () => {
  server.use(
    http.get("/server_info", () =>
      HttpResponse.json({
        uptime: 0,
        idle_time: 0,
        version: "1.17.0",
        // No available_tools field
      }),
    ),
  );

  await OptionService.getConfig();

  expect(isAgentServerToolAvailable("browser_tool_set")).toBe(true);
  expect(isAgentServerToolAvailable("terminal")).toBe(true);
});

This would explicitly verify the fallback behavior for older servers that don't advertise capabilities.

[RISK ASSESSMENT]

⚠️ Risk Assessment: 🟢 LOW

This is a clean infrastructure improvement that adds proper server capability detection. The change:

  • Only gates tools based on actual server support
  • Maintains full backward compatibility with older servers via sensible fallback (allow all tools when metadata is missing)
  • Has good test coverage for the new gating behavior
  • Makes no changes to agent behavior, prompts, or evaluation-sensitive code

The module-level cache pattern is appropriate for a frontend that connects to a single backend per session. Cache invalidation happens correctly on errors.

VERDICT:

Worth merging: Solid implementation with proper fallback behavior. The optional test suggestion above would strengthen the backward compatibility coverage, but existing tests are sufficient.

KEY INSIGHT:

Elegant additive feature detection that degrades gracefully for older servers - new servers get fine-grained tool gating, old servers preserve existing env-flag behavior.

@neubig
Copy link
Copy Markdown
Collaborator Author

neubig commented May 1, 2026

@OpenHands QA to confirm that this PR works with OpenHands/software-agent-sdk#3028 in the case that the browser is available or unavailable.

@openhands-ai
Copy link
Copy Markdown

openhands-ai Bot commented May 1, 2026

I'm on it! neubig can track my progress at all-hands.dev

Co-authored-by: openhands <openhands@all-hands.dev>
Copy link
Copy Markdown
Collaborator Author

neubig commented May 1, 2026

QA complete.

I found one integration mismatch while checking against OpenHands/software-agent-sdk#3028: that PR currently exposes /server_info.usable_tools, while this frontend branch was only reading available_tools. I fixed that in b548f3b by preferring usable_tools, falling back to available_tools, and preserving the old "allow all tools" fallback when neither field is present.

What I verified:

  • Browser advertised: against a live checkout of software-agent-sdk PR #3028 (273e911), /server_info.usable_tools included browser_tool_set, isAgentServerToolAvailable("browser_tool_set") returned true, and the frontend kept browser_tool_set in the conversation start payload.
  • Browser withheld: I ran the same SDK checkout with the /server_info.usable_tools response patched to omit browser_tool_set (this sandbox has /usr/bin/chromium, so I had to emulate the no-browser environment). In that case, isAgentServerToolAvailable("browser_tool_set") returned false, and the frontend omitted browser_tool_set from the conversation start payload.
  • Regression coverage: added tests for usable_tools, legacy available_tools, and the no-metadata fallback.
  • Repo verification: npm test, npm run typecheck, and npm run build all pass after the fix.

AI-generated PR update/comment by OpenHands on behalf of the user.

@openhands-ai
Copy link
Copy Markdown

openhands-ai Bot commented May 1, 2026

Final summary:

Completion status

  • The PR comment request was fully addressed.
  • QA was completed for both cases requested:
    • browser support advertised by the SDK server
    • browser support not advertised by the SDK server
  • The integration mismatch discovered during QA was fixed on this branch.
  • The fix was committed and pushed (b548f3b).
  • The PR description was updated to match the actual SDK contract.
  • A QA results comment was posted on PR Gate browser tool requests on server support #64.

New changes since my last summary

  • No additional code changes were made after the previous update.
  • I only performed final cleanup/confirmation:
    • stopped the temporary QA servers used for live verification
    • confirmed the branch is clean and synced with origin

Conciseness check

  • The final code changes remain focused and minimal:
    • compatibility handling for usable_tools with available_tools fallback
    • targeted tests for the metadata cases
    • mock metadata alignment
    • one AGENTS.md memory note
  • No extraneous working-tree changes remain.

Copy link
Copy Markdown
Collaborator Author

neubig commented May 1, 2026

Follow-up update: per review, I removed the frontend fallback to available_tools and merged the latest main before pushing.

Current behavior on this branch:

  • use /server_info.usable_tools when present
  • omit browser_tool_set if it is not advertised there
  • keep the existing allow-all behavior only when the server does not advertise any tool metadata

I re-ran:

  • npm test
  • npm run typecheck
  • npm run build

AI-generated PR update/comment by OpenHands on behalf of the user.

@neubig
Copy link
Copy Markdown
Collaborator Author

neubig commented May 2, 2026

This still needs live testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants