Skip to content

feat(mcp): experimental --modal flag for browser_annotate (compare singleton vs per-request window)#40525

Draft
Skn0tt wants to merge 5 commits intomicrosoft:mainfrom
Skn0tt:feat-browser-annotate-modal
Draft

feat(mcp): experimental --modal flag for browser_annotate (compare singleton vs per-request window)#40525
Skn0tt wants to merge 5 commits intomicrosoft:mainfrom
Skn0tt:feat-browser-annotate-modal

Conversation

@Skn0tt
Copy link
Copy Markdown
Member

@Skn0tt Skn0tt commented Apr 30, 2026

Spike branch to A/B-compare two browser_annotate designs:

Merges pavel-dash (#40506) on top of feat-browser-annotate-cli (#40520) and adds the toggle without touching the default code path. Intentionally not for merge — the toggle exists so we can pick a winner and delete the other.

Findings from a manual same-page concurrent test:

  • Singleton: both parallel requests received the same annotation.
  • Modal: distinct windows / distinct answers, but second window hits Screencast is already started since both attach a screencast to the same page.

pavelfeldman and others added 5 commits April 29, 2026 20:17
Extract dashboard discovery behind a SessionProvider abstraction.
RegistrySessionProvider keeps the existing serverRegistry-driven
behavior; IdentitySessionProvider wraps a single user-provided
BrowserContext. Expose openDashboardForContext(context) so callers
can open the dashboard for a context they already have.
Extracts `show --annotate` into a dedicated `annotate` CLI command that forwards to the `browser_annotate` MCP tool. The daemon now propagates an AbortSignal per connection so that disconnecting an in-flight annotate cancels it on the daemon side as well.
# Conflicts:
#	packages/playwright-core/src/tools/dashboard/dashboardApp.ts
#	packages/playwright-core/src/tools/dashboard/dashboardController.ts
Adds a temporary modal: true input on browser_annotate (and matching
--modal CLI option) that opens a dedicated dashboard window per request
via Pavel's IdentitySessionProvider, instead of routing through the
singleton dashboard daemon. Used to A/B-compare the two designs.

The default (modal: false) path is unchanged.
@Skn0tt
Copy link
Copy Markdown
Member Author

Skn0tt commented Apr 30, 2026

Hands-on findings

Played with both modes by having two agents request annotations simultaneously.

Modal Dashboard

Liked

  • After submitting both annotations, my desktop is empty again.

Disliked

  • Both windows opened stacked on top of each other — I had no idea there were two. Hard to prioritise which to do first.
  • If I don't want to deal with them, I have to minimise multiple windows.

Singleton Dashboard

Liked

  • Easy to discover more from the dashboard (one place to look).

Disliked

  • We don't actually have inbox/queueing implemented — concurrent requests collapse onto each other.
  • Upon submitting feedback, the window does not close.

Preference

Singleton, with these changes:

  • Implement inbox / queueing for concurrent annotation requests.
  • Auto-minimise the window upon submission, if it was already minimised (preserve the user's window state).

@github-actions
Copy link
Copy Markdown
Contributor

Test results for "MCP"

9 failed
❌ [chrome] › mcp/autowait.spec.ts:19 › racy navigation destroys context @mcp-windows-latest-chrome
❌ [firefox] › mcp/cli-core.spec.ts:31 › close @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-core.spec.ts:37 › click button @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-core.spec.ts:51 › click link @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-core.spec.ts:64 › dblclick @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-core.spec.ts:71 › type @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-core.spec.ts:80 › fill @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-core.spec.ts:89 › fill numeric @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-core.spec.ts:98 › hover @mcp-windows-latest-firefox

6877 passed, 927 skipped


Merge workflow run.

@github-actions
Copy link
Copy Markdown
Contributor

Test results for "tests 1"

7 flaky ⚠️ [chromium-library] › library/video.spec.ts:719 › screencast › should work with video+trace `@ubuntu-22.04-chromium-tip-of-tree`
⚠️ [chromium-library] › library/video.spec.ts:719 › screencast › should work with video+trace `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/video.spec.ts:719 › screencast › should work with video+trace `@chromium-ubuntu-22.04-node22`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1080 › cli codegen › should not throw csp directive violation errors `@firefox-ubuntu-22.04-node20`
⚠️ [webkit-library] › library/video.spec.ts:275 › screencast › should capture navigation `@webkit-ubuntu-22.04-node20`
⚠️ [playwright-test] › ui-mode-test-output.spec.ts:118 › should collapse repeated console messages for test `@ubuntu-latest-node22`
⚠️ [playwright-test] › ui-mode-test-watch.spec.ts:185 › should watch new file `@windows-latest-node20`

41578 passed, 784 skipped


Merge workflow run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants