Skip to content

Hold modifier keys on agent tap and double tap - #430

Merged
kblok merged 2 commits into
mainfrom
tap-modifiers
Oct 17, 2026
Merged

kblok merged 2 commits into
mainfrom
tap-modifiers

Conversation

@kblok

@kblok kblok commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

The agent could not hold a key while it clicked, so it could not Shift-click to extend a selection or Control-click to toggle one item. Now tap and double_tap, in act and over MCP, take an optional modifiers list (Shift, Control, Alt, Meta, ControlOrMeta) on an engine that declares the new TapModifiers capability, which the web engine does. The held keys are recorded in the trace and replayed from the cache. Custom executors see Target.TapModifiers and pass ClickOptions to TapAsync and DoubleTapAsync.

.NET-only: KeyModifier, KeyModifiers (upstream's KEY_MODIFIERS and isKeyModifier), ClickOptions, EngineCapabilities.TapModifiers, RecordedAction.HasValidModifiers, and the test Chromium_holds_the_modifiers_of_a_tap_the_agent_sends. Not ported: right_click and secondaryTap (no engine has them), so the four upstream its that name it are partial: "offers modifiers...", "keeps modifiers out of the vocabulary...", "threads modifiers through..." (no right_click, schemas not closed), and "rejects a trace whose modifiers are malformed..." (a stored null reads as none). "Says what keys a tap held" runs through DefaultAgent.Describe.

Closes #399

🤖 Generated with Claude Code

https://claude.ai/code/session_01KWUdrjfarTEM4dmBbK6GaV

kblok and others added 2 commits October 17, 2026 15:34
The agent's tap and double_tap tools take a modifiers list on an engine that declares TapModifiers, as upstream's tools do on an engine that declares tapModifiers. The web engine holds the keys through Playwright. The held keys are recorded in the trace and replayed from the cache.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KWUdrjfarTEM4dmBbK6GaV
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KWUdrjfarTEM4dmBbK6GaV
@kblok
kblok merged commit e50f9d0 into main Oct 17, 2026
1 check passed
@kblok
kblok deleted the tap-modifiers branch October 17, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hold modifier keys on tap, double tap, and right click

1 participant