fix(control): hold the pointer, drop stale moves, settle before clicking - #62
Merged
Merged
Conversation
Three symptoms from a real Mac-to-KDE session — "the mouse leaves the window", "I can't click anything", "it's super slow" — with three separate causes. The host's debug log has all of them. **The pointer escapes the video.** Taking control did not capture the pointer, so the guest's real cursor walked out of the picture the moment they pushed toward an edge. Reaching a menu bar, a corner, or a window's close button — most of what taking control is for — was impossible. Pointer lock existed but only on the desktop viewer, only inside fullscreen, behind a hover button in a corner; the web viewer had none at all. Now clicking the picture takes the pointer and Escape gives it back, no fullscreen involved, with the two words of instruction on screen while control is granted but not yet taken. The click that acquires control is not forwarded — the guest is reaching for the window, not for whatever sits under the pointer. If the browser refuses the lock, it falls back to absolute coordinates rather than leaving the guest unable to act. **Clicks landing nowhere.** On Wayland, positioning the pointer and pressing the button are two separate ydotool invocations writing to uinput, and the compositor processes that stream asynchronously — so the press could be delivered while the pointer was still where the host left it. The guest's cursor sits over a button, they click, nothing happens. The nut.js backend has waited a frame here since the same bug was found on macOS; this path never got it. The host log shows the two commands issued back to back with nothing between them. **Everything lagging.** Every injection is serialized, and on Wayland each one costs a process spawn. A viewer streams pointer movement at their display's refresh rate, which arrives faster than that drains, so the queue grew without bound: the pointer trailed further and further behind and a click sat behind hundreds of stale positions before it was even attempted. Moves that a newer position has already replaced are now dropped before they can queue. Nothing is lost — a move says where the pointer *is*, not how it got there — and clicks, scrolls and keystrokes are never dropped, nor is any move during a drag, where the path is the whole point. The count is reported in diagnostics, so a host that feels slow with nothing coalesced points somewhere else. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| const backend = new WaylandYdotoolInputBackend( | ||
| run, | ||
| { hasBinary: true, hasSocket: true, socketPath: '/tmp/.ydotool_socket' }, |
ThreatCrush Security Scan170 finding(s) HIGH/CRITICAL: 16 | MEDIUM: 48 | LOW: 106
…and 120 more. Full results in the Security tab. Snippets are redacted; ThreatCrush never prints matched credential material. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Diagnosed from the Mac guest console and the KDE/Wayland host's
pairux-input.log. Three reported symptoms — "the mouse leaves the window", "I can't click anything", "it's super slow" — turned out to have three separate causes, all visible in the logs.1. The pointer escapes the video
Taking control never captured the pointer. The guest's real cursor walked out of the picture the moment they pushed toward an edge, so reaching a menu bar, a corner, or a window's close button — most of what taking control is for — was impossible.
Pointer lock did exist, but only on the desktop viewer, only inside fullscreen, behind a hover button in a corner. The web viewer had none at all.
Now: click the picture to take the pointer, Esc to give it back. No fullscreen involved, with
Click to control · Esc to releaseon screen while control is granted but not yet taken. Ported to the web viewer, which had nothing.Two details that matter:
2. Clicks landing nowhere
The host log shows the two commands issued back to back:
Those are two separate ydotool invocations writing to uinput, and the compositor processes that stream asynchronously — so the press can be delivered while the pointer is still where the host left it. The guest's cursor sits over a button, they click, nothing happens.
The nut.js backend has waited a frame between positioning and pressing since the same bug was found on macOS. This path never got it.
3. Everything lagging
Every injection is serialized through one promise chain, and on Wayland each one costs a process spawn (~10ms). A viewer streams pointer movement at their display's refresh rate — far faster than that drains — so the queue grew without bound. The pointer trailed further and further behind, and a click sat behind hundreds of stale positions before it was even attempted.
Moves that a newer position has already replaced are now dropped before they can queue. Nothing is lost: a move says where the pointer is, not how it got there. Never dropped: clicks, scrolls, keystrokes, and any move during a drag, where the path is the whole point.
stats.coalescedis in diagnostics, so a host that feels slow with nothing coalesced points somewhere else.Also in the log, not fixed here
The host's
[RemoteInput] Capture bounds primary displayline lands before[RemoteInput] Screen size: 1920x1080— the capture source is resolved before the injector has initialised, so the multi-monitor mapping from #59 silently falls back to the primary display. It does not affect this single-monitor host, but it will affect any host sharing a second screen. Worth its own fix.lint0 errors,typecheck9/9. 185 remote-input tests, 585 desktop, 648 web — +8 covering the coalescing rules, the settle ordering, and the new capture flow.Not run on real hardware; the three causes are each confirmed from the logs and covered by tests.
🤖 Generated with Claude Code