Keep the npm download out of the pyright tests' time budget

[christophwille]

Symptom

PythonInterpreterTests.AnImportResolvesIntoAnEnvironmentThatIsNotInTheWorktree failed on Windows in run 33840465175 and on Windows and Linux in run 33841144458, each time after exactly three minutes:

System.Threading.Tasks.TaskCanceledException : A task was canceled.
   at Stampeded.Core.Lsp.LspConnection.RequestAsync(...) LspConnection.cs:219
   at Stampeded.Core.Lsp.LspConnection.StartAsync(...) LspConnection.cs:115
   at PythonInterpreterTests.AnImportResolvesIntoAnEnvironmentThatIsNotInTheWorktree() PythonInterpreterTests.cs:87

The branch those runs built (#10) changes only build.yml and dotnet-tools.json. Its merge base 7b0e186 passed the same test on all three OSes in run 33785386015 the evening before. Nothing in the repository, the runner image (ubuntu-24.04 20260831.293.1 in both the passing and the failing Linux job) or the pyright version (1.1.413 since 2026-08-14) changed in between.

Cause

The TRX artifacts keep each test's own log, including for passed tests. On a runner with node and no pyright on PATH, LanguageServers.Python() resolves to npx --yes --package pyright -- pyright-langserver --stdio, and the first npx start downloads the package from the npm registry before the server says a word. The gap between -> started (pid ...) and Pyright language server 1.1.413 starting is that download, and that gap is what grew:

Run (UTC)	Linux	macOS	Windows
33785386015 (Sep 3, 17:35)	17 s	16 s	37 s
33840465175 (Sep 4, 05:27)	29 s	75 s	> 180 s, failed
33841144458 (Sep 4, 05:38)	> 180 s, failed	109 s	> 180 s, failed

The test creates one CancellationTokenSource(TimeSpan.FromMinutes(3)) and hands it to LspConnection.StartAsync, so the token meant to bound a language server's handshake also bounded a network download from a registry the test does not control. Every CI runner starts with an empty npm cache, so the download happens every run, and this fixture is the first pyright test alphabetically, so it is the one that paid for it. PythonLspTests and PythonProjectConfigTests share the same three-minute budget and the same exposure; they passed only because npx had the package (at least partially) cached by the time they ran, after the first attempt was killed.

Why now

npm's status page lists Intermittent Failures Impacting npm Publish from 2026-09-03 21:42 to 23:21 UTC with Package installation degraded. The passing run was before it; the failing runs were the next morning, and the download times above show installs were still slow then. The test was always one slow registry away from red.

The app is not affected: ReviewWorkspace.TryStartPythonAsync passes the session token, which has no timeout, so a slow download only makes "Starting pyright" take longer.

Fix

A new test helper, PythonServer.ResolveAsync(), replaces the direct LanguageServers.Python() call in the three pyright tests. When the spec is the npx form, it first awaits a shared Lazy<Task> that runs

npx --yes --package pyright -- pyright --version

once per test assembly through ExternalTool.RunAsync, with a ten-minute budget of its own. npx keys its cache on the package spec, so this puts the package in the same _npx/<hash> directory pyright-langserver is then started from. RunAsync already logs the command with its elapsed time, so the download shows up in the test output as a number:

07:54:51.789 [C:\Program Files\nodejs\npx.CMD] --yes --package pyright -- pyright --version -> exit 0 (4262 ms)

The three-minute tokens stay as they are; they now measure pyright and not the registry. A download that fails or runs out its budget fails every pyright test with that reason instead of an opaque cancellation inside initialize. It is deliberately not an Assert.Ignore: a registry that cannot be reached is a fact about the run, not about the machine.

Rejected alternatives

• Raising the three-minute budget. Hides the cause, and makes every real failure take longer to report.
• A warm-up step in build.yml only. Fixes CI but leaves the test measuring the wrong thing; a developer's first run on a fresh npm cache has the same exposure.
• Pinning --package pyright@<version>. Does nothing for download speed, and the unpinned spec is what the app itself uses.

Verification

• Full suite locally: 257 passed, the one failure is the known symlink-privilege test on a non-admin Windows account. The TRX shows the warm-up line once, in the first pyright test, and not in the later ones.
• CI on this branch: see the Build run on this PR.

🤖 Generated with Claude Code

[christophwille]

Build run 33842356786 on this branch is green on all three OSes. The warm-up line in each TRX gives the npm download time the tests used to absorb inside their three-minute budget:

OS	npx ... pyright --version
Linux	421 s
Windows	221 s
macOS	45 s

Two of the three would have failed the old test again this run. The registry is evidently still slow; 421 s is inside the ten-minute warm-up budget, but not by a wide margin.