test: fix all 182 failures unmasked by #2172 + closeTab last-tab race#2230
Open
time-attack wants to merge 7 commits into
Open
test: fix all 182 failures unmasked by #2172 + closeTab last-tab race#2230time-attack wants to merge 7 commits into
time-attack wants to merge 7 commits into
Conversation
Several browse and design test files armed `setTimeout(() => process.exit(0), 500)` inside afterAll as a workaround for a browser close() that can hang. That assumes each test file runs in its own process. It doesn't: `bun test` runs EVERY file in one shared process. The 500ms timer therefore fired partway through a LATER test file and terminated the entire run with exit code 0 and no summary. Because the exit code was 0, the suite could not gate — only a fraction of the files ran, and every downstream failure was silently masked. The fix removes the delayed process.exit from teardown and instead time-boxes the resource cleanup: browser teardowns race `close()` against a short timeout and abandon it if it hangs (the child is reaped at real process exit), and the design daemon /shutdown test stubs `process.exit` around the shutdown timers so they fire harmlessly. A new static-guard test, test/no-suicide-exit.test.ts, scans every *.test.ts and fails if any file schedules a delayed process.exit, so the pattern cannot silently return. test/user-slug-fallback.test.ts is made deterministic via HOME isolation. Verified: test/no-suicide-exit.test.ts + test/user-slug-fallback.test.ts pass (15/15); browse/test/commands.test.ts passes standalone (243/243) and now runs to a full summary; design/test/daemon.test.ts passes (34/34) with no early exit. Honest note: with the suite now able to run to completion, it surfaces pre-existing failures that the early exit had been hiding — e.g. design/test/feedback-roundtrip.test.ts fails identically on unmodified origin/main (a `session.clearLoadedHtml` stub gap), independent of this change. These are not introduced here; this change only makes them visible so they can be gated on and fixed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ins the race The page 'close' event handler can run while page.close() is awaited and reassign activeTabId before closeTab's post-close active check, so the last-tab auto-create branch never ran and the manager was left with zero tabs. Capture wasActive before close(); only reassign activeTabId when it no longer points at a live tab. Event-dispatch order varies across Playwright/Chromium versions, which is why this raced on some machines only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The v1.14 sidebar rip (PR garrytan#1216) replaced the chat-queue path with the claude PTY terminal but missed these test files; the teardown process.exit bug then hid their failures until the suite could run to completion again. - sidebar-integration.test.ts, security-sidepanel-dom.test.ts: deleted — every test drives removed endpoints (/sidebar-command, /sidebar-chat, /sidebar-agent/*) or the ripped #security-banner chat UI. PTY-path coverage lives in terminal-agent-integration.test.ts; the rip itself is pinned as absence tests in sidebar-tabs.test.ts. - sidebar-ux.test.ts: removed stale blocks pinning the dead architecture; re-pinned invariants that survived in new form (identity-based killAgentByRecord, PTY cleanup-prompt injection, typed DOMException catch, lazy terminal spawn). - sidebar-tabs.test.ts: two assertions updated to the current source shape. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ate restore - commands.test.ts: describe-scoped beforeAll(goto) hooks become beforeEach — bun 1.2.x runs describe-scoped beforeAll eagerly at file start in file order, so the three navigations clobbered each other and cascaded into 55 wrong-page / dead-session failures; beforeEach is correctly scoped on all bun versions. - content-security.test.ts: restore the module-load urlBlocklistFilter after the content-filter describe — the module instance is shared across files, and leaving the registry empty broke security-integration.test.ts downstream. - watchdog.test.ts: hardcoded ports 34901-34903 → BROWSE_PORT=0 (random), ending EADDRINUSE collisions with sibling worktree runs. - gstack-update-check.test.ts: pin GSTACK_HOME/GSTACK_STATE_ROOT to the test's state dir so leaked env from other files can't redirect config. - terminal-agent(.integration).test.ts: re-pin static-grep anchors to the current maybeSpawnPty/canDispatchOverTunnel source; feature-detected skip for Bun.spawn PTY support (absent on bun 1.2.x); success-path WebSocket handshake via raw TCP (fetch() never resolves 101 upgrades on bun 1.2.x). - dual-listener.test.ts: update pinned literal to the args-aware canDispatchOverTunnel(body?.command, body?.args) form. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ent capabilities - gstack-memory-helpers.test.ts: env vars were saved in beforeEach but restored in afterAll, so the restore leaked the previous test's dead temp dir into GSTACK_HOME/GBRAIN_HOME/HOME/PATH for every file that ran later (broke gstack-config and make-pdf format-gate in full-suite runs). Capture once at collection time instead. - gstack-config.test.ts: scrub inherited GSTACK_STATE_ROOT/GSTACK_HOME in run() unless a test passes them explicitly (defense in depth). - gbrain-detect-install.test.ts: append a bun-only shim dir to SAFE_PATH — the script's '#!/usr/bin/env -S bun run' shebang needs bun on PATH, but ~/.bun/bin also carries a real gbrain on dev machines. - gen-skill-docs / openclaw-native-skills: skipIf(!Bun.YAML) — parser lands in bun 1.3; CI pins bun latest so strict-YAML coverage remains. - codex-resume-flag-semantics: availability probe upgraded from 'which codex' to 'codex --version' exit 0 (half-installed wrapper resolved but could not execute). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…runcation - feedback-roundtrip.test.ts: handleWrite/ReadCommand's third param is a TabSession, not the BrowserManager — the manager delegates enough of the session surface to limp through setup but lacks clearLoadedHtml, so goto threw on every test. Call via bm.getActiveSession() like commands.test.ts does. - variants-retry-after.test.ts: the Retry-After HTTP-date was built from Date.now()+3000, and toUTCString() truncates milliseconds — shaving up to 999ms off the encoded wait and failing whenever the clock was late in a second. Align the base to a whole second; assertions unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
This is the follow-up to #2172. That PR stopped test teardowns from silently killing the
bun testprocess partway through the run (exit 0, no summary, ~34 of 434 files executed). With the suite finally able to run to completion, it exposed 182 pre-existing test failures that the early exit had been masking — this PR fixes all of them, plus one real product bug the investigation uncovered.This branch contains #2172 (merged locally at its head,
e7c37d76) because the suite cannot run far enough to verify these fixes without it. Merge #2172 first; this PR's diff then reduces to its own five commits.Before / after (full free suite, identical command)
The two residual "fails" in the after-run summary are module-load artifacts (see "Not fixed here" below) that appear identically in the before-run — the before-run's summary said 184 while printing 182 real failure lines; same two phantoms.
The five commits
1.
fix(browse)— closeTab race loses last-tab auto-create (product code, the only source change)BrowserManager.closeTabcheckedtabId === this.activeTabIdafterawait page.close(). But the page'close'event handler (wired inwirePageEvents) can run during that await and reassignactiveTabIdfirst. When it won the race, the "closed the last tab → auto-create a blank one" branch never executed, leaving the manager with zero tabs and every subsequent command failing withNo active page. Event-dispatch order varies across Playwright/Chromium versions and machines, which is why this raced only on some setups. Fix: capturewasActivebefore closing, and only reassignactiveTabIdif it no longer points at a live tab (respecting a valid choice the event handler already made).2.
test(sidebar)— 92 failures: tests for an architecture that no longer existsThe v1.14 sidebar rip (#1216) replaced the chat-queue path (
sidebar-agent.ts,/sidebar-command,/sidebar-chat,/sidebar-agent/*) with the interactive claude PTY terminal, and deleted the main chat-queue test files — but missed these. They have been failing invisibly ever since.sidebar-integration.test.ts— every test drives removed endpoints; its two "passing" tests only asserted 401s on paths that no longer exist. PTY-path integration coverage lives interminal-agent-integration.test.ts.security-sidepanel-dom.test.ts— drives the#security-bannerchat-pane DOM that was ripped with the chat queue (sidepanel.jsdocuments the SEC shield as "not driven yet"). Server-side security-stack coverage (content-security, security, adversarial suites) is untouched.sidebar-ux.test.ts(1670 → ~960 lines) — removed blocks pinning the dead architecture (the rip itself is already pinned as absence tests insidebar-tabs.test.ts); re-pinned invariants that survived in new form: shutdown now asserts identity-basedkillAgentByRecord(notpkill), cleanup prompt asserts PTY injection viagstackInjectToTerminal, CSSOM guard asserts the typedDOMExceptioncatch.sidebar-tabs.test.ts— two static-grep anchors updated to current source.3.
test(browse)— 63 failures: bun-version-dependent hook semantics + hermeticityThe dominant root cause (55 failures in
commands.test.ts): bun 1.2.x runs describe-scopedbeforeAllhooks eagerly at file load, in file order; newer bun runs them lazily. ThreebeforeAll(goto …)navigation hooks clobbered each other on 1.2.x, tests ran against the wrong page, 5s bun timeouts left stale 30s Playwright waits whose delayed rejections got mis-attributed to later tests, and a mid-cascaderestoreStatefailure killed the shared session for the rest of the file. Converted the three hooks tobeforeEach(identical semantics on every bun version; the local fixture server makes it ~ms per test). This is why the file passed 243/243 on machines with newer bun but failed 55 deterministically on bun 1.2.19.Also in this commit:
content-security.test.ts: the content-filter describe calledclearContentFilters()and never restored the module-load-registeredurlBlocklistFilter. The module instance is shared across every test file in the process, sosecurity-integration.test.tsfailed downstream. Added anafterAllrestore; the downstream assertion (whose job is to catch exactly this unregistration) is untouched.watchdog.test.ts: hardcoded ports 34901–34903 →BROWSE_PORT=0. The fixed ports collided (EADDRINUSE) with sibling worktree runs of the same file; the tests never dial the port.gstack-update-check.test.ts: pinGSTACK_HOME/GSTACK_STATE_ROOTto the test's state dir so env leaked by other files can't redirect the script's config resolution.terminal-agent.test.ts/terminal-agent-integration.test.ts: static-grep anchors re-pinned after themaybeSpawnPty/ args-awarecanDispatchOverTunnelrefactors (the new anchors pin the equivalent, stronger invariants); the PTY round-trip test gets a feature-detectedskipIf(!BUN_HAS_PTY)(bun 1.2.xBun.spawnhas noterminaloption — probed, not version-sniffed); the Sec-WebSocket-Protocol success-path handshake is rewritten as a raw TCP handshake vianode:netbecausefetch()with Upgrade headers never resolves for 101 responses on bun 1.2.x (same assertions, truer to the test's stated intent).dual-listener.test.ts: pinned literal updated to the current args-aware tunnel-dispatch source.4.
test(hermetic)— 20 failures: an env-restore leak and machine-state assumptionsThe best bug of the batch:
gstack-memory-helpers.test.tssavedGSTACK_HOME/GBRAIN_HOME/HOME/PATHinbeforeEachbut restored inafterAll— so after test #1, the "saved" value was the previous test's temp dir, and the restore permanently leaked a deletedgstack-test-engine-*temp path intoprocess.envfor every file that ran after it. That single leak caused 11 failures in two unrelated files (gstack-config.test.ts, make-pdfformat-gate.test.ts). Proven both directions: the exact failures reproduce by running the leaky file first pre-fix, and everything is green post-fix including a cold-daemon run. Fixed by capturing once at collection time (same pattern asbrain-cache-roundtrip.test.ts), plus defense-in-depth env scrubbing ingstack-config.test.ts's runner.Also:
gbrain-detect-install.test.tsappends a bun-only shim dir to its deliberate SAFE_PATH (the script's#!/usr/bin/env -S bun runshebang needs bun, but~/.bun/binalso carries a realgbrainon dev machines);gen-skill-docs/openclaw-native-skillsstrict-YAML tests gate onBun.YAMLavailability (lands in bun 1.3; CI pins latest so coverage is preserved where enforced);codex-resume-flag-semanticsupgrades its availability probe fromwhich codextocodex --versionexit-0 (a half-installed npm wrapper resolves but cannot execute).5.
test(design)— 7 failures: interface drift and a millisecond truncationfeedback-roundtrip.test.ts(0/6 passing, fails identically on unmodified main): the test passed theBrowserManagerwherehandleWriteCommand/handleReadCommandnow take aTabSession. The manager delegates enough of the session surface to limp through setup but lacksclearLoadedHtml, whichgotocalls — henceTypeErroron every test. Now calls viabm.getActiveSession()exactly like the canonical caller incommands.test.ts. Product code was correct and is untouched.variants-retry-after.test.ts: the test built its Retry-After HTTP-date fromDate.now() + 3000, andtoUTCString()truncates milliseconds — silently shaving up to 999ms off the encoded wait, failing whenever the wall clock was >~500ms into a second. A coin-flip test. The base timestamp is now aligned to a whole second; assertions unchanged.Ground rules the fixes followed
bun-version: latest, still runs everything)./shippass.Verification
commands.test.ts243/243 in 28.9s; whole browse-core cluster in one process: 449 pass / 1 skip / 0 fail).test:evals) were not run locally; the only product-code change is thecloseTabfix, which CI's diff-based selection will cover.Known issues deliberately not fixed here (pre-existing, identical before/after)
test/skill-e2e-*/skill-llm-evalfiles on bun 1.2.x (thetest(name, {timeout}, fn)options-object signature needs newer bun). These files are--ignored by the free-suite script and run in CI with latest bun; they still load when the free suite is invoked with positional dir filters.test/gbrain-refresh-install-render.test.ts(source-anchor miss) andios-qa/daemon/test/daemon-integration.test.ts(afterEachundefined at load on bun 1.2.x).TUNNEL_PATHSinbrowse/src/server.tsstill allowlists the removed/sidebar-chatroute (requests 404; editing the tunnel security surface was out of scope), andpackage.jsonenginesstill claimsbun >=1.0.0while parts of the suite genuinely want ≥1.3.🤖 Generated with Claude Code