Skip to content

test(ports): isolate the launch test's python - #398

Merged
AbysmalBiscuit merged 1 commit into
mainfrom
launch-test-diagnostics
Oct 16, 2026
Merged

AbysmalBiscuit merged 1 commit into
mainfrom
launch-test-diagnostics

Conversation

@AbysmalBiscuit

Copy link
Copy Markdown
Owner

Agent summary

launch_non_blocking_returns_before_readiness_then_status_flips intermittently fails on Windows. The Python server it spawns never listens within 10 s and leaves an empty log, so it neither bound its port nor raised an error. Other tests kept finishing during the wait, so the runner was not stalled. The root cause is unknown.

  • Python ran in the shared temp directory, which python -c puts first on sys.path, so import socket scanned whatever other tests had left there. It now runs in the test's own empty directory.
  • -X importtime writes each import to the log, and the failure message says whether the process was still running at the deadline. A recurrence then shows whether Python started and where it stopped.
  • The server is stopped before the assertion, so a failure no longer leaves it running.

Proof

  1. CI runs launch_non_blocking_returns_before_readiness_then_status_flips on ubuntu, macOS and Windows; it must pass on all three.
  2. The failure it targets is intermittent (2 of the last 60 CI runs, both Windows), so a green run does not prove the cause is gone. The new failure message does: it says whether Python was still running and how far its imports got.

🤖 Generated with Claude Opus 5.5 via Claude Code

The launch test intermittently fails on Windows: the Python server it
spawns never listens within 10 s and leaves an empty log, so it neither
bound nor raised. The cause is unknown.

Python ran in the shared temp directory, which `-c` puts first on
sys.path, so its imports scanned whatever tests had left there. It now
runs in the test's own empty directory. With `-X importtime` the log
records each import, and on failure the message says whether the
process was still running, so a recurrence shows where Python stopped.
The server is stopped before the assertion, so a failure no longer
leaves it running.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚡ Grok Review

Reviewed commit: 22c3f69d24

❌ Patch is incorrect: 1 finding.

The new Windows-only import-time diagnostics can fail the launch test even when the server is healthy, because the log drain treats a still-running Python process as a hard error.

P1 Badge Do not fail a healthy launch when Python is still running

crates/devkit-ports/tests/lifecycle.rs#L640-L646

On Windows, launch_non_blocking_returns_before_readiness_then_status_flips now starts Python with -X importtime and requires server_log to contain import time: before the readiness poll. output_or_running returns Err whenever try_wait is None, and wait_for_line treats that as fatal even if the process is still writing. A server that binds and stays up therefore fails the test whenever the drain races the first import-time line, which is the opposite of the flake this change is meant to diagnose. Drain until the marker appears or the deadline passes, and only then report whether the process is still running.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-16T17:57:53.597751Z 22c3f69 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@AbysmalBiscuit
AbysmalBiscuit dismissed github-actions[bot]’s stale review October 16, 2026 18:03

False positive: the finding cites crates/devkit-ports/tests/lifecycle.rs and the functions server_log, output_or_running and wait_for_line. None of them exist at 22c3f69 (no lifecycle.rs under crates/devkit-ports/tests; rg finds none of the names). The PR changes only the in-crate test in src/run.rs, which never waits for an import-time line: it polls the port, then reads the log once for the failure message. The test passed on ubuntu, macOS and Windows in this PR's CI.

@AbysmalBiscuit
AbysmalBiscuit merged commit 3884090 into main Oct 16, 2026
23 checks passed
@AbysmalBiscuit
AbysmalBiscuit deleted the launch-test-diagnostics branch October 16, 2026 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant