Fix race condition in stateful pages marker with atomic writes

[masenf]

Type of change

• Bug fix (non-breaking change which fixes an issue)

Description

Fixes a startup race condition in backend-only mode with multiple workers where a worker could read a truncated .web/backend/stateful_pages.json file and crash with JSONDecodeError.

Root cause: The marker file was written directly, so concurrent writers could produce partial/corrupted JSON that readers would encounter.

Solution: Write the marker atomically by:

1. Writing to a temporary file in the same directory
2. Using Path.replace() to atomically swap it into place
3. Ensuring readers only ever see a complete, valid marker (or no marker at all)

Additionally, refactored the marker reading logic to handle the case where the marker doesn't exist yet (another worker may be writing it), which correctly falls through to evaluating all pages rather than assuming "no marker" means "no stateful pages".

Changes

reflex/app.py:

• Modified _write_stateful_pages_marker() to use atomic writes via tempfile.mkstemp() and Path.replace()
• Always writes the marker (even for stateless apps with an empty list), ensuring backend workers have a definitive signal

reflex/compiler/compiler.py:

• Extracted marker reading into _read_stateful_pages_marker() helper that returns None if the marker doesn't exist yet
• Simplified compile_app() logic: only skip full page evaluation if the marker exists and is readable; missing marker falls through to normal compilation

Tests:

• Added test_write_stateful_pages_marker_never_truncates_final_path(): verifies the marker is never opened for writing (only swapped into place)
• Added test_write_stateful_pages_marker_is_always_written(): verifies stateless apps write an empty marker
• Added test_write_stateful_pages_marker_concurrent_readers_see_valid_json(): stress test with 4 concurrent writers and 4 concurrent readers, ensuring no JSONDecodeError or partial reads
• Updated test_compile_registers_plugin_routes_on_backend_early_return() to correctly expect all pages evaluated when marker is missing

Test Plan

All new unit tests pass and cover the atomic write behavior and concurrent access patterns. Existing tests updated to reflect the corrected behavior when the marker is absent.

https://claude.ai/code/session_01GSjqov3yBj4cBasJqzyrrQ

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; the previously reported marker correctness and regression-test concerns are resolved in the current code.

Summary

This PR makes the stateful-page marker safe for concurrent backend-worker startup.

• Writes complete marker data to a same-directory temporary file before atomically replacing the final path.
• Preserves cross-account readability and retries transient Windows sharing violations.
• Treats missing, truncated, or temporarily unavailable markers as a request for full page evaluation.
• Avoids modifying the marker during dry-run compilation.
• Adds regression coverage for corruption recovery, permissions, cleanup, concurrency, and dry-run behavior.

_{Reviews (7) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..."}

[codspeed]

Merging this PR will not alter performance

✅ 53 untouched benchmarks
⏩ 9 skipped benchmarks¹

---

_{Comparing claude/upbeat-brown-jglset (e7664b6) with main (aaa44a0)}

Footnotes

1. 9 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[cubic-dev-ai]

Review completed against the latest diff

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[FarhanAliRaza]

Validated end to end in a real example app (stateful + stateless + ComponentState pages): truncated marker crashes main with JSONDecodeError and recovers cleanly on this branch; two concurrent markerless backend workers start without errors and leave a valid marker; browser state behavior unregressed; reflex export succeeds and preserves the marker. Inline comments are non-blocking notes only.

---

Generated by Claude Code