[Richard Lundeen (richlundeen)]

Description

Replace the shared scenario-progress refresh lock with one lock per run. A slow storage read for one run no longer blocks refresh for unrelated runs. Requests for the same run remain serialized across cache validation, page hydration, and snapshot construction.

Keep lock ownership separate from replaceable cache data. Track owners and waiters so eviction, plan changes, cancellation, and storage failures cannot split a run's lock. Protect active entries from LRU eviction, allow temporary overflow only when protected entries exceed the normal 32-run limit, and trim the cache as requests finish. Remove unused lock state when the last caller exits.

Progress calculations, keyset queries, page size, REST response shapes, and queue behavior are unchanged. No background refresh tasks or new shutdown duties are added. Calculation and metrics-file optimizations remain out of scope.

This implements item 3 of the async memory backend improvements plan, after #2953 and #3046.

Tests and Documentation

Add event-gated tests for cross-run overlap, same-run serialization, owner and waiter cancellation, partial-page recovery after storage failures, protected-entry eviction, temporary overflow cleanup, snapshot failures, and plan invalidation. Review also identified missing coverage for an owner exiting while same-run callers are waiting; the new test covers both cancellation and storage failure, retains the saved cursor, and verifies that a later caller uses the same lock.

Update the read-model and helper descriptions to explain per-run ownership and the cache-overflow rule. Component responsibilities and public APIs do not change.

Validation:

uv run --no-sync pytest -q tests\unit\backend\test_scenario_progress_read_model.py tests\unit\backend\test_scenario_run_service.py::test_get_progress_keeps_live_snapshot_across_storage_await tests\unit\backend\test_scenario_run_service.py::test_get_progress_cache_only_maps_new_storage_rows tests\unit\backend\test_scenario_run_service.py::test_get_progress_cache_refreshes_identifier_enriched_after_insert tests\unit\backend\test_scenario_run_service.py::test_get_progress_refreshes_enriched_rows_across_storage_pages tests\unit\analytics\test_scenario_statistics_parity.py

Passed: 69 tests.

$env:UV_NO_SYNC = '1'; uv run --no-sync pre-commit run --all-files

Passed: all hooks, including the full project type check, memory migration checks, and documentation checks. UV_DEFAULT_INDEX was preserved and uv.lock is unchanged.

JupyText execution and the full unit suite were not run. No notebooks changed; validation used the scoped regression and statistics-parity tests above.