fix: PLTF-2895 dedicated read session for RemoteSandboxService.get_sandbox (avoids idle-in-txn without breaking delete atomicity) - #142
Open
aivong-openhands wants to merge 1 commit into
Conversation
…void idle-in-transaction (PLTF-2895) get_sandbox() read the stored sandbox on the shared, request-scoped AsyncSession and then made a (slow) runtime API call, leaving that session "idle in transaction" — pinning a pooled connection and blocking autovacuum on v1_remote_sandbox. Instead of committing the shared session before the network call (the approach taken in OpenHands/OpenHands#14769), read on a short-lived, isolated session so the runtime API call never spans the shared transaction. Committing the shared session would also flush sibling services' still-pending writes (e.g. the raw DELETEs queued by delete_app_conversation), breaking the atomicity of those multi-step flows — the regression identified in the review of #14769 (finding #1). Adds a real-session regression test (TestSharedSessionAtomicityDuringDelete) that seeds an uncommitted sibling DELETE on the shared session, calls get_sandbox, and asserts from a separate connection that the sibling write was not committed. Refs OpenHands/OpenHands#14769 and its review feedback. Co-authored-by: openhands <openhands@all-hands.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RemoteSandboxService.get_sandbox()reads the stored sandbox on the shared, request-scopedAsyncSessionand then makes a (slow) runtime API call. That leaves the session "idle in transaction" for the duration of the network call — pinning a pooled connection and blocking autovacuum onv1_remote_sandbox.This PR reads the stored sandbox on a short-lived, isolated session (
_isolated_read_session) so the runtime API call never spans the shared transaction. The shared request session is left completely untouched.Background — why not just commit the shared session?
This is the same "idle in transaction" problem addressed by OpenHands/OpenHands#14769 (
fix: PLTF-2895 release DB transactions before runtime API I/O), which was opened before the app-server code moved to this repo. That PR's approach was tocommit()the shared session before the runtime call.Review of #14769 found that committing the shared session is unsafe (finding #1): the session is shared by every service in a request, so committing it also flushes sibling services' still-pending writes. Concretely,
delete_app_conversationinterleaves, per conversation:so committing inside
get_sandboxwould commit the previous iteration's still-pendingDELETEs — making the multi-step delete no longer all-or-nothing.The dedicated-read-session approach here achieves #14769's goal (no transaction spans runtime I/O) without that atomicity regression, because the shared session is never read on or committed.
Test
Adds
TestSharedSessionAtomicityDuringDelete— a real SQLAlchemy (SQLite) regression test (no DB mocking) that:DELETEon the shared session,get_sandbox,This test fails if
get_sandboxever commits the shared session again. Fulltest_remote_sandbox_service.pysuite: 114 passed;ruff check+ruff format --checkclean.Scope / follow-ups
get_sandbox(the path exercised by the conversation-delete loop). The other read paths (search_sandboxes,get_sandbox_by_session_api_key,batch_get_sandboxes,pause_old_sandboxes) should get the same treatment — tracked in RemoteSandboxService: extend dedicated read-session fix to remaining read paths (follow-up to #14769, finding #1) OpenHands#16374.delete_sandbox's write/revoke-around-network-I/O ordering is a separate design decision (review finding build(deps): bump python from 3.13.7-slim-trixie to 3.14.6-slim-trixie in /containers/app #2), not addressed here._isolated_read_sessionrecovers the async engine from the shared session's sync bind; a cleaner version would inject anasync_sessionmakerfromDbSessionInjector.References
This PR was created by an AI agent (OpenHands) on behalf of @aivong-openhands.