feat(sdk/io): add PostgreSQLFileStore for persistent EventLog#2902
feat(sdk/io): add PostgreSQLFileStore for persistent EventLog#2902StatPan wants to merge 10 commits intoOpenHands:mainfrom
Conversation
|
@OpenHands /codereview /github-pr-review 🤖 This comment was automatically posted by the pr-triage skill (OpenHands) on behalf of a maintainer. See PR #2906 for skill details. |
|
I'm on it! xingyaoww can track my progress at all-hands.dev |
xingyaoww
left a comment
There was a problem hiding this comment.
🟡 Acceptable — useful feature, backward-compatible plumbing, but has a real SQL injection vector and testing gaps that should be addressed before merging.
This review was generated by an AI assistant (OpenHands) on behalf of the maintainers.
SummaryI performed a code review on PR #2902 ("feat(sdk/io): add PostgreSQLFileStore for persistent EventLog") as requested by the What was done:
Key findings posted:
The review was posted as a COMMENT (not APPROVE) because there are issues at "important" level or higher that should be addressed before merging. |
|
Thanks for the thorough and speedy review! Didn't expect feedback this fast — almost like you had a robot reading it for you. Oh wait. 😄 All points are well taken. Here's my plan: Will fix in next push:
Will address with a note / follow-up:
Working on the fixes now — should have a push up today. Thanks again for the detailed feedback! |
a7dda78 to
cfe7473
Compare
|
Hi @xingyaoww, all 7 review items have been addressed in cfe7473:
Would appreciate a re-review when you have a moment. Thank you! |
c531606 to
a57b87a
Compare
Adds PostgreSQLFileStore — a new FileStore implementation backed by PostgreSQL — and a file_store injection parameter on ConversationState and LocalConversation, enabling callers to supply any FileStore without being constrained to the local filesystem. Motivation: in ephemeral environments (Cloud Run, containers without persistent volumes), LocalFileStore is unavailable and the SDK falls back to InMemoryFileStore, losing all event history across requests. Full conversation resume — including tool_call/tool_result events — requires persisting the EventLog to a durable store. Changes: - openhands/sdk/io/postgresql.py: PostgreSQLFileStore - background event-loop thread + asyncpg pool (sync FileStore interface) - per-path threading.Lock for EventLog sequential index (single-process) - optional asyncpg import; no hard dependency added to core SDK - openhands/sdk/io/__init__.py: conditional re-export of PostgreSQLFileStore - openhands/sdk/conversation/state.py: file_store param on create() - openhands/sdk/conversation/impl/local_conversation.py: file_store param - pyproject.toml: optional extra [postgresql] = asyncpg>=0.29.0 - tests/sdk/io/test_postgresql_filestore.py: 10 unit tests (mocked pool) All changes are backward-compatible: file_store=None preserves existing LocalFileStore/InMemoryFileStore behaviour. Co-authored-by: openhands <openhands@all-hands.dev>
…tributed safety Lock key is derived from namespace + path via SHA-256 (64-bit signed int). Uses a dedicated connection per lock call (asyncpg.connect) to avoid PoolConnectionProxy vs Connection type mismatch and to keep the session-level advisory lock lifecycle self-contained. Safe for concurrent writers across multiple processes and instances. Co-authored-by: openhands <openhands@all-hands.dev>
…separation, context manager, tests - Validate table name against strict identifier regex (SQL injection fix) - Wrap _setup() failure in try/except to clean up background thread/loop - Add __enter__/__exit__ for context manager support - Extract DDL to ensure_schema() classmethod; add auto_create_schema flag - Document connection-per-lock trade-off in _lock_acquire docstring - Add file_store + persistence_dir interaction docs in state.py - Relabel existing tests as contract tests, add integration test suite - Register integration pytest marker in pyproject.toml
1a31d47 to
5eb70c8
Compare
|
[Automatic Post]: It has been a while since there was any activity on this PR. @StatPan, are you still working on it? If so, please go ahead, if not then please request review, close it, or request that someone else follow up. |
Co-authored-by: openhands <openhands@all-hands.dev>
|
Resolved the merge conflicts by merging the latest upstream/main into feat/postgresql-filestore and pushed the update in 2570149.
|
Co-authored-by: openhands <openhands@all-hands.dev>
Closes #2903
Summary
PostgreSQLFileStore— aFileStorebacked by PostgreSQL for environments where the local filesystem is unavailable or ephemeral (Cloud Run, containers without persistent volumes)file_store: FileStore | None = Noneparameter toConversationState.create()andLocalConversation.__init__()so callers can inject anyFileStoreimplementationMotivation
In ephemeral environments,
LocalFileStorerequires a filesystem path that doesn't persist across requests. The SDK falls back toInMemoryFileStore, losing all event history. Without a persistent EventLog, full multi-turn resume — includingtool_call/tool_resultevents — is impossible.Changes
openhands/sdk/io/postgresql.pyPostgreSQLFileStoreimplementationopenhands/sdk/io/__init__.pyopenhands/sdk/conversation/state.pyfile_storeparam onConversationState.create()openhands/sdk/conversation/impl/local_conversation.pyfile_storeparam onLocalConversation.__init__()pyproject.toml[postgresql]=asyncpg>=0.29.0tests/sdk/io/test_postgresql_filestore.pyAll changes are backward-compatible:
file_store=Nonepreserves existingLocalFileStore/InMemoryFileStoreselection logic.Design notes
FileStoreinterface to async asyncpg without creating a new loop per call and without interfering with the host app's event looppg_advisory_lock-based locking: safe for concurrent writers across multiple processes and instances (Cloud Run, autoscaled deployments). Lock key = SHA-256(namespace + "\x00" + path)[:8] as signed int64. Usespg_try_advisory_lockwith 50 ms polling and a dedicated connection perlock()call to keep the session-level advisory lock lifecycle self-contained.asyncpgis not a hard dependency.PostgreSQLFileStoreis re-exported fromopenhands.sdk.ioonly when asyncpg is installed;__init__.pyusestry/except ImportErrorUsage
Docs
OpenHands/docs#465
🤖 Generated with Claude Code