Skip to content

Close remote databases after one-shot replication - #75

Open
apple-ouyang wants to merge 1 commit into
vrtmrz:mainfrom
apple-ouyang:codex/close-one-shot-remote-db
Open

Close remote databases after one-shot replication#75
apple-ouyang wants to merge 1 commit into
vrtmrz:mainfrom
apple-ouyang:codex/close-one-shot-remote-db

Conversation

@apple-ouyang

Copy link
Copy Markdown
Contributor

Summary

  • Close the temporary remote PouchDB database after one-shot replication settles.
  • Release the database before reduced-batch or resurrection retries begin.
  • Close the database when setup fails after the connection has been acquired.

Why

Periodic replication creates a fresh HTTP PouchDB instance for every one-shot run. The replication handler was cancelled when it settled, but the database itself was never closed. PouchDB therefore retained every instance in its constructor-level destruction listener registry.

A renderer heap snapshot taken after approximately 58 hours with a 60-second periodic interval contained 3,046 remote HTTP database instances, compared with four in a shorter-run snapshot. The growth rate closely followed the configured periodic replication interval.

The investigation and reproduction details are in obsidian-livesync#1034.

Implementation

openOneShotReplication() now owns the temporary remote database from the point at which connectivity succeeds and closes it in finally. Continuous replication is unchanged.

Validation

  • Targeted LiveSyncReplicator.unit.spec.ts: 5 passed.
  • Full unit suite: 88 files, 1,507 tests passed.
  • TypeScript check: passed.
  • ESLint: passed with one unrelated existing warning in TrysteroReplicatorP2PServer.ts.
  • Svelte check: passed with existing warnings only.
  • Production build: passed.
  • iOS 15 compatibility check: passed.

@vrtmrz

vrtmrz commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Thank you very much for tracing this resource retention and preparing the fix. The diagnosis is convincing, and I agree that a temporary remote database should have an explicit owner and be closed after one-shot replication.

I am currently finalising #76, which establishes the resource-ownership and replacement rules used by Commonlib, including the Trystero replicator, and substantially changes the surrounding lifecycle structure.

During the rewrite and its end-to-end testing, quite a few issues have come to light in the P2P functionality. I still think the underlying idea was sound, but I have to admit that parts of the original implementation were not written particularly well.

Could you revisit this PR after #76 has landed, review the lifecycle pattern there, and then rebase and update this fix against the resulting structure?

The current tests already establish the central close-after-replication behaviour. When adapting it, I think we should also cover:

  • closing a connection when checkReplicationConnectivity() cannot transfer ownership to its caller;
  • completing the close before a retry or resurrection starts; and
  • logging a close failure without replacing the underlying replication result or preventing the retry.

I am very glad that you found this long-running resource leak. Thank you for the careful investigation and contribution.

@apple-ouyang

Copy link
Copy Markdown
Contributor Author

Understood. I will wait until #76 has merged, then rebase #75 onto the resulting main branch and adapt the fix to the ownership pattern established there.

I will extend the coverage to verify that:

  • a connection is closed when checkReplicationConnectivity() cannot transfer ownership;
  • closing completes before retry or resurrection begins; and
  • a close failure is logged without replacing the replication result or preventing the retry.

I will update the existing PR rather than opening a replacement.

Temporary remote PouchDB instances now have explicit ownership through
connectivity checks and one-shot replication. Disposal completes before a
retry or resurrection begins, and close failures are logged without changing
the replication result.
@apple-ouyang
apple-ouyang force-pushed the codex/close-one-shot-remote-db branch from 437a0c0 to 84e48e3 Compare July 27, 2026 02:32
@apple-ouyang

Copy link
Copy Markdown
Contributor Author

Updated this PR after #76 landed and force-pushed the existing branch with --force-with-lease.

The revised ownership flow now:

  • closes the remote database inside checkReplicationConnectivity() whenever the connection cannot be transferred to its caller;
  • always awaits one-shot database disposal before a retry or resurrection begins; and
  • logs and suppresses a close failure so that it neither replaces the replication result nor prevents the deferred retry.

Validation completed:

  • npm run verify:package: 65 test files and 1,210 tests passed, together with the boundary, release-selection, package build, and packed-consumer checks;
  • npm run test:integration:managed: 2 integration files and 4 tests passed against managed CouchDB and MinIO; and
  • Self-hosted LiveSync PR #1033 with the locally packed Commonlib artefact: 73 unit-test files and 565 tests passed; npm run check and npm run build passed. The check retained one pre-existing Obsidian settings-search lint warning and reported no errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants