Skip to content

Fix sqlite delete hang by deferring rowid deletes#174

Merged
staticlibs merged 1 commit intoduckdb:mainfrom
jprafael:main
Mar 15, 2026
Merged

Fix sqlite delete hang by deferring rowid deletes#174
staticlibs merged 1 commit intoduckdb:mainfrom
jprafael:main

Conversation

@jprafael
Copy link
Copy Markdown
Contributor

This attempts to fix the underlying cause for duckdb/ducklake#805.

The debugging and implementation was done by an LLM.
I don't know enough of the code base to know if this is the correct solution or if there is something else that should be done instead but it does fix the deadlock.

LLM Generated:

Summary:

  • Captured a stack trace during the hang: the main thread is stuck in SQLite’s busy handler (sqliteDefaultBusyCallback → sqlite3BtreeDelete), indicating a self-deadlock between the scan and delete on the same SQLite table.
  • Implemented a fix in the sqlite extension: buffer rowids during the scan, then perform deletes after the scan completes. This avoids the read cursor holding locks while deletes execute.
  • The new regression test now passes in the C++ test runner.

Changes

  • Updated delete implementation to buffer rowids and delete after scan completes: src/storage/sqlite_delete.cpp
  • Added regression test: test/sql/storage/attach_delete_hang_repro.test

Tests

env SQLITE_TPCH_GENERATED=1 timeout 120s ./build/release/test/unittest "test/sql/storage/attach_delete_hang_repro.test"
Result: pass

env SQLITE_TPCH_GENERATED=1 timeout 120s ./build/release/test/unittest "test/sql/storage/attach_delete.test"
Result: pass

@staticlibs
Copy link
Copy Markdown
Collaborator

Hi, thanks for the PR! I will take a look.

@staticlibs
Copy link
Copy Markdown
Collaborator

Thanks!

Perhaps we don't need to keep the statement on a global state at all here, but the current version should be no worse than before.

@staticlibs staticlibs merged commit af2892a into duckdb:main Mar 15, 2026
12 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants