Skip to content

Synchronize packed validity updates in writable vectors - #904

Merged
staticlibs merged 2 commits into
duckdb:mainfrom
hutiefang76:frank/nextten-dd-validity-20261009-01a11186
Oct 10, 2026
Merged

staticlibs merged 2 commits into
duckdb:mainfrom
hutiefang76:frank/nextten-dd-validity-20261009-01a11186

Conversation

@hutiefang76

Copy link
Copy Markdown
Contributor

Parallel producers writing different rows of a DuckDBWritableVector can still update the same packed 64-bit validity word. The non-atomic read/modify/write drops other producers' bits, so valid scalar-function results are returned as SQL NULL (and concurrent null writes can be lost).

Synchronize only the validity word update on the existing buffer, preserving parallel data writes and Java 8 API compatibility. Extend the existing parallel stream regression and add a barrier-based test covering concurrent valid/null writes.

Validation on JDK 17:

  • The existing parallel test and the new regression each failed 3/3 against the original implementation.
  • Both tests passed 20 repeated runs each after the fix.
  • The complete test runner passed all 541 tests. After setting DuckDB threads=1 in the new test to isolate its Java worker barriers, both focused tests passed again; the production change was unchanged apart from comments.
  • The two changed files pass clang-format.

AI assistance: DeepSeek assisted implementation and first review; the final diff and actual test output were checked before submission.

Signed-off-by: hutiefang76 <137664623+hutiefang76@users.noreply.github.com>

@staticlibs staticlibs left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, this is a good catch! The change looks good to me, just a one request to change the lock handling.

Comment thread src/main/java/org/duckdb/DuckDBWritableVector.java Outdated
Signed-off-by: hutiefang76 <137664623+hutiefang76@users.noreply.github.com>
@staticlibs
staticlibs merged commit b2175e5 into duckdb:main Oct 10, 2026
16 checks passed
staticlibs added a commit that referenced this pull request Oct 10, 2026
This is a backport of the PR #904 to `v2.0-cyanoptera` branch.

Co-Authored-By: hutiefang76 <137664623+hutiefang76@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants