Conversation
Contributor
Author
|
@pitrou - Ready for review when you get a chance :) |
pitrou
approved these changes
Aug 26, 2026
| if (!dictionary_span.HasValidityBitmap()) { | ||
| // The dictionary has nulls but no validity bitmap to inspect, which means all | ||
| // of its values are null (a null-typed dictionary). Every index, valid or not, | ||
| // is then logically null. |
Member
There was a problem hiding this comment.
Perhaps DCHECK that the dictionary's type is Type::NA?
github-actions
Bot
added
awaiting committer review
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
The dictionary of a
dictionary(..., null())array is aNullArray, which reportsnull_count == lengthbut has no validity bitmap.dict_util::LogicalNullCountonly skipped its per-index scan when the dictionary's null count was 0, so for these arrays the scan read through a null bitmap pointer and crashed.What changes are included in this PR?
Return the array's length when the dictionary has nulls but no validity bitmap. Every value in such a dictionary is null, so there is nothing to scan.
Are these changes tested?
Yes, in
TestArray.TestNullCountfor all index types, and inTestArray.TestSlicePreservesAllNullCountfor a slice. Two of the new cases crash without the fix.Are there any user-facing changes?
ComputeLogicalNullCount()no longer crashes on these arrays, and neither does thecountkernel, which calls it.This PR contains a "Critical Fix". It fixes a crash on valid input.