-
Notifications
You must be signed in to change notification settings - Fork 4.3k
GH-17211: [C++] Add hash32 and hash64 scalar compute functions
#45001
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kszucs
wants to merge
89
commits into
apache:main
Choose a base branch
from
kszucs:scalar-hash
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+2,838
−11
Open
Changes from all commits
Commits
Show all changes
89 commits
Select commit
Hold shift + click to select a range
04776a6
GH-17211: refresh history for scalar_hash kernel
drin bef8c0b
GH-17211: refresh history for updates to key_hash
drin a1adc88
GH-17211: refresh history for new compute fn infra
drin 98aaaf3
GH-17211: refresh history of scalar hash benchmark
drin 362a5b5
added KeyColumnArray alias
drin ab02dd1
task: revert style and other minor changes
drin b427d5e
task: reducing support to some nested types
drin 4502af2
minor: added type check for ListArray
drin 77b42eb
Update the implementation to use column oriented hash-combine.
kszucs 0e947ae
add hypothesis test case and fix list hashing
kszucs 43d2be4
ensure to keep newly allocated arrays alive until hash gets computed
kszucs 6a52388
minor cleanup
kszucs 02ad712
wrong include
kszucs 776c201
add hash32 function
kszucs 484ae79
add hash32 function and address review comments
kszucs bf6d0d7
update testing suite
kszucs 6d2d913
generated struct tests
kszucs 910361c
prepare adding hash32 tests
kszucs fe2fb0a
add tests for hash32, dictionary encoded type, extension type
kszucs dfe2bcc
ensure that NULL and 0 hashes to different numbers
kszucs cfd66e8
fix the benchmarks
kszucs db499a6
update docs to highlight 0 return for NULL values
kszucs 6465886
use resize instead of reserve
kszucs c037429
fix R compute function filtering test
kszucs 38340a1
address review comments
kszucs 374d566
update boolean test case
kszucs 3200d6d
update exec checks
kszucs 8c25799
use Hasher::kHashBatchTempStackUsage
kszucs 28aa8a4
ensure that slices properly work
kszucs 516abfd
add tests for fixed_size_binary()
kszucs 1c167a5
add rudimentary quality tests
kszucs 7a812f6
test temporal types too
kszucs 6feca33
test temporal and decimal types
kszucs 6e8a096
add type matcher to explicitly raise NotImplemented for unsupported t…
kszucs 8df07a9
update error message
kszucs 6293b44
update docs
kszucs 1ea5599
set a tolerance for the randomly generated numbers
kszucs 93c4add
make msvc happy
kszucs 141d3d4
Rename DCHECK macros to ARROW_DCHECK
kszucs 59d254e
Missing blank lines
kszucs a9686c8
chore: move key_hash benchmarks to key_hash_benchmark.cc under compute
kszucs 63aaf30
Apply suggestions from code review
kszucs 4d2a8e3
use type singleton
kszucs 06a6773
run clang format on scalar_hash_benchmark.cc
kszucs 2914268
run cmake format
kszucs e60dbd7
Address review comments and fix missing-declarations build failure
kszucs 901c6dd
Fix unreachable slicing check and remove unused hashing_rng in benchm…
kszucs 96e7fcc
Fix nested-type null and hash-corruption bugs in hash32/hash64
kszucs c5e12b7
Remove dead list-like branches from ToColumnArray
kszucs 96039e5
Fix independently-null nested fields hashing as non-zero garbage
kszucs 5ffc10d
Size key_hash_benchmark's TempVectorStack from kHashBatchTempStackUsage
kszucs 1e8d5ed
Hoist KeyColumnArray construction out of key_hash_benchmark's timed loop
kszucs 85b090a
Fix GCC -Werror=range-loop-construct in scalar_hash_test.cc
kszucs 7872dce
Guard against a null values buffer in ToColumnArray's binary path
kszucs a099eed
Add R test asserting hash32/hash64 are discoverable via list_compute_…
kszucs d090d5e
Avoid hashing unreferenced child data for sliced list/map arrays, fix…
kszucs 31101f9
Add hypothesis tests asserting hash32/hash64 respect Array.slice()
kszucs b01857d
Revert kAddend change to the shared HashIntImp; scope the null/0 fix …
kszucs 6b65e6d
Unwrap extension types in HashableMatcher so unsupported storage type…
kszucs 3193d05
Add test coverage for zero/null collision across byte widths and FIXE…
kszucs 859414c
Fix HashChild reusing a nested field's unshifted validity buffer
kszucs 5bdf006
Avoid the bitmap copy in HashChild by omitting the validity buffer en…
kszucs 1af6e2c
Fix double blank lines around the new Hash Functions doc section
kszucs 3ef5649
Avoid hashing unreferenced struct nested-field data, avoid gtest macr…
kszucs 372f0b1
Hoist the per-column vector out of KeyHashIntegers benchmark loops
kszucs 9877f6b
Fix Hash32/Hash64 API docstrings
kszucs b6691d5
Fix values-child offset cancellation bug in list/map hashing
kszucs ed7c83e
Fix valid struct rows coincidentally colliding with the null hash sen…
kszucs 746050c
Loosen RandomPrimitive's hash-quality tolerance for cross-platform RN…
kszucs 57f24ac
Fix independently-null fields of multi-field structs not hashing to 0
kszucs 11bfed4
Dedupe Hash64Struct{Small,Medium,Large}Strings benchmarks
kszucs 015485e
Make hash32/hash64 null-aware instead of reserving 0 for nulls
kszucs 1e62e95
Fix zero-width fixed_size_binary hashing, and simplify
kszucs 22e3e0b
Fix NullType struct fields not invalidating their row's hash
kszucs b64054c
Add DoNotOptimize/ClobberMemory to key hash benchmarks
kszucs 3c6c2b7
Bulk-copy hash output validity, and fix doc nits
kszucs 01e2916
Reject unsupported nested child types during hash dispatch
kszucs 0e2d8b0
Drop unused builder_primitive.h include from hashing_benchmark
kszucs 28fb6a9
Route zero-width fixed_size_binary struct fields through HashArray
kszucs 8615f2f
Add the new scalar-hash sources to the Meson build
kszucs c65c43e
Distinguish a null child element from one hashing to zero
kszucs 987c5d3
Hash map keys even when the item is null
kszucs 1e06942
Move zero-width struct comment to its own test
kszucs a3c1c8b
Close scalar-hash test coverage gaps found in review
kszucs defc465
Add missing standard-library includes flagged in PR review
kszucs dd8323f
Cover a scalar argument to hash32/hash64
kszucs ddba847
Document the scalar and chunked input shapes for hash32/hash64
kszucs f32786c
Spell out the null and stability semantics in the hash FunctionDocs
kszucs e052771
Feed HashMultiColumn row counts it can represent
kszucs File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.