[test](nereids) stabilize flaky prune_bucket_with_bucket_shuffle_join#64530
Open
924060929 wants to merge 1 commit into
Open
[test](nereids) stabilize flaky prune_bucket_with_bucket_shuffle_join#64530924060929 wants to merge 1 commit into
924060929 wants to merge 1 commit into
Conversation
With enable_nereids_distribute_planner=true the RIGHT OUTER JOIN distribution is non-deterministic between BUCKET_SHUFFLE and PARTITIONED. The choice is sticky within a JDBC connection, so the existing retry(120, 1000) (which reuses the same connection) cannot escape PARTITIONED once a connection lands there, and the BUCKET_SHUFFLE assertion fails on a large fraction of runs. Both plans are correct; BUCKET_SHUFFLE just saves one exchange. Only run the bucket-shuffle-specific checks when the planner actually chose BUCKET_SHUFFLE; otherwise return early.
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
Stabilize the flaky regression test
prune_bucket_with_bucket_shuffle_join.Problem
With
enable_nereids_distribute_planner=true, theRIGHT OUTER JOINin this case has anon-deterministic distribution: it can be planned as either
BUCKET_SHUFFLEorPARTITIONED.Both plans are correct —
BUCKET_SHUFFLEjust has one fewer exchange.The choice is sticky within a JDBC connection: every
explainon the same connectionreturns the same distribution. The regression framework reuses one connection per suite
(
SuiteContext.getConnection()caches it in aThreadLocal), so the existingretry(120, 1000)retries on the same sticky connection and can never flipPARTITIONED→BUCKET_SHUFFLE. Once a run lands onPARTITIONED, theassertTrue(result.contains("RIGHT OUTER JOIN(BUCKET_SHUFFLE)"))assertion fails for all120 retries → flaky failure.
Fix
After enabling the distribute planner, explain once and check whether the plan actually
chose
BUCKET_SHUFFLE:pruning, result check);
This is a test-only change; it does not touch FE/BE planner behavior. Both distributions
already produce correct results.
Further comments
The underlying non-determinism (benign tie-break vs. whether the planner should
deterministically prefer
BUCKET_SHUFFLE) is a separate planner question and is leftas-is here; this PR only removes the flakiness from the regression case.