branch-4.2:[fix](cost model) scale partitioned scalar aggregate cost by BE number #66354 - #68368
Merged
Merged
Conversation
…en partitioned (apache#66354) Problem Summary: A scalar count(distinct) runs its global aggregate hash-shuffled by the distinct key (non-empty partitionExpressions), i.e. in parallel across BEs. But the cost model set the parallelism factor from group-by keys alone, so with no group by it used factor = 1, costing the aggregate as single-point. This overestimated the parallel plan and made CBO reject it. Solution: Treat a global aggregate as partitioned when it has group by keys or non-empty partitionExpressions, and scale its cost by beNumber accordingly. --------- Co-authored-by: yangtao555 <yangtao555@jd.com> (cherry picked from commit 051b9ac)
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
feiniaofeiafei
changed the title
feiniaofeiafei
changed the title
Collaborator
Author
|
run buildall |
englefly
added a commit
to englefly/incubator-doris
that referenced
this pull request
Sep 23, 2026
…tputs after rebase Rebasing onto the current branch-4.2 brought in b58e421 [fix](cost model) scale partitioned scalar aggregate cost by BE number (apache#68368) which changes the distributed plan of tpcds q16/q94/q95. Those 25 baselines were touched by both sides, so the rebase conflicted on 22 of them and the merge left them without the table alias. Regenerated the whole shape_check tree with -forceGenOut on the rebased tree (1001 suites, 0 failures) so the baselines carry both the new cost-model plan and the table alias. Only these 25 files changed: 75 insertions / 75 deletions, every changed line a pure alias insertion. Verification: - re-run without -forceGenOut: shape_check 1001 suites, 0 failures - the 22 suites from the previous commit: 0 failures (cost model change does not affect them) - alias-stripped, all 998 shape_check .out are byte-identical to origin/branch-4.2's committed baselines, i.e. the only delta is the alias
englefly
added a commit
to englefly/incubator-doris
that referenced
this pull request
Sep 23, 2026
…tputs after rebase Rebasing onto the current branch-4.2 brought in b58e421 [fix](cost model) scale partitioned scalar aggregate cost by BE number (apache#68368) which changes the distributed plan of tpcds q16/q94/q95. Those 25 baselines were touched by both sides, so the rebase conflicted on 22 of them and the merge left them without the table alias. Regenerated the whole shape_check tree with -forceGenOut on the rebased tree (1001 suites, 0 failures) so the baselines carry both the new cost-model plan and the table alias. Only these 25 files changed: 75 insertions / 75 deletions, every changed line a pure alias insertion. Verification: - re-run without -forceGenOut: shape_check 1001 suites, 0 failures - the 22 suites from the previous commit: 0 failures (cost model change does not affect them) - alias-stripped, all 998 shape_check .out are byte-identical to origin/branch-4.2's committed baselines, i.e. the only delta is the alias
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
picked from #66354