Skip to content

[fix](function) Preserve DISTINCT in median percentile conversion - #68386

Open
Mryange wants to merge 1 commit into
apache:masterfrom
Mryange:fix-median-distinct
Open

Mryange wants to merge 1 commit into
apache:masterfrom
Mryange:fix-median-distinct

Conversation

@Mryange

@Mryange Mryange commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

median(DISTINCT expr) was rewritten to a non-distinct percentile(expr, 0.5), causing duplicate values to affect the result. The conversion now preserves the MEDIAN aggregate's DISTINCT flag when constructing PERCENTILE.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Mryange
Mryange requested a review from yujun777 as a code owner September 22, 2026 09:11
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Mryange

Mryange commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

/review

@Mryange

Mryange commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

run buildall

@github-actions

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: You've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Sep 28th, 2026 2:04 AM.
Workflow run: https://github.com/apache/doris/actions/runs/35708966512

The selected account is excluded until 2026-09-28T02:04:00Z. Please trigger /review again; another configured account may be available.

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 27722 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 93086215a287d0057b6f4501e1d11859457a7370, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17662	3884	3874	3874
q2	2201	369	307	307
q3	10047	1406	787	787
q4	4680	482	350	350
q5	7471	821	544	544
q6	177	170	134	134
q7	768	792	591	591
q8	9315	1505	1511	1505
q9	5392	4159	4126	4126
q10	6835	1322	1002	1002
q11	422	272	251	251
q12	649	416	300	300
q13	18031	2613	2023	2023
q14	267	259	242	242
q15	q16	744	722	677	677
q17	1787	1193	1026	1026
q18	6498	5633	5550	5550
q19	1316	1229	901	901
q20	494	394	257	257
q21	5817	3324	2966	2966
q22	452	360	309	309
Total cold run time: 101025 ms
Total hot run time: 27722 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4679	4773	4479	4479
q2	727	561	531	531
q3	4724	5132	4655	4655
q4	2223	2373	1463	1463
q5	4508	4366	4562	4366
q6	226	173	125	125
q7	1811	1704	1482	1482
q8	2284	2090	2020	2020
q9	7358	7289	7168	7168
q10	3715	3629	3150	3150
q11	564	366	339	339
q12	701	700	501	501
q13	2293	2608	1996	1996
q14	267	273	248	248
q15	q16	668	686	603	603
q17	7314	6708	6687	6687
q18	11906	11027	11755	11027
q19	1084	979	997	979
q20	2193	2204	1917	1917
q21	5031	4149	4350	4149
q22	509	460	396	396
Total cold run time: 64785 ms
Total hot run time: 58281 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 152822 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 93086215a287d0057b6f4501e1d11859457a7370, data reload: false

query5	4319	608	444	444
query6	441	211	190	190
query7	4867	570	295	295
query8	346	178	163	163
query9	8826	4016	4020	4016
query10	461	298	251	251
query11	5786	3542	3203	3203
query12	147	92	87	87
query13	1265	600	467	467
query14	6560	4546	4263	4263
query14_1	3993	3979	3970	3970
query15	203	197	178	178
query16	981	461	422	422
query17	912	679	555	555
query18	2438	463	339	339
query19	201	186	147	147
query20	83	80	80	80
query21	219	133	121	121
query22	13126	12973	12951	12951
query23	14085	12979	12451	12451
query23_1	12934	12713	12619	12619
query24	7409	1189	693	693
query24_1	676	689	689	689
query25	561	442	381	381
query26	1266	317	173	173
query27	2686	579	342	342
query28	4557	1979	1974	1974
query29	1628	741	543	543
query30	308	225	189	189
query31	917	768	643	643
query32	148	101	96	96
query33	537	324	263	263
query34	1168	1122	638	638
query35	725	749	642	642
query36	809	778	700	700
query37	147	104	92	92
query38	1848	1774	1728	1728
query39	709	691	651	651
query39_1	660	684	662	662
query40	228	128	106	106
query41	71	69	69	69
query42	96	95	94	94
query43	338	352	303	303
query44	1386	713	715	713
query45	182	185	165	165
query46	1076	1227	713	713
query47	1508	1510	1381	1381
query48	399	380	286	286
query49	583	406	288	288
query50	980	337	255	255
query51	10308	10476	10142	10142
query52	88	87	83	83
query53	228	257	177	177
query54	253	221	209	209
query55	78	80	74	74
query56	231	209	208	208
query57	1531	1440	1412	1412
query58	276	257	258	257
query59	2000	2088	1864	1864
query60	281	242	214	214
query61	142	153	152	152
query62	404	321	264	264
query63	217	177	176	176
query64	2784	1034	822	822
query65	3455	3425	3420	3420
query66	1792	419	304	304
query67	19809	20086	19767	19767
query68	3142	1522	896	896
query69	407	309	261	261
query70	898	816	795	795
query71	291	234	212	212
query72	2607	2541	2210	2210
query73	830	783	410	410
query74	4629	4506	4312	4312
query75	2319	2301	1975	1975
query76	2327	1111	787	787
query77	364	406	298	298
query78	9030	9070	8509	8509
query79	1349	1205	726	726
query80	912	452	372	372
query81	589	324	282	282
query82	651	171	123	123
query83	292	231	188	188
query84	323	143	118	118
query85	1064	480	380	380
query86	381	232	233	232
query87	1999	1974	1850	1850
query88	3610	2728	2700	2700
query89	359	285	246	246
query90	1749	183	184	183
query91	172	157	129	129
query92	103	92	92	92
query93	1443	1474	873	873
query94	610	336	325	325
query95	645	452	341	341
query96	989	787	327	327
query97	2437	2440	2357	2357
query98	161	147	163	147
query99	746	742	610	610
Total cold run time: 236669 ms
Total hot run time: 152822 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 23.89 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 93086215a287d0057b6f4501e1d11859457a7370, data reload: false

query1	0.01	0.01	0.01
query2	0.09	0.04	0.04
query3	0.26	0.14	0.14
query4	1.61	0.14	0.15
query5	0.24	0.23	0.22
query6	1.16	0.93	0.95
query7	0.04	0.01	0.01
query8	0.06	0.04	0.04
query9	0.39	0.34	0.32
query10	0.56	0.53	0.59
query11	0.20	0.15	0.15
query12	0.20	0.15	0.16
query13	0.46	0.47	0.47
query14	0.98	0.93	0.96
query15	0.59	0.58	0.61
query16	0.31	0.34	0.33
query17	1.11	1.07	1.13
query18	0.22	0.20	0.19
query19	2.06	1.93	1.88
query20	0.02	0.02	0.01
query21	15.42	0.22	0.12
query22	4.77	0.05	0.05
query23	16.14	0.31	0.13
query24	3.01	0.42	0.33
query25	0.10	0.05	0.06
query26	0.71	0.21	0.14
query27	0.03	0.03	0.03
query28	3.52	0.83	0.33
query29	12.55	4.12	3.20
query30	0.27	0.15	0.15
query31	2.77	0.55	0.32
query32	3.22	0.59	0.49
query33	3.12	3.29	3.18
query34	15.50	3.91	3.27
query35	3.26	3.20	3.25
query36	0.55	0.44	0.40
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.04
query40	0.18	0.14	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 96.03 s
Total hot run time: 23.89 s

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants