[fix](iceberg) Allow nested decimal precision widening by OIiveirra · Pull Request #68378 · apache/doris

OIiveirra · 2026-09-22T07:57:26Z

fix Allow nested decimal precision widening

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Iceberg allows increasing DECIMAL precision when the scale is unchanged, but Doris rejected this change when the decimal field was nested in a STRUCT, ARRAY, or MAP. The rejection caused the nested decimal schema-evolution cases in the Trino Iceberg compatibility suite to fail. This change delegates nested primitive promotion validation to Iceberg's TypeUtil, so the connector accepts only promotions supported by Iceberg, including same-scale decimal precision widening, while continuing to reject narrowing and incompatible changes.

The regression coverage creates an Iceberg table with STRUCT<amount:DECIMAL(5,3)>, widens it to STRUCT<amount:DECIMAL(10,3)>, verifies the resulting schema, and verifies that the pre-existing value 12.345 remains readable.

Release note

Iceberg complex column changes now support safe nested decimal precision widening.

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
Tests:
- IcebergNestedColumnEvolutionTest: 40 passed
- CatalogBackedIcebergCatalogOpsColumnEvolutionTest: 44 passed
- external_table_p0/iceberg/test_iceberg_nested_schema_evolution_ddl.groovy: 1 suite passed locally
- Added nested decimal precision-widening schema and old-row-read assertions to the regression suite.
- The six original cross-engine Trino product-test cases are left to TeamCity CI validation.
Behavior changed:
- No.
- Yes. Nested Iceberg decimal precision widening with an unchanged scale is now accepted.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2026-09-22T07:57:31Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: Iceberg allows a DECIMAL precision increase when the scale is unchanged, but the connector's complex-column diff only allowed integer and floating-point promotions. As a result, ALTER TABLE MODIFY COLUMN rejected STRUCT<field:DECIMAL(5,3)> to STRUCT<field:DECIMAL(10,3)>, breaking Trino Iceberg compatibility tests. Delegate nested primitive promotion validation to Iceberg TypeUtil and cover the persisted top-level complex MODIFY path. Update the existing test that previously asserted the now-supported decimal widening must fail. ### Release note Iceberg complex column changes now support safe nested decimal precision widening. ### Check List (For Author) - Test: FE Iceberg unit tests (84 tests) and external_table_p0/iceberg/test_iceberg_nested_schema_evolution_ddl.groovy (1 suite) - Behavior changed: Yes (safe nested Iceberg decimal precision widening is accepted) - Does this need documentation: No

OIiveirra · 2026-09-22T09:00:45Z

run buildall

hello-stephen · 2026-09-22T10:37:41Z

TPC-H: Total hot run time: 28162 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 96055e4e6f7cab9da7184695717a9fbb619b868b, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17604	3905	3960	3905
q2	2206	387	306	306
q3	10033	1485	891	891
q4	4688	479	362	362
q5	7517	829	548	548
q6	179	166	136	136
q7	733	781	609	609
q8	9332	1536	1576	1536
q9	5447	4222	4194	4194
q10	6824	1315	1028	1028
q11	430	267	256	256
q12	631	420	290	290
q13	18058	2624	1993	1993
q14	261	258	237	237
q15	q16	728	722	651	651
q17	1775	1177	1162	1162
q18	6520	5630	5543	5543
q19	1176	1261	968	968
q20	479	402	263	263
q21	5410	3450	2979	2979
q22	461	383	305	305
Total cold run time: 100492 ms
Total hot run time: 28162 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4633	4687	4360	4360
q2	736	590	540	540
q3	4781	5198	4560	4560
q4	2318	2369	1476	1476
q5	4480	4422	4653	4422
q6	224	171	132	132
q7	1801	1714	1475	1475
q8	2404	2074	2051	2051
q9	7319	7241	7327	7241
q10	3671	3556	3105	3105
q11	535	391	373	373
q12	702	698	502	502
q13	2277	2607	1983	1983
q14	263	275	249	249
q15	q16	661	681	630	630
q17	7342	6811	6721	6721
q18	11891	11068	11809	11068
q19	1090	987	1008	987
q20	2216	2195	1927	1927
q21	5031	4202	4397	4202
q22	507	460	400	400
Total cold run time: 64882 ms
Total hot run time: 58404 ms

hello-stephen · 2026-09-22T10:47:36Z

TPC-DS: Total hot run time: 152733 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 96055e4e6f7cab9da7184695717a9fbb619b868b, data reload: false

query5	4303	609	453	453
query6	438	224	192	192
query7	4821	553	286	286
query8	326	175	170	170
query9	8795	3996	3976	3976
query10	441	310	260	260
query11	5823	3522	3211	3211
query12	149	93	88	88
query13	1270	621	414	414
query14	6565	4526	4240	4240
query14_1	3984	3968	3918	3918
query15	203	206	185	185
query16	1030	491	379	379
query17	937	674	553	553
query18	2443	462	345	345
query19	205	182	140	140
query20	85	82	80	80
query21	228	132	116	116
query22	13029	12909	12728	12728
query23	14032	13091	12479	12479
query23_1	12612	12408	12468	12408
query24	7211	1160	684	684
query24_1	675	684	695	684
query25	551	443	382	382
query26	1283	321	164	164
query27	2685	582	345	345
query28	4533	2000	2003	2000
query29	1611	743	537	537
query30	305	220	187	187
query31	892	765	635	635
query32	162	102	97	97
query33	548	307	254	254
query34	1188	1130	640	640
query35	726	745	636	636
query36	794	795	721	721
query37	156	105	95	95
query38	1826	1767	1674	1674
query39	677	688	666	666
query39_1	635	649	652	649
query40	231	123	104	104
query41	83	72	70	70
query42	96	99	95	95
query43	332	342	296	296
query44	1401	720	717	717
query45	188	178	173	173
query46	1068	1235	742	742
query47	1482	1503	1378	1378
query48	377	381	294	294
query49	584	397	289	289
query50	975	353	286	286
query51	10465	10281	10257	10257
query52	86	86	83	83
query53	243	253	177	177
query54	258	201	187	187
query55	81	74	67	67
query56	227	214	213	213
query57	1472	1392	1426	1392
query58	298	259	255	255
query59	1985	2041	1854	1854
query60	274	261	227	227
query61	149	148	145	145
query62	402	323	267	267
query63	215	178	179	178
query64	2797	997	820	820
query65	3466	3407	3419	3407
query66	1789	415	306	306
query67	20117	20307	19982	19982
query68	3077	1465	982	982
query69	392	299	279	279
query70	868	809	815	809
query71	315	230	206	206
query72	2641	2498	2291	2291
query73	793	746	437	437
query74	4634	4490	4296	4296
query75	2296	2286	1940	1940
query76	2334	1133	757	757
query77	363	396	298	298
query78	9219	9161	8505	8505
query79	1405	1187	760	760
query80	763	446	366	366
query81	554	327	281	281
query82	919	168	119	119
query83	293	242	202	202
query84	315	142	110	110
query85	1003	472	377	377
query86	397	244	228	228
query87	1981	1993	1832	1832
query88	3618	2748	2700	2700
query89	360	283	251	251
query90	1724	174	180	174
query91	174	162	126	126
query92	102	83	82	82
query93	1469	1435	981	981
query94	605	334	295	295
query95	650	370	419	370
query96	1006	804	343	343
query97	2423	2443	2343	2343
query98	158	162	144	144
query99	738	733	625	625
Total cold run time: 236599 ms
Total hot run time: 152733 ms

hello-stephen · 2026-09-22T10:52:28Z

ClickBench: Total hot run time: 23.79 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 96055e4e6f7cab9da7184695717a9fbb619b868b, data reload: false

query1	0.01	0.01	0.01
query2	0.10	0.05	0.05
query3	0.27	0.14	0.14
query4	1.61	0.13	0.14
query5	0.24	0.23	0.21
query6	1.15	0.89	0.93
query7	0.04	0.01	0.01
query8	0.06	0.03	0.03
query9	0.38	0.34	0.34
query10	0.53	0.56	0.53
query11	0.20	0.15	0.14
query12	0.20	0.15	0.14
query13	0.47	0.47	0.48
query14	0.96	0.93	0.94
query15	0.60	0.58	0.60
query16	0.30	0.31	0.31
query17	1.10	1.08	1.07
query18	0.20	0.19	0.19
query19	2.01	2.01	1.86
query20	0.02	0.01	0.02
query21	15.50	0.18	0.13
query22	5.06	0.05	0.05
query23	16.08	0.31	0.13
query24	2.99	0.40	0.34
query25	0.12	0.05	0.05
query26	0.74	0.21	0.14
query27	0.03	0.04	0.03
query28	3.54	0.81	0.35
query29	12.48	4.04	3.22
query30	0.27	0.15	0.15
query31	2.76	0.57	0.32
query32	3.23	0.59	0.49
query33	3.18	3.30	3.12
query34	15.45	4.02	3.29
query35	3.20	3.18	3.21
query36	0.55	0.42	0.43
query37	0.10	0.06	0.07
query38	0.05	0.04	0.03
query39	0.04	0.03	0.03
query40	0.17	0.16	0.14
query41	0.08	0.03	0.03
query42	0.04	0.02	0.03
query43	0.05	0.03	0.03
Total cold run time: 96.16 s
Total hot run time: 23.79 s

hello-stephen · 2026-09-22T12:18:26Z

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

OIiveirra · 2026-09-23T04:19:58Z

run buildall

hello-stephen · 2026-09-23T04:58:10Z

TPC-H: Total hot run time: 27229 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3cb1726293219f2652367a569e4459e5f2ef581e, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17625	3820	3813	3813
q2	2155	359	320	320
q3	10083	1441	813	813
q4	4683	473	344	344
q5	7467	805	551	551
q6	173	162	131	131
q7	724	761	594	594
q8	9294	1376	1481	1376
q9	5302	4110	4090	4090
q10	6818	1315	1024	1024
q11	428	266	237	237
q12	634	411	292	292
q13	18040	2607	1981	1981
q14	267	259	241	241
q15	q16	733	711	651	651
q17	1782	1144	994	994
q18	6469	5612	5544	5544
q19	1196	1212	1054	1054
q20	484	396	259	259
q21	5433	2876	2628	2628
q22	413	349	292	292
Total cold run time: 100203 ms
Total hot run time: 27229 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4136	4043	4034	4034
q2	726	547	509	509
q3	4443	4800	4357	4357
q4	2190	2328	1451	1451
q5	4202	4087	4065	4065
q6	221	171	124	124
q7	1686	1619	1423	1423
q8	2147	1843	2238	1843
q9	7560	7310	7362	7310
q10	3711	3669	3199	3199
q11	553	404	367	367
q12	723	746	539	539
q13	2463	2775	2137	2137
q14	308	310	271	271
q15	q16	687	728	634	634
q17	7821	7161	7095	7095
q18	11890	11129	11721	11129
q19	1175	1092	1061	1061
q20	2300	2213	1934	1934
q21	5291	4350	4298	4298
q22	501	449	410	410
Total cold run time: 64734 ms
Total hot run time: 58190 ms

hello-stephen · 2026-09-23T05:08:03Z

TPC-DS: Total hot run time: 152014 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3cb1726293219f2652367a569e4459e5f2ef581e, data reload: false

query5	4311	592	455	455
query6	425	204	193	193
query7	4812	531	293	293
query8	323	175	161	161
query9	8783	3951	3932	3932
query10	450	302	257	257
query11	5822	3543	3200	3200
query12	148	87	84	84
query13	1356	548	382	382
query14	6503	4479	4214	4214
query14_1	3948	3971	3947	3947
query15	196	196	182	182
query16	1003	457	436	436
query17	935	689	561	561
query18	2423	467	339	339
query19	209	187	145	145
query20	86	82	80	80
query21	226	140	115	115
query22	12972	12914	12746	12746
query23	13981	13032	12485	12485
query23_1	12535	12520	12681	12520
query24	7282	1142	664	664
query24_1	711	690	765	690
query25	553	425	375	375
query26	1274	326	171	171
query27	2714	571	333	333
query28	4620	2028	1978	1978
query29	1591	722	531	531
query30	297	223	190	190
query31	880	773	642	642
query32	147	99	97	97
query33	533	321	254	254
query34	1171	1098	646	646
query35	719	788	641	641
query36	781	774	719	719
query37	142	105	95	95
query38	1825	1763	1701	1701
query39	667	686	672	672
query39_1	635	646	673	646
query40	220	121	103	103
query41	71	68	67	67
query42	103	94	92	92
query43	334	346	301	301
query44	1380	721	726	721
query45	187	177	164	164
query46	1032	1194	737	737
query47	1479	1536	1388	1388
query48	457	377	298	298
query49	584	402	293	293
query50	963	361	251	251
query51	10465	10212	10463	10212
query52	87	88	73	73
query53	235	267	181	181
query54	247	204	207	204
query55	76	74	69	69
query56	211	206	205	205
query57	1559	1504	1356	1356
query58	276	264	246	246
query59	1994	2066	1848	1848
query60	274	231	218	218
query61	146	138	143	138
query62	402	320	261	261
query63	210	174	178	174
query64	2799	998	799	799
query65	3478	3396	3403	3396
query66	1804	431	299	299
query67	20217	19999	19887	19887
query68	3242	1581	881	881
query69	405	300	252	252
query70	883	841	826	826
query71	293	229	208	208
query72	2635	2475	2264	2264
query73	853	806	414	414
query74	4613	4523	4305	4305
query75	2297	2296	1922	1922
query76	2302	1113	705	705
query77	348	398	294	294
query78	9027	9021	8446	8446
query79	1387	1159	686	686
query80	638	444	368	368
query81	547	323	274	274
query82	634	161	126	126
query83	285	218	202	202
query84	308	139	114	114
query85	842	472	377	377
query86	315	238	232	232
query87	1993	1949	1846	1846
query88	3595	2711	2693	2693
query89	354	283	241	241
query90	1890	184	176	176
query91	163	159	132	132
query92	101	92	91	91
query93	1400	1482	831	831
query94	523	336	296	296
query95	673	354	327	327
query96	1079	783	346	346
query97	2417	2404	2319	2319
query98	167	152	148	148
query99	707	732	619	619
Total cold run time: 235909 ms
Total hot run time: 152014 ms

hello-stephen · 2026-09-23T05:12:54Z

ClickBench: Total hot run time: 24.04 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3cb1726293219f2652367a569e4459e5f2ef581e, data reload: false

query1	0.01	0.01	0.00
query2	0.10	0.05	0.05
query3	0.26	0.14	0.14
query4	1.61	0.14	0.14
query5	0.24	0.23	0.22
query6	1.16	0.95	0.97
query7	0.04	0.01	0.01
query8	0.05	0.04	0.03
query9	0.39	0.32	0.34
query10	0.56	0.55	0.55
query11	0.20	0.14	0.14
query12	0.18	0.15	0.14
query13	0.46	0.47	0.47
query14	0.95	0.97	0.94
query15	0.59	0.59	0.60
query16	0.32	0.32	0.31
query17	1.11	1.12	1.11
query18	0.21	0.19	0.20
query19	2.03	1.94	1.88
query20	0.02	0.01	0.01
query21	15.49	0.20	0.14
query22	4.78	0.05	0.05
query23	16.14	0.30	0.13
query24	2.97	0.42	0.30
query25	0.10	0.05	0.03
query26	0.72	0.20	0.15
query27	0.04	0.04	0.03
query28	3.49	0.76	0.35
query29	12.47	4.15	3.30
query30	0.27	0.15	0.15
query31	2.79	0.56	0.30
query32	3.23	0.59	0.49
query33	3.14	3.16	3.12
query34	15.46	3.93	3.29
query35	3.25	3.25	3.24
query36	0.56	0.43	0.42
query37	0.10	0.07	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.02
query40	0.18	0.15	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 95.93 s
Total hot run time: 24.04 s

OIiveirra requested review from 924060929, CalvinKirs, Gabriel39 and yiguolei as code owners September 22, 2026 07:57

OIiveirra changed the title ~~Fix iceberg nested decimal~~ [fix](iceberg) Allow nested decimal precision widening Sep 22, 2026

OIiveirra force-pushed the fix-iceberg-nested-decimal branch from cfe5d1b to 96055e4 Compare September 22, 2026 08:40

[test](regression) Add nested decimal evolution coverage

3cb1726

Navigation Menu

[fix](iceberg) Allow nested decimal precision widening - #68378

[fix](iceberg) Allow nested decimal precision widening#68378
OIiveirra wants to merge 2 commits into
apache:masterfrom
OIiveirra:fix-iceberg-nested-decimal

OIiveirra commented Sep 22, 2026 •

edited

Loading

Uh oh!

hello-stephen commented Sep 22, 2026

Uh oh!

OIiveirra commented Sep 22, 2026

Uh oh!

hello-stephen commented Sep 22, 2026

Uh oh!

hello-stephen commented Sep 22, 2026

Uh oh!

hello-stephen commented Sep 22, 2026

Uh oh!

hello-stephen commented Sep 22, 2026

Uh oh!

OIiveirra commented Sep 23, 2026

Uh oh!

hello-stephen commented Sep 23, 2026

Uh oh!

hello-stephen commented Sep 23, 2026

Uh oh!

hello-stephen commented Sep 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

OIiveirra commented Sep 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Sep 22, 2026

Uh oh!

OIiveirra commented Sep 22, 2026

Uh oh!

hello-stephen commented Sep 22, 2026

Uh oh!

hello-stephen commented Sep 22, 2026

Uh oh!

hello-stephen commented Sep 22, 2026

Uh oh!

hello-stephen commented Sep 22, 2026

FE UT Coverage Report

Uh oh!

OIiveirra commented Sep 23, 2026

Uh oh!

hello-stephen commented Sep 23, 2026

Uh oh!

hello-stephen commented Sep 23, 2026

Uh oh!

hello-stephen commented Sep 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

OIiveirra commented Sep 22, 2026 •

edited

Loading