Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
### What problem does this PR solve? Issue Number: None Related PR: apache#67002 Problem Summary: A Broker Load fragment may report DATA_QUALITY_ERROR together with its error URL and first error message. Coordinator handles the failure and releases the completion latch before storing those diagnostics. The loading task can then snapshot empty values into the final job state. SHOW LOAD consequently loses the URL or FirstErrorMsg even though the backend supplied them. Publish both fields before cancellation wakes the loading task in the legacy Coordinator and Nereids LoadProcessor. Use the pre-status hook for Nereids diagnostics while retaining final report aggregation in its existing position. The related PR addresses the URL race; this change also preserves FirstErrorMsg and tests the production cancellation path. The unit tests inspect diagnostics at the cancellation/latch boundary, so they exercise the race deterministically without sleeps. They also cover normal completion and reports that omit diagnostic fields. Scope and compatibility: - Change diagnostic publication order only; retain transaction, counter, retry, and persistence behavior. - Add no SQL syntax, configuration, RPC field, or storage format. - This fixes diagnostics carried by the report that triggers failure; it does not add collection of reports arriving after task teardown. ### Release note Fix missing error URLs and first error messages in SHOW LOAD when a Broker Load data quality error report includes those diagnostics. ### Check List (For Author) - Test - [ ] Regression test - [x] Unit Test - [ ] Manual test - [ ] No need to test or manual test - Behavior changed: - [ ] No. - [x] Yes. Preserve reported diagnostics when load tasks wake on failure. - Does this need documentation? - [x] No. Restore existing SHOW LOAD diagnostic fields without introducing a new user-facing interface. - [ ] Yes. ### Validation Local validation passed with JDK 17, Maven 3.9.9, and Thrift 0.24.0: - CoordinatorLoadDiagnosticsTest: 2 tests - LoadProcessorTest: 3 tests - AbstractJobProcessorTest: 2 tests - QeProcessorImplReportAckTest: 7 tests Total: 14 tests, 0 failures, 0 errors, and 0 skipped; 5 tests are new. FE Checkstyle reported 0 violations. git diff --check passed. No cluster regression test was run. ```bash bash ./run-fe-ut.sh --run \ 'org.apache.doris.qe.CoordinatorLoadDiagnosticsTest,'\ 'org.apache.doris.qe.runtime.LoadProcessorTest,'\ 'org.apache.doris.qe.AbstractJobProcessorTest,'\ 'org.apache.doris.qe.QeProcessorImplReportAckTest' ```
yonchicy
force-pushed
the
fix_broker_load_err_url_miss
branch
from
September 22, 2026 06:53
d2b8476 to
35f81a5
Compare
Author
Local pipeline review — ✅ PASSschema: doris-repo-review/v1
status: PASS
pr: apache/doris#68371
commit: 35f81a5b1569ae997c73099499dd8cd614e6c9fe
base: 5c4dbdd07fc43ea287168e1a203beaff4998517f
reviewed_at: 2026-09-22T15:24+08:00
reviewer: yonchicy
model: gpt-6-astra
effort: xhigh
findings: {blocker: 0, major: 0, minor: 0, nit: 0}
rounds: 1
converged: trueNotes for maintainers
Reviewed locally with the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: #67002
Problem Summary:
A Broker Load fragment may report DATA_QUALITY_ERROR together with its
error URL and first error message. Coordinator handles the failure and
releases the completion latch before storing those diagnostics. The
loading task can then snapshot empty values into the final job state.
SHOW LOAD consequently loses the URL or FirstErrorMsg even though the
backend supplied them.
Publish both fields before cancellation wakes the loading task in the
legacy Coordinator and Nereids LoadProcessor. Use the pre-status hook
for Nereids diagnostics while retaining final report aggregation in its
existing position. The related PR addresses the URL race; this change
also preserves FirstErrorMsg and tests the production cancellation path.
The unit tests inspect diagnostics at the cancellation/latch boundary,
so they exercise the race deterministically without sleeps. They also
cover normal completion and reports that omit diagnostic fields.
Scope and compatibility:
retry, and persistence behavior.
it does not add collection of reports arriving after task teardown.
Release note
Fix missing error URLs and first error messages in SHOW LOAD when a
Broker Load data quality error report includes those diagnostics.
Check List (For Author)
Test
Behavior changed:
failure.
Does this need documentation?
introducing a new user-facing interface.
Validation
Local validation passed with JDK 17, Maven 3.9.9, and Thrift 0.24.0:
Total: 14 tests, 0 failures, 0 errors, and 0 skipped; 5 tests are new.
FE Checkstyle reported 0 violations. git diff --check passed.
No cluster regression test was run.
Check List (For Reviewer who merge this PR)