Log and count protected upsert revert failures

[KKcorps]

PR flow

Add logging and metering for protected upsert revert failures in base and concurrent map handlers, preserving existing exception propagation or key removal.

flowchart TD
  N0["Base class#58; catch RuntimeException in removeSegment #40;F2#41;"]:::stModified
  N1["ConcurrentMap#58; catch Exception in revertAndRemoveSegment #40;F3#41;"]:::stModified
  N2["Log #39;UPSERT#95;METADATA#95;REVERT#95;FAILED#39; and exception #40;F2#44; F3#41;"]:::stModified
  N3["Increment UPSERT#95;METADATA#95;REVERT#95;FAILURES meter #40;F2#44; F3#41;"]:::stModified
  N4["Rethrow exception #40;F2#41;"]:::stModified
  N5["Return null #40;F3#41;"]:::stModified
  N0 -->|"logs failure"| N2
  N1 -->|"logs failure"| N2
  N2 -->|"increments meter"| N3
  N3 -->|"propagates exception"| N4
  N3 -->|"returns null for fallback"| N5
  classDef stAdded fill:#dafbe1,stroke:#1a7f37,color:#1f2328,stroke-width:2px
  classDef stModified fill:#fff8c5,stroke:#9a6700,color:#1f2328,stroke-width:2px
  classDef stRemoved fill:#ffebe9,stroke:#cf222e,color:#1f2328,stroke-width:2px
  classDef stUnchanged fill:#f6f8fa,stroke:#656d76,color:#1f2328,stroke-width:1px

Loading

AI-generated · Green: added · Yellow: modified · Red: removed · Gray: existing

Diff evidence

• F2: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java — before · after
• F3: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java — before · after

• Regenerate PR flow

Protected upsert revert can lose a previous key location while the existing ConcurrentMap
reader/bitmap fallback still completes normally. Add UPSERT_METADATA_REVERT_FAILURES and the
UPSERT_METADATA_REVERT_FAILED log marker to those handled fallbacks and to exceptions already
escaping the protected revert operation.

Preserve existing OSS behavior: handled fallbacks still remove the key and return normally;
unhandled runtime exceptions still propagate unchanged. No new segment-error-cache writes,
Helix state changes, query fences, or ingestion gates are added. doOffload() is unchanged.

The existing eligibility check requires PROTECTED mode, a mutable segment, and a table
configuration needing revert (partial upsert, dropOutOfOrderRecord, or an out-of-order record
column). Ordinary removal paths are unchanged. The meter counts failure events, not distinct
segments; one segment can have several key fallbacks. Successful reverts do not increment it.

The signals identify suspect metadata; they do not repair it. Manual reconstruction and replay
remain the recovery procedure, including later sequences in the affected partition for pauseless
partial upsert. The operator runbook is maintained separately from this PR.

Validation: 64 OSS tests passed: ConcurrentMapPartitionUpsertMetadataManagerTest (34) and
RealtimeSegmentDataManagerTest (30). Five added cases cover protected removal/replacement reporting,
a non-protected control, and the handled reader/bitmap fallbacks. Existing exception objects and
key-removal behavior are preserved without segment-error-cache writes. No cluster recovery test or
benchmark was run. Spotless, Checkstyle, license checks and git diff --check passed.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.75%. Comparing base (7f024b6) to head (8de0e54).
⚠️ Report is 10 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #19505      +/-   ##
============================================
+ Coverage     67.73%   67.75%   +0.01%     
  Complexity     1430     1430              
============================================
  Files          3489     3489              
  Lines        224637   224682      +45     
  Branches      35468    35468              
============================================
+ Hits         152160   152227      +67     
  Misses        60450    60450              
+ Partials      12027    12005      -22     

Flag	Coverage Δ
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-25	67.75% <100.00%> (+0.01%)	⬆️
lane-a	100.00% <ø> (ø)
lane-b	0.00% <ø> (ø)
temurin	67.75% <100.00%> (+0.01%)	⬆️
unittests	67.74% <100.00%> (+0.01%)	⬆️
unittests1	57.82% <7.14%> (+0.03%)	⬆️
unittests2	39.44% <100.00%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.