Skip to content

[improvement](be) Apply trash sweep policy to shutdown tablets - #66395

Draft
wenzhenghu wants to merge 2 commits into
apache:masterfrom
wenzhenghu:wzh/master-shutdown-tablet-direct-delete
Draft

[improvement](be) Apply trash sweep policy to shutdown tablets#66395
wenzhenghu wants to merge 2 commits into
apache:masterfrom
wenzhenghu:wzh/master-shutdown-tablet-direct-delete

Conversation

@wenzhenghu

@wenzhenghu wenzhenghu commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Proposed changes

Issue Number: None

Related PR: HYDCP#75

Shutdown tablets are processed after the DataDir trash sweep. Previously, even when a manual cleanup or high disk usage made the current sweep delete trash immediately, newly resolved shutdown tablet paths could still be moved back into trash according to the global retention configuration. This delayed disk-space reclamation and recreated trash entries during an urgent cleanup.

This PR:

  • builds one immutable sweep policy per DataDir and shares it between trash cleanup and shutdown-tablet resolution;
  • deletes eligible shutdown tablet paths directly when trash retention is disabled, during manual cleanup, or above the existing high-watermark threshold;
  • preserves move-to-trash behavior for normal retention sweeps;
  • defers shutdown tablets on DataDirs that failed their health checks, avoiding further filesystem I/O against a broken path;
  • reloads and processes deferred shutdown tablets after an operator restores the path, clears its persisted broken state, and restarts the BE; permanently retired paths remain owned by the existing disk-retirement workflow;
  • keeps the existing transition, active-tablet, path, UID, TabletMeta, failure requeue, reference-count, and resolved-budget semantics;
  • adds direct-delete result, timing, and reason metrics plus an unused-DataDir deferred gauge;
  • adds BE unit coverage for the policy matrix, explicit filesystem modes, mixed DataDir policies, direct-delete failure retry, path-success/meta-failure retry, and intentional-skip metric semantics.

No FE-BE protocol, persistent format, or new configuration is introduced.

Release note

Shutdown tablet paths now follow the current DataDir trash sweep policy. During immediate cleanup conditions they are deleted directly instead of being moved into trash again.

Check List

  • Formatting: ./build-support/clang-format.sh and ./build-support/check-format.sh passed
  • Static checks: target-file clang-format 16 dry-run and git diff --check passed
  • BE build: not completed; the local build was interrupted during third-party dependency preparation before source compilation
  • BE unit tests: coverage added but not executed per the requested validation scope
  • Regression tests: not run
  • Behavior changed: Yes. Immediate-cleanup conditions directly delete eligible shutdown tablet paths; shutdown tablets on runtime-broken DataDirs are deferred to avoid further filesystem I/O until operator recovery and BE restart
  • Documentation required: No

### What problem does this PR solve?

Issue Number: None

Related PR: HYDCP#75

Problem Summary: Shutdown tablets were moved into trash even when the current sweep was deleting trash immediately for manual cleanup, disabled retention, or high disk usage. This delayed disk-space reclamation and could recreate trash entries during an urgent cleanup. Build one immutable policy per DataDir and use it for both trash expiration and shutdown-tablet path resolution. Eligible shutdown tablet paths are deleted directly during immediate cleanup conditions and continue to move to trash during normal retention sweeps. Shutdown tablets on unused DataDirs remain deferred, and existing transition, UID, path, metadata, reference-count, and failure-requeue safeguards are preserved.

### Release note

Shutdown tablet paths now follow the current per-DataDir trash sweep policy. During immediate cleanup conditions they are deleted directly instead of being moved into trash again.

### Check List (For Author)

- Test: Unit Test coverage added but not executed per requested validation scope
    - Static checks: clang-format 16, build-support/check-format.sh, and git diff --check passed
    - BE build: Not completed; interrupted during third-party dependency preparation before source compilation
- Behavior changed: Yes. Shutdown tablet paths may be deleted directly during immediate trash cleanup conditions.
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@wenzhenghu
wenzhenghu marked this pull request as draft August 3, 2026 14:05
### What problem does this PR solve?

Issue Number: None

Related PR: HYDCP#75

Problem Summary: Clarify that shutdown tablets on a DataDir which failed its health check are deferred until the operator restores the path, clears its broken state, and restarts the BE. Remove an unused trash-expiration assignment and document the idempotent parent-directory check used by GC retries.

### Release note

None

### Check List (For Author)

- Test: No need to test (static-only review follow-up; formatting and diff checks passed)
- Behavior changed: No
- Does this need documentation: No
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants