CAS: parallel delete blobs - #2351
Conversation
Signed-off-by: Konstantin Morozov <just.morozov.k@gmail.com>
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8e0ec8de87
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| /// The entry left the pipeline — drop its in-process condemn-marker confirmation. | ||
| meta_writer->forgetCondemnMarker(entry.ref, entry.token); | ||
| } | ||
| redeleteBlobs(redelete_now, layout, op, new_round, generation, round_work_budget, report, outcomes[shard]); |
There was a problem hiding this comment.
When cas_gc_shards > 1, this invocation completes and waits for one shard before the loop reaches the next shard, so re-deletes from different shards never overlap. If each shard has fewer than cas_gc_redelete_min_batch_size pending entries, the entire phase remains sequential despite cas_gc_redelete_concurrency > 1; even larger per-shard batches cannot use more concurrency than one shard contains. Schedule the phase's eligible entries before a single wait so the configured fan-out applies across shards.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Default value gc_shards = 1 . Yes, in some cases we could obtain seq execution. But in these cases we don;t have a lot blobs. I suggest skip this impovment now.
| CasOperation job_op = store->openRequests().resume(gen); | ||
| io_results[i] = performRedeleteIo(entries[i], layout, job_op); |
There was a problem hiding this comment.
When parallel re-delete is enabled, performRedeleteIo executes the blob HEAD and conditional DELETE on these pool threads, while GcPhaseTimer snapshots only the round thread's ProfileEvents. As a result, the pending_deletes row in system.cas_gc_log omits the underlying request and retry counters precisely when cas_gc_redelete_concurrency > 1, breaking per-phase operation accounting; collect the worker deltas or expose equivalent explicit phase metrics.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
This is a regression only when the feature is enabled. It is a known limitation that read-ahead also has, and it will be fixed in a separate PR for both places at once.
1 participant
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Added cas_gc_redelete_concurrency to run blob deletes in the CAS GC pending_deletes phase in parallel.
Documentation entry for user-facing changes
...
CI/CD Options
Exclude tests:
Regression jobs to run: