Conversation
A failed delete leaves the record eligible, so the same batch came back on the next iteration while the recorded size never dropped. Against a backend refusing deletes the pass retried forever, burning CPU and flooding the log. A canceled context made it faster: every delete fails at once, so the loop spun as quickly as the database could answer. The pass now ends when a batch clears no record, and returns as soon as the context is canceled. Progress counts records rather than bytes, because a cached record can have no size recorded. A pass can therefore end with the cache still above its limit, leaving the records that failed for the next pass to retry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes git-pkgs#341.
A failed delete leaves the record eligible, so the same batch comes back on the next iteration while the recorded size never drops. Against a backend refusing deletes the pass retries forever, burning CPU and flooding the log. A canceled context makes it faster, since every delete fails at once.
The pass now ends when a batch clears no record, and returns as soon as the context is canceled. Progress counts records rather than bytes, because a cached record can have no size recorded. A pass can therefore end with the cache still above its limit, leaving the failed records for the next pass.
Two disclosures. The zero-size test the issue asks for pins the behaviour but is not load-bearing: its sizeless record shares a batch with a sized one, so a bytes-based rule would still pass it. Separately, the storage interface documents Delete as returning nil for a missing path; the file backend honours that but GCS returns the raw error, so there a record whose bytes are already gone can become a permanently unevictable LRU head. That looks worth its own fix rather than widening this one.
CI here fails on the pre-existing handler test build, which git-pkgs#362 fixes. This same commit stacked on that fix is green across all six jobs: https://github.com/montehurd/proxy/actions/runs/35656065103