analyze: replace --by-name by --group-by - #10301
Merged
Merged
Conversation
--by-name decomposed the repository by archive name, which is a grouping with the keys hardwired. Archive series names are not unique in a repository shared by multiple machines or users, so different hosts backing up a "home" series ended up in one row: its exclusive size was the combined cost of unrelated data sets and could not be attributed to either host. It also no longer matched what prune does. prune groups by name,host, and the number analyze reports per row - "what would deleting this free" - is only actionable if the row is the unit that prune applies its retention rules to. --by-name is replaced by --group-by KEYS, taking the same comma-separated archive attributes (name, host, user, tags) and using the same grouping as prune, so "--group-by name" is the previous behaviour. As before, giving the option selects the decomposition mode, and it can not be combined with archive filters, because the shared and unreferenced rows need the repository-wide view. Grouping by more attributes moves chunks from the group rows into the shared row: if two hosts back up identical content, --group-by name reports it as exclusive to that name, while --group-by name,host correctly shows that deleting either host's archives alone would not free it. The JSON output changes accordingly, without keeping compatibility: the by_name key becomes by_group, its names list becomes groups, and each entry has a group object (attribute -> value) instead of a name. The list of attributes grouped by is in the new group_by key.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #10301 +/- ##
=======================================
Coverage 87.65% 87.66%
=======================================
Files 103 103
Lines 18702 18717 +15
Branches 2877 2881 +4
=======================================
+ Hits 16393 16408 +15
+ Misses 1609 1607 -2
- Partials 700 702 +2 ☔ View full report in Codecov by Harness. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
--by-namewas already a grouping, with the keys hardwired toname:Two problems with that:
It is wrong in a shared repository. Archive series names are not unique across hosts, so two
machines backing up a
homeseries collapse into one row. That row's exclusive size is the combinedcost of two unrelated data sets, attributable to neither, and the report's own claim — "deleting all
archives of that name would free them" — silently becomes a statement about both hosts' data.
It no longer matches prune. analyze's headline number is "what would I free by dropping this
row", which is only actionable if the row is the unit prune actually operates on. Since #10291 that
unit is (name, host), while analyze's was (name) — they stopped lining up in exactly the repositories
where the question is worth asking.
The change
--by-nameis replaced by--group-by KEYS, taking the same comma-separated archive attributes asprune (
name,host,user,tags) and the samearchive_group_key().--group-by nameis theprevious behaviour. As before, giving the option selects the decomposition mode and it cannot be
combined with archive filters, since the shared and unreferenced rows need the repository-wide view.
An empty
--group-by ""is rejected: for prune "one group" is meaningful, here it would be a singlerow covering the whole repository, which says nothing.
There is a second, less obvious payoff. Grouping by more attributes moves chunks from the group rows
into the shared row, where they belong: if two hosts back up identical content,
--group-by namereports it as exclusive to that name, while
--group-by name,hostcorrectly shows that deletingeither host's archives alone would not free it. The old report overstated per-name reclaimable space.
Text report row labels change from the bare name to the formatted group key, the same one prune and
repo-list print:
becomes
JSON
Changed without keeping compatibility, as requested:
by_name→by_groupnameslist →groupsnamestring → agroupobject mapping each grouped-by attribute to this group'svalue, e.g.
{"name": "home", "host": "host1"}group_bykey listing the attributes that were grouped bydocs/internals/frontends.rstis updated to match.MAX_NAMESbecameMAX_GROUPS(same 2^18−2limit, which now bounds distinct groups — more attributes mean more groups, so the guard earns its
keep).
Tests
test_analyze_group_by_name— the old--by-nametest, now asserting the group-labelled rowstest_analyze_group_by_name_and_host— new: both hosts use ahomeseries;--group-by nameshows one group with everything exclusive to it,
--group-by name,hostshows two groups with thecommon file correctly moved into the shared row
test_analyze_group_by_rejects_filters,test_analyze_group_by_needs_a_keytest_analyze_json_group_by— the new JSON shape, includinggroup_byand thegroupobjectsFull test suite: 2935 passed, 986 skipped.
ruff checkclean.Worth a CHANGES entry as a breaking change, since
--by-nameshipped in 2.0.0b23 and the JSON keychanges with it: