Skip to content

feat(cluster): original-vs-clustered comparison accessors (fixes #728)#734

Merged
FBumann merged 1 commit into
mainfrom
docs/clustering-compare-v7
Jul 21, 2026
Merged

feat(cluster): original-vs-clustered comparison accessors (fixes #728)#734
FBumann merged 1 commit into
mainfrom
docs/clustering-compare-v7

Conversation

@FBumann

@FBumann FBumann commented Jul 20, 2026

Copy link
Copy Markdown
Member

Motivation

Discussion #728 asks how to compare original vs clustered profiles now that fs_clustered.clustering.plot.compare() was removed in v7 (#655). The v7 migration guide pointed at aggregation_result, but using it directly exposes a papercut: flixopt renames period_period before calling tsam_xarray, and aggregation_result returns that raw result, so users must .sel(_period=...).

Changes

New convenience accessors on Clustering (pre-serialization):

  • compare(variable=None) → tidy xr.Dataset(original, clustered) on the original time axis, plot-ready — no reshaping.
  • original / reconstructed / residuals / accuracy.

These un-rename the internal dims (_periodperiod) and align original/reconstructed dim order, so selection uses natural coord names. Works across variables × periods × scenarios.

Plotting is left to the user. compare() returns tidy data, so a chart is a one-liner:

px.line(clustering.compare(var).to_dataframe()[['original', 'clustered']])

No plot.compare() method is added — it would be maintained plotting surface (styling, kind=, plotly dep) duplicating what one line of the user's own library does. The tidy-data helper this builds on is now upstream in tsam_xarray (#92, completed).

Docs:

  • migration-guide-v7.md — compare recipe (timeseries + duration curve) + multi-variable/period/scenario notes.
  • clustering.md — replaced the stale "Clustering Quality Metrics" section (documented the removed clustering.metrics) with clustering.accuracy / compare().
  • 08c-clustering notebook — demo cell plotting compare() with plotly express.
  • Fixed broken tsam_xarray repo links (FZJ-IEK3-VSAFBumann).

Tests

TestClusteringCompare: dim alignment, residuals identity, tidy compare(), plot-ready columns, accuracy exposure, period (not _period) naming, pre-serialization guards. All green; ruff clean; mkdocs build --strict passes.

The wider tests/test_clustering/ suite has pre-existing failures on main in the dev env from the linopy v1 Constraint.lhs deprecation — unrelated (no modeling files touched).

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added clustering accessors for original, reconstructed, residuals, and accuracy metrics.
    • Added compare() to generate tidy original-versus-clustered datasets, with optional variable selection.
    • Preserves natural time-axis alignment and dimensions for analysis/plotting.
  • Documentation

    • Updated the v7 migration guide and clustering docs to use the new quality/access API and compare() workflow.
    • Clearly documented that these results are pre-serialization only and must be regenerated after loading/saving.
    • Refreshed notebook examples and updated reference links.
  • Tests

    • Added integration coverage for the new accessors, compare(), and serialization-gated behavior.

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@FBumann, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 36 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 788bd301-af7f-4f36-828d-86ce2c4fccac

📥 Commits

Reviewing files that changed from the base of the PR and between 2efb3bd and 1d4eb31.

📒 Files selected for processing (6)
  • docs/notebooks/08c-clustering.ipynb
  • docs/user-guide/migration-guide-v7.md
  • docs/user-guide/optimization/clustering.md
  • flixopt/clustering/base.py
  • flixopt/plot_result.py
  • tests/test_clustering/test_integration.py
📝 Walkthrough

Walkthrough

Changes

Clustering comparison API

Layer / File(s) Summary
Full-data comparison accessors
flixopt/clustering/base.py
Adds original, reconstructed, residuals, accuracy, and compare() over full aggregation results, including dimension remapping and variable filtering.
Comparison behavior validation
tests/test_clustering/test_integration.py, flixopt/plot_result.py
Tests dimensions, residuals, accuracy metrics, period selection, serialization-state validation, and plotting data compatibility; updates a plotting example.
Migration and usage documentation
docs/user-guide/migration-guide-v7.md, docs/user-guide/optimization/clustering.md, docs/notebooks/08c-clustering.ipynb
Documents the new APIs, multidimensional usage, raw aggregation access, upstream links, notebook examples, and pre-serialization restrictions.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant Clustering
  participant AggregationResult
  participant xrDataset
  User->>Clustering: call compare(variable)
  Clustering->>AggregationResult: retrieve full aggregation data
  AggregationResult-->>Clustering: return original and reconstructed arrays
  Clustering->>xrDataset: build original and clustered variables
  xrDataset-->>User: return comparison dataset
Loading

Possibly related PRs

  • flixOpt/flixopt#554: Introduces the clustering and aggregation structure extended by these accessors.
  • flixOpt/flixopt#555: Adds the core clustering/aggregation framework used by the comparison API.
  • flixOpt/flixopt#571: Refactors the clustering result paths extended by the new accessors.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the new original-vs-clustered clustering accessors.
Description check ✅ Passed The description is detailed and covers motivation, changes, docs, and tests, though it doesn't follow the template headings exactly.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/clustering-compare-v7

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/user-guide/migration-guide-v7.md`:
- Around line 93-96: The migration guide should describe
fs_clustered.clustering.plot.compare() as deprecated but still temporarily
supported with a deprecation warning, and recommend clustering.compare() for new
code. In docs/user-guide/optimization/clustering.md lines 154-156, replace
“removed” with “deprecated” and explicitly mention backward compatibility.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f95e344a-1fcc-4b06-b572-baf5ef3518ac

📥 Commits

Reviewing files that changed from the base of the PR and between 3cd03a5 and a7191f4.

📒 Files selected for processing (5)
  • docs/user-guide/migration-guide-v7.md
  • docs/user-guide/optimization/clustering.md
  • flixopt/clustering/base.py
  • flixopt/plot_result.py
  • tests/test_clustering/test_integration.py

Comment thread docs/user-guide/migration-guide-v7.md Outdated
@FBumann
FBumann force-pushed the docs/clustering-compare-v7 branch 2 times, most recently from c9707d4 to f797738 Compare July 20, 2026 12:49
@FBumann
FBumann force-pushed the docs/clustering-compare-v7 branch from f797738 to 0ccc76d Compare July 20, 2026 19:38

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/notebooks/08c-clustering.ipynb`:
- Around line 149-161: Update the notebook snippet import to include
ExtremeConfig alongside ClusterConfig, so the existing ExtremeConfig usage in
the clustering example is defined when copied and executed.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 44bb2bdc-81ee-473e-baeb-e2d668508ae3

📥 Commits

Reviewing files that changed from the base of the PR and between a7191f4 and 0ccc76d.

📒 Files selected for processing (6)
  • docs/notebooks/08c-clustering.ipynb
  • docs/user-guide/migration-guide-v7.md
  • docs/user-guide/optimization/clustering.md
  • flixopt/clustering/base.py
  • flixopt/plot_result.py
  • tests/test_clustering/test_integration.py
🚧 Files skipped from review as they are similar to previous changes (4)
  • flixopt/plot_result.py
  • docs/user-guide/optimization/clustering.md
  • tests/test_clustering/test_integration.py
  • flixopt/clustering/base.py

Comment thread docs/notebooks/08c-clustering.ipynb
@FBumann
FBumann force-pushed the docs/clustering-compare-v7 branch 3 times, most recently from 81e6b11 to b9f3fa5 Compare July 20, 2026 20:48

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/user-guide/migration-guide-v7.md`:
- Around line 142-143: Update the migration guide example around
clustering.compare(var) to define var before it is used, or replace it with a
representative variable-name string consistent with the earlier example, so the
snippet runs without a NameError.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2516c709-7ddb-44d0-a990-844200016a5c

📥 Commits

Reviewing files that changed from the base of the PR and between 0ccc76d and b9f3fa5.

📒 Files selected for processing (6)
  • docs/notebooks/08c-clustering.ipynb
  • docs/user-guide/migration-guide-v7.md
  • docs/user-guide/optimization/clustering.md
  • flixopt/clustering/base.py
  • flixopt/plot_result.py
  • tests/test_clustering/test_integration.py
🚧 Files skipped from review as they are similar to previous changes (3)
  • docs/user-guide/optimization/clustering.md
  • flixopt/plot_result.py
  • docs/notebooks/08c-clustering.ipynb

Comment thread docs/user-guide/migration-guide-v7.md Outdated
@FBumann
FBumann force-pushed the docs/clustering-compare-v7 branch from b9f3fa5 to 2efb3bd Compare July 21, 2026 06:20
Restore a first-class way to compare original vs clustered profiles, which
regressed when clustering.plot.compare() was removed in v7 (#655). Reported in
discussion #728.

New accessors on Clustering (pre-serialization):
- compare(variable=None) -> tidy Dataset(original, clustered), plot-ready
- original / reconstructed / residuals / accuracy

These un-rename tsam_xarray's internal dims (_period -> period) and align
original/reconstructed dim order, so users select with the natural coord names
instead of the raw aggregation_result's _period.

Plotting is left to the user: compare() returns tidy data, so a chart is a
one-liner (px.line(compare(var).to_dataframe()[['original','clustered']])).
The tidy data helper is now upstream in tsam_xarray (FBumann/tsam_xarray#92).

Docs:
- migration-guide-v7: compare recipe (timeseries + duration curve) and
  multi-variable/period/scenario notes
- clustering.md: replace stale clustering.metrics section (removed in v7) with
  clustering.accuracy / compare()
- 08c-clustering notebook: demo cell plotting compare() with plotly express
- fix broken tsam_xarray repo links (FZJ-IEK3-VSA -> FBumann)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@FBumann
FBumann force-pushed the docs/clustering-compare-v7 branch from 2efb3bd to 1d4eb31 Compare July 21, 2026 06:44
@FBumann
FBumann merged commit 82f2be1 into main Jul 21, 2026
10 of 11 checks passed
@FBumann FBumann mentioned this pull request Jul 21, 2026
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant