Add signal-decomposition degradation analysis - #518
Conversation
Ports the CVXPY seasonal-trend decomposition prototype into a new rdtools.signal_decomposition module and wires it end-to-end through TrendAnalysis. New module (rdtools/signal_decomposition.py): - Public degradation() entry point returning (Rd_pct, Rd_CI, sd_trend_results) - Three trend types: linear, pwl, monotone - Four loss functions: l2, l1, huber, quantile - Missing data handled natively via masked equality constraint - Stability analysis: analyze_fit_stability, get_valid_endpoints, plot_stability - Plotting/display: plot_decomposition, animate_degradation, format_degradation_report - log_transform path via extract_degradation_rate_log TrendAnalysis wiring (rdtools/analysis_chains.py): - _signal_decomposition_degradation private method - sensor_analysis / clearsky_analysis accept 'signal_decomposition' in analyses list and sd_kwargs passthrough - plot_signal_decomposition_summary method Dependencies: - cvxpy >= 1.7.5 and spcqe >= 0.3.0 added as hard runtime dependencies - scipy floor raised from 1.8.1 to 1.13.0 (required by cvxpy's transitive deps) - cvxpy and spcqe excluded from check_dependencies.py min-pin enforcement (their transitive scipy requirement is incompatible with pinning an exact lower-bound version alongside scipy==1.13.0 on Python 3.10) Tests and docs: - rdtools/test/signal_decomposition_test.py (29 tests) - Additions to analysis_chains_test.py (fixtures, plot tests, bad-case coverage) - api.rst updated with Signal Decomposition section and new TrendAnalysis method
|
Just finishing up examples and package dependency testing. Ready for general review. Open item: minimum scipy version support. |
…repeated calls to analysis_chains.TrendAnalysis.sensor_analysis
…r the end of the trend window.
analyze_fit_stability and animate_degradation both call make_problem(y_full[:n], **make_problem_kwargs). When callers pass sd_trend_results['args'] directly (which includes 'y' because _get_kwargs captures the full make_problem signature), this caused a "multiple values for argument 'y'" TypeError. Strip 'y' from the dict at the top of each function so passing args through is always safe.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## development #518 +/- ##
===============================================
- Coverage 96.79% 94.99% -1.81%
===============================================
Files 12 13 +1
Lines 2342 3437 +1095
===============================================
+ Hits 2267 3265 +998
- Misses 75 172 +97 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…or signal decomposition models
dict of CI arrays rather than a single [lo, hi] for the overall rate. degradation() merges these into sd_trend_results as ci_<rate_key> entries, giving ci_rate_pre/post_pct_yr for pwl and ci_rate_yearly_pct_yr (shape 2×n_years) for monotone. Rd_CI is unchanged.
- add validated convex soiling component and coherence selector - return a structural null when coherent soiling is not detected - report overall and quarterly losses and local soiling rates - bootstrap uncertainty using the selected terminal model - pass insolation through TrendAnalysis for weighted losses - add selector, metric, integration, and plotting tests - add a PVDAQ soiling comparison marimo notebook - ignore local agent and prototype artifacts
Refine positively detected soiling components with two deterministic cleaning-interval IRL1 solves to reduce L1 amplitude shrinkage. Apply the same refinement to bootstrap fits, preserve the structural-null path, expose refinement diagnostics, and document the intended dry-soiling scope and upstream outage requirement.
Drop Python 3.10 support and move the minimum test environment to Python 3.11. Raise dependency floors to a coherent stack that builds and passes the full test suite, including matching minimum pins for CVXPY and SPCQE. Regenerate the Pixi lockfile, pin the Pixi version used in CI, and update the supported-version documentation and release notes. Make the PVLib bootstrap regression test deterministic without leaking global RNG state.
Fully qualify signal decomposition function names so Sphinx does not resolve them relative to the module twice during autosummary generation.
Add a first-class signal decomposition guide covering the basic trend model, loss functions, expanded trend diagnostics, and SD++ soiling. Use log-transformed Huber loss with M=0.05 as the default SD model while preserving explicit overrides, and record the resolved transform in result metadata. Add reproducible PVDAQ decomposition and synthetic year-1 breakpoint figures, align the marimo example defaults, and test the new preset.
Test the animation workflow, soiling insolation validation, missing-quarter warnings, and the public trend plot. Use deterministic component doubles for animation rendering so the test exercises frame updates and writer selection without invoking an external encoder or duplicating solver tests. Plot normalized energy directly with Matplotlib to avoid incompatible datetime converters.
Document synthetic dry-soiling recovery with a reproducible PVDAQ figure and generated degradation and soiling report. Reduce the ordinary signal-decomposition seasonal default from six to three Fourier harmonic pairs, matching the SD++ preset while preserving explicit configuration. Regenerate the basic and year-1 breakpoint figures under the updated default and extend the preset test.
|
We updated the minimum supported environment to Python 3.11 and defined The newest package in this minimum environment is CVXPY 1.7.5, released December 5, 2025. As of August 25, 2026, the environment is therefore about 8 months and 20 days old. Most other minimum dependencies date from 2022–2024, with a few newer requirements introduced by the signal-decomposition stack. This gives us a realistic, reproducible minimum environment rather than a collection of independently old version floors that may not build or work together. |
|
I am currently working on raising the "age" of the minimum environment, which is currently about 8 months. The new target is 2 years. Cvxpy has been downgraded successfully, and I'm now testing additional binding packages. |
Lower the tested floors to CVXPY 1.5.3, pvlib 0.11.0, and SPCQE 0.0.3, then regenerate the Pixi lockfile. The newest signal-decomposition floor is SPCQE 0.0.3, released October 28, 2024—1 year, 9 months, and 28 days old as of August 25, 2026. The complete environment’s newest direct requirement remains bayesian-filters 1.4.5, released October 20, 2025. Confirm the complete minimum-version suite passes with 491 tests.
|
Follow-up: The minimum-version review is complete. We lowered the tested floors to CVXPY 1.5.3, pvlib 0.11.0, and SPCQE 0.0.3, and regenerated the Pixi lockfile. The complete The newest signal-decomposition dependency floor is now SPCQE 0.0.3, released October 28, 2024—1 year, 9 months, and 28 days old as of August 25, 2026. This is close to the two-year target while retaining the first stable SPCQE release; the older pre-cutoff release is an alpha with an incomplete public API. The complete environment still contains one newer direct requirement: |
|
Last remaining open item is to update the changelog, per the PR template |
|
I have no more planned commits for this PR. Ready for final review! |
Ports the CVXPY seasonal-trend decomposition prototype into a new
rdtools.signal_decompositionmodule and wires it end-to-end throughTrendAnalysis. The implementation now provides a unified framework for conventional degradation analysis, expanded lifecycle trend analysis, and SD++ soiling analysis.New module (
rdtools/signal_decomposition.py):degradation()entry point returning(Rd_pct, Rd_CI, sd_trend_results)plot_decomposition,plot_trend,plot_stability,animate_degradation, andformat_degradation_reportSD++ soiling:
TrendAnalysis wiring (
rdtools/analysis_chains.py):_signal_decomposition_degradationprivate integration methodsensor_analysisandclearsky_analysisacceptsignal_decompositionin theanalyseslist and pass configuration throughsd_kwargsplot_signal_decomposition_summarymethodDependencies and minimum environment:
check-depsverifies that all direct dependency pins match their declared lower boundstest-minbuilds the Python 3.11 lower-bound environment and runs the complete test suiteTests and documentation:
signal_decomposition_test.pycoverage for trend models, losses, bootstrap uncertainty, structural soiling selection, IRL1 refinement, null behavior, metrics, plots, and reportsanalysis_chains_test.pyapi.rstand the v3.3.0 changelogTodo list:
TrendAnalysis[ ] New functions added to__init__.pyThis is intentionally not being done. The function should be imported as
rdtools.signal_decomposition.degradationorfrom rdtools.signal_decomposition import degradationto avoid clashing with the existingrdtools.degradationmodule.api.rstand the other Sphinx documentation pages are up to date