[Mike Tokic (mitokic)]

This pull request improves support for custom hierarchical reconciliation with Spark (lazy table) backends in the finnts package, ensuring that custom coverage validation, error handling, and output are consistent and robust across both local and Spark workflows. It introduces more comprehensive validation for custom candidate coverage, adds detailed grouped statistics for lazy tables, and enhances the test suite to cover Spark-specific behaviors.

Custom Hierarchical Reconciliation Improvements:

• Enhanced the custom_reconciliation_coverage function to support and validate Spark lazy tables by collecting distinct identities and grouped statistics, rather than loading all forecast rows into memory. This ensures robust coverage checks for custom candidates and prevents incomplete or invalid data from proceeding. [1] [2]
• Updated documentation in hierarchy.R to clarify that Spark forecast tables remain lazy during preflight, and grouped statistics are used for validation.

Error Handling and Output Consistency:

• Improved error handling in reconcile_hierarchical_data to ensure that errors in custom reconciliation with Spark are properly surfaced and that model names are restored from the custom run manifest. [1] [2] [3]

Testing Enhancements:

• Added a hierarchy_lazy_collector test helper to simulate Spark lazy table collection, enabling precise control and validation of grouped queries in tests.
• Expanded test coverage to verify that custom Spark reconciliation validates lazy coverage, rejects incomplete or invalid candidates, and floors near-zero forecasts consistently on both local and Spark routes. [1] [2]

Documentation and Versioning:

• Bumped the package version to 0.7.0.9019 and updated NEWS.md to reflect the new development version. [1] [2]

These changes collectively ensure that custom hierarchical reconciliation is robust, memory-efficient, and consistent across execution environments, with comprehensive validation and strong test coverage.

[Copilot]

🟡 Changes recommended

The test collector always reports zero invalid-date groups, leaving date-membership validation untested and requiring out-of-calendar and missing-Date rejection cases before writes, alongside a valid multi-date split.

1 open issue

• The collector always returns zero invalid-date groups, regardless of the fixture or generated… · New

What changed in this PR

This PR extends custom hierarchical reconciliation to Spark lazy tables, using grouped validation instead of collecting full forecast data into memory.

Changes:

• Adds lazy-table coverage checks using distinct identities and grouped statistics.
• Propagates custom reconciliation errors and restores readable model names in Spark output.
• Expands local/Spark regression tests and advances the development version to 0.7.0.9019.

File	Description
tests/​testthat/​test-hierarchical.R	Adds lazy Spark coverage to shared reconciliation route tests.
tests/​testthat/​test-custom-model-hierarchy.R	Adds a lazy collector helper, coverage checks, and near-zero forecast tests.
R/​hierarchy.R	Implements lazy preflight validation, error propagation, and model-name restoration.
NEWS.md	Advances the existing development-version heading.
DESCRIPTION	Updates the package version to 0.7.0.9019.

_{Review effort: Balanced}

[smdp2000ofc]

ghcp flagged some runtime multiple rescans. please have a look