feat(downloader): add download-resilience module (resilience.py)#387
Merged
Conversation
…2.8.0) Port the download-resilience decision tree from bart26l-vorlesung /_team4_resilience.py into src/spotforecast2_safe/downloader/resilience.py with logic unchanged. Public names: download_with_fallback, DownloadResult, ZoneOutcome, make_store, ZONE_COLUMNS, SNAPSHOT_TTL (and private _ZONE_AREAS, _fmt_age). All sf2-safe imports remain lazy inside functions so monkeypatching works in tests and configure_environment() runs before any get_data_home() call. Add top-level re-exports of download_with_fallback, DownloadResult, ZoneOutcome in spotforecast2_safe.__init__. Expose resilience as a submodule in spotforecast2_safe.downloader.__init__. Add 18 scenario + unit tests in tests/downloader/test_resilience.py covering: all-four-live, per-zone snapshot fallback (age, gate flag, WARNING log), multi-zone cache, combined fallback (live / cache / missing), expired-TTL handling, --no-fallback skips reads but still writes, _fmt_age, report(), make_store, and public API surface. All 2709 tests pass. Bump version 22.7.0 -> 22.8.0 in pyproject.toml; update CHANGELOG.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the new public downloader.resilience module into the quartodoc API
reference and harden its docstrings to the sf2-safe standard.
quartodoc / _quarto.yml:
* Add download_with_fallback, DownloadResult, ZoneOutcome, make_store to
the Downloader quartodoc.sections contents and matching website.sidebar
entries. Closes the doc-coverage gap for the 22.8.0 public API.
Docstrings (src/spotforecast2_safe/downloader/resilience.py):
* make_store: add a Returns section.
* download_with_fallback: add a non-executed Quarto Examples block
(#| eval: false), matching the network-dependent sibling pattern in
downloader.entsoe (download_zone_loads / ZoneResult).
* Convert the module-level docstring from RST (:: literal blocks, ----
underlines) to the project's Google-style prose so griffe parses it.
Generated stubs (docs/reference/*.qmd, index.qmd, _freeze/) regenerated via
docs/quartodoc_build.py. DownloadResult.report / fallback_gate are embedded
on the class page (no separate listing, matching repo convention).
The existing tests/downloader/test_resilience.py already mirrors the
download_with_fallback / make_store example logic; all 18 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ol-managed) semantic-release owns pyproject version and CHANGELOG.md; the feat commit drives the 22.8.0 minor bump on release. Keep the branch at 22.7.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…E table, retry tests
1. Remove DownloadResult/ZoneOutcome/download_with_fallback re-exports
from spotforecast2_safe/__init__.py and its __all__. Canonical consumer
path is `from spotforecast2_safe.downloader import resilience as resil`.
2. Add STRIDE threat-model table to resilience.py module docstring covering
two data flows: (a) outbound ENTSO-E HTTPS delegated to entsoe.py with
countermeasures noted there; (b) SnapshotStore.restore filesystem trust
boundary with TTL + atomic-write countermeasures and operator permission
note.
3. Remove dead `from pathlib import Path` import (Path not used at module
level); ruff check is now clean.
4. Change module logger from getLogger("team4_resilience") to
getLogger(__name__) = "spotforecast2_safe.downloader.resilience";
update caplog logger name in test_tennet_fails_valid_snapshot.
5. Add two retry/backoff tests (max_retries=2):
- test_retry_zone_succeeds_on_second_attempt: zone fails attempt 1,
succeeds attempt 2; asserts status "live" and sleep called once with
backoff=5.0 (injected sleep=list.append).
- test_retry_only_retries_failed_zones: tracks zones arg per call;
first call requests all four, second call requests only the still-
failing zone; zone ends "cache" via snapshot fallback.
Suite: 2710 passed, 1 skipped. ruff clean. REUSE compliant (725/725).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…nce tests Reference ZoneResult via the entsoe module alias instead of a parallel 'from ... import ZoneResult', so the module is imported a single way. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Adds
spotforecast2_safe.downloader.resilienceas a first-class public module: the per-zone ENTSO-E download → snapshot-fallback decision tree, previously living as_team4_resilience.pyin thebart26l-vorlesunglecture scripts. Upstreaming it lets the lecture scripts and a new standalone book import one canonical, tested, documented implementation instead of duplicating the file.How it works
download_with_fallback(...)orchestrates existing sf2-safe primitives (downloader.entsoe.download_zone_loads/download_new_data,utils.snapshot_store.SnapshotStore): per-zone collect-mode retry with exponential backoff → per-zone snapshot fallback within TTL → combined DE-total fallback →modedecision (four_zone/combined/None). Returns aDownloadResult(ZoneOutcomeper zone); never raises on download failure. Logic is byte-faithful to the original.download_with_fallback,DownloadResult,ZoneOutcome,make_store,ZONE_COLUMNS,SNAPSHOT_TTL(canonical pathfrom spotforecast2_safe.downloader import resilience as resil; no package-root re-export, so__all__is unchanged).Safety / compliance
entsoe.py) and theSnapshotStore.restorefilesystem trust boundary (Tampering → TTL bound + atomic writes).Tests & docs
tests/downloader/test_resilience.py(scenario matrix + retry/backoff +DownloadResult.report+ API surface). Full suite: 2710 passed, 1 skipped.ruffclean;reuse lintcompliant.No breaking changes.
feat→ minor bump (22.8.0) via semantic-release.🤖 Generated with Claude Code