release: 22.6.0 — ENTSO-E gap repair + resume/cooldown fixes#383
Merged
Conversation
… country_code DE
Reconciles the long-lived fix/entsoe-gap-repair work onto current develop
(develop had independently reworked download_new_data, so this is a re-apply,
not a mechanical rebase).
Field incident: ENTSO-E published no data for 2026-06-01/02, leaving an interior
hole in interim/energy_load.csv that the pipeline could neither detect nor heal.
New API (spotforecast2_safe.downloader):
- repair_data_gaps(): detect interior gaps, heal from already-downloaded raw
CSVs first, then issue targeted downloads for intervals still missing; never
invents values, raises by default (on_unavailable='use_existing' opts into
proceeding with gapped data).
- find_missing_intervals(): report interior gaps of a DatetimeIndex (mode-based
step, so hourly and 15-min data both work).
- download_new_data(..., on_unavailable=...): opt-in to keep operating on the
existing interim data when ENTSO-E stays unreachable after the retry budget.
Fixes in download_new_data() (kept develop's _MAX_BACKFILL_DAYS heal + timeout):
- Resume (start=None) now reads the interim file via fetch_data(filename=...).
The bare fetch_data() call raised ValueError ("filename must be specified")
unconditionally, so resume silently fell into the 7-day fallback on every
incremental run -- which also made develop's backfill heal dead code. Now both
work.
- Cooldown is keyed on the recency of the newest raw entsoe_load_* file, not the
WIDTH of the requested window (the old check silently skipped sub-24h
backfills). A window overlapping a known interim gap bypasses the cooldown.
- end=None now means "now" instead of "today 00:00 UTC" (the latter made
incremental runs no-ops for most of the day once resume works).
- end <= start with both bounds explicit raises ValueError; a derived empty
window is an "already up to date" no-op.
- An empty/all-NaN ENTSO-E response is logged at WARNING and writes no raw file
instead of masking the gap as a successful download.
- default country_code "FR" -> "DE", matching download_renewable_forecast (DE)
/ download_day_ahead_price (DE_LU) and the package's German-energy focus.
Tests: new tests/test_entsoe_gap_repair.py (375 lines); the two width-semantics
cooldown tests updated to the recency model. quartodoc reference + sidebar
regenerated for the two new public functions. Full suite 2665 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ntsoe-gap-repair feat(downloader): ENTSO-E gap repair + resume/cooldown fixes (country_code default DE)
## [22.6.0-rc.1](v22.5.0...v22.6.0-rc.1) (2026-06-13) ### Features * **downloader:** ENTSO-E gap repair + resume/cooldown fixes; default country_code DE ([848027f](848027f))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promote
develop→mainto release 22.6.0.Since 22.5.0 (#382):
repair_data_gaps(),find_missing_intervals(), anddownload_new_data(on_unavailable=...).download_new_data: resume now actually reads the interim file (the barefetch_data()raised unconditionally → silent 7-day fallback every run, also making develop's backfill heal dead code); recency-based cooldown (was width-based, silently skipped sub-24h backfills) with gap-overlap bypass;end=None→ "now";end<=startguard; empty/all-NaN response no longer masked as a successful download.country_code"FR"→"DE"(consistency with sibling downloaders + package focus).Full suite 2665 passed; ruff clean; quartodoc reference in sync.
🤖 Generated with Claude Code