Add typed historical market and operational data

[kvkenyon]

Adds typed historical access to the Public Reports client merged in #7, including archives older than the row-query API and publications available only in monthly bundles. The client now has 289 historical table readers across 113 products, covering all 242 report-query tables plus additional CSV, workbook, and supported PDF tables.

Data access

• Complete retained wind backfill: all 21,676,139 report rows from 108,275 original publications match independent CSV decoding across five layouts. Separate original-file accounting and a fresh bounded archive-listing check also pass. The source period fields span April 2014–September 2026, including overlapping and future forecast periods; this does not claim continuous actual observations. No wind decoder changes were needed.

• Public LMP snapshots preserve all settlement-point prices when ERCOT displays - for an unavailable five-minute LMP change. lmpChange is now Decimal | None; complete-source regression coverage checks all 1,123 points, including the three missing changes.

• Demand and combined wind/solar dashboards remain usable before day-ahead publication: eight not-yet-published fields are typed as nullable decimals. Captured original responses preserve available values and distinguish missing forecasts from observed zero generation.

• Five anonymous market-display methods add dated DAM/RT hub-load-zone prices, DAM ancillary prices, and actual forecast/weather-zone load. Finite series keys and decimal values are typed; original DST period labels are preserved. Live comparisons cover 6,268 values across 13 tables plus two unavailable/wrong-day DAM responses rejected explicitly. Public URL retention is sampled, not claimed continuous.

• monthly_forecast_performance reads all 15 linked monthly Forecast/Backcast workbooks. All 3,330 monthly records (3,255 numeric values) match the original cells, and fresh downloads match every fixture hash. Raw scales and original dates remain available; explicit percentage formatting supports a typed percent conversion. Forecast errors, backcasts and Goal/Stretch targets are distinct.

• load_forecast_performance adds typed hourly actual/selected/model-code series and separate error summaries from all 48 linked metric workbooks for ERCOT and the eight weather zones. Complete source-cell comparisons cover 316,123 hourly/calculation records, 21,150 summaries and 4,412,079 numeric values. Missing actuals, malformed Hour numbers and spreadsheet-error markers remain explicit; these files do not establish forecast issuance times.

• ESRClient adds the separate ESR service using ERCOT_ESR_SUBSCRIPTION_KEY: one generated four-second query with sync/async pagination and a typed archive reader. It shares the existing transport and history implementation without new dependencies. All 13 filters were tested live; 225 rows across oldest/middle/newest complete archives match original cells, paginated queries, downloads and publication-aware reads. The service currently returns May–December 2025 data, so this is not a current battery telemetry feed.

• backfill() combines monthly bundles and remaining archive documents. Original document IDs avoid repeated downloads; distinct corrections and repeated source rows remain separate. Unbounded selection includes bundle-only history and returns bundle rows before requesting archive listings; uncovered archive IDs are streamed afterward.

• publications() retains original posting metadata beside typed rows for forecast vintages and delayed disclosures. Publication bounds and delivery dates remain separate.

• Generated readers preserve historical fields, missing values, DST flags, nested ZIP members, and changing tables in shared disclosure reports. Unsupported selected data raises instead of disappearing from the result.

• Direct public readers cover load, fuel mix, retail profiles, settlement loss factors, 4CP, CRR/POLR data, resource outlooks, and capacity changes. Typed public dashboards expose prices, demand, generation, battery/system capacity, and reserve conditions.

• generation_profiles reads modeled wind/solar planning series with typed hourly MW values, embedded metadata and separately published site-key tables. Dates, source clocks and repeated hours are preserved. Older side-by-side wind tables retain independent dates and a source-block identifier; hypothetical-solar tracking metadata is typed. These are retrospective scenarios, not observed generation or point-in-time forecasts.

• peak_demand_forecasts exposes all 11 linked public summer-peak scenario workbooks with typed weather-year MW values, separate gross/net/PV tables, P50/P90 fields and large-load assumptions. Full source comparisons cover 155 forecast-year rows and 2,712 numeric values.

• hourly_load_forecasts adds all six main public hourly workbooks from the 2021–2025 vintages, including XLSB and both 2025 TSP/adjusted scenarios through target year 2044. Independent complete-source comparisons matched 701,294 rows and 32,084,130 values. Concrete weather-zone fields preserve base/gross/net demand, EVs, rooftop PV and separate flexible/contract/officer-letter loads, missing totals, original hour labels and the source Date/YMD disagreements. Calamine is confined to the optional files extra. Offline fixtures retain original XML/binary row excerpts; the winter reliability-standard hourly workbook remains an explicit gap.

• hourly_load_scenarios adds all eight regional weather-year forecast workbooks, with 45 typed weather-year predictions per hour plus separate EV/PV/flexible/contract/officer-letter adjustments. Complete original-cell comparisons cover 771,233 rows and 38,271,930 numeric values, with target dates through 2035. The streaming reader preserves North's different calendar order, missing Date/zone/component columns, West's 515 blank predictions, and all seven #REF! errors and five literal dot markers from South. Original marker rows are included in the compact offline fixtures. No new dependencies are added.

• monthly_load_forecasts adds 11 public monthly workbooks with 1,705 source rows and 3,408 peak/energy values. Dates, scenario titles, declared units and the 2024 large-load assumption remain typed; the 2025 missing date, blank final values and unspecified units are preserved without repairing the source. Both forecast readers share discovery and source-file handling.

• seasonal_peak_forecasts and weekly_peak_forecasts add 24 public workbooks, preserving historical versus forecast sections, gross/net/PV, coincidence, provider assumptions and percentile provenance. Every source date/period and zone/total value was compared across 615 seasonal and 263 weekly rows (7,902 peak values).

• Runnable examples export a market day, forecast vintages, or maximum retained DAM/RT settlement-price history. The price-history example filters delivery dates after combining both sources.

Review guide

Start with README.md and docs/market-data.md for the market-facing workflow and verified coverage. docs/usage.md retains the detailed examples; docs/data-coverage.md records source-specific layouts and limitations.

Most added text is source fixtures and evidence under tools/inputs/; generated models/methods are in tinyercot/_generated.py. The shared archive/bundle behavior is in tinyercot/_history.py, with transport/listing support in _client.py. Specialized public readers live in separate private modules. Fixtures support tests and generation and are excluded from the installed wheel. Workbook/PDF parsers remain optional extras.

The scope is public data useful for pricing, dispatch, hedging, load forecasting, settlement and resource planning. The old idr_compliance reader, its public model/export, dedicated tests, fixtures and examples have been removed from this PR. The complete implementation is preserved on wip/legacy-idr-compliance. Historical weather and zonal energy remain available as market-analysis inputs.

Validation

• The last full suite passed 1,274 tests before the LMP-marker fix. That localized change passes 325 transport/dashboard tests, including all 1,123 captured settlement points; strict typing and live retrieval also pass.
• Strict mypy passes for 43 source files; Ruff and offline regeneration pass. Public Reports generated output is unchanged; the same generator now emits the ESR client.
• Complete retained-source comparisons cover 81,236,426 DAM-price rows, 461,736 DAM ancillary-price rows, and 108,285 weather-zone actual-load rows. Additional source comparisons and their exact scope are linked from docs/data-coverage.md. Other products have sampled historical-layout checks; full intervening coverage is not implied.
• A complete-file decoder check covers 28,881,274 rows in 221 distinct intermediate/annual CSV samples across 92 historical readers, closing the header-only coverage gap for those saved files. Every file decoded and its source row count matched; this does not imply every retained publication or independently verified numerical values. Decoder/source hashes and counts are recorded in tools/inputs/history/intermediate-full-file-evidence.json and its CSV manifest. No runtime changes were needed.
• Complete generation-profile comparisons cover 789,073 hourly rows and 18,187,437 values in eight original downloads, plus embedded metadata for 377 workbook sites. Twenty-five CSV header/first-two-row samples also pass. The four site-key workbooks now have full comparisons across 1,828 site/unit rows and 32 summary rows, including queued-model flags and an explicit same-vintage unit-code spelling mismatch. Older wind archive comparisons additionally cover 40 distinct CSV bodies (499,446 records and 40,418,046 MW values); two hypothetical-solar workbooks add 35,088 records and 5,228,112 values. The two older wind-shape workbooks add 262,968 records and 29,714,904 values, with original site IDs and summaries preserved. Early solar weather-year/TMY3 tables and the remaining Excel profile vintages are not yet covered.
• A live real-time-price startup check matched the first 1,000 rows against the original CSV after one bundle listing and one download, with no archive requests. Regression cases cover lazy archive pagination, duplicate IDs, bounded selection and cross-source overlap. This check does not establish full RT historical coverage.
• The new DAM/RT export cases use original CSV samples and mocked listings to verify delivery-date filtering, bundle-only and archive-only publications, overlap handling, point aliases, decimals, and DST flags.
• Wheel and source-distribution builds pass. The approximately 242 KB wheel contains the runtime and typing metadata, excluding examples, tests, docs, and fixtures. The source distribution retains the new guides and runnable examples.

Source gaps remain explicit; no missing observations are synthesized. MIS remains excluded, including separately listed annual DAM/RT hub/load-zone and DAM ancillary-price reports whose published routes use ICE/MIS. Their oldest years were not queried; Public Reports archive bounds are not the earliest price history ERCOT necessarily holds. Further peripheral legacy-document expansion is paused; the unfinished wind-report reader is preserved on a separate local branch and is not part of this PR. Existing specialized readers remain documented. This PR is for review; no merge is included.