diff --git a/README.md b/README.md index aa70f88..c7f99dd 100644 --- a/README.md +++ b/README.md @@ -90,7 +90,7 @@ generated query and its history reader through `ESRClient`, using and source freshness. MIS remains outside the current scope. Direct public services include `hourly_load`, `fuel_mix`, `load_profiles`, -`loss_factors`, `load_forecast_performance`, and `dashboards`. Install `tinyercot[files]` for XLS/XLSX/XLSB readers +`loss_factors`, `load_forecast_performance`, `ancillary_requirements`, and `dashboards`. Install `tinyercot[files]` for XLS/XLSX/XLSB readers or `tinyercot[pdf]` for supported PDF tables. Detailed source-specific examples remain in the [usage reference](docs/usage.md). diff --git a/docs/data-coverage.md b/docs/data-coverage.md index 958fdad..84d0165 100644 --- a/docs/data-coverage.md +++ b/docs/data-coverage.md @@ -1704,3 +1704,36 @@ extra hourly observation. The complete original workbook is a regression fixture under `tools/inputs/winter-forecasts/source.zip`, excluded from the installed wheel. [Evidence](../tools/inputs/winter-forecasts/evidence.json) records the original URL, hash, source bounds and comparison scope. The reader adds no dependencies. + +## Ancillary-service requirement workbooks + +`Client.ancillary_requirements` discovers the direct public methodology archive +on ERCOT's DAM page and reads its numerical workbooks as typed revisions. A fresh +anonymous download on September 8, 2026 contains **15 workbooks for 2016–2026**, +including the September 1, 2026 revision. All original workbook hashes match the +fixtures. This is separate from API DAM AS plans and actual procurement reports. + +The original worksheets contain **17,832 hourly quantities, 4,368 RRS allocation +rows, 4,608 adjustment values and 1,115 supporting values**. Regression comparisons +account for **all 51,515 numeric cells**, including hour-axis labels and supporting +calculations, using original worksheet coordinates. They compare period labels, +RRS components, adjustment bases and 277 located notes as well. Numeric values +follow openpyxl's stored-number interpretation, not exact XML numeric lexemes. + +Revisions, split-month periods, individual June 2021 dates, footnote markers and +source worksheet positions remain distinct. The 2016 RRS workbook uses `0` for +some hour labels; these are retained. A 2018 table explicitly contains RRS +changes rather than requirements. Totals and unlabelled calculations remain +separate supporting values with no inferred unit. Adjustment bases retain their +wind/solar capacity denominator or forced-outage qualification. The 2025–2026 +regulation adjustment sheets' external `#REF!` helper cells remain located notes; +the published numeric adjustment tables are readable. + +`effectiveDate` uses only the explicit effective date in the workbook name. +The 2020 workbook has none, so its value is `None`; no revision establishes an +original publication timestamp. No adjustment, revision or difference is applied +automatically. See the [usage example](usage.md#ancillary-service-requirement-revisions). +The [source manifest](../tools/inputs/as-requirements/sources.json) and +[comparison receipt](../tools/inputs/as-requirements/evidence.json) record scope, +counts and hashes. Tests retain all 15 unchanged original numerical workbooks; +they stay outside the installed wheel. Narrative Word documents are excluded. diff --git a/docs/market-data.md b/docs/market-data.md index 79b4290..d70f0fb 100644 --- a/docs/market-data.md +++ b/docs/market-data.md @@ -44,6 +44,8 @@ settlement load zones are different geographies. constraints, outages, and renewable/load forecasts. Use the public `dashboards.sced_capacity()`, `dashboards.ancillary_capacity()`, and `dashboards.energy_storage()` snapshots for their published operating context. +Public `ancillary_requirements` adds versioned annual requirement schedules, RRS +components and adjustment tables; see the [requirement workflow](usage.md#ancillary-service-requirement-revisions). Aggregate system data do not establish an individual battery's feasible dispatch. The public SCED ESR disclosure is part of Public Reports; it is distinct from the separate ESR API, now available through `ESRClient`; see the diff --git a/docs/usage.md b/docs/usage.md index f91f988..3523956 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -1547,3 +1547,42 @@ explanation (75 in this workbook). Source dates, hour labels, file metadata, worksheet positions and explanatory notes remain available. These are forecasts for winter exposure analysis, not observed load or archived operational forecast vintages. Use `read(saved_bytes)` for an already downloaded workbook. + + +## Ancillary-service requirement revisions + +Install `tinyercot[files]` to read the public requirement workbooks. This service +uses the direct download linked from ERCOT's DAM page and needs no credentials: + +```python +from datetime import date +from tinyercot import Client + +with Client() as ercot: + for revision in ercot.ancillary_requirements.rows( + where=lambda r: r.effectiveDate == date(2026, 9, 1), + ): + for quantity in revision.quantities: + if quantity.service == "ECRS" and quantity.period.strip() == "Sep": + print(quantity.hourEnding, quantity.quantityMW) +``` + +`files()` discovers the current archive link. `download(file)` returns its bytes; +`read(data, filename=...)` reads a saved archive or individual workbook. Each +result is one `AncillaryServiceRequirements` revision with typed `quantities`, +`rrsAllocations`, `adjustments`, `supportingValues` and located `notes`. + +These are published requirement schedules, not actual DAM/RT procurement. +`effectiveDate` comes from an explicit workbook filename; it is `None` when the +filename has none and does not establish when the publication became available. +Keep revisions separate. Period strings retain partial months, dates and footnote +markers. `kind="change"` identifies the 2018 RRS difference table; changes and +wind/solar/forced-outage adjustments are never applied automatically. Read each +adjustment's original `basis` for its denominator and applicability. + +RRS components and ratios retain their source meanings; `fractionFromLrs` keeps +the stored fraction. Original hour labels, including `0` in the 2016 RRS tables, +are unchanged. `supportingValues` retains totals and unlabelled calculations by +worksheet position without assigning a unit; `notes` retains qualifications and +spreadsheet error strings outside the data tables. The source `Total` rows sum +hourly schedule values and are not monthly energy totals. diff --git a/tests/test_as_requirements.py b/tests/test_as_requirements.py new file mode 100644 index 0000000..01af5e8 --- /dev/null +++ b/tests/test_as_requirements.py @@ -0,0 +1,186 @@ +import hashlib +import json +from datetime import date, datetime +from decimal import Decimal +from io import BytesIO +from pathlib import Path +from zipfile import ZipFile + +import httpx +import openpyxl +import pytest + +from tinyercot import AncillaryServiceRequirements, Client + +INPUTS = Path(__file__).resolve().parents[1] / "tools/inputs/as-requirements" +SOURCES = json.loads((INPUTS / "sources.json").read_text()) +RRS_FIELDS = [ + "totalRrsMW", + "pfrsMW", + "lrsMW", + "equivalencyRatio", + "fractionFromLrs", + "totalEquivalentPfrsMW", +] + + +def original(member): + with ZipFile(INPUTS / "originals.zip") as z: + return z.read(member) + + +def number(value): + return None if value is None else Decimal(str(value)) + + +def label(value): + return value.date().isoformat() if isinstance(value, datetime) else str(value) + + +@pytest.mark.parametrize( + "source", SOURCES["files"], ids=lambda s: s["member"].split("/")[-1] +) +def test_all_original_requirement_cells(source): + data = original(source["member"]) + assert hashlib.sha256(data).hexdigest() == source["sha256"] + with Client() as c: + (doc,) = c.ancillary_requirements.read(data, filename=source["member"]) + assert ( + AncillaryServiceRequirements.model_validate_json(doc.model_dump_json()) == doc + ) + workbook = openpyxl.load_workbook(BytesIO(data), read_only=True, data_only=True) + matrices = {s.title: list(s.values) for s in workbook} + workbook.close() + numeric = { + (s, r + 1, c + 1) + for s, rows in matrices.items() + for r, row in enumerate(rows) + for c, v in enumerate(row) + if type(v) in (int, float) + } + accounted = set() + for q in doc.quantities: + rows = matrices[q.sourceSheet] + assert q.quantityMW == number(rows[q.sourceRow - 1][q.sourceColumn - 1]) + assert q.hourEnding == rows[q.sourceRow - 1][0] + header = next( + row for row in reversed(rows[: q.sourceRow - 1]) if row[0] == "HE" + ) + assert q.period == label(header[q.sourceColumn - 1]) + assert q.kind == ( + "change" if q.sourceSheet.startswith("Change") else "requirement" + ) + accounted.update( + [ + (q.sourceSheet, q.sourceRow, q.sourceColumn), + (q.sourceSheet, q.sourceRow, 1), + ] + ) + for a in doc.rrsAllocations: + rows = matrices[a.sourceSheet] + header_index = next( + r + for r in reversed(range(a.sourceRow - 1)) + if rows[r][a.sourceColumn - 1] == "HE" + ) + assert a.period == label(rows[header_index - 1][a.sourceColumn - 1]) + assert a.hourEnding == rows[a.sourceRow - 1][a.sourceColumn - 1] + accounted.add((a.sourceSheet, a.sourceRow, a.sourceColumn)) + for offset, field in enumerate(RRS_FIELDS, 1): + col = a.sourceColumn - 1 + offset + present = col < len(rows[header_index]) and rows[header_index][col] not in ( + None, + "", + "HE", + ) + assert getattr(a, field) == ( + number(rows[a.sourceRow - 1][col]) if present else None + ) + if present: + accounted.add((a.sourceSheet, a.sourceRow, col + 1)) + for a in doc.adjustments: + rows = matrices[a.sourceSheet] + assert a.adjustmentMW == number(rows[a.sourceRow - 1][a.sourceColumn - 1]) + assert a.period == rows[a.sourceRow - 1][0] + header_index = next( + r for r in reversed(range(a.sourceRow - 1)) if rows[r][0] == "Month" + ) + assert a.hourEnding == rows[header_index][a.sourceColumn - 1] + assert a.basis in [row[0] for row in rows[:header_index] if row] + accounted.update( + [ + (a.sourceSheet, a.sourceRow, a.sourceColumn), + (a.sourceSheet, header_index + 1, a.sourceColumn), + ] + ) + for v in doc.supportingValues: + raw = matrices[v.sourceSheet][v.sourceRow - 1] + assert v.value == number(raw[v.sourceColumn - 1]) + assert v.label == (raw[0] if isinstance(raw[0], str) else None) + accounted.add((v.sourceSheet, v.sourceRow, v.sourceColumn)) + assert numeric - accounted == set(), "Unaccounted source numeric cells" + for note in doc.notes: + assert ( + note.text + == matrices[note.sourceSheet][note.sourceRow - 1][note.sourceColumn - 1] + ) + + +def test_vintages_partial_months_changes_and_adjustment_units(): + with Client() as c: + docs = list( + c.ancillary_requirements.read((INPUTS / "originals.zip").read_bytes()) + ) + assert len(docs) == 15 + assert {d.year for d in docs} == set(range(2016, 2027)) + current = next(d for d in docs if d.effectiveDate == date(2026, 9, 1)) + assert {q.service for q in current.quantities} == { + "RegUp", + "RegDown", + "RRS", + "NSRS", + "ECRS", + } + assert len(current.adjustments) == 1152 + assert any("Solar" in a.basis and "1000 MW" in a.basis for a in current.adjustments) + assert any(n.text == "#REF!" for n in current.notes) + revisions = [d for d in docs if d.year == 2021] + assert {d.effectiveDate for d in revisions} == { + date(2021, 1, 1), + date(2021, 7, 12), + date(2021, 9, 1), + } + july = next(d for d in revisions if d.effectiveDate.month == 7) + assert {"Jul 1 - 11", "Jul 12 - 31"} <= {q.period for q in july.quantities} + assert "2021-06-01" in {q.period for q in july.quantities} + old = next(d for d in docs if d.year == 2016) + assert 0 in {r.hourEnding for r in old.rrsAllocations} + assert any(q.kind == "change" for d in docs for q in d.quantities) + assert next(d for d in docs if d.year == 2020).effectiveDate is None + assert len({d.sourceMember for d in docs}) == 15 + + +def test_anonymous_discovery_download_and_revision_filter(): + calls = [] + + def handle(request): + calls.append(str(request.url)) + if request.url.path == "/mktinfo/dam": + return httpx.Response( + 200, + text=f'Methodology for Determining Minimum Ancillary Service Requirements', + ) + assert str(request.url) == SOURCES["url"] + return httpx.Response(200, content=(INPUTS / "originals.zip").read_bytes()) + + with ( + httpx.Client(transport=httpx.MockTransport(handle)) as http, + Client(client=http) as c, + ): + docs = list( + c.ancillary_requirements.rows( + where=lambda d: d.effectiveDate == date(2026, 9, 1) + ) + ) + assert len(docs) == 1 + assert calls == ["https://www.ercot.com/mktinfo/dam", SOURCES["url"]] diff --git a/tests/typing_client.py b/tests/typing_client.py index 8882d8a..97cba83 100644 --- a/tests/typing_client.py +++ b/tests/typing_client.py @@ -4,6 +4,9 @@ from typing import Literal, assert_type from tinyercot import ( + AncillaryServiceAdjustment, + AncillaryServiceQuantity, + AncillaryServiceRequirements, Archive, CapacityProject, CapacityTotals, @@ -46,6 +49,7 @@ PolrUsage, Publication, PublicFile, + ResponsiveReserveAllocation, RetailTransactionMonth, ScheduledGeneration, SeasonalPeakForecast, @@ -980,3 +984,18 @@ def hourly_load_scenario_typing(client: Client) -> None: assert_type(wind_solar.stppfDayAhead, Decimal | None) assert_type(wind_solar.pvgrppDayAhead, Decimal | None) assert_type(wind_solar.stwpf, Decimal) + + +with Client() as client: + assert_type(client.ancillary_requirements.files(), list[PublicFile]) + assert_type( + client.ancillary_requirements.read(b""), Iterator[AncillaryServiceRequirements] + ) + requirements = next( + client.ancillary_requirements.rows(where=lambda d: d.year == 2026) + ) + assert_type(requirements.effectiveDate, date | None) + assert_type(requirements.quantities, list[AncillaryServiceQuantity]) + assert_type(requirements.rrsAllocations, list[ResponsiveReserveAllocation]) + assert_type(requirements.adjustments, list[AncillaryServiceAdjustment]) + assert_type(requirements.quantities[0].quantityMW, Decimal | None) diff --git a/tinyercot/__init__.py b/tinyercot/__init__.py index 6d40d0b..9a175f4 100644 --- a/tinyercot/__init__.py +++ b/tinyercot/__init__.py @@ -1,5 +1,17 @@ """Tiny, fully typed access to ERCOT public data.""" +from ._as_requirements import AncillaryServiceAdjustment as AncillaryServiceAdjustment +from ._as_requirements import AncillaryServiceQuantity as AncillaryServiceQuantity +from ._as_requirements import ( + AncillaryServiceRequirementNote as AncillaryServiceRequirementNote, +) +from ._as_requirements import ( + AncillaryServiceRequirements as AncillaryServiceRequirements, +) +from ._as_requirements import ( + AncillaryServiceSupportingValue as AncillaryServiceSupportingValue, +) +from ._as_requirements import ResponsiveReserveAllocation as ResponsiveReserveAllocation from ._capacity import CapacityProject as CapacityProject from ._capacity import CapacityTotals as CapacityTotals from ._client import Document as Document diff --git a/tinyercot/_as_requirements.py b/tinyercot/_as_requirements.py new file mode 100644 index 0000000..0dd5bfd --- /dev/null +++ b/tinyercot/_as_requirements.py @@ -0,0 +1,272 @@ +"""Published ancillary-service requirements and their historical revisions.""" + +from __future__ import annotations + +import re +from collections.abc import Iterator +from datetime import date, datetime +from decimal import Decimal +from typing import Literal + +from pydantic import BaseModel, ConfigDict, Field + +from ._legacy_load import _number +from ._load import _sheets, _workbooks +from ._public_tables import _PublicTable + +Service = Literal["RegUp", "RegDown", "RRS", "NSRS", "ECRS"] + + +class _SourceCell(BaseModel): + model_config = ConfigDict(extra="forbid") + sourceSheet: str + sourceRow: int + sourceColumn: int + + +class AncillaryServiceQuantity(_SourceCell): + """Published hourly requirement, or explicitly labelled change, in MW. + + period retains month, partial-month and individual-date labels. These are + requirement schedules, not quantities actually procured in DAM or RTM. + """ + + service: Service + kind: Literal["requirement", "change"] = "requirement" + period: str + hourEnding: int + quantityMW: Decimal | None + + +class ResponsiveReserveAllocation(_SourceCell): + """RRS components and source ratios; fractions are not multiplied by 100.""" + + period: str + hourEnding: int + totalRrsMW: Decimal | None + pfrsMW: Decimal | None + lrsMW: Decimal | None + equivalencyRatio: Decimal | None + fractionFromLrs: Decimal | None = None + totalEquivalentPfrsMW: Decimal | None = None + + +class AncillaryServiceAdjustment(_SourceCell): + """An adjustment in MW with its original basis, applied by the caller.""" + + service: Service + basis: str + period: str + hourEnding: int + adjustmentMW: Decimal | None + + +class AncillaryServiceSupportingValue(_SourceCell): + """Published totals or unlabelled worksheet values, with no inferred unit.""" + + label: str | None + value: Decimal + + +class AncillaryServiceRequirementNote(_SourceCell): + text: str + + +class AncillaryServiceRequirements(BaseModel): + """One workbook revision; effectiveDate comes from its name, not issue time. + + Revisions are never merged or applied automatically. Read notes alongside + the tables for qualifications, adjustment bases and source spreadsheet errors. + """ + + model_config = ConfigDict(extra="forbid") + year: int + effectiveDate: date | None + sourceMember: str + quantities: list[AncillaryServiceQuantity] = Field(default_factory=list) + rrsAllocations: list[ResponsiveReserveAllocation] = Field(default_factory=list) + adjustments: list[AncillaryServiceAdjustment] = Field(default_factory=list) + supportingValues: list[AncillaryServiceSupportingValue] = Field( + default_factory=list + ) + notes: list[AncillaryServiceRequirementNote] = Field(default_factory=list) + + +def _service(sheet: str) -> Service: + name = sheet.lower() + services: tuple[tuple[str, Service], ...] = ( + ("regulation-up", "RegUp"), + ("regulation up", "RegUp"), + ("reg-up", "RegUp"), + ("regulation-down", "RegDown"), + ("regulation down", "RegDown"), + ("rrs", "RRS"), + ("nsrs", "NSRS"), + ("ecrs", "ECRS"), + ) + for text, service in services: + if text in name: + return service + raise ValueError(f"Unknown ancillary service: {sheet}") + + +def _label(value: object) -> str: + return value.date().isoformat() if isinstance(value, datetime) else str(value) + + +_RRS = { + "Total RRS MW": "totalRrsMW", + "PFRS": "pfrsMW", + "LRs": "lrsMW", + "Equivalency Ratio": "equivalencyRatio", + "%RRS from LRs": "fractionFromLrs", + "Total Equivalent PFRs": "totalEquivalentPfrsMW", +} + + +class AncillaryRequirements(_PublicTable[AncillaryServiceRequirements]): + """Direct public AS requirement workbooks; requires tinyercot[files].""" + + index_url = "https://www.ercot.com/mktinfo/dam" + title_pattern = ( + r"Methodology for Determining Minimum Ancillary Service Requirements" + ) + + def _read( + self, data: bytes, filename: str + ) -> Iterator[AncillaryServiceRequirements]: + found = False + for member, content in _workbooks(data): + if member == "workbook.xlsx": + member = filename + sheets = [ + (name, list(rows)) for name, rows in _sheets(content, date_columns=()) + ] + year = re.search( + r"\b20\d{2}\b", member.replace("_", " ") + " " + sheets[0][0] + ) + if year is None: + raise ValueError(f"{member}: no requirement year") + effective = re.search(r"Effective (\d{8}|\d{6})(?!\d)", member) + effective_date = None + if effective: + stamp = effective[1] + effective_date = date( + int(stamp[4:]) + (2000 if len(stamp) == 6 else 0), + int(stamp[:2]), + int(stamp[2:4]), + ) + document = AncillaryServiceRequirements( + year=int(year[0]), effectiveDate=effective_date, sourceMember=member + ) + for sheet, rows in sheets: + used: set[tuple[int, int]] = set() + basis = "" + service: Service | None = None + for r, row in enumerate(rows): + text = str(row[0] or "") if row else "" + if "Incremental MW Adjustment" in text: + basis = text + service = ( + _service(sheet) if not sheet.startswith("Reg") else None + ) + if text in ("RegUp", "RegDown"): + service = "RegUp" if text == "RegUp" else "RegDown" + if text == "Month" and tuple(row[1:25]) == tuple(range(1, 25)): + if not basis or service is None: + raise ValueError( + f"{member}/{sheet}: adjustment basis or service absent" + ) + for i in range(r + 1, r + 13): + for c in range(1, 25): + document.adjustments.append( + AncillaryServiceAdjustment( + service=service, + basis=basis, + period=_label(rows[i][0]), + hourEnding=c, + adjustmentMW=_number(rows[i][c]), + sourceSheet=sheet, + sourceRow=i + 1, + sourceColumn=c + 1, + ) + ) + used.add((i, c)) + used.add((i, 0)) + used.update((r, c) for c in range(25)) + elif "HE" in row: + starts = [c for c, value in enumerate(row) if value == "HE"] + for start in starts: + stop = start + 1 + while stop < len(row) and row[stop] not in (None, ""): + stop += 1 + headers = row[start + 1 : stop] + rrs = str(headers[0]).strip() == "Total RRS MW" + for i in range(r + 1, r + 25): + hour = int(str(rows[i][start])) + if rrs: + fields = { + _RRS[str(header).strip()]: _number(rows[i][c]) + for c, header in enumerate(headers, start + 1) + } + document.rrsAllocations.append( + ResponsiveReserveAllocation.model_validate( + dict( + fields, + period=_label(rows[r - 1][start]), + hourEnding=hour, + sourceSheet=sheet, + sourceRow=i + 1, + sourceColumn=start + 1, + ) + ) + ) + else: + for c, period in enumerate(headers, start + 1): + document.quantities.append( + AncillaryServiceQuantity( + service=_service(sheet), + kind="change" + if sheet.startswith("Change") + else "requirement", + period=_label(period), + hourEnding=hour, + quantityMW=_number(rows[i][c]), + sourceSheet=sheet, + sourceRow=i + 1, + sourceColumn=c + 1, + ) + ) + used.update((i, c) for c in range(start, stop)) + used.update((r, c) for c in range(start, stop)) + for r, row in enumerate(rows): + for c, value in enumerate(row): + if (r, c) not in used and isinstance(value, (int, float)): + document.supportingValues.append( + AncillaryServiceSupportingValue( + label=row[0] if isinstance(row[0], str) else None, + value=Decimal(str(value)), + sourceSheet=sheet, + sourceRow=r + 1, + sourceColumn=c + 1, + ) + ) + if ( + (r, c) not in used + and isinstance(value, str) + and value.strip() + ): + document.notes.append( + AncillaryServiceRequirementNote( + text=value, + sourceSheet=sheet, + sourceRow=r + 1, + sourceColumn=c + 1, + ) + ) + if not document.quantities and not document.rrsAllocations: + raise ValueError(f"{member}: no ancillary requirement tables") + found = True + yield document + if not found: + raise ValueError("Download contains no ancillary requirement workbooks") diff --git a/tinyercot/_client.py b/tinyercot/_client.py index 4468330..21c6bfa 100644 --- a/tinyercot/_client.py +++ b/tinyercot/_client.py @@ -15,6 +15,7 @@ from httpx_retries import Retry, RetryTransport from pydantic import AliasChoices, BaseModel, ConfigDict, Field +from ._as_requirements import AncillaryRequirements from ._capacity import CapacityChanges from ._coincident_peaks import CoincidentPeaks from ._dashboards import Dashboards @@ -448,6 +449,10 @@ def polr(self) -> PolrHistory: def retail_transactions(self) -> RetailTransactions: return RetailTransactions(self._http) + @property + def ancillary_requirements(self) -> AncillaryRequirements: + return AncillaryRequirements(self._http) + @property def indicative_ordc(self) -> IndicativeOrdcHistory: return IndicativeOrdcHistory(self._http) diff --git a/tools/inputs/README.md b/tools/inputs/README.md index 1dc4549..efb0b7c 100644 --- a/tools/inputs/README.md +++ b/tools/inputs/README.md @@ -5,6 +5,9 @@ client offline, without ERCOT credentials or a separately initialized submodule. - `operations.json` and the root `api_response_fields.json` describe the API used by the generator. Overrides and field mappings record deliberate corrections. +- `as-requirements/` retains all 15 original numerical workbooks from the public + ancillary-service methodology archive, their hashes and comparison receipt. + Narrative Word documents are not included in this fixture. - `esr/` contains the separate ESR operation/response contracts, archive mapping, original nested ZIPs, API captures and the live verification receipt. It feeds the same generator and ESR routing/history regression tests. diff --git a/tools/inputs/as-requirements/evidence.json b/tools/inputs/as-requirements/evidence.json new file mode 100644 index 0000000..47e8bc1 --- /dev/null +++ b/tools/inputs/as-requirements/evidence.json @@ -0,0 +1,196 @@ +{ + "checkedAt": "2026-09-08T22:23:27.957589+00:00", + "url": "https://www.ercot.com/files/docs/2022/06/07/2026_Methodology_for_Determining_Minimum_AS_Reqs.zip", + "archiveSha256": "0c5a13c8f4558102a415daf52fea5330872a3cd2316d91877da70368bdd79d34", + "files": [ + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2021 Effective 071221.xlsx", + "year": 2021, + "effectiveDate": "2021-07-12", + "sha256": "4270a2eac4ecc261c9d1953b8af98316d1000bb2386db0fe3a515b96fe9c3025", + "quantities": 1272, + "rrsAllocations": 312, + "adjustments": 0, + "supportingValues": 77, + "notes": 18, + "sourceNumericCells": 3629 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2024 Effective 01012024.xlsx", + "year": 2024, + "effectiveDate": "2024-01-01", + "sha256": "a7a7ba0bf1a0d5a1859f5b414a956abb991da692006f62b054c97a532fc44faf", + "quantities": 1440, + "rrsAllocations": 288, + "adjustments": 0, + "supportingValues": 60, + "notes": 12, + "sourceNumericCells": 3348 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2021 Effective 010121.xlsx", + "year": 2021, + "effectiveDate": "2021-01-01", + "sha256": "0f28e0bb51268d7c25dd66f3f5b9b86f06643544c76f9859801f1d220e25bb73", + "quantities": 1152, + "rrsAllocations": 288, + "adjustments": 0, + "supportingValues": 72, + "notes": 10, + "sourceNumericCells": 3312 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2019 Effective 010119.xlsx", + "year": 2019, + "effectiveDate": "2019-01-01", + "sha256": "7abcb54114235e1f01053769af744651e85fb254d16ea6ad92c895379d280063", + "quantities": 864, + "rrsAllocations": 288, + "adjustments": 0, + "supportingValues": 36, + "notes": 7, + "sourceNumericCells": 2700 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2026 Effective 01012026.xlsx", + "year": 2026, + "effectiveDate": "2026-01-01", + "sha256": "0d44b7e87e1304b8248e20f2a95ae79a90937664ea392fc63791146488db2873", + "quantities": 1440, + "rrsAllocations": 288, + "adjustments": 1152, + "supportingValues": 60, + "notes": 38, + "sourceNumericCells": 4596 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Requirements_2016 Effective 010116.xlsx", + "year": 2016, + "effectiveDate": "2016-01-01", + "sha256": "3b276608d953b3be76881ebbfa407485fbdd0b03e27e5406ed3c33ce1b0d6db4", + "quantities": 864, + "rrsAllocations": 288, + "adjustments": 0, + "supportingValues": 36, + "notes": 22, + "sourceNumericCells": 2412 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2025 Effective 01012025.xlsx", + "year": 2025, + "effectiveDate": "2025-01-01", + "sha256": "24816856d75e706106bc5588c2a5a6dc022e19d7e26add9e93571f5432dc778f", + "quantities": 1440, + "rrsAllocations": 288, + "adjustments": 2304, + "supportingValues": 60, + "notes": 46, + "sourceNumericCells": 5844 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2018 Effective 060118.xlsx", + "year": 2018, + "effectiveDate": "2018-06-01", + "sha256": "63da36db856eb76bcbd10f1e9942b92e4c7dd28a8feacaadaf47eaf68f777ac7", + "quantities": 1152, + "rrsAllocations": 288, + "adjustments": 0, + "supportingValues": 36, + "notes": 9, + "sourceNumericCells": 3012 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2020.xlsx", + "year": 2020, + "effectiveDate": null, + "sha256": "94b52ad2afbc0d46e54cb8bbc2b6fdd3d9daf5f15cc78c1c8975af07132e4eaf", + "quantities": 1152, + "rrsAllocations": 288, + "adjustments": 0, + "supportingValues": 360, + "notes": 8, + "sourceNumericCells": 3336 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2018 Effective 010118.xlsx", + "year": 2018, + "effectiveDate": "2018-01-01", + "sha256": "727d9d592cea96be2f92340833acae41f934c0176af24bac396a02c997f0cf3a", + "quantities": 864, + "rrsAllocations": 288, + "adjustments": 0, + "supportingValues": 36, + "notes": 7, + "sourceNumericCells": 2412 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2022 Effective 01012022.xlsx", + "year": 2022, + "effectiveDate": "2022-01-01", + "sha256": "0b8cfe5b0ecf141829fcc08db4b5dbb4e9c6e8216014f1a11bbc3f05b64704f7", + "quantities": 1152, + "rrsAllocations": 288, + "adjustments": 0, + "supportingValues": 48, + "notes": 10, + "sourceNumericCells": 3024 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2023 Effective 01012023_v1_2.xlsx", + "year": 2023, + "effectiveDate": "2023-01-01", + "sha256": "d2ba7ed9c5c147fd58bddee693c882c55a9acf95c10ef052752d85c8e784543a", + "quantities": 1464, + "rrsAllocations": 288, + "adjustments": 0, + "supportingValues": 61, + "notes": 14, + "sourceNumericCells": 3253 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2017 Effective 010117.xlsx", + "year": 2017, + "effectiveDate": "2017-01-01", + "sha256": "ef27c20b406ea853faa36d84047ed6aa510e6309d0a44e2a41b1a234cacc7513", + "quantities": 864, + "rrsAllocations": 288, + "adjustments": 0, + "supportingValues": 36, + "notes": 20, + "sourceNumericCells": 2412 + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2021 Effective 090121.xlsx", + "year": 2021, + "effectiveDate": "2021-09-01", + "sha256": "f918dfe800286e933bb09680aa012d2a193907c69805037bc9bab361cd016689", + "quantities": 1272, + "rrsAllocations": 312, + "adjustments": 0, + "supportingValues": 77, + "notes": 18, + "sourceNumericCells": 3629 + }, + { + "member": "ERCOT AS Quantities 2026 Effective 09012026.xlsx", + "year": 2026, + "effectiveDate": "2026-09-01", + "sha256": "f5a3001ffd4b8972b0ee573ef7c800c64418ead252409dd78ae95d1d44d7e88f", + "quantities": 1440, + "rrsAllocations": 288, + "adjustments": 1152, + "supportingValues": 60, + "notes": 38, + "sourceNumericCells": 4596 + } + ], + "totals": { + "quantities": 17832, + "rrsAllocations": 4368, + "adjustments": 4608, + "supportingValues": 1115, + "notes": 277, + "sourceNumericCells": 51515 + }, + "comparison": "Every original numeric cell is accounted for and compared at its source coordinates in tests/test_as_requirements.py. Numbers follow openpyxl stored-number interpretation; dates, labels, adjustment bases and notes are compared separately. The full original workbooks are fixtures." +} diff --git a/tools/inputs/as-requirements/originals.zip b/tools/inputs/as-requirements/originals.zip new file mode 100644 index 0000000..1ac53eb Binary files /dev/null and b/tools/inputs/as-requirements/originals.zip differ diff --git a/tools/inputs/as-requirements/sources.json b/tools/inputs/as-requirements/sources.json new file mode 100644 index 0000000..368385a --- /dev/null +++ b/tools/inputs/as-requirements/sources.json @@ -0,0 +1,81 @@ +{ + "url": "https://www.ercot.com/files/docs/2022/06/07/2026_Methodology_for_Determining_Minimum_AS_Reqs.zip", + "archiveSha256": "0c5a13c8f4558102a415daf52fea5330872a3cd2316d91877da70368bdd79d34", + "files": [ + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2021 Effective 071221.xlsx", + "bytes": 43471, + "sha256": "4270a2eac4ecc261c9d1953b8af98316d1000bb2386db0fe3a515b96fe9c3025" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2024 Effective 01012024.xlsx", + "bytes": 43241, + "sha256": "a7a7ba0bf1a0d5a1859f5b414a956abb991da692006f62b054c97a532fc44faf" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2021 Effective 010121.xlsx", + "bytes": 37839, + "sha256": "0f28e0bb51268d7c25dd66f3f5b9b86f06643544c76f9859801f1d220e25bb73" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2019 Effective 010119.xlsx", + "bytes": 32177, + "sha256": "7abcb54114235e1f01053769af744651e85fb254d16ea6ad92c895379d280063" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2026 Effective 01012026.xlsx", + "bytes": 64431, + "sha256": "0d44b7e87e1304b8248e20f2a95ae79a90937664ea392fc63791146488db2873" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Requirements_2016 Effective 010116.xlsx", + "bytes": 27966, + "sha256": "3b276608d953b3be76881ebbfa407485fbdd0b03e27e5406ed3c33ce1b0d6db4" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2025 Effective 01012025.xlsx", + "bytes": 61061, + "sha256": "24816856d75e706106bc5588c2a5a6dc022e19d7e26add9e93571f5432dc778f" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2018 Effective 060118.xlsx", + "bytes": 32016, + "sha256": "63da36db856eb76bcbd10f1e9942b92e4c7dd28a8feacaadaf47eaf68f777ac7" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2020.xlsx", + "bytes": 36463, + "sha256": "94b52ad2afbc0d46e54cb8bbc2b6fdd3d9daf5f15cc78c1c8975af07132e4eaf" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2018 Effective 010118.xlsx", + "bytes": 26324, + "sha256": "727d9d592cea96be2f92340833acae41f934c0176af24bac396a02c997f0cf3a" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2022 Effective 01012022.xlsx", + "bytes": 35403, + "sha256": "0b8cfe5b0ecf141829fcc08db4b5dbb4e9c6e8216014f1a11bbc3f05b64704f7" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2023 Effective 01012023_v1_2.xlsx", + "bytes": 44788, + "sha256": "d2ba7ed9c5c147fd58bddee693c882c55a9acf95c10ef052752d85c8e784543a" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2017 Effective 010117.xlsx", + "bytes": 29296, + "sha256": "ef27c20b406ea853faa36d84047ed6aa510e6309d0a44e2a41b1a234cacc7513" + }, + { + "member": "Archive ERCOT Methodologies for Determining Ancillary Service Requirements/ERCOT AS Quantities 2021 Effective 090121.xlsx", + "bytes": 44421, + "sha256": "f918dfe800286e933bb09680aa012d2a193907c69805037bc9bab361cd016689" + }, + { + "member": "ERCOT AS Quantities 2026 Effective 09012026.xlsx", + "bytes": 64820, + "sha256": "f5a3001ffd4b8972b0ee573ef7c800c64418ead252409dd78ae95d1d44d7e88f" + } + ] +}