Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ generated query and its history reader through `ESRClient`, using
and source freshness. MIS remains outside the current scope.

Direct public services include `hourly_load`, `fuel_mix`, `load_profiles`,
`loss_factors`, `load_forecast_performance`, and `dashboards`. Install `tinyercot[files]` for XLS/XLSX/XLSB readers
`loss_factors`, `load_forecast_performance`, `ancillary_requirements`, and `dashboards`. Install `tinyercot[files]` for XLS/XLSX/XLSB readers
or `tinyercot[pdf]` for supported PDF tables. Detailed source-specific examples
remain in the [usage reference](docs/usage.md).

Expand Down
33 changes: 33 additions & 0 deletions docs/data-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -1704,3 +1704,36 @@ extra hourly observation. The complete original workbook is a regression fixture
under `tools/inputs/winter-forecasts/source.zip`, excluded from the installed wheel.
[Evidence](../tools/inputs/winter-forecasts/evidence.json) records the original URL,
hash, source bounds and comparison scope. The reader adds no dependencies.

## Ancillary-service requirement workbooks

`Client.ancillary_requirements` discovers the direct public methodology archive
on ERCOT's DAM page and reads its numerical workbooks as typed revisions. A fresh
anonymous download on September 8, 2026 contains **15 workbooks for 2016–2026**,
including the September 1, 2026 revision. All original workbook hashes match the
fixtures. This is separate from API DAM AS plans and actual procurement reports.

The original worksheets contain **17,832 hourly quantities, 4,368 RRS allocation
rows, 4,608 adjustment values and 1,115 supporting values**. Regression comparisons
account for **all 51,515 numeric cells**, including hour-axis labels and supporting
calculations, using original worksheet coordinates. They compare period labels,
RRS components, adjustment bases and 277 located notes as well. Numeric values
follow openpyxl's stored-number interpretation, not exact XML numeric lexemes.

Revisions, split-month periods, individual June 2021 dates, footnote markers and
source worksheet positions remain distinct. The 2016 RRS workbook uses `0` for
some hour labels; these are retained. A 2018 table explicitly contains RRS
changes rather than requirements. Totals and unlabelled calculations remain
separate supporting values with no inferred unit. Adjustment bases retain their
wind/solar capacity denominator or forced-outage qualification. The 2025–2026
regulation adjustment sheets' external `#REF!` helper cells remain located notes;
the published numeric adjustment tables are readable.

`effectiveDate` uses only the explicit effective date in the workbook name.
The 2020 workbook has none, so its value is `None`; no revision establishes an
original publication timestamp. No adjustment, revision or difference is applied
automatically. See the [usage example](usage.md#ancillary-service-requirement-revisions).
The [source manifest](../tools/inputs/as-requirements/sources.json) and
[comparison receipt](../tools/inputs/as-requirements/evidence.json) record scope,
counts and hashes. Tests retain all 15 unchanged original numerical workbooks;
they stay outside the installed wheel. Narrative Word documents are excluded.
2 changes: 2 additions & 0 deletions docs/market-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ settlement load zones are different geographies.
constraints, outages, and renewable/load forecasts. Use the public
`dashboards.sced_capacity()`, `dashboards.ancillary_capacity()`, and
`dashboards.energy_storage()` snapshots for their published operating context.
Public `ancillary_requirements` adds versioned annual requirement schedules, RRS
components and adjustment tables; see the [requirement workflow](usage.md#ancillary-service-requirement-revisions).
Aggregate system data do not establish an individual battery's feasible dispatch.
The public SCED ESR disclosure is part of Public Reports; it is distinct from
the separate ESR API, now available through `ESRClient`; see the
Expand Down
39 changes: 39 additions & 0 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -1547,3 +1547,42 @@ explanation (75 in this workbook). Source dates, hour labels, file metadata,
worksheet positions and explanatory notes remain available. These are forecasts
for winter exposure analysis, not observed load or archived operational forecast
vintages. Use `read(saved_bytes)` for an already downloaded workbook.


## Ancillary-service requirement revisions

Install `tinyercot[files]` to read the public requirement workbooks. This service
uses the direct download linked from ERCOT's DAM page and needs no credentials:

```python
from datetime import date
from tinyercot import Client

with Client() as ercot:
for revision in ercot.ancillary_requirements.rows(
where=lambda r: r.effectiveDate == date(2026, 9, 1),
):
for quantity in revision.quantities:
if quantity.service == "ECRS" and quantity.period.strip() == "Sep":
print(quantity.hourEnding, quantity.quantityMW)
```

`files()` discovers the current archive link. `download(file)` returns its bytes;
`read(data, filename=...)` reads a saved archive or individual workbook. Each
result is one `AncillaryServiceRequirements` revision with typed `quantities`,
`rrsAllocations`, `adjustments`, `supportingValues` and located `notes`.

These are published requirement schedules, not actual DAM/RT procurement.
`effectiveDate` comes from an explicit workbook filename; it is `None` when the
filename has none and does not establish when the publication became available.
Keep revisions separate. Period strings retain partial months, dates and footnote
markers. `kind="change"` identifies the 2018 RRS difference table; changes and
wind/solar/forced-outage adjustments are never applied automatically. Read each
adjustment's original `basis` for its denominator and applicability.

RRS components and ratios retain their source meanings; `fractionFromLrs` keeps
the stored fraction. Original hour labels, including `0` in the 2016 RRS tables,
are unchanged. `supportingValues` retains totals and unlabelled calculations by
worksheet position without assigning a unit; `notes` retains qualifications and
spreadsheet error strings outside the data tables. The source `Total` rows sum
hourly schedule values and are not monthly energy totals.
186 changes: 186 additions & 0 deletions tests/test_as_requirements.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
import hashlib
import json
from datetime import date, datetime
from decimal import Decimal
from io import BytesIO
from pathlib import Path
from zipfile import ZipFile

import httpx
import openpyxl
import pytest

from tinyercot import AncillaryServiceRequirements, Client

INPUTS = Path(__file__).resolve().parents[1] / "tools/inputs/as-requirements"
SOURCES = json.loads((INPUTS / "sources.json").read_text())
RRS_FIELDS = [
"totalRrsMW",
"pfrsMW",
"lrsMW",
"equivalencyRatio",
"fractionFromLrs",
"totalEquivalentPfrsMW",
]


def original(member):
with ZipFile(INPUTS / "originals.zip") as z:
return z.read(member)


def number(value):
return None if value is None else Decimal(str(value))


def label(value):
return value.date().isoformat() if isinstance(value, datetime) else str(value)


@pytest.mark.parametrize(
"source", SOURCES["files"], ids=lambda s: s["member"].split("/")[-1]
)
def test_all_original_requirement_cells(source):
data = original(source["member"])
assert hashlib.sha256(data).hexdigest() == source["sha256"]
with Client() as c:
(doc,) = c.ancillary_requirements.read(data, filename=source["member"])
assert (
AncillaryServiceRequirements.model_validate_json(doc.model_dump_json()) == doc
)
workbook = openpyxl.load_workbook(BytesIO(data), read_only=True, data_only=True)
matrices = {s.title: list(s.values) for s in workbook}
workbook.close()
numeric = {
(s, r + 1, c + 1)
for s, rows in matrices.items()
for r, row in enumerate(rows)
for c, v in enumerate(row)
if type(v) in (int, float)
}
accounted = set()
for q in doc.quantities:
rows = matrices[q.sourceSheet]
assert q.quantityMW == number(rows[q.sourceRow - 1][q.sourceColumn - 1])
assert q.hourEnding == rows[q.sourceRow - 1][0]
header = next(
row for row in reversed(rows[: q.sourceRow - 1]) if row[0] == "HE"
)
assert q.period == label(header[q.sourceColumn - 1])
assert q.kind == (
"change" if q.sourceSheet.startswith("Change") else "requirement"
)
accounted.update(
[
(q.sourceSheet, q.sourceRow, q.sourceColumn),
(q.sourceSheet, q.sourceRow, 1),
]
)
for a in doc.rrsAllocations:
rows = matrices[a.sourceSheet]
header_index = next(
r
for r in reversed(range(a.sourceRow - 1))
if rows[r][a.sourceColumn - 1] == "HE"
)
assert a.period == label(rows[header_index - 1][a.sourceColumn - 1])
assert a.hourEnding == rows[a.sourceRow - 1][a.sourceColumn - 1]
accounted.add((a.sourceSheet, a.sourceRow, a.sourceColumn))
for offset, field in enumerate(RRS_FIELDS, 1):
col = a.sourceColumn - 1 + offset
present = col < len(rows[header_index]) and rows[header_index][col] not in (
None,
"",
"HE",
)
assert getattr(a, field) == (
number(rows[a.sourceRow - 1][col]) if present else None
)
if present:
accounted.add((a.sourceSheet, a.sourceRow, col + 1))
for a in doc.adjustments:
rows = matrices[a.sourceSheet]
assert a.adjustmentMW == number(rows[a.sourceRow - 1][a.sourceColumn - 1])
assert a.period == rows[a.sourceRow - 1][0]
header_index = next(
r for r in reversed(range(a.sourceRow - 1)) if rows[r][0] == "Month"
)
assert a.hourEnding == rows[header_index][a.sourceColumn - 1]
assert a.basis in [row[0] for row in rows[:header_index] if row]
accounted.update(
[
(a.sourceSheet, a.sourceRow, a.sourceColumn),
(a.sourceSheet, header_index + 1, a.sourceColumn),
]
)
for v in doc.supportingValues:
raw = matrices[v.sourceSheet][v.sourceRow - 1]
assert v.value == number(raw[v.sourceColumn - 1])
assert v.label == (raw[0] if isinstance(raw[0], str) else None)
accounted.add((v.sourceSheet, v.sourceRow, v.sourceColumn))
assert numeric - accounted == set(), "Unaccounted source numeric cells"
for note in doc.notes:
assert (
note.text
== matrices[note.sourceSheet][note.sourceRow - 1][note.sourceColumn - 1]
)


def test_vintages_partial_months_changes_and_adjustment_units():
with Client() as c:
docs = list(
c.ancillary_requirements.read((INPUTS / "originals.zip").read_bytes())
)
assert len(docs) == 15
assert {d.year for d in docs} == set(range(2016, 2027))
current = next(d for d in docs if d.effectiveDate == date(2026, 9, 1))
assert {q.service for q in current.quantities} == {
"RegUp",
"RegDown",
"RRS",
"NSRS",
"ECRS",
}
assert len(current.adjustments) == 1152
assert any("Solar" in a.basis and "1000 MW" in a.basis for a in current.adjustments)
assert any(n.text == "#REF!" for n in current.notes)
revisions = [d for d in docs if d.year == 2021]
assert {d.effectiveDate for d in revisions} == {
date(2021, 1, 1),
date(2021, 7, 12),
date(2021, 9, 1),
}
july = next(d for d in revisions if d.effectiveDate.month == 7)
assert {"Jul 1 - 11", "Jul 12 - 31"} <= {q.period for q in july.quantities}
assert "2021-06-01" in {q.period for q in july.quantities}
old = next(d for d in docs if d.year == 2016)
assert 0 in {r.hourEnding for r in old.rrsAllocations}
assert any(q.kind == "change" for d in docs for q in d.quantities)
assert next(d for d in docs if d.year == 2020).effectiveDate is None
assert len({d.sourceMember for d in docs}) == 15


def test_anonymous_discovery_download_and_revision_filter():
calls = []

def handle(request):
calls.append(str(request.url))
if request.url.path == "/mktinfo/dam":
return httpx.Response(
200,
text=f'<a href="{SOURCES["url"]}">Methodology for Determining Minimum Ancillary Service Requirements</a>',
)
assert str(request.url) == SOURCES["url"]
return httpx.Response(200, content=(INPUTS / "originals.zip").read_bytes())

with (
httpx.Client(transport=httpx.MockTransport(handle)) as http,
Client(client=http) as c,
):
docs = list(
c.ancillary_requirements.rows(
where=lambda d: d.effectiveDate == date(2026, 9, 1)
)
)
assert len(docs) == 1
assert calls == ["https://www.ercot.com/mktinfo/dam", SOURCES["url"]]
19 changes: 19 additions & 0 deletions tests/typing_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
from typing import Literal, assert_type

from tinyercot import (
AncillaryServiceAdjustment,
AncillaryServiceQuantity,
AncillaryServiceRequirements,
Archive,
CapacityProject,
CapacityTotals,
Expand Down Expand Up @@ -46,6 +49,7 @@
PolrUsage,
Publication,
PublicFile,
ResponsiveReserveAllocation,
RetailTransactionMonth,
ScheduledGeneration,
SeasonalPeakForecast,
Expand Down Expand Up @@ -980,3 +984,18 @@ def hourly_load_scenario_typing(client: Client) -> None:
assert_type(wind_solar.stppfDayAhead, Decimal | None)
assert_type(wind_solar.pvgrppDayAhead, Decimal | None)
assert_type(wind_solar.stwpf, Decimal)


with Client() as client:
assert_type(client.ancillary_requirements.files(), list[PublicFile])
assert_type(
client.ancillary_requirements.read(b""), Iterator[AncillaryServiceRequirements]
)
requirements = next(
client.ancillary_requirements.rows(where=lambda d: d.year == 2026)
)
assert_type(requirements.effectiveDate, date | None)
assert_type(requirements.quantities, list[AncillaryServiceQuantity])
assert_type(requirements.rrsAllocations, list[ResponsiveReserveAllocation])
assert_type(requirements.adjustments, list[AncillaryServiceAdjustment])
assert_type(requirements.quantities[0].quantityMW, Decimal | None)
12 changes: 12 additions & 0 deletions tinyercot/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
"""Tiny, fully typed access to ERCOT public data."""

from ._as_requirements import AncillaryServiceAdjustment as AncillaryServiceAdjustment
from ._as_requirements import AncillaryServiceQuantity as AncillaryServiceQuantity
from ._as_requirements import (
AncillaryServiceRequirementNote as AncillaryServiceRequirementNote,
)
from ._as_requirements import (
AncillaryServiceRequirements as AncillaryServiceRequirements,
)
from ._as_requirements import (
AncillaryServiceSupportingValue as AncillaryServiceSupportingValue,
)
from ._as_requirements import ResponsiveReserveAllocation as ResponsiveReserveAllocation
from ._capacity import CapacityProject as CapacityProject
from ._capacity import CapacityTotals as CapacityTotals
from ._client import Document as Document
Expand Down
Loading