Skip to content

Freeze legacy API and add public-only offline catalog - #1

Closed
kvkenyon wants to merge 1 commit into
mainfrom
feat/public-foundation
Closed

Freeze legacy API and add public-only offline catalog#1
kvkenyon wants to merge 1 commit into
mainfrom
feat/public-foundation

Conversation

@kvkenyon

@kvkenyon kvkenyon commented Sep 5, 2026

Copy link
Copy Markdown
Owner

The generator previously read a moving ERCOT URL and could emit empty Row models when response metadata was missing. This PR freezes legacy reproduction and adds an opt-in offline public catalog. It adds no current typed or raw data retrieval.

Covered now

  • Preserve _client.py, _generated.py, and __init__.py byte for byte against audit base d1daad25df42d3fff41f88b907d99ef325b970e0: 35 product exports, configure, 510 method signatures/categories, 204 nested models, and the base response model. Preserve legacy exceptions, pagination, token behavior, and DataFrames.
  • Generate the exact legacy source from compact local inputs with source URLs, capture times, hashes, ordered contracts, and explicit unknown provenance. Fail on missing/unverified schemas, unsupported types/formats, invalid names, collisions, changed input, or changed output. --check is read-only. Remove the authenticated developer --refresh and --cache-products commands; metadata refresh needs a separate reviewed tool.
  • Add tinyercot.catalog: immutable offline observations of 257 public API operations, including 243 data GET paths (242 public-reports and one ESR). Record verbs/media types, legacy path membership, 39 changed query contracts, and one order-only change.
  • All 243 current data paths remain untyped/raw-unsupported: 203 have cached but unverified fields; 40 lack cached fields. metadata_only does not claim retrieval coverage. Add selected API/MIS/live delivery metadata and explicit access boundaries.
  • Add offline tests, CI on Python 3.11/3.14, sdist input/fixture packaging, and installed-wheel catalog validation. Update obsolete developer documentation.

The inventory uses ERCOT's public Public Reports OpenAPI export and ESR export, captured on 2026-09-05. Source hashes and projection recipes are in tools/inputs/; scope and source links are in docs/public-foundation.md.

Restricted, unavailable, and deferred

Secure, Certified, participant-only, EWS, private telemetry, private bids/COP/awards, settlements, customer data, and other entitlement-bound services remain restricted and excluded. The catalog marks ERCOT's 2001 hourly load gap unavailable. Missing or unrecognized access labels remain unknown. Public aggregates and delayed disclosures do not grant access to original private records. Public API data requests still require ERCOT account authentication; offline metadata does not.

Current response verification, full EMIL metadata, new clients/errors/status handling, raw transport, MIS/archive downloads, live feeds, lifecycle metadata, bounded history, temporal/DST, revision/as-of semantics, dependency matrix coverage, and installed-wheel typing remain deferred. No new adapter or placeholder client is added.

No authenticated ERCOT request or historical bulk extraction occurred. No ERCOT credentials were read or used. No paid data access, secret copying, or forecaster work occurred. Tests use fixtures, synthetic credentials, and mocked transport with network access blocked. No no-mistakes workflow ran. This is a direct PR; do not merge.

Validation

873 tests passed on Python 3.14.0 and 3.11.14. Exact commands:

uv sync --frozen --group dev
uv run python tools/generate_client.py --check
uv run pytest -q
UV_PROJECT_ENVIRONMENT=/tmp/tinyercot-py311 uv run --frozen --python 3.11 pytest -q
uv run ruff check tools/generate_client.py tinyercot/catalog.py tests
uv run ruff format --check tools/generate_client.py tinyercot/catalog.py tests
uv build
uv export --frozen --no-dev --no-emit-project --output-file /tmp/tinyercot-requirements.txt
uv venv /tmp/tinyercot-wheel --python 3.14
uv pip sync --python /tmp/tinyercot-wheel/bin/python /tmp/tinyercot-requirements.txt
uv pip install --python /tmp/tinyercot-wheel/bin/python --no-deps dist/tinyercot-0.2.2-py3-none-any.whl
/tmp/tinyercot-wheel/bin/python -I tests/wheel_smoke.py
git diff --check
git diff --cached --check

Generation reproduced SHA-256 aaf91cb373b7557113e5343e4366e0d331566d82327ffabe4b9562937f3c2b5c. Scoped lint/format passed for nine Python files. The sdist/wheel build and isolated installed-wheel check passed. The final wheel was reinstalled with uv pip install --python /tmp/tinyercot-wheel/bin/python --no-deps --reinstall dist/tinyercot-0.2.2-py3-none-any.whl and the smoke check passed again. Runtime dependency versions remain fixed; uv corrected the stale local project version in its lock.

Exploratory full-tree Ruff checks found pre-existing findings in examples/streaming_stats.py (RUF013, DTZ011) and frozen tinyercot/__init__.py (RUF100), plus existing formatting in the example and _client.py. Those files remain unchanged. CI checks the authored/changed Python scope. New Python documentation was reviewed for Google-style sections; the existing repo has no type-check command or source license-header convention.

The initial catalog test exposed the omitted order-only drift case, and the first wheel omitted JSON metadata. Both were fixed and retested. docs/task-note.md records exact scope, evidence, commands, outcomes, and deferred work. No implementation blocker remains.

Handoff

Head commit: 4b26c8a91efbba668dbbc40f564bbe812d4ae792.
GitHub offline compatibility run passed for Python 3.11 and 3.14. The worktree is clean. This PR remains open and unmerged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant