kvkenyon · kvkenyon · Sep 5, 2026
diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml
@@ -19,12 +19,13 @@ jobs:
       - uses: astral-sh/setup-uv@v6
         with:
           python-version: ${{ matrix.python }}
-      - run: uv sync --frozen --group dev
+      - run: uv sync --frozen --group dev --extra files
       - run: uv run python tools/generate_client.py --check
-      - run: uv run pytest -q
+      - run: uv run python tools/generate_public.py --check
+      - run: uv run --extra files pytest -q
       # Keep the byte-frozen runtime and unrelated examples out of formatting.
-      - run: uv run ruff check tools/generate_client.py tinyercot/catalog.py tests
-      - run: uv run ruff format --check tools/generate_client.py tinyercot/catalog.py tests
+      - run: uv run ruff check tools/generate_client.py tools/generate_public.py tools/probe_public.py tinyercot/catalog.py tinyercot/public tests
+      - run: uv run ruff format --check tools/generate_client.py tools/generate_public.py tools/probe_public.py tinyercot/catalog.py tinyercot/public tests
       - run: uv build
       - run: uv export --frozen --no-dev --no-emit-project --output-file /tmp/tinyercot-requirements.txt
       - run: uv venv /tmp/tinyercot-wheel

diff --git a/.gitignore b/.gitignore
@@ -32,3 +32,6 @@ tests/cassettes/*.yaml
 
 # claude
 .remember/
+
+# Local bounded retrieval evidence
+.local-evidence/
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -2,16 +2,18 @@
 
 TinyERCOT preserves its legacy public Python API. The opt-in `tinyercot.catalog`
 module supplies offline metadata only. Read `docs/public-foundation.md` before
-adding current data capabilities.
+adding current data capabilities. Read `docs/public-retrieval.md` for the opt-in
+two-endpoint API client, annual file adapter, and ESR website feed.
 
 ## Local checks
 
 ```bash
-uv sync --frozen --group dev
+uv sync --frozen --group dev --extra files
 uv run python tools/generate_client.py --check
-uv run pytest -q
-uv run ruff check tools/generate_client.py tinyercot/catalog.py tests
-uv run ruff format --check tools/generate_client.py tinyercot/catalog.py tests
+uv run python tools/generate_public.py --check
+uv run --extra files pytest -q
+uv run ruff check tools/generate_client.py tools/generate_public.py tools/probe_public.py tinyercot/catalog.py tinyercot/public tests
+uv run ruff format --check tools/generate_client.py tools/generate_public.py tools/probe_public.py tinyercot/catalog.py tinyercot/public tests
 uv build
 ```
 
@@ -29,6 +31,13 @@ It has no `--refresh`, `--cache-products`, or `--pandas` option.
 - `api_response_fields.json` is the frozen legacy field cache. It does not verify
   current schemas. `products.json` is historical evidence, not a generation input.
 - `tinyercot/catalog.py` reads bundled public metadata without data requests.
+- `tinyercot/public/` contains opt-in clients and separate errors and row models.
+- `tools/generate_public.py` generates current models from pinned response fields.
+  New endpoints need actual field evidence. Cached legacy rows do not qualify.
+
+Real integration tests require explicit opt-in paths and an installed wheel.
+The normal suite blocks sockets and skips them. Do not print credential files,
+auth responses, tokens, or HTTPX request objects from real requests.
 
 Do not edit the generated file manually. Keep the three legacy runtime files
 unchanged for this milestone. New clients, errors, schema policies, file

diff --git a/MANIFEST.in b/MANIFEST.in
@@ -1,4 +1,4 @@
 include api_response_fields.json uv.lock
 recursive-include tools *.py *.json
 recursive-include tests *.py *.json
-recursive-include docs *.md
+recursive-include docs *.md *.json
diff --git a/README.md b/README.md
@@ -2,17 +2,18 @@
 
 Python client with 102 legacy typed ERCOT Public API endpoint families.
 
-Legacy imports, signatures, and behavior stay fixed. Current server compatibility
-is not verified. The opt-in offline catalog describes observed public sources
-and access boundaries; it adds no data retrieval.
+Legacy imports, signatures, and behavior stay fixed. Legacy server compatibility
+is not verified. The opt-in `tinyercot.public` surface supports two current DAM
+price endpoints, bounded annual DAM file samples, and the rolling website ESR
+feed. The separate offline catalog records observed sources and access boundaries.
 
 ## Install
 
 ```bash
 uv add tinyercot
 ```
 
-## Setup
+## Legacy setup
 
 ```bash
 export ERCOT_USERNAME="your-username"
@@ -110,16 +111,50 @@ Secure, Certified, EWS, private participant records, and customer data are
 restricted and excluded. Public API data requests still need an ERCOT account.
 See [scope, provenance, and adapter boundaries](docs/public-foundation.md).
 
+## Opt-in public retrieval
+
+The new client has **2 generated typed operations in 2 products out of 243
+observed public data paths across 98 product namespaces**. This is partial
+coverage. Unknown row schemas fail closed. Restricted services remain excluded.
+
+```python
+from datetime import date
+from tinyercot.public import Credentials, PublicClient, WebClient, coverage
+
+scopes = coverage()  # Offline; no credentials or network requests.
+with PublicClient(Credentials.from_env()) as client:
+    page = client.dam_prices(
+        start=date(2026, 9, 4), end=date(2026, 9, 4),
+        settlement_point="HB_HOUSTON", size=2,
+    )
+    capacity = client.dam_capacity_prices(
+        start=date(2026, 9, 4), end=date(2026, 9, 4),
+        ancillary_type="REGUP", size=2,
+    )
+    receipt = page.receipt  # Public URL, UTC retrieval time, byte count, SHA-256.
+
+with WebClient() as client:
+    snapshot = client.esr()  # Anonymous website feed, separate from Public Data API.
+    stale = snapshot.is_stale()
+```
+
+Use an existing secure environment injection method for credentials. The new
+client does not load `.env` or authenticate on import. It has per-instance
+request and byte limits. Its errors, row models, and pagination are separate
+from the legacy DataFrame and exception contracts.
+See [exact coverage, annual files, evidence, and limits](docs/public-retrieval.md).
+
 ## Development and legacy generation
 
 Generation uses hash-pinned local inputs. It never fetches an upstream URL.
 
 ```bash
-uv sync --frozen --group dev
+uv sync --frozen --group dev --extra files
 uv run python tools/generate_client.py --check
-uv run pytest -q
-uv run ruff check tools/generate_client.py tinyercot/catalog.py tests
-uv run ruff format --check tools/generate_client.py tinyercot/catalog.py tests
+uv run python tools/generate_public.py --check
+uv run --extra files pytest -q
+uv run ruff check tools/generate_client.py tools/generate_public.py tools/probe_public.py tinyercot/catalog.py tinyercot/public tests
+uv run ruff format --check tools/generate_client.py tools/generate_public.py tools/probe_public.py tinyercot/catalog.py tinyercot/public tests
 uv build
 ```
 
@@ -128,4 +163,6 @@ Use `--output /tmp/legacy.py` to write a review copy. Paths do not depend on the
 working directory. `--check` never writes. Missing inputs or changed hashes fail
 before output is written. The former authenticated `--refresh` and
 `--cache-products` developer commands are removed. Metadata refresh and current
-API generation need a separate reviewed tool; neither is part of this milestone.
+API generation use separate inputs. `tools/generate_public.py` generates only
+the two source-backed current models from `tools/inputs/current/`. It never
+downloads upstream specifications or overwrites the legacy generated file.
diff --git a/docs/evidence/fixture-provenance.json b/docs/evidence/fixture-provenance.json
@@ -0,0 +1,171 @@
+{
+  "fixtures": [
+    {
+      "fixture": "capacity-current.json",
+      "sha256": "3fa8d075dadb0baf6cb0b4656ec87e887a7c85c6d96ab2923dc3e42bd8373251",
+      "transformation": "Exact public response bytes.",
+      "source": {
+        "source_url": "https://api.ercot.com/api/public-reports/np4-188-cd/dam_clear_price_for_cap?deliveryDateFrom=2026-09-04&deliveryDateTo=2026-09-04&ancillaryType=REGUP&size=2&page=1&sort=deliveryDate&dir=desc",
+        "retrieved_at": "2026-09-05 20:54:00.360875+00:00",
+        "sha256": "3fa8d075dadb0baf6cb0b4656ec87e887a7c85c6d96ab2923dc3e42bd8373251",
+        "byte_count": 1336,
+        "status": 200
+      }
+    },
+    {
+      "fixture": "capacity-oldest.json",
+      "sha256": "ed8731e3b8bb8e9c682403f11b0722f6c61f935d5a1ce23dac219b164623f855",
+      "transformation": "Exact public response bytes.",
+      "source": {
+        "source_url": "https://api.ercot.com/api/public-reports/np4-188-cd/dam_clear_price_for_cap?ancillaryType=REGUP&size=1&page=1&sort=deliveryDate&dir=asc",
+        "retrieved_at": "2026-09-05 20:54:03.804946+00:00",
+        "sha256": "ed8731e3b8bb8e9c682403f11b0722f6c61f935d5a1ce23dac219b164623f855",
+        "byte_count": 1237,
+        "status": 200
+      }
+    },
+    {
+      "fixture": "annual-cells.json",
+      "sha256": "3be74d1ea482d14e14318431a68e9a52433296be07ecc79fe5171732156bee70",
+      "transformation": "Header and first four rows from 2010 Dec_1 and 2026 Aug; tests construct small XLSX archives from these observed cells.",
+      "sources": [
+        {
+          "source_url": "https://www.ercot.com/misdownload/servlets/mirDownload?doclookupId=283205860&reportTypeId=13060",
+          "retrieved_at": "2026-09-05T20:29:36.736580+00:00",
+          "bytes": 241081,
+          "sha256": "aca9db5d9dbb124ec7a5afcf13d245b7f984c93958d7e81a9b86a9008f695312",
+          "status": 200,
+          "document_id": "283205860",
+          "report_type_id": "13060",
+          "source_published_at": "2013-03-01T13:17:18-05:00",
+          "friendly_name": "DAMLZHBSPP_2010",
+          "members": [
+            {
+              "name": "rpt.00013060.0000000000000000.DAMLZHBSPP_2010.xlsx",
+              "bytes": 291305
+            }
+          ]
+        },
+        {
+          "source_url": "https://www.ercot.com/misdownload/servlets/mirDownload?doclookupId=1268441709&reportTypeId=13060",
+          "retrieved_at": "2026-09-05T20:29:39.667899+00:00",
+          "bytes": 1356199,
+          "sha256": "6ca3b9751f3868b00bc8b48e9fef66350cf090170b23fff9a747a26923486a34",
+          "status": 200,
+          "document_id": "1268441709",
+          "report_type_id": "13060",
+          "source_published_at": "2026-08-30T08:02:17-05:00",
+          "friendly_name": "DAMLZHBSPP_2026",
+          "members": [
+            {
+              "name": "rpt.00013060.0000000000000000.DAMLZHBSPP_2026.xlsx",
+              "bytes": 1950568
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "fixture": "dam-annual-list.json",
+      "sha256": "43495cb7a429a314c43a97121aae818c30d688651a19af6f077e0a6ad47d7091",
+      "source": {
+        "source_url": "https://www.ercot.com/misapp/servlets/IceDocListJsonWS?reportTypeId=13060",
+        "retrieved_at": "2026-09-05T20:27:20.742164+00:00",
+        "status": 200,
+        "bytes": 7797,
+        "sha256": "43495cb7a429a314c43a97121aae818c30d688651a19af6f077e0a6ad47d7091",
+        "content_type": null,
+        "http_last_modified": null
+      },
+      "transformation": "Exact public response bytes."
+    },
+    {
+      "fixture": "dam-current-page1.json",
+      "sha256": "dc70cef89b0d7ce0fb53a6441f8d5bdd0f79b5d60fff6125253046017cf37d18",
+      "source": {
+        "source_url": "https://api.ercot.com/api/public-reports/np4-190-cd/dam_stlmnt_pnt_prices",
+        "query": {
+          "deliveryDateFrom": "2026-09-04",
+          "deliveryDateTo": "2026-09-04",
+          "settlementPoint": "HB_HOUSTON",
+          "size": 2,
+          "page": 1,
+          "sort": "deliveryDate",
+          "dir": "desc"
+        },
+        "retrieved_at": "2026-09-05T20:28:34.745619+00:00",
+        "bytes": 1384,
+        "sha256": "dc70cef89b0d7ce0fb53a6441f8d5bdd0f79b5d60fff6125253046017cf37d18",
+        "status": 200
+      },
+      "transformation": "Exact public response bytes."
+    },
+    {
+      "fixture": "dam-current-page2.json",
+      "sha256": "62e38531d8bf6a2e8ab858f0dd005982c4a145e229a6197188386731f905efa9",
+      "source": {
+        "source_url": "https://api.ercot.com/api/public-reports/np4-190-cd/dam_stlmnt_pnt_prices",
+        "query": {
+          "deliveryDateFrom": "2026-09-04",
+          "deliveryDateTo": "2026-09-04",
+          "settlementPoint": "HB_HOUSTON",
+          "size": 2,
+          "page": 2,
+          "sort": "deliveryDate",
+          "dir": "desc"
+        },
+        "retrieved_at": "2026-09-05T20:28:36.897114+00:00",
+        "bytes": 1384,
+        "sha256": "62e38531d8bf6a2e8ab858f0dd005982c4a145e229a6197188386731f905efa9",
+        "status": 200
+      },
+      "transformation": "Exact public response bytes."
+    },
+    {
+      "fixture": "dam-oldest.json",
+      "sha256": "c887c6583e05dede55a538e49f1d19136d6b733df1542eaf87f7b24672357ef6",
+      "source": {
+        "source_url": "https://api.ercot.com/api/public-reports/np4-190-cd/dam_stlmnt_pnt_prices",
+        "query": {
+          "settlementPoint": "HB_HOUSTON",
+          "size": 1,
+          "page": 1,
+          "sort": "deliveryDate",
+          "dir": "asc"
+        },
+        "retrieved_at": "2026-09-05T20:28:42.357641+00:00",
+        "bytes": 1279,
+        "sha256": "c887c6583e05dede55a538e49f1d19136d6b733df1542eaf87f7b24672357ef6",
+        "status": 200
+      },
+      "transformation": "Exact public response bytes."
+    },
+    {
+      "fixture": "dam-product.json",
+      "sha256": "39af5e785f5d8da682d2552a1c5aab34b6d8a129f0396536d2257be9527eef74",
+      "source": {
+        "source_url": "https://api.ercot.com/api/public-reports/np4-190-cd",
+        "query": {},
+        "retrieved_at": "2026-09-05T20:28:32.451913+00:00",
+        "bytes": 1372,
+        "sha256": "39af5e785f5d8da682d2552a1c5aab34b6d8a129f0396536d2257be9527eef74",
+        "status": 200
+      },
+      "transformation": "Exact public response bytes."
+    },
+    {
+      "fixture": "esr-live.json",
+      "sha256": "04d60f3488471982e2190123f0be5998c6bd0acbb00e74a5304d7b28a577c007",
+      "source": {
+        "source_url": "https://www.ercot.com/api/1/services/read/dashboards/energy-storage-resources.json",
+        "retrieved_at": "2026-09-05T20:27:20.661195+00:00",
+        "status": 200,
+        "bytes": 88668,
+        "sha256": "51095e4a3a63915ea4d73ee77831461d8e5da20f866fadece1f5c23a80b23633",
+        "content_type": "application/json",
+        "http_last_modified": null
+      },
+      "transformation": "First two and last two rows from each source day; source timestamps preserved."
+    }
+  ]
+}