From b353c0080d8cd3433c21606324ce17f429eca764 Mon Sep 17 00:00:00 2001 From: Jamie Date: Thu, 27 Aug 2026 10:43:07 -0400 Subject: [PATCH] feat: add least-privilege Lighthouse report diagnostics Add optional report-read auth, a fixed CEO diagnostic helper, governance documentation, tests, and the v1.30.0 release bundle. --- .dev.vars.example | 3 + AGENTS.md | 12 +- CHANGELOG.md | 10 + OPERATIONS.md | 50 +++-- README.md | 28 ++- SOT.md | 42 ++-- package-lock.json | 4 +- package.json | 3 +- scripts/read-ceo-report.mjs | 300 ++++++++++++++++++++++++++++ src/index.ts | 50 ++++- tests/phase2.test.mjs | 2 +- tests/phase3.test.mjs | 2 +- tests/release-control.test.mjs | 7 +- tests/report-auth.test.mjs | 257 ++++++++++++++++++++++++ tests/report-helper.test.mjs | 345 +++++++++++++++++++++++++++++++++ 15 files changed, 1064 insertions(+), 51 deletions(-) create mode 100644 scripts/read-ceo-report.mjs create mode 100644 tests/report-auth.test.mjs create mode 100644 tests/report-helper.test.mjs diff --git a/.dev.vars.example b/.dev.vars.example index e717bde..71bbdca 100644 --- a/.dev.vars.example +++ b/.dev.vars.example @@ -1,4 +1,7 @@ # Local development placeholders only. Never commit real secret values. +# The CEO diagnostic helper does not load this file; it uses LIGHTHOUSE_REPORT_READ_TOKEN or a hidden prompt. +# A real REPORT_READ_TOKEN must be random, 32-128 URL-safe ASCII characters, and differ from ADMIN_TOKEN. ADMIN_TOKEN=replace-with-local-admin-token +REPORT_READ_TOKEN=replace-with-a-distinct-local-report-read-token CF_API_TOKEN=replace-with-local-cloudflare-token TELEMETRY_RATE_LIMIT_SECRET=replace-with-a-long-random-local-secret diff --git a/AGENTS.md b/AGENTS.md index a7f6f74..05a139f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -36,6 +36,8 @@ Authority sources in descending order: - Any Cloudflare Workers Builds configuration change requires explicit external-change approval and a read-only verification receipt before relying on it. - When code depends on a D1 schema change, a migration requires explicit approval and remote verification before any Worker deployment. - Do not print or commit secret values. +- The optional `REPORT_READ_TOKEN` is a secret-binding capability, not repository data. Do not generate, provision, inspect, remove, or rotate its production value without explicit secret-operation approval. It must be independently generated and cryptographically random, contain 32 to 128 URL-safe ASCII characters (`A-Z`, `a-z`, `0-9`, `_`, and `-`), and differ from `ADMIN_TOKEN`; an identical non-empty configuration intentionally fails every protected read and write closed. +- `npm run --silent diagnostic:ceo` is the sole canonical operator helper for a Lighthouse production CEO read. Its fixed, non-echoing behavior does not grant endpoint approval; every invocation still requires explicit production-read scope. - Cross-repository contracts with Agent Smith, BUS Core, buscore-site, or the leads database must be documented when touched. ## Canonical Analytics Access and Diagnostics @@ -45,8 +47,8 @@ For any Lighthouse alert, WATCH, analytics question, report failure, service-hea 1. Read `SOT.md`, then `OPERATIONS.md`, then the relevant `CHANGELOG.md` entry and contract/fixture. 2. Treat `OPERATIONS.md` as the canonical procedure for choosing a diagnostic surface and classifying its side effects. It is subordinate to `SOT.md` and cannot authorize behavior absent from the SOT. 3. Default to local zero-mutation diagnosis. Production, Cloudflare, Discord, GitHub, D1, or other external access requires explicit scope approval. -4. If the approved endpoint, credential source, account context, or tool access is missing, report `ACCESS_BLOCKED`. Do not improvise an endpoint, credential, direct SQL query, or alternate service. -5. Do not fan out across every endpoint. Start with supplied evidence; when live access is approved, use the stored-data `GET /report?view=ceo` diagnostic first and narrow from there. +4. If the approved endpoint, credential source, account context, deployed read-token capability, or tool access is missing, report `ACCESS_BLOCKED`. Do not improvise an endpoint, credential, direct SQL query, raw request command, or alternate service. +5. Do not fan out across every endpoint. Start with supplied evidence; when a production CEO read and its credential mechanism are explicitly approved, run exactly `npm run --silent diagnostic:ceo` and narrow from its validated stored-data result. Do not add arguments, substitute a URL/header, redirect its output to a file, or retry automatically. Canonical Cloudflare identity: account `eb1a8dd5723031d94e57642e3eaaebda`, Worker `buscore-lighthouse`, primary D1 binding `DB` at database ID `e46f2daa-7e97-45a3-9bf0-49003a42850c` named `lighthouse`, and Production Custom Domain `lighthouse.buscore.ca`. An empty Worker Routes list is expected for this Custom Domain. If another account context lacks the Worker, correct the account rather than diagnosing an outage. @@ -54,7 +56,11 @@ Operational constraints: - `GET /report?view=source_health` is ingestion-integrity evidence, not service-probe truth. - `WATCH` is not synonymous with outage; Agent Smith owns WATCH/ALERT/UNAVAILABLE wording, while Lighthouse owns facts and availability. -- The current `ADMIN_TOKEN` is broad: it protects report reads and the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes. Possession does not authorize writes. +- `ADMIN_TOKEN` remains broad: it protects report reads and is the only credential accepted by the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes. Optional `REPORT_READ_TOKEN`, when separately provisioned, is accepted only through `X-Report-Token` for `GET /report`; the admin credential remains a backward-compatible report fallback and the read credential never authorizes a write. +- The report-read split does not make report requests zero-mutation. Bare/fleet/site reports refresh stored traffic, and every report view—including CEO—can increment `metrics_daily.errors` when assembly fails. The helper is therefore Class 2 and approval-gated. +- Treat the helper's three static failure lines literally: `access blocked` is credential/access evidence, `report unavailable` is application evidence with a possible error-counter increment, and generic `diagnostic failed` is an unclassified transport/safety/contract failure. None authorizes a retry or alternate probe. +- The local 1.30.0 bundle contains no secret value, provisioning, deployment, or production verification. Do not claim production read-token availability until separately approved external activation and readback prove it. +- Agent Smith still carries `LIGHTHOUSE_ADMIN_TOKEN` for report reads and monthly snapshot writes. Do not remove or rotate that credential until an owner-approved snapshot-specific authorization split or write removal is implemented and verified in the owning repositories. - Many GET/HEAD surfaces change evidence. Bare/fleet/site reports refresh stored traffic; report failures can increment errors; artifact HEAD records raw/HEAD truth; update, redirect, artifact, telemetry, and admin-write routes are not passive probes. - Git publication is not zero-mutation. The 1.29.3 `main` merge proved that the former Workers Builds production command could promote active traffic independently of the checked-in gate. Durable readback on 2026-08-26 confirmed that both non-production and production publication now use `npx wrangler versions upload`. Pushes still create version or preview state, while active production promotion is reserved for the explicitly approved manual GitHub workflow. Any future external-setting drift reinstates `BLOCKED BEFORE MERGE`. - Lighthouse, Agent Smith, BUS Core, buscore-site, and tgc-site are separate failure domains. Do not infer a cross-service outage from one unavailable layer. diff --git a/CHANGELOG.md b/CHANGELOG.md index c8c2063..56ac9db 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,15 @@ # Changelog +## [1.30.0] - 2026-08-26 + +- Added the optional `REPORT_READ_TOKEN` Worker secret and exact `X-Report-Token` authentication path for every `GET /report` view. The read secret must be independently generated, cryptographically random, contain 32 to 128 URL-safe ASCII characters, and differ from `ADMIN_TOKEN`; existing exact `X-Admin-Token`/`ADMIN_TOKEN` report access remains backward-compatible when the configuration is distinct. +- Kept `POST /campaign`, `POST /notes`, and `POST /report/snapshot` administrative-only. The report-read header is rejected by all three writes, missing/blank/malformed credentials fail closed, and the existing `401 {"ok":false,"error":"unauthorized"}` contract is preserved. An identical non-empty admin/read-secret configuration now fails closed for every protected read and write before database or deferred work instead of silently defeating least privilege. +- Added the fixed `npm run --silent diagnostic:ceo` operator helper for at most one non-echoing request to `https://lighthouse.buscore.ca/report?view=ceo`. It uses `X-Report-Token`, enforces the same credential format, accepts a hidden interactive credential or `LIGHTHOUSE_REPORT_READ_TOKEN` automation input, removes the automation variable from the helper process after capture, rejects arguments, URL overrides, redirects, retries, user-supplied credential/report files, and `.env` loading, and enforces a 15-second timeout plus 1 MiB response cap. +- Made the helper validate the existing strict CEO contract `1.1` before writing pretty-printed JSON plus one newline to stdout. Noninteractive failures use only three static categories: access blocked for missing/invalid credentials and `401`/`403`; report unavailable with an explicit possible-`metrics_daily.errors` warning for `503`; and a generic diagnostic failure for transport, safety, parse, schema, or output failures. Response bodies, numeric HTTP statuses, schema diagnostics, and credentials are suppressed. No browser CORS permission was added for `X-Report-Token`. +- Preserved report payloads, views, schemas, route status semantics, D1 schema and data semantics, retention, cron cadence, integrations, and metric definitions. Stored-data CEO/TGC/source-health/asset/monthly reads still skip the traffic refresh but can best-effort increment `metrics_daily.errors` if report assembly fails; bare/fleet/site reads retain their existing traffic refresh/upsert behavior. +- Documented a rollback-safe phased rollout: keep administrative report fallback intact, separately approve and verify distinct secret provisioning and Worker deployment, verify the read path with one explicitly approved CEO request, and treat a rollback as restoration of the prior admin-only read contract. No secret value, secret provisioning, endpoint request, deployment, migration, or production interaction is part of this local bundle. +- Recorded the remaining cross-service boundary: Agent Smith still holds `LIGHTHOUSE_ADMIN_TOKEN` because it both reads Lighthouse reports and performs the monthly `POST /report/snapshot` write. Full least privilege requires a later owner-approved snapshot-specific authorization split or removal of that write before removing the broad Smith credential or rotating `ADMIN_TOKEN`. + ## [1.29.4] - 2026-08-26 - Recorded the owner-approved read-only Cloudflare audit showing that Workers Builds build `793715ef-6123-4d98-a4a7-797634d07812`, sourced from merged `main` commit `59231d09084d0fa4db71012b6f29550886c5b605`, automatically ran `npx wrangler deploy` and promoted Worker version `ba611ac1-653d-47a2-a465-a85f4124b6b6` to 100% of production traffic. diff --git a/OPERATIONS.md b/OPERATIONS.md index 94f7bb2..3ff6a79 100644 --- a/OPERATIONS.md +++ b/OPERATIONS.md @@ -2,11 +2,11 @@ - Status: current operational runbook - Scope: Lighthouse analytics access, evidence interpretation, incident diagnosis, and release-control boundaries -- Repository baseline: Lighthouse `1.29.4` release-control reconciliation; CEO report and metric-definition contract `1.1` +- Repository baseline: Lighthouse `1.30.0` least-privilege report diagnostics local governed bundle; CEO report and metric-definition contract `1.1` - Active production baseline: Lighthouse `1.29.3`, Worker version `ba611ac1-653d-47a2-a465-a85f4124b6b6`, 100% traffic, created `2026-08-26T22:57:30.628Z` from `main` commit `59231d09084d0fa4db71012b6f29550886c5b605` through Cloudflare build `793715ef-6123-4d98-a4a7-797634d07812` - Last reconciled: 2026-08-26 -- Diagnostic access used for this reconciliation: owner-approved authenticated Cloudflare control-plane metadata reads in the verified owning account -- Production interaction: control-plane metadata only; no Lighthouse endpoint, D1 query, log tail, scheduled invocation, deployment, route change, secret operation, or traffic change +- Diagnostic access used for the 1.30.0 bundle: local repository inspection and tests only; the retained infrastructure baseline comes from the separately recorded, owner-approved 1.29.4 Cloudflare control-plane audit +- Production interaction for the 1.30.0 bundle: none; no Lighthouse endpoint, Cloudflare read, D1 query, log tail, scheduled invocation, deployment, route change, secret operation, or traffic change - Evidence side effects: no Lighthouse application evidence was refreshed, counted, persisted, archived, or otherwise changed - Audit snapshot: the preceding 24-hour Cloudflare overview showed 67 invocations and zero Cloudflare platform/runtime errors. This is historical platform telemetry, not proof of Lighthouse endpoint, report, source, probe, or delivery health @@ -34,6 +34,7 @@ The default diagnostic posture is local and zero-mutation: - Do not treat `git push` as local-only. Durable readback on 2026-08-26 confirmed that both non-production and production Workers Builds use `npx wrangler versions upload`. Pushes can therefore upload Worker versions and create previews, although they must not promote active traffic. - Do not run migrations, deployments, scheduled handlers, retention jobs, report snapshots, notes, campaign writes, telemetry submissions, or release downloads during passive diagnosis. - Never print, persist, or commit secret values, raw lead data, identifiers, IP material, or other private payloads. +- The existence of `npm run --silent diagnostic:ceo` is not production-access approval. Run it only for an explicitly approved production CEO read; that read remains Class 2 and can increment error evidence if report assembly fails. If the approved endpoint, credential source, account context, or tool authorization is unavailable, record `ACCESS_BLOCKED` and stop that diagnostic branch. Do not substitute guessed URLs, unrelated credentials, direct D1 queries, or broader probes. @@ -92,7 +93,7 @@ The following production identities were verified by approved read-only control- | Cron | `5 0 * * *` | Active at `00:05 UTC`; matches checked-in configuration | | Persistent logs and traces | Disabled | No retained Worker log/trace evidence is available unless separately approved and enabled | -The active Worker secret names are `ADMIN_TOKEN`, `CF_API_TOKEN`, `CF_ZONE_TAG`, `DISCORD_WEBHOOK_URL`, `IGNORED_IP`, `PRICE_GUARD_KEY`, and `TELEMETRY_RATE_LIMIT_SECRET`. Values were not read or exposed. `DISCORD_WEBHOOK_URL` and `PRICE_GUARD_KEY` are not referenced by current source; retaining, rotating, or removing them is a separate secret operation requiring explicit approval. +The last approved control-plane inventory found active Worker secret names `ADMIN_TOKEN`, `CF_API_TOKEN`, `CF_ZONE_TAG`, `DISCORD_WEBHOOK_URL`, `IGNORED_IP`, `PRICE_GUARD_KEY`, and `TELEMETRY_RATE_LIMIT_SECRET`. Values were not read or exposed. Version 1.30.0 adds the optional `REPORT_READ_TOKEN` secret-binding contract locally, but this bundle does not create, provision, or verify a production value. Until a separately approved secret operation and readback prove otherwise, do not claim that production has the new credential. `DISCORD_WEBHOOK_URL` and `PRICE_GUARD_KEY` are not referenced by current source; retaining, rotating, or removing them is a separate secret operation requiring explicit approval. Cloudflare currently displays the historical repository name `True-Good-Craft/buscore-lighthouse`; GitHub redirects it to `True-Good-Craft/lighthouse`. Package metadata now names the canonical repository, but changing the external integration label is a separate control-plane action. @@ -142,7 +143,9 @@ The pre-merge Workers Builds repair was completed on 2026-08-26: the prior produ | Name | Where used | Boundary | |---|---|---| -| `ADMIN_TOKEN` | Lighthouse runtime | Exact-match credential accepted through `X-Admin-Token` | +| `ADMIN_TOKEN` | Lighthouse runtime | Exact-match credential accepted through `X-Admin-Token`; authorizes reports and the three protected writes | +| `REPORT_READ_TOKEN` | Lighthouse runtime | Optional independently generated, cryptographically random, distinct 32-to-128-character URL-safe-ASCII secret accepted through `X-Report-Token` for `GET /report` only; never accepted by an administrative write | +| `LIGHTHOUSE_REPORT_READ_TOKEN` | Local diagnostic helper process | Optional automation input satisfying the same format for `npm run --silent diagnostic:ceo`; captured and removed from that process environment immediately, never loaded from `.env` | | `LIGHTHOUSE_ADMIN_TOKEN` | Agent Smith runtime | Consumer-side name for the Lighthouse admin credential | | `LIGHTHOUSE_REPORT_URL` | Agent Smith runtime | Canonical report endpoint; checked-in production value points to `https://lighthouse.buscore.ca/report` | | `CF_API_TOKEN` and `CF_ZONE_TAG` | Lighthouse scheduled traffic capture | Cloudflare GraphQL traffic source, not report authentication | @@ -150,7 +153,23 @@ The pre-merge Workers Builds repair was completed on 2026-08-26: the prior produ | `GITHUB_REPO` and optional `GITHUB_TOKEN` | Lighthouse scheduled GitHub snapshot/probe configuration | Repository selection and optional API quota; not Lighthouse report authentication | | `TELEMETRY_RATE_LIMIT_SECRET` | Lighthouse ingestion/release counting | Keys rotating abuse-control identifiers; never a diagnostic credential | -`ADMIN_TOKEN` is not a read-only credential. The same token authorizes protected report reads and the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes. Possession of the token does not authorize those writes. Do not paste it into commands, logs, chat, files, or screenshots. +`ADMIN_TOKEN` is not a read-only credential. It remains a backward-compatible report credential and is the only credential accepted by the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes. `REPORT_READ_TOKEN` grants GET-report authorization only when it is independently generated, cryptographically random, contains 32 to 128 URL-safe ASCII characters (`A-Z`, `a-z`, `0-9`, `_`, and `-`), and differs from `ADMIN_TOKEN`, but it does not make the report implementation zero-write. A malformed report secret disables that path while preserving a distinct admin fallback; identical non-empty admin/read secrets intentionally fail every protected read and write closed before database or deferred work. Possession of either credential does not itself authorize a production request. Do not paste a production value into commands, logs, chat, files, screenshots, or `.env`; `.dev.vars.example` contains non-secret local placeholders and is not read by the diagnostic helper. + +### 1.30.0 rollout and rollback boundary + +The local governed bundle adds the credential contract and helper but performs no secret generation, provisioning, deployment, production request, or readback. Production remains on the previously verified admin-only report contract until separate evidence proves activation. + +Roll out in reversible stages, each under its own approval: + +1. Validate the complete local Code + SOT + CHANGELOG + Version bundle. +2. Independently generate and provision a distinct cryptographically random 32-to-128-character URL-safe-ASCII production `REPORT_READ_TOKEN` through an approved non-echoing mechanism without removing or rotating `ADMIN_TOKEN`; record only secret-name/version metadata and the format/distinctness assertion, never the value. +3. Deploy the approved Worker bundle through the sole authorized production workflow and obtain its deployment receipt. +4. With separate Class 2 endpoint approval, run exactly one `npm run --silent diagnostic:ceo` request and verify the strict CEO `1.1` output without exposing the credential. +5. Migrate individual consumers only through their owning repository's governed change. Keep administrative fallback until each consumer is verified. + +Rolling back the Lighthouse Worker restores the prior admin-only report-read behavior and does not roll back secret state or another repository. A read-token-only client will receive `401` after such a rollback; use the retained, separately authorized admin fallback only as an intentional compatibility path, not an automatic retry. Secret removal or rotation is a separate destructive security operation and is not implied by code rollback. + +Agent Smith is not least-privilege after this Lighthouse-only change: it still needs `LIGHTHOUSE_ADMIN_TOKEN` for `POST /report/snapshot`. Splitting its report reads without first separating that write leaves the broad credential in the same runtime. A snapshot-specific credential or removal of the archive write requires a later owner-approved Agent Smith/Lighthouse contract change before the broad Smith secret can be removed and `ADMIN_TOKEN` rotated. ## Diagnostic Access Classes @@ -185,6 +204,8 @@ These protected views skip the best-effort Cloudflare traffic refresh and read c These are read-mostly, not guaranteed zero-write: if report assembly throws, Lighthouse best-effort increments `metrics_daily.errors` before returning `503 report_unavailable`. +For the CEO row only, the canonical operator mechanism is `npm run --silent diagnostic:ceo`. The helper has a fixed production URL, `Accept: application/json`, and `X-Report-Token`; makes at most one request with no retry or redirect; rejects non-200 responses; times out after 15 seconds; caps the response at 1 MiB; and validates strict CEO contract `1.1` before emitting pretty-printed JSON plus one newline. It accepts a 32-to-128-character URL-safe-ASCII credential through either a hidden TTY prompt or `LIGHTHOUSE_REPORT_READ_TOKEN` supplied by an approved non-echoing automation environment, and removes that environment variable immediately after capture. It accepts no arguments, URL override, token file, report file, or `.env`. Noninteractive failures are nonzero and finite: missing/invalid credentials or HTTP `401`/`403` emit `Lighthouse CEO diagnostic access blocked.`; HTTP `503` emits `Lighthouse CEO report unavailable; metrics_daily.errors may have been incremented.`; every other transport, response-safety, parse, schema, or output failure emits `Lighthouse CEO diagnostic failed.`. No response body, numeric HTTP status, schema detail, or token reaches output. These controls protect credential handling but do not authorize the request or remove the possible error-counter side effect. + Agent Smith surfaces inherit downstream behavior: | Smith surface | Current behavior and diagnostic boundary | @@ -225,9 +246,9 @@ Use this order for a Lighthouse `WATCH`, `ALERT`, unavailable report, or analyti 1. **Preserve the initiating evidence.** Record the exact message, timestamp, timezone, delivery lane, requested window, and whether it came from Agent Smith `/health`, `/report`, a scheduled Discord message, Cloudflare, or another surface. 2. **Classify the layer before probing.** Separate report preparation, Lighthouse facts, scheduled service probes, source ingestion, Agent Smith presentation, Discord delivery, and Cloudflare access. 3. **Read local authority.** Read `AGENTS.md`, `SOT.md`, this runbook, the relevant changelog entry, and the CEO schema/fixture matching the observed state. If `source_health` or supplied evidence identifies a producer, also read that producer's canonical files listed above. -4. **Confirm authorization and access material.** Use only the approved production endpoint and an owner-approved, non-echoing credential mechanism. Agent Smith's Cloudflare runtime binding is not a user-facing diagnostic credential source. Never reveal the credential. If the endpoint, mechanism, or authorization is unavailable, report `ACCESS_BLOCKED` and continue only with local evidence. -5. **Read the CEO view first only when both production access and a safe credential mechanism are approved.** Fetch exactly `GET https://lighthouse.buscore.ca/report?view=ceo` with the `X-Admin-Token` header without placing the token literal in command arguments, files, logs, chat, or screenshots. Check HTTP status, JSON parse, `report_contract_version`, `metric_definition_version`, `generated_at`, exact windows, `sources`, `details.service_probes`, and `limitations` before interpreting totals. -6. **Narrow to one secondary view only when evidence requires it.** Use `source_health` for producer-ingestion integrity, `asset` for stored probe/GitHub/rollup detail, or `tgc` for the TGC compatibility diagnostic. Do not fan out across every surface. +4. **Confirm authorization and access material.** Use only the approved production endpoint and the repository's fixed non-echoing helper with an owner-approved report-read credential mechanism. Agent Smith's Cloudflare runtime binding is not a user-facing diagnostic credential source. Never reveal or copy a credential. If the endpoint, mechanism, authorization, or deployed/provisioned `REPORT_READ_TOKEN` is unavailable, report `ACCESS_BLOCKED` and continue only with local evidence. +5. **Read the CEO view first only when both production access and a safe credential mechanism are approved.** Run exactly `npm run --silent diagnostic:ceo`; do not recreate its request with a broader command, substitute an admin credential, add arguments, redirect output to a file, or retry automatically. On validated success, inspect `report_contract_version`, `metric_definition_version`, `generated_at`, exact windows, `sources`, `details.service_probes`, and `limitations` before interpreting totals. Classify `Lighthouse CEO diagnostic access blocked.` as `ACCESS_BLOCKED`; classify `Lighthouse CEO report unavailable; metrics_daily.errors may have been incremented.` as report-unavailable application evidence with the stated possible mutation; classify generic `Lighthouse CEO diagnostic failed.` as an unresolved transport/safety/contract failure. None permits response exposure, an alternate endpoint, or an automatic retry. +6. **Narrow to one secondary view only when evidence requires it.** Use `source_health` for producer-ingestion integrity, `asset` for stored probe/GitHub/rollup detail, or `tgc` for the TGC compatibility diagnostic. The fixed CEO helper cannot select these views, so their production access remains `ACCESS_BLOCKED` unless a separately approved endpoint scope and non-echoing mechanism are provided; do not alter the helper or improvise a raw request. Do not fan out across every surface. 7. **Check Agent Smith separately.** Confirm configured mode and active lane. Production is checked in as `ceo_v1`. A Smith `/health` response can show its configuration and CEO readiness; it does not independently prove Discord delivery, Lighthouse cron completion, TGC source freshness, or monthly archive success. 8. **Correlate by timestamp and ownership.** Compare the alert time, CEO `generated_at`, each source's `data_through`, probe `checked_at` values, and the applicable complete or partial window. Do not compare partial today with a complete day as though they were equivalent. 9. **Report diagnosis with confidence and gaps.** State what is proven, what is inferred, what is unavailable, whether any diagnostic action could have changed evidence, and the smallest next check requiring approval. @@ -299,7 +320,7 @@ These are BUS Core-oriented liveness probes. They do not probe `truegoodcraft.ca Additional failure boundaries: -- HTTP `401 unauthorized` from `/report` proves credential mismatch/missing configuration at that request boundary; it does not prove the Worker is down. +- HTTP `401 unauthorized` from `/report` proves a missing/malformed/mismatched credential or a fail-closed admin/read-secret collision at that request boundary; it does not prove the Worker is down. - HTTP `503 report_unavailable` means report assembly threw and may have incremented `metrics_daily.errors`. - A Cloudflare/Wrangler authorization or account-context error—including the previously observed `7403` during a control-plane read—is classified as access/tooling failure until independent service evidence says otherwise. The number alone is not a Lighthouse application status. - Producer-side `sendBeacon`/`fetch` behavior, whether apparently queued or failed, does not create delivery proof. Lighthouse acceptance must be established from Lighthouse evidence. @@ -325,8 +346,9 @@ Never report a WATCH as an outage, an access failure as application failure, a s ## Known Gaps — Do Not Work Around -- Lighthouse has no least-privilege report-read credential. The current `ADMIN_TOKEN` also authorizes three write routes. -- This repository has no owner-approved, non-echoing production report helper or canonical operator secret-retrieval mechanism. Until one is separately designed and approved, lack of a safe mechanism is `ACCESS_BLOCKED`. +- Version 1.30.0 defines a least-privilege report-read credential and fixed non-echoing CEO helper locally, but no production `REPORT_READ_TOKEN` value, deployment, or verification receipt exists in this bundle. Until separately approved external activation is proven, production read-token access is `ACCESS_BLOCKED`. +- The helper safely consumes a credential but is not a credential-retrieval system. There is still no owner-approved operator secret-retrieval mechanism; do not copy a value from Agent Smith, Cloudflare displays, shell history, files, or another service. +- Agent Smith still holds the broad `LIGHTHOUSE_ADMIN_TOKEN` because it archives monthly snapshots. Full cross-service least privilege remains blocked on a later snapshot-specific credential split or removal of that write. - The active `lighthouse.buscore.ca` attachment is an externally managed Production Custom Domain rather than checked-in route configuration. It was verified on 2026-08-26, but future attachment changes require another control-plane read. - Cloudflare Workers Builds commands remain externally managed rather than checked into this repository. Durable readback on 2026-08-26 verified both production Deploy and Version commands as `npx wrangler versions upload`. If that setting becomes inaccessible or differs, report `BLOCKED BEFORE MERGE` until it is reconciled. - The checked-in manual deployment workflow is the sole authorized production path. The availability and scope of its GitHub `CLOUDFLARE_API_TOKEN` have not been proven by this local bundle. @@ -340,19 +362,21 @@ Never report a WATCH as an outage, an access failure as application failure, a s - Browser producers are fail-soft and provide no end-to-end delivery receipt. - Agent Smith has no independent Discord receipt ledger; a post attempt, log line, or archived monthly snapshot is not receipt proof. -Any helper, token split, further route/config change, cross-repository index repair, receipt mechanism, or automated drift check is a separate future change requiring its owning repository's approval and governance bundle. +Any additional helper, further token split, further route/config change, cross-repository index repair, receipt mechanism, or automated drift check is a separate future change requiring its owning repository's approval and governance bundle. ## Approval Boundaries Separate approval is required before: - Any production endpoint or external-service request. +- Any invocation of `npm run --silent diagnostic:ceo` against its fixed production endpoint, even when a read token is already available; it is a Class 2 request and can increment error evidence on report-assembly failure. - Any Cloudflare control-plane, log, or D1 access, including `release:status` and `release:history`. - Any endpoint in Class 3. - Any commit, branch push, or PR merge. A branch push creates external Worker-version or preview state. - Any `release:upload`; it creates Cloudflare version state even though it does not intentionally promote traffic. - Any manual production deployment-workflow dispatch; it is an active-production deployment operation. Direct `wrangler deploy` is not an authorized alternate path. - Any Cloudflare Workers Builds setting change, custom-domain or route change, observability change, secret operation, direct SQL, migration, scheduled invocation, release, or cross-repository edit. +- Provisioning, verifying, removing, or rotating `REPORT_READ_TOKEN` or `ADMIN_TOKEN`, and any Agent Smith credential migration or snapshot-auth change. - Any merge when the last approved Workers Builds readback is missing, superseded by a later setting change, or differs from exactly `npx wrangler versions upload`. Report that state as `BLOCKED BEFORE MERGE`. The last durable verification was 2026-08-26. With the Workers Builds production command verified as `npx wrangler versions upload`, a merge still requires explicit owner approval and still creates external version state, but it must not be treated as a production release. Production promotion remains a separate, explicit manual action. diff --git a/README.md b/README.md index e7fb7a1..f08ca46 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,13 @@ # buscore-lighthouse +## 1.30.0 least-privilege report diagnostics + +Repository version 1.30.0 adds an optional report-read credential to the local governed bundle while preserving the existing administrative contract. An independently generated cryptographically random `REPORT_READ_TOKEN` containing 32 to 128 URL-safe ASCII characters and differing from `ADMIN_TOKEN` is accepted only through `X-Report-Token` for `GET /report`; `X-Admin-Token`/`ADMIN_TOKEN` remains a backward-compatible report credential and remains the only credential accepted by `POST /campaign`, `POST /notes`, and `POST /report/snapshot`. An identical non-empty admin/read-secret configuration fails every protected read and write closed before database or deferred work. The new read token changes authorization scope only: report views, response payloads, status semantics, schemas, D1 data, retention, schedules, and metrics are unchanged. + +The fixed `npm run --silent diagnostic:ceo` helper makes at most one non-echoing request to the canonical CEO view, validates the strict CEO `1.1` response, and prints pretty-printed JSON only after validation. It has no URL override, arguments, redirect following, retry, user-supplied credential/report file, or `.env` input. Its finite static failures distinguish access blockage, HTTP `503` report unavailability with a possible `metrics_daily.errors` increment, and an otherwise unclassified diagnostic failure without printing response bodies or dynamic details. Production use still requires explicit access approval and is read-mostly rather than zero-write. + +This local bundle creates or exposes no secret value and performs no secret provisioning, production request, upload, deployment, migration, or active-traffic change. Production activation requires separately approved secret provisioning, deployment, and readback. Agent Smith remains on its broad `LIGHTHOUSE_ADMIN_TOKEN` until a later snapshot-specific authorization split allows its monthly archive write to be separated safely. + ## 1.29.4 release-control reconciliation Repository version 1.29.4 records the verified 1.29.3 production promotion and repairs repository-controlled release authority. Active production is Worker version `ba611ac1-653d-47a2-a465-a85f4124b6b6`, promoted from merged `main` commit `59231d09084d0fa4db71012b6f29550886c5b605` by Cloudflare build `793715ef-6123-4d98-a4a7-797634d07812`. The canonical production hostname is `lighthouse.buscore.ca`, attached as a Production Custom Domain rather than a Worker Route. The additional public `workers.dev` surface and branch/version previews are not canonical diagnostic endpoints. The cron remains `5 0 * * *`; CEO report and metric-definition contract remain `1.1`. @@ -14,7 +22,13 @@ Repository version 1.29.3 adds the canonical operations and diagnostics runbook ## Operations and diagnostics -Use [`OPERATIONS.md`](OPERATIONS.md) as the canonical runbook for Lighthouse alerts, analytics diagnosis, endpoint selection, credential boundaries, and request side effects. Read it after `SOT.md` and before contacting production or Cloudflare. `PHASE2_ANALYTICS_NOTES.md` and `PHASE3_ANALYTICS_NOTES.md` are historical implementation records, not current runbooks. +Use [`OPERATIONS.md`](OPERATIONS.md) as the canonical runbook for Lighthouse alerts, analytics diagnosis, endpoint selection, credential boundaries, and request side effects. Read it after `SOT.md` and before contacting production or Cloudflare. When a CEO production read is explicitly approved, use the fixed helper and enter the credential through its hidden prompt: + +```bash +npm run --silent diagnostic:ceo +``` + +Automation may supply a 32-to-128-character URL-safe-ASCII `LIGHTHOUSE_REPORT_READ_TOKEN` through an approved non-echoing environment mechanism. Never place the value in command arguments, files, `.env`, logs, chat, or screenshots. The helper removes the variable from its process after capture, does not grant authorization, and must not be run merely because the command exists. `PHASE2_ANALYTICS_NOTES.md` and `PHASE3_ANALYTICS_NOTES.md` are historical implementation records, not current runbooks. ## 1.29.2 service-probe truth repair @@ -270,7 +284,7 @@ Notes: `GET /report?view=ceo` is the preferred decision-report input. Its strict schema is `contracts/ceo-v1/report.schema.json`. The window keys are `today`, `latest_complete_day`, `last_7_complete_days`, `previous_7_complete_days`, and `last_30_complete_days`. Every metric carries those same keys and is either a finite aggregate or `null` when its source is unavailable. -Source state distinguishes `available` from `unavailable`, `fresh` from `stale` or `unknown`, and `full`, `partial`, or `unavailable` coverage. The contract retains `full` for a future source with explicit completeness proof. Current sparse event/counter sources do not prove that every day in a decision window was observable, so available current windows remain partial; they do not claim full coverage from a recent watermark alone. Trusted artifact-click interest begins on `2026-08-10`: earlier intent rows are excluded from sums and the watermark, wholly earlier windows are `null`, and spanning or later windows contain partial totals from the boundary forward. The `buscore_site` source definition reflects that boundary and the limitations explicitly state that pre-definition history is excluded. A successful aggregate query can return numeric zero, but without an all-history watermark its source is `unknown` with `source_history_missing`; an old watermark is `stale` with `source_data_stale`. When a source is unavailable, dependent inquiry-attribution, product-version/failure, or service-probe detail is `null`, not an empty list. Voluntary-inquiry attribution uses only 14 fixed privacy-safe buckets and never emits a raw label. The endpoint is independently guarded per source, aggregate-only, admin-token protected, and contains no PII or persistent identifiers. Its configured-leads path uses nine D1 statements in batches of at most three; client-supplied app versions are SQL-ranked and limited to ten before reaching Worker memory. Strict Ajv 2020 tests validate all fixtures plus representative live producer outputs. Existing views remain available for diagnostics and rollback. +Source state distinguishes `available` from `unavailable`, `fresh` from `stale` or `unknown`, and `full`, `partial`, or `unavailable` coverage. The contract retains `full` for a future source with explicit completeness proof. Current sparse event/counter sources do not prove that every day in a decision window was observable, so available current windows remain partial; they do not claim full coverage from a recent watermark alone. Trusted artifact-click interest begins on `2026-08-10`: earlier intent rows are excluded from sums and the watermark, wholly earlier windows are `null`, and spanning or later windows contain partial totals from the boundary forward. The `buscore_site` source definition reflects that boundary and the limitations explicitly state that pre-definition history is excluded. A successful aggregate query can return numeric zero, but without an all-history watermark its source is `unknown` with `source_history_missing`; an old watermark is `stale` with `source_data_stale`. When a source is unavailable, dependent inquiry-attribution, product-version/failure, or service-probe detail is `null`, not an empty list. Voluntary-inquiry attribution uses only 14 fixed privacy-safe buckets and never emits a raw label. The endpoint is independently guarded per source, aggregate-only, protected by the current dual report-credential contract, and contains no PII or persistent identifiers. Its configured-leads path uses nine D1 statements in batches of at most three; client-supplied app versions are SQL-ranked and limited to ten before reaching Worker memory. Strict Ajv 2020 tests validate all fixtures plus representative live producer outputs. Existing views remain available for diagnostics and rollback. The current protected report families are bare legacy `/report`, `view=fleet`, `view=site`, `view=tgc`, `view=source_health`, `view=asset`, `view=monthly`, and `view=ceo`. `view=site` requires a registered `site_key`; the specialized views retain their existing required parameters and response contracts. Explicit `view=legacy` is invalid; omit `view` for the legacy contract. @@ -742,7 +756,8 @@ Pageview ingestion notes: Required bindings/secrets: - `DB` - `MANIFEST_R2` -- `ADMIN_TOKEN` +- `ADMIN_TOKEN` (required for protected writes and retained as a backward-compatible report credential) +- `REPORT_READ_TOKEN` (optional distinct, cryptographically random 32-to-128-character URL-safe-ASCII secret enabling GET-report-only `X-Report-Token` authentication) - `IGNORED_IP` (optional) - `CF_API_TOKEN` (required for scheduled Buscore traffic capture) - `CF_ZONE_TAG` (required for scheduled Buscore traffic capture) @@ -751,7 +766,7 @@ Required bindings/secrets: - `GITHUB_REPO` (optional; defaults to `True-Good-Craft/TGC-BUS-Core` for the scheduled GitHub snapshot and latest-release probe) - `GITHUB_TOKEN` (optional secret; raises scheduled GitHub API snapshot rate limits and is not required by the public latest-release HEAD probe) -`ADMIN_TOKEN` is a broad administrative credential, not a read-only token. It protects report reads and the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes. Do not expose its value or treat report-read approval as write approval. +`ADMIN_TOKEN` remains broad: it authorizes report reads and the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes. `REPORT_READ_TOKEN` is accepted only for `GET /report`, only through `X-Report-Token`, and only when it is independently generated, cryptographically random, contains 32 to 128 URL-safe ASCII characters (`A-Z`, `a-z`, `0-9`, `_`, and `-`), and differs from `ADMIN_TOKEN`. A malformed report secret disables only that read path while preserving a distinct admin fallback; identical non-empty admin/read secrets fail every protected read and write closed. Production credential values never belong in source, `wrangler.toml`, command arguments, files, logs, chat, or screenshots; `.dev.vars.example` contains local placeholders only and is not a helper credential source. A report-read approval never authorizes a write. No new bindings or secrets are introduced by pageview ingestion. @@ -802,7 +817,7 @@ Configure that environment's returned database ID explicitly. The production `DB ### 4. Apply migrations -Local migration uses the stable binding name and cannot reach remote D1. A new remote environment must use its explicit configuration. The checked-in configuration targets production, so its remote command requires separate production-migration approval. Version 1.29.4 has no migration. +Local migration uses the stable binding name and cannot reach remote D1. A new remote environment must use its explicit configuration. The checked-in configuration targets production, so its remote command requires separate production-migration approval. Version 1.30.0 has no migration. ```bash # local (for wrangler dev) @@ -817,10 +832,11 @@ npx wrangler d1 migrations apply DB --remote ### 5. Set secrets -Production secrets already exist; ordinary setup must not recreate, reveal, or rotate them. For a separately approved new environment, target its explicit configuration: +The production secrets in the last verified inventory already exist; `REPORT_READ_TOKEN` is not part of that verified production state. Ordinary setup must not recreate, reveal, or rotate production secrets. For a separately approved new environment, independently generate a distinct cryptographically random 32-to-128-character URL-safe-ASCII report secret through a non-echoing mechanism and target its explicit configuration: ```bash npx wrangler secret put ADMIN_TOKEN --config YOUR_ENVIRONMENT_WRANGLER_CONFIG +npx wrangler secret put REPORT_READ_TOKEN --config YOUR_ENVIRONMENT_WRANGLER_CONFIG npx wrangler secret put CF_API_TOKEN --config YOUR_ENVIRONMENT_WRANGLER_CONFIG npx wrangler secret put CF_ZONE_TAG --config YOUR_ENVIRONMENT_WRANGLER_CONFIG npx wrangler secret put TELEMETRY_RATE_LIMIT_SECRET --config YOUR_ENVIRONMENT_WRANGLER_CONFIG diff --git a/SOT.md b/SOT.md index ea39327..ac2206a 100644 --- a/SOT.md +++ b/SOT.md @@ -1,5 +1,17 @@ # Lighthouse — Source of Truth +## Least-privilege report diagnostics — v1.30.0 + +Version 1.30.0 adds an optional, additive report-read credential without removing the established administrative contract. A usable `REPORT_READ_TOKEN` Worker secret is an independently generated cryptographically random string of 32 to 128 URL-safe ASCII characters (`A-Z`, `a-z`, `0-9`, `_`, and `-`), is distinct from `ADMIN_TOKEN`, and is accepted for any `GET /report` view only through an exact `X-Report-Token` match. Existing callers may continue to authenticate report reads through the exact `X-Admin-Token`/`ADMIN_TOKEN` match. `POST /campaign`, `POST /notes`, and `POST /report/snapshot` remain administrative writes and accept only `X-Admin-Token`; the report-read header never authorizes them. A missing, blank, malformed, or incorrect report credential fails closed with the existing `401 {"ok":false,"error":"unauthorized"}` response while a distinct administrative fallback remains usable. If the two configured secrets are identical and non-empty, every protected report read and administrative write fails closed with that same `401` before database or deferred work; this prevents a provisioning collision from silently granting write authority to the nominal read credential. Report routes, views, payloads, schemas, status semantics, storage, retention, and scheduling are otherwise unchanged. + +`REPORT_READ_TOKEN` is an authorization-scope boundary, not a promise of zero evidence mutation. Bare `/report`, `view=fleet`, and `view=site` retain their best-effort previous-completed-day Cloudflare traffic capture/upsert. The stored-data `view=ceo`, `view=tgc`, `view=source_health`, `view=asset`, and `view=monthly` paths continue to skip that refresh, but any report-assembly exception may still best-effort increment `metrics_daily.errors`. Production access therefore still requires explicit approval, and the canonical first diagnostic remains the stored-data CEO view. + +The repository provides one fixed, non-echoing operator helper: `npm run --silent diagnostic:ceo`. With a credential satisfying the same 32-to-128 URL-safe-ASCII contract, it performs at most one `GET https://lighthouse.buscore.ca/report?view=ceo` request with `Accept: application/json` and `X-Report-Token`. It accepts no URL or command-line override, retry, redirect, user-supplied credential/report file, or `.env` input; uses a hidden interactive prompt or the `LIGHTHOUSE_REPORT_READ_TOKEN` automation environment variable; removes that variable from the helper process environment immediately after capture; enforces a 15-second timeout and 1 MiB response limit; rejects redirects and non-200 responses; and writes pretty-printed JSON plus one newline to stdout only after fatal UTF-8 decoding, JSON parsing, reflected-token suppression, and validation against the existing strict CEO contract `1.1`. Noninteractive failures are nonzero and use only three finite static lines: missing/invalid credentials and HTTP `401`/`403` emit `Lighthouse CEO diagnostic access blocked.`; HTTP `503` emits `Lighthouse CEO report unavailable; metrics_daily.errors may have been incremented.`; every other transport, response-safety, parse, schema, or output failure emits `Lighthouse CEO diagnostic failed.`. The helper never emits the credential, response body, numeric HTTP status, or schema detail. It does not grant production authorization and cannot make a CEO request zero-write when report assembly fails. + +This local governed bundle changes behavior, auth, configuration, and operator workflow, and documents the remaining cross-service boundary. It adds the optional `REPORT_READ_TOKEN` secret-binding capability, but introduces no endpoint, report field, D1 migration, D1/R2 binding or resource change, retention change, scheduled behavior, secret value, secret provisioning, deployment, or active-production interaction. `ADMIN_TOKEN` remains required for administrative writes and remains a backward-compatible report credential. Production use of `REPORT_READ_TOKEN` requires separately approved secret provisioning, deployment, and readback. Rollback to the prior Worker version restores the prior admin-only read contract, so the administrative fallback must remain available until read-token consumers have been verified. + +Agent Smith remains outside this least-privilege completion boundary. Its current runtime still uses `LIGHTHOUSE_ADMIN_TOKEN` for report reads and the administrative monthly snapshot write. Migrating its reads alone would leave the broad credential present. Removing that credential from Agent Smith requires a later owner-approved snapshot-specific authorization split (or removal of that write), coordinated cross-repository documentation/version bundles, external secret changes, verification, and only then any administrative-token rotation. + ## Release-control and infrastructure reconciliation — v1.29.4 Version 1.29.4 records the owner-approved read-only Cloudflare control-plane audit performed on 2026-08-26 and aligns repository-controlled release tooling with the verified infrastructure. The audit established that the merge of `main` commit `59231d09084d0fa4db71012b6f29550886c5b605` caused Cloudflare Workers Builds build `793715ef-6123-4d98-a4a7-797634d07812` to run the configured production command `npx wrangler deploy` without a validation build command and promote Worker version `ba611ac1-653d-47a2-a465-a85f4124b6b6` to 100% of production traffic. That version was created at `2026-08-26T22:57:30.628Z`. The canonical production hostname is `lighthouse.buscore.ca`, attached as a Production Custom Domain rather than a Worker Route. The additional production `workers.dev` surface and branch/version previews are enabled but are not canonical diagnostic endpoints. The active cron is `5 0 * * *` (`00:05 UTC`). @@ -110,7 +122,7 @@ Existing public-site `/metrics/event` and legacy `/metrics/pageview` behavior re - Lighthouse is a single Cloudflare Worker that acts as a minimal, privacy-first, aggregate-first stats source with a multi-site event ingestion spine and a legacy BUS Core pageview ingestion path. - Lighthouse is a generic, deterministic metrics primitive; BUS Core is a current observed client/use-case, not a runtime dependency. -- It serves/proxies manifest data from R2, records daily aggregate counters in D1, records daily Buscore traffic snapshots in D1, accepts first-party pageview events into D1, and exposes an admin-protected multi-view `GET /report` endpoint. +- It serves/proxies manifest data from R2, records daily aggregate counters in D1, records daily Buscore traffic snapshots in D1, accepts first-party pageview events into D1, and exposes a credential-protected multi-view `GET /report` endpoint. - It does not post reports to Discord. Discord/operator report requirements are satisfied by authenticated local report payloads unless an outbound sender is explicitly approved in this SOT. - Runtime surface: Worker `fetch` handler plus one scheduled daily job whose independently fail-soft tasks capture the previous completed traffic day, write rollups and public GitHub/probe snapshots, and prune bounded-retention data. @@ -165,8 +177,8 @@ Four additive D1 tables and their scheduled writers: - `health_checks` — active funnel liveness probes, once per daily cron (low frequency). Each probe is isolated and never throws; a failure records `ok = 0` with a note and cannot break reporting or the rest of the scheduled run. Pruned to ~90 days. New routes/views: -- `POST /campaign` — admin-token-protected (same `ADMIN_TOKEN` as `/report`). Operator/aggregate data only. `201 {ok,id}` on success; `401` without token; `400 invalid_json`/`invalid_campaign`; `503 campaign_insert_failed`. -- `GET /report?view=asset` — admin-protected read of stored Phase 2 aggregates: latest + recent `daily_rollup`, latest `github_snapshots`, latest-per-target `health_checks`, and recent `campaign_log` with downstream event/lead counts joined by `tagged_src`/`utm_campaign`. Skips the best-effort traffic refresh (reads stored aggregates only). Existing `legacy`/`fleet`/`site`/`source_health` views are unchanged. +- `POST /campaign` — administrative-write protected through `X-Admin-Token`/`ADMIN_TOKEN`; `REPORT_READ_TOKEN` is never accepted. Operator/aggregate data only. `201 {ok,id}` on success; `401` without valid admin authentication; `400 invalid_json`/`invalid_campaign`; `503 campaign_insert_failed`. +- `GET /report?view=asset` — protected read of stored Phase 2 aggregates: latest + recent `daily_rollup`, latest `github_snapshots`, latest-per-target `health_checks`, and recent `campaign_log` with downstream event/lead counts joined by `tagged_src`/`utm_campaign`. It accepts the current report-read contract defined in the v1.30.0 section and skips the best-effort traffic refresh (reads stored aggregates only). Existing `legacy`/`fleet`/`site`/`source_health` views are unchanged. Scheduling: the existing daily cron `5 0 * * *` now also runs, after traffic capture (so the rollup sees the day's traffic row), the daily-rollup / github-snapshot / health-check / prune writers. Each is independently fail-soft; one failing cannot break the others or core reporting. @@ -194,10 +206,10 @@ Deterministic scoring (pure, exported functions; documented weights): - Each returns `{ score: number|null, available, reason, weight, inputs }`. - **Honesty invariants (non-negotiable):** a score is `null` (never faked) when its primary input is missing, with a reason such as `awaiting first scheduled rollup` / `insufficient data`; every score carries its raw `inputs` (raw numbers are never hidden); a score is explicitly **not a valuation**; Acquisition Readiness is **capped by Reliability** and returns `null` if Reliability is unavailable; **stars are weighted ≤10%** of GitHub Trust. Downloads are not users; update checks are not active users. -New routes/views (all admin-token-protected, same `ADMIN_TOKEN` as `/report`): -- `GET /report?view=monthly` — previous completed calendar month's structured asset data: wQPI MoM, downloads, attributed leads + lead quality, known-version check-in average + adoption (labelled proxy), community posts → downstream (per channel), reliability (uptime/errors/freshness), GitHub health, the five scores with inputs, previous-month Acquisition Readiness for the delta, and recent operator notes. Skips the traffic refresh (reads stored aggregates). Missing pieces are `null`/`awaiting first scheduled rollup`, never faked. -- `POST /notes` — insert an operator note `{ note, tag? }`. -- `POST /report/snapshot` — archive a generated brief `{ kind (daily|weekly|monthly), status?, wqpi?, summary_json?, narrative? }`. +New routes/views: +- `GET /report?view=monthly` — protected by the current report-read contract defined in the v1.30.0 section; returns the previous completed calendar month's structured asset data: wQPI MoM, downloads, attributed leads + lead quality, known-version check-in average + adoption (labelled proxy), community posts → downstream (per channel), reliability (uptime/errors/freshness), GitHub health, the five scores with inputs, previous-month Acquisition Readiness for the delta, and recent operator notes. Skips the traffic refresh (reads stored aggregates). Missing pieces are `null`/`awaiting first scheduled rollup`, never faked. +- `POST /notes` — administrative-write protected through `X-Admin-Token`/`ADMIN_TOKEN`; insert an operator note `{ note, tag? }`. +- `POST /report/snapshot` — administrative-write protected through `X-Admin-Token`/`ADMIN_TOKEN`; archive a generated brief `{ kind (daily|weekly|monthly), status?, wqpi?, summary_json?, narrative? }`. Privacy: `report_snapshots`, `operator_notes`, and `view=monthly` are aggregate/operator-authored. No emails, `bc_uid`/`bc_sid`, `anon_user_id`/`session_id`, raw or hashed IPs, user-agent, or fingerprints. `summary_json` carries compact aggregate numbers only. @@ -360,10 +372,10 @@ The following rules are non-negotiable unless this SOT is explicitly revised: ### Reporting - `GET /report` - - Requires header `X-Admin-Token`. - - Auth check is exact equality against `env.ADMIN_TOKEN`: - - `const token = request.headers.get("X-Admin-Token")` - - `if (!env.ADMIN_TOKEN || !token || token !== env.ADMIN_TOKEN) { ...401 unauthorized... }` + - Accepts either the exact usable `X-Report-Token`/`env.REPORT_READ_TOKEN` match or the backward-compatible exact non-empty `X-Admin-Token`/`env.ADMIN_TOKEN` match. + - `REPORT_READ_TOKEN` is optional and usable only when it is an independently generated cryptographically random string containing 32 to 128 URL-safe ASCII characters (`A-Z`, `a-z`, `0-9`, `_`, and `-`). Absence or malformed configuration never disables a distinct administrative fallback and never authorizes a request. + - If non-empty `REPORT_READ_TOKEN` and `ADMIN_TOKEN` values are identical, both report-read paths and all three administrative writes fail closed with `401` before database or deferred work. + - Header names are not interchangeable: the read credential is accepted only through `X-Report-Token`, and placing it in `X-Admin-Token` does not grant administrative access. - On auth failure: returns `401` JSON `{ "ok": false, "error": "unauthorized" }`. - If `view` is omitted, blank, or absent, `/report` preserves the legacy response shape with `today`, `yesterday`, `last_7_days`, additive top-level `last_30_days`, `month_to_date`, `trends`, additive top-level `traffic`, additive top-level `human_traffic`, additive top-level `identity`, additive top-level `site_events`, and additive top-level `release_signals`. - Bare legacy `/report` continues to support `site_key` with optional flags `exclude_test_mode` (default `true`) and `production_only` (default from tracked-site `production_only_default`) for the additive `site_events` block only. @@ -466,7 +478,8 @@ Required bindings/secrets used by code: - `DB` - `MANIFEST_R2` -- `ADMIN_TOKEN` +- `ADMIN_TOKEN` — required exact-match administrative credential accepted through `X-Admin-Token` for the three protected writes and retained as a backward-compatible report credential. +- `REPORT_READ_TOKEN` — optional secret; when it is an independently generated cryptographically random string containing 32 to 128 URL-safe ASCII characters (`A-Z`, `a-z`, `0-9`, `_`, and `-`) and differs from `ADMIN_TOKEN`, its exact value is accepted only through `X-Report-Token` for `GET /report`. It is never accepted by a write route and is not configured in `[vars]`. An identical non-empty admin/read configuration disables every protected read and write until corrected. - `IGNORED_IP` — optional; if set, requests whose `CF-Connecting-IP` exactly matches this value skip counter increments but receive normal responses. - `CF_API_TOKEN` — required for the approved daily Buscore traffic capture job. - `CF_ZONE_TAG` — required for the approved daily Buscore traffic capture job. @@ -684,8 +697,9 @@ Rules: - Anonymous continuity values are first-party random UUID-like values and remain independent from `ip_hash` and `user_agent_hash`. - Lighthouse must not combine `anon_user_id` with `ip_hash` or `user_agent_hash` into synthetic identity. - Traffic capture uses Cloudflare aggregate analytics only; no raw request logging is introduced outside the documented narrow pageview ingestion path. -- `/report` is protected by `X-Admin-Token` exact match to `env.ADMIN_TOKEN`. -- `ADMIN_TOKEN` is a broad administrative credential, not a read-only diagnostic token. The same exact-match credential protects `GET /report` and the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes. Authorization to read a report does not imply authorization to call those writes. +- `GET /report` accepts an exact `X-Report-Token` match to an optional 32-to-128-character URL-safe-ASCII `env.REPORT_READ_TOKEN`, or the backward-compatible exact non-empty `X-Admin-Token` match to `env.ADMIN_TOKEN`, only while the two configured secrets do not collide. +- `ADMIN_TOKEN` remains a broad administrative credential. It protects the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes and remains accepted for report reads. `REPORT_READ_TOKEN` is GET-report-only and is never valid for those writes. Authorization to read a report does not imply authorization to call a write. +- `X-Report-Token` is not added to browser CORS allow-headers. Version 1.30.0 does not create a browser-readable report surface. ## 8. Explicit Non-Features diff --git a/package-lock.json b/package-lock.json index 4e30ce8..9241c2b 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "buscore-lighthouse", - "version": "1.29.4", + "version": "1.30.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "buscore-lighthouse", - "version": "1.29.4", + "version": "1.30.0", "license": "ISC", "devDependencies": { "@cloudflare/workers-types": "^4.20260305.0", diff --git a/package.json b/package.json index 5e12b7b..b5b5bf1 100644 --- a/package.json +++ b/package.json @@ -1,9 +1,10 @@ { "name": "buscore-lighthouse", - "version": "1.29.4", + "version": "1.30.0", "description": "Standalone deterministic metrics worker: manifest proxy + fixed daily counters + protected on-demand reporting.", "scripts": { "dev": "wrangler dev", + "diagnostic:ceo": "node scripts/read-ceo-report.mjs", "release:upload": "wrangler versions upload", "release:status": "wrangler deployments status --json", "release:history": "wrangler deployments list --json", diff --git a/scripts/read-ceo-report.mjs b/scripts/read-ceo-report.mjs new file mode 100644 index 0000000..f7c0537 --- /dev/null +++ b/scripts/read-ceo-report.mjs @@ -0,0 +1,300 @@ +import { readFileSync } from "node:fs"; +import { resolve } from "node:path"; +import { StringDecoder } from "node:string_decoder"; +import { pathToFileURL } from "node:url"; + +import Ajv2020 from "ajv/dist/2020.js"; +import addFormats from "ajv-formats"; + +export const CEO_REPORT_URL = "https://lighthouse.buscore.ca/report?view=ceo"; +export const REPORT_TOKEN_ENV = "LIGHTHOUSE_REPORT_READ_TOKEN"; +export const REPORT_TOKEN_HEADER = "X-Report-Token"; +export const REQUEST_TIMEOUT_MS = 15_000; +export const MAX_RESPONSE_BYTES = 1024 * 1024; +export const MIN_REPORT_TOKEN_BYTES = 32; +export const MAX_REPORT_TOKEN_BYTES = 128; +export const FAILURE_LINE = "Lighthouse CEO diagnostic failed.\n"; +export const ACCESS_BLOCKED_LINE = "Lighthouse CEO diagnostic access blocked.\n"; +export const REPORT_UNAVAILABLE_LINE = "Lighthouse CEO report unavailable; metrics_daily.errors may have been incremented.\n"; +export const TOKEN_PROMPT = "Lighthouse report token: "; + +const CEO_SCHEMA_URL = new URL("../contracts/ceo-v1/report.schema.json", import.meta.url); +const REPORT_TOKEN_PATTERN = /^[A-Za-z0-9_-]{32,128}$/; +const FAILURE_KIND = Object.freeze({ + ACCESS_BLOCKED: "access_blocked", + DIAGNOSTIC: "diagnostic", + REPORT_UNAVAILABLE: "report_unavailable", +}); + +class DiagnosticFailure extends Error { + constructor(kind = FAILURE_KIND.DIAGNOSTIC) { + super("diagnostic_failed"); + this.name = "DiagnosticFailure"; + this.kind = kind; + } +} + +let reportValidator; + +function fail(kind = FAILURE_KIND.DIAGNOSTIC) { + throw new DiagnosticFailure(kind); +} + +export function isUsableReportToken(token) { + if (typeof token !== "string" || !REPORT_TOKEN_PATTERN.test(token)) return false; + const bytes = Buffer.byteLength(token, "utf8"); + return bytes >= MIN_REPORT_TOKEN_BYTES && bytes <= MAX_REPORT_TOKEN_BYTES; +} + +function getReportValidator() { + if (reportValidator) return reportValidator; + + const schema = JSON.parse(readFileSync(CEO_SCHEMA_URL, "utf8")); + const ajv = new Ajv2020({ strict: true, allErrors: true }); + addFormats(ajv); + reportValidator = ajv.compile(schema); + return reportValidator; +} + +export function takeEnvironmentToken(environment) { + if (!environment || typeof environment !== "object") fail(); + + const hadToken = Object.prototype.hasOwnProperty.call(environment, REPORT_TOKEN_ENV); + const value = environment[REPORT_TOKEN_ENV]; + if (!Reflect.deleteProperty(environment, REPORT_TOKEN_ENV)) fail(); + if (!hadToken) return null; + if (!isUsableReportToken(value)) fail(FAILURE_KIND.ACCESS_BLOCKED); + return value; +} + +export function readHiddenToken({ input, output }) { + return new Promise((resolveToken, rejectToken) => { + if ( + !input?.isTTY + || !output?.isTTY + || typeof input.setRawMode !== "function" + || typeof input.on !== "function" + || typeof input.removeListener !== "function" + ) { + rejectToken(new DiagnosticFailure(FAILURE_KIND.ACCESS_BLOCKED)); + return; + } + + const wasRaw = Boolean(input.isRaw); + const wasPaused = typeof input.isPaused === "function" ? input.isPaused() : false; + const decoder = new StringDecoder("utf8"); + let token = ""; + let settled = false; + + const restoreInput = () => { + input.removeListener("data", onData); + input.removeListener("end", onEnd); + input.removeListener("error", onError); + try { + input.setRawMode(wasRaw); + } catch { + // The caller still receives only the static failure contract. + } + if (wasPaused && typeof input.pause === "function") input.pause(); + }; + + const settle = (ok) => { + if (settled) return; + settled = true; + restoreInput(); + try { + output.write("\n"); + } catch { + rejectToken(new DiagnosticFailure()); + return; + } + if (ok && isUsableReportToken(token)) resolveToken(token); + else rejectToken(new DiagnosticFailure(FAILURE_KIND.ACCESS_BLOCKED)); + }; + + function onData(chunk) { + const text = decoder.write(Buffer.from(chunk)); + for (const character of text) { + if (character === "\u0003" || character === "\u0004") { + settle(false); + return; + } + if (character === "\r" || character === "\n") { + settle(true); + return; + } + if (character === "\b" || character === "\u007f") { + token = Array.from(token).slice(0, -1).join(""); + continue; + } + if (character < " ") continue; + + const nextToken = `${token}${character}`; + if (Buffer.byteLength(nextToken, "utf8") > MAX_REPORT_TOKEN_BYTES) { + settle(false); + return; + } + token = nextToken; + } + } + + function onEnd() { + settle(false); + } + + function onError() { + settle(false); + } + + try { + input.setRawMode(true); + if (typeof input.resume === "function") input.resume(); + output.write(TOKEN_PROMPT); + input.on("data", onData); + input.on("end", onEnd); + input.on("error", onError); + } catch { + settle(false); + } + }); +} + +async function readBoundedBody(response) { + const contentLength = response.headers?.get?.("content-length"); + if (typeof contentLength === "string" && /^\d+$/.test(contentLength.trim())) { + const advertisedBytes = Number(contentLength); + if (!Number.isSafeInteger(advertisedBytes) || advertisedBytes > MAX_RESPONSE_BYTES) fail(); + } + + if (!response.body || typeof response.body.getReader !== "function") fail(); + + const reader = response.body.getReader(); + const chunks = []; + let totalBytes = 0; + + while (true) { + const { done, value } = await reader.read(); + if (done) break; + if (!(value instanceof Uint8Array)) fail(); + + totalBytes += value.byteLength; + if (totalBytes > MAX_RESPONSE_BYTES) { + try { + await reader.cancel(); + } catch { + // Cancellation failure does not alter the static diagnostic failure. + } + fail(); + } + chunks.push(value); + } + + const body = new Uint8Array(totalBytes); + let offset = 0; + for (const chunk of chunks) { + body.set(chunk, offset); + offset += chunk.byteLength; + } + return body; +} + +export async function fetchValidatedCeoReport({ + token, + fetchImpl = globalThis.fetch, + scheduleTimeout = setTimeout, + cancelTimeout = clearTimeout, +}) { + if (!isUsableReportToken(token)) fail(FAILURE_KIND.ACCESS_BLOCKED); + if (typeof fetchImpl !== "function") fail(); + + const controller = new AbortController(); + const timeout = scheduleTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS); + + try { + const response = await fetchImpl(CEO_REPORT_URL, { + method: "GET", + headers: { + Accept: "application/json", + [REPORT_TOKEN_HEADER]: token, + }, + redirect: "manual", + signal: controller.signal, + }); + + if (!response || response.redirected === true) fail(); + if (response.status === 401 || response.status === 403) fail(FAILURE_KIND.ACCESS_BLOCKED); + if (response.status === 503) fail(FAILURE_KIND.REPORT_UNAVAILABLE); + if (response.status !== 200) fail(); + + const body = await readBoundedBody(response); + const text = new TextDecoder("utf-8", { fatal: true }).decode(body); + if (text.includes(token)) fail(); + + const report = JSON.parse(text); + if (!getReportValidator()(report)) fail(); + + const output = `${JSON.stringify(report, null, 2)}\n`; + if (output.includes(token)) fail(); + return output; + } catch (error) { + if (error instanceof DiagnosticFailure) throw error; + fail(); + } finally { + cancelTimeout(timeout); + } +} + +function writeStaticFailure(stderr, kind) { + const line = kind === FAILURE_KIND.ACCESS_BLOCKED + ? ACCESS_BLOCKED_LINE + : kind === FAILURE_KIND.REPORT_UNAVAILABLE + ? REPORT_UNAVAILABLE_LINE + : FAILURE_LINE; + try { + stderr?.write?.(line); + } catch { + // There is no secondary output channel and dynamic errors are forbidden. + } +} + +export async function runDiagnostic({ + argv = [], + environment = process.env, + stdin = process.stdin, + stdout = process.stdout, + stderr = process.stderr, + fetchImpl = globalThis.fetch, + promptImpl = readHiddenToken, + scheduleTimeout = setTimeout, + cancelTimeout = clearTimeout, +} = {}) { + let environmentToken = null; + + try { + environmentToken = takeEnvironmentToken(environment); + if (!Array.isArray(argv) || argv.length !== 0) fail(); + + const token = environmentToken ?? await promptImpl({ input: stdin, output: stderr }); + if (!isUsableReportToken(token)) fail(FAILURE_KIND.ACCESS_BLOCKED); + + const output = await fetchValidatedCeoReport({ + token, + fetchImpl, + scheduleTimeout, + cancelTimeout, + }); + stdout.write(output); + return 0; + } catch (error) { + const kind = error instanceof DiagnosticFailure ? error.kind : FAILURE_KIND.DIAGNOSTIC; + writeStaticFailure(stderr, kind); + return 1; + } finally { + environmentToken = null; + } +} + +const invokedPath = process.argv[1] ? pathToFileURL(resolve(process.argv[1])).href : null; +if (invokedPath === import.meta.url) { + process.exitCode = await runDiagnostic({ argv: process.argv.slice(2) }); +} diff --git a/src/index.ts b/src/index.ts index 6d31793..5503113 100644 --- a/src/index.ts +++ b/src/index.ts @@ -15,6 +15,7 @@ export interface Env { BUSCORE_LEADS_DB?: D1Database; MANIFEST_R2: R2Bucket; ADMIN_TOKEN: string; + REPORT_READ_TOKEN?: string; IGNORED_IP: string; CF_API_TOKEN: string; CF_ZONE_TAG: string; @@ -5147,6 +5148,43 @@ function safeRatio(numerator: number, denominator: number): number { return numerator / Math.max(1, denominator); } +const REPORT_READ_TOKEN_PATTERN = /^[A-Za-z0-9_-]{32,128}$/; + +function hasExactHeaderToken(request: Request, headerName: string, expectedToken: string | undefined): boolean { + if (!expectedToken) return false; + const presentedToken = request.headers.get(headerName); + return presentedToken !== null && presentedToken === expectedToken; +} + +function hasCredentialCollision(env: Env): boolean { + return Boolean( + env.ADMIN_TOKEN && + env.REPORT_READ_TOKEN && + env.ADMIN_TOKEN === env.REPORT_READ_TOKEN + ); +} + +function hasUsableReportReadToken(env: Env): env is Env & { REPORT_READ_TOKEN: string } { + return Boolean(env.REPORT_READ_TOKEN && REPORT_READ_TOKEN_PATTERN.test(env.REPORT_READ_TOKEN)); +} + +function hasAdminAccess(request: Request, env: Env): boolean { + if (hasCredentialCollision(env)) return false; + return hasExactHeaderToken(request, "X-Admin-Token", env.ADMIN_TOKEN); +} + +function hasReportReadAccess(request: Request, env: Env): boolean { + if (hasCredentialCollision(env)) return false; + return ( + hasAdminAccess(request, env) || + ( + request.method === "GET" && + hasUsableReportReadToken(env) && + hasExactHeaderToken(request, "X-Report-Token", env.REPORT_READ_TOKEN) + ) + ); +} + function withCors(request: Request, response: Response, allowMethods: string = "GET, OPTIONS"): Response { const headers = new Headers(response.headers); @@ -6587,8 +6625,7 @@ export default { // Admin-token-protected operator route for logging community posts. // Operator-authored aggregate/annotation data only; no user data, no PII. if (url.pathname === "/campaign" && request.method === "POST") { - const token = request.headers.get("X-Admin-Token"); - if (!env.ADMIN_TOKEN || !token || token !== env.ADMIN_TOKEN) { + if (!hasAdminAccess(request, env)) { return withCors(request, Response.json({ ok: false, error: "unauthorized" }, { status: 401 })); } @@ -6615,8 +6652,7 @@ export default { // Admin-token-protected operator note insert (feeds the monthly narrative). if (url.pathname === "/notes" && request.method === "POST") { - const token = request.headers.get("X-Admin-Token"); - if (!env.ADMIN_TOKEN || !token || token !== env.ADMIN_TOKEN) { + if (!hasAdminAccess(request, env)) { return withCors(request, Response.json({ ok: false, error: "unauthorized" }, { status: 401 })); } let body: unknown; @@ -6639,8 +6675,7 @@ export default { // Admin-token-protected report archival (Agent Smith archives what it posts). if (url.pathname === "/report/snapshot" && request.method === "POST") { - const token = request.headers.get("X-Admin-Token"); - if (!env.ADMIN_TOKEN || !token || token !== env.ADMIN_TOKEN) { + if (!hasAdminAccess(request, env)) { return withCors(request, Response.json({ ok: false, error: "unauthorized" }, { status: 401 })); } let body: unknown; @@ -6903,8 +6938,7 @@ export default { } if (url.pathname === "/report") { - const token = request.headers.get("X-Admin-Token"); - if (!env.ADMIN_TOKEN || !token || token !== env.ADMIN_TOKEN) { + if (!hasReportReadAccess(request, env)) { return withCors(request, Response.json({ ok: false, error: "unauthorized" }, { status: 401 })); } diff --git a/tests/phase2.test.mjs b/tests/phase2.test.mjs index bfcb60d..5ea57b8 100644 --- a/tests/phase2.test.mjs +++ b/tests/phase2.test.mjs @@ -244,7 +244,7 @@ test("GET /report?view=asset returns rollup, github, health, campaigns with down assert.doesNotMatch(JSON.stringify(body), PII, "asset report must contain no PII"); }); -test("GET /report?view=asset requires the admin token", async () => { +test("GET /report?view=asset rejects a missing report credential", async () => { const response = await worker.fetch(new Request("https://lighthouse.buscore.ca/report?view=asset"), assetEnv(), ctx); assert.equal(response.status, 401); }); diff --git a/tests/phase3.test.mjs b/tests/phase3.test.mjs index 3cadb27..ad09241 100644 --- a/tests/phase3.test.mjs +++ b/tests/phase3.test.mjs @@ -188,7 +188,7 @@ test("GET /report?view=monthly with data computes scores, shows sub-scores + inp assert.doesNotMatch(JSON.stringify(body), PII); }); -test("GET /report?view=monthly requires the admin token", async () => { +test("GET /report?view=monthly rejects a missing report credential", async () => { const res = await worker.fetch(new Request("https://lighthouse.buscore.ca/report?view=monthly"), monthlyEnv(), ctx); assert.equal(res.status, 401); }); diff --git a/tests/release-control.test.mjs b/tests/release-control.test.mjs index 85c9897..fa2e462 100644 --- a/tests/release-control.test.mjs +++ b/tests/release-control.test.mjs @@ -46,16 +46,19 @@ test("production deployment is manual, main-only, serialized, gated, and receipt test("operator scripts expose reads and upload without a direct production bypass", () => { assert.equal(packageJson.scripts.deploy, undefined); assert.equal(packageJson.scripts["release:deploy"], undefined); + assert.equal(packageJson.scripts["diagnostic:ceo"], "node scripts/read-ceo-report.mjs"); assert.equal(packageJson.scripts["release:upload"], "wrangler versions upload"); assert.equal(packageJson.scripts["release:status"], "wrangler deployments status --json"); assert.equal(packageJson.scripts["release:history"], "wrangler deployments list --json"); assert.equal(packageJson.repository.url, "git+https://github.com/True-Good-Craft/lighthouse.git"); }); -test("the mandatory release-control bundle records the verified external receipt", () => { - assert.equal(packageJson.version, "1.29.4"); +test("the current governed bundle and historical release-control receipt stay synchronized", () => { + assert.equal(packageJson.version, "1.30.0"); assert.equal(packageLock.version, packageJson.version); assert.equal(packageLock.packages[""].version, packageJson.version); + assert.match(sot, /^## Least-privilege report diagnostics — v1\.30\.0$/m); + assert.match(changelog, /^## \[1\.30\.0\] - 2026-08-26$/m); assert.match(sot, /^## Release-control and infrastructure reconciliation — v1\.29\.4$/m); assert.match(changelog, /^## \[1\.29\.4\] - 2026-08-26$/m); assert.match( diff --git a/tests/report-auth.test.mjs b/tests/report-auth.test.mjs new file mode 100644 index 0000000..d76edb4 --- /dev/null +++ b/tests/report-auth.test.mjs @@ -0,0 +1,257 @@ +import test from "node:test"; +import assert from "node:assert/strict"; + +import * as workerModule from "../dist/index.js"; + +const worker = workerModule.default?.fetch + ? workerModule.default + : workerModule.default?.default ?? workerModule.default; + +const ADMIN_SECRET = "admin-secret"; +const REPORT_SECRET = "0123456789abcdef0123456789abcdef0123456789abcdef"; + +function makeDb() { + const calls = { prepare: 0, run: 0 }; + const db = { + prepare() { + calls.prepare += 1; + const statement = { + bind() { + return statement; + }, + async first() { + return null; + }, + async all() { + return { results: [] }; + }, + async run() { + calls.run += 1; + return { success: true, meta: { changes: 1 } }; + }, + }; + return statement; + }, + }; + return { db, calls }; +} + +function makeEnv({ + db, + includeReportToken = true, + reportToken = REPORT_SECRET, + adminToken = ADMIN_SECRET, +} = {}) { + const env = { + DB: db ?? makeDb().db, + MANIFEST_R2: {}, + ADMIN_TOKEN: adminToken, + IGNORED_IP: "", + CF_API_TOKEN: "", + CF_ZONE_TAG: "", + }; + if (includeReportToken) env.REPORT_READ_TOKEN = reportToken; + return env; +} + +function makeContext() { + const calls = { waitUntil: 0 }; + return { + calls, + ctx: { + waitUntil() { + calls.waitUntil += 1; + }, + }, + }; +} + +async function assertUnauthorized(response) { + assert.equal(response.status, 401); + assert.deepEqual(await response.json(), { ok: false, error: "unauthorized" }); +} + +test("GET /report accepts the exact report-read token without changing the report contract", async () => { + const { db, calls } = makeDb(); + const { ctx } = makeContext(); + const response = await worker.fetch( + new Request("https://lighthouse.test/report?view=ceo", { + headers: { "X-Report-Token": REPORT_SECRET }, + }), + makeEnv({ db }), + ctx + ); + + assert.equal(response.status, 200); + const payload = await response.json(); + assert.equal(payload.view, "ceo"); + assert.equal(payload.report_contract_version, "1.1"); + assert.ok(calls.prepare > 0, "an authorized report reaches stored-data reads"); + assert.equal(calls.run, 0, "the stored-data CEO view performs no writes"); + assert.equal(response.headers.get("Access-Control-Allow-Headers"), "Content-Type"); +}); + +test("GET /report retains exact admin-token compatibility when the distinct read binding is absent or malformed", async () => { + const configurations = [ + { includeReportToken: false }, + { includeReportToken: true, reportToken: "short-report-secret" }, + ]; + + for (const configuration of configurations) { + const { db } = makeDb(); + const response = await worker.fetch( + new Request("https://lighthouse.test/report?view=ceo", { + headers: { "X-Admin-Token": ADMIN_SECRET }, + }), + makeEnv({ db, ...configuration }), + makeContext().ctx + ); + + assert.equal(response.status, 200); + assert.equal((await response.json()).view, "ceo"); + } +}); + +test("GET /report rejects missing, blank, malformed, unbound, and non-exact report credentials before side effects", async () => { + const cases = [ + { name: "missing header", headers: {}, includeReportToken: true }, + { name: "blank header", headers: { "X-Report-Token": "" }, includeReportToken: true }, + { name: "missing binding", headers: { "X-Report-Token": REPORT_SECRET }, includeReportToken: false }, + { name: "blank binding", headers: { "X-Report-Token": REPORT_SECRET }, includeReportToken: true, reportToken: "" }, + { + name: "short binding", + headers: { "X-Report-Token": "short-report-secret" }, + includeReportToken: true, + reportToken: "short-report-secret", + }, + { + name: "space-containing binding", + headers: { "X-Report-Token": `${"a".repeat(16)} ${"b".repeat(16)}` }, + includeReportToken: true, + reportToken: `${"a".repeat(16)} ${"b".repeat(16)}`, + }, + { + name: "punctuation-containing binding", + headers: { "X-Report-Token": `${"a".repeat(31)}!` }, + includeReportToken: true, + reportToken: `${"a".repeat(31)}!`, + }, + { + name: "oversized binding", + headers: { "X-Report-Token": "a".repeat(129) }, + includeReportToken: true, + reportToken: "a".repeat(129), + }, + { name: "wrong value", headers: { "X-Report-Token": "wrong" }, includeReportToken: true }, + { name: "value with suffix", headers: { "X-Report-Token": `${REPORT_SECRET}-extra` }, includeReportToken: true }, + { name: "report credential in admin header", headers: { "X-Admin-Token": REPORT_SECRET }, includeReportToken: true }, + { name: "wrong admin value", headers: { "X-Admin-Token": "wrong" }, includeReportToken: true }, + ]; + + for (const authCase of cases) { + const { db, calls } = makeDb(); + const { ctx, calls: contextCalls } = makeContext(); + const response = await worker.fetch( + new Request("https://lighthouse.test/report?view=invalid", { headers: authCase.headers }), + makeEnv({ + db, + includeReportToken: authCase.includeReportToken, + reportToken: authCase.reportToken, + }), + ctx + ); + + await assertUnauthorized(response); + assert.equal(calls.prepare, 0, `${authCase.name}: no database read or error-counter access`); + assert.equal(calls.run, 0, `${authCase.name}: no database write`); + assert.equal(contextCalls.waitUntil, 0, `${authCase.name}: no deferred work`); + } +}); + +test("REPORT_READ_TOKEN cannot turn /report into a non-GET surface", async () => { + const { db, calls } = makeDb(); + const { ctx, calls: contextCalls } = makeContext(); + const response = await worker.fetch( + new Request("https://lighthouse.test/report?view=ceo", { + method: "POST", + headers: { + "Content-Type": "application/json", + "X-Report-Token": REPORT_SECRET, + }, + body: JSON.stringify({ method_probe: "must not be authorized" }), + }), + makeEnv({ db }), + ctx + ); + + assert.equal(response.status, 405); + assert.deepEqual(await response.json(), { ok: false, error: "method_not_allowed" }); + assert.equal(calls.prepare, 0); + assert.equal(calls.run, 0); + assert.equal(contextCalls.waitUntil, 0); +}); + +test("REPORT_READ_TOKEN cannot authorize campaign, note, or snapshot writes", async () => { + const requests = [ + ["/campaign", { channel: "community", tagged_src: "test" }], + ["/notes", { note: "must not be inserted" }], + ["/report/snapshot", { kind: "monthly", summary_json: { test: true } }], + ]; + + for (const [path, body] of requests) { + const { db, calls } = makeDb(); + const { ctx, calls: contextCalls } = makeContext(); + const response = await worker.fetch( + new Request(`https://lighthouse.test${path}`, { + method: "POST", + headers: { + "Content-Type": "application/json", + "X-Report-Token": REPORT_SECRET, + }, + body: JSON.stringify(body), + }), + makeEnv({ db }), + ctx + ); + + await assertUnauthorized(response); + assert.equal(calls.prepare, 0, `${path}: authorization fails before database access`); + assert.equal(calls.run, 0, `${path}: no insert is attempted`); + assert.equal(contextCalls.waitUntil, 0, `${path}: no deferred work`); + } +}); + +test("colliding admin and report-read secrets fail closed for reads and writes", async () => { + const collisionSecret = "abcdef0123456789abcdef0123456789abcdef0123456789"; + const requests = [ + new Request("https://lighthouse.test/report?view=ceo", { + headers: { "X-Report-Token": collisionSecret }, + }), + new Request("https://lighthouse.test/report?view=ceo", { + headers: { "X-Admin-Token": collisionSecret }, + }), + new Request("https://lighthouse.test/campaign", { + method: "POST", + headers: { + "Content-Type": "application/json", + "X-Admin-Token": collisionSecret, + }, + body: JSON.stringify({ channel: "community", tagged_src: "collision" }), + }), + ]; + + for (const request of requests) { + const { db, calls } = makeDb(); + const { ctx, calls: contextCalls } = makeContext(); + const response = await worker.fetch( + request, + makeEnv({ db, adminToken: collisionSecret, reportToken: collisionSecret }), + ctx + ); + + await assertUnauthorized(response); + assert.equal(calls.prepare, 0); + assert.equal(calls.run, 0); + assert.equal(contextCalls.waitUntil, 0); + } +}); diff --git a/tests/report-helper.test.mjs b/tests/report-helper.test.mjs new file mode 100644 index 0000000..74cf2cb --- /dev/null +++ b/tests/report-helper.test.mjs @@ -0,0 +1,345 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { EventEmitter } from "node:events"; +import { readFileSync } from "node:fs"; + +const originalFetch = globalThis.fetch; +let importFetchCalls = 0; +let helper; +try { + globalThis.fetch = async () => { + importFetchCalls += 1; + throw new Error("unexpected import-time fetch"); + }; + helper = await import("../scripts/read-ceo-report.mjs"); +} finally { + globalThis.fetch = originalFetch; +} + +const { + ACCESS_BLOCKED_LINE, + CEO_REPORT_URL, + FAILURE_LINE, + MAX_REPORT_TOKEN_BYTES, + MAX_RESPONSE_BYTES, + MIN_REPORT_TOKEN_BYTES, + REPORT_TOKEN_ENV, + REPORT_TOKEN_HEADER, + REPORT_UNAVAILABLE_LINE, + REQUEST_TIMEOUT_MS, + TOKEN_PROMPT, + isUsableReportToken, + readHiddenToken, + runDiagnostic, +} = helper; + +const VALID_REPORT_TOKEN = "0123456789abcdef0123456789abcdef0123456789abcdef"; +const SECOND_VALID_REPORT_TOKEN = "abcdef0123456789abcdef0123456789abcdef0123456789"; + +const fixtureText = readFileSync( + new URL("../contracts/ceo-v1/healthy-zero.json", import.meta.url), + "utf8", +); +const fixture = JSON.parse(fixtureText); + +function writer({ isTTY = false } = {}) { + const chunks = []; + return { + isTTY, + chunks, + write(value) { + chunks.push(String(value)); + return true; + }, + get text() { + return chunks.join(""); + }, + }; +} + +function successfulResponse(value = fixtureText) { + return new Response(value, { + status: 200, + headers: { "Content-Type": "application/json" }, + }); +} + +test("helper import is local-only and the diagnostic surface is fixed", () => { + assert.equal(importFetchCalls, 0); + assert.equal(CEO_REPORT_URL, "https://lighthouse.buscore.ca/report?view=ceo"); + assert.equal(REPORT_TOKEN_ENV, "LIGHTHOUSE_REPORT_READ_TOKEN"); + assert.equal(REPORT_TOKEN_HEADER, "X-Report-Token"); + assert.equal(REQUEST_TIMEOUT_MS, 15_000); + assert.equal(MAX_RESPONSE_BYTES, 1024 * 1024); + assert.equal(MIN_REPORT_TOKEN_BYTES, 32); + assert.equal(MAX_REPORT_TOKEN_BYTES, 128); + assert.equal(isUsableReportToken("a".repeat(32)), true); + assert.equal(isUsableReportToken("a".repeat(128)), true); + assert.equal(isUsableReportToken("a".repeat(31)), false); + assert.equal(isUsableReportToken("a".repeat(129)), false); + assert.equal(isUsableReportToken(`a${"b".repeat(31)} `), false); + assert.equal(isUsableReportToken(`a${"b".repeat(30)}!`), false); + assert.equal(isUsableReportToken(`a${"b".repeat(30)}é`), false); +}); + +test("automation credential is deleted before one fixed validated GET reaches stdout", async () => { + const secret = VALID_REPORT_TOKEN; + const environment = { [REPORT_TOKEN_ENV]: secret }; + const stdout = writer(); + const stderr = writer(); + const calls = []; + let scheduledDelay = null; + let clearedTimer = null; + const timerHandle = Symbol("timer"); + + const exitCode = await runDiagnostic({ + argv: [], + environment, + stdin: { isTTY: false }, + stdout, + stderr, + fetchImpl: async (url, options) => { + calls.push({ url, options }); + return successfulResponse(); + }, + scheduleTimeout(_callback, delay) { + scheduledDelay = delay; + return timerHandle; + }, + cancelTimeout(handle) { + clearedTimer = handle; + }, + }); + + assert.equal(exitCode, 0); + assert.equal(Object.hasOwn(environment, REPORT_TOKEN_ENV), false); + assert.equal(calls.length, 1); + assert.equal(calls[0].url, CEO_REPORT_URL); + assert.equal(calls[0].options.method, "GET"); + assert.equal(calls[0].options.redirect, "manual"); + assert.equal(calls[0].options.headers[REPORT_TOKEN_HEADER], secret); + assert.equal(calls[0].options.headers.Accept, "application/json"); + assert.ok(calls[0].options.signal instanceof AbortSignal); + assert.equal(scheduledDelay, REQUEST_TIMEOUT_MS); + assert.equal(clearedTimer, timerHandle); + assert.deepEqual(JSON.parse(stdout.text), fixture); + assert.equal(stdout.text.endsWith("\n"), true); + assert.equal(stderr.text, ""); + assert.equal(`${stdout.text}${stderr.text}`.includes(secret), false); +}); + +test("hidden TTY prompt does not echo the interactive credential", async () => { + class FakeTty extends EventEmitter { + constructor() { + super(); + this.isTTY = true; + this.isRaw = false; + this.paused = true; + this.rawTransitions = []; + } + isPaused() { return this.paused; } + pause() { this.paused = true; } + resume() { this.paused = false; } + setRawMode(value) { + this.isRaw = value; + this.rawTransitions.push(value); + } + } + + const input = new FakeTty(); + const output = writer({ isTTY: true }); + const tokenPromise = readHiddenToken({ input, output }); + input.emit("data", Buffer.from(`${VALID_REPORT_TOKEN}x`)); + input.emit("data", Buffer.from("\b")); + input.emit("data", Buffer.from("\r")); + + assert.equal(await tokenPromise, VALID_REPORT_TOKEN); + assert.equal(output.text, `${TOKEN_PROMPT}\n`); + assert.equal(output.text.includes(VALID_REPORT_TOKEN), false); + assert.deepEqual(input.rawTransitions, [true, false]); + assert.equal(input.paused, true); +}); + +test("missing credential and any CLI argument fail closed without a request", async (t) => { + await t.test("missing credential", async () => { + const environment = {}; + const stdout = writer(); + const stderr = writer(); + let fetchCalls = 0; + + const exitCode = await runDiagnostic({ + argv: [], + environment, + stdin: { isTTY: false }, + stdout, + stderr, + fetchImpl: async () => { + fetchCalls += 1; + return successfulResponse(); + }, + }); + + assert.equal(exitCode, 1); + assert.equal(fetchCalls, 0); + assert.equal(stdout.text, ""); + assert.equal(stderr.text, ACCESS_BLOCKED_LINE); + }); + + await t.test("arguments cannot override the fixed request", async () => { + const secret = SECOND_VALID_REPORT_TOKEN; + const environment = { [REPORT_TOKEN_ENV]: secret }; + const stdout = writer(); + const stderr = writer(); + let fetchCalls = 0; + + const exitCode = await runDiagnostic({ + argv: ["https://example.invalid/report"], + environment, + stdout, + stderr, + fetchImpl: async () => { + fetchCalls += 1; + return successfulResponse(); + }, + }); + + assert.equal(exitCode, 1); + assert.equal(Object.hasOwn(environment, REPORT_TOKEN_ENV), false); + assert.equal(fetchCalls, 0); + assert.equal(stdout.text, ""); + assert.equal(stderr.text, FAILURE_LINE); + assert.equal(stderr.text.includes(secret), false); + }); + + await t.test("invalid automation credential format is deleted and fails before a request", async () => { + const environment = { [REPORT_TOKEN_ENV]: "too-short" }; + const stdout = writer(); + const stderr = writer(); + let fetchCalls = 0; + + const exitCode = await runDiagnostic({ + environment, + stdout, + stderr, + fetchImpl: async () => { + fetchCalls += 1; + return successfulResponse(); + }, + }); + + assert.equal(exitCode, 1); + assert.equal(Object.hasOwn(environment, REPORT_TOKEN_ENV), false); + assert.equal(fetchCalls, 0); + assert.equal(stdout.text, ""); + assert.equal(stderr.text, ACCESS_BLOCKED_LINE); + }); +}); + +test("redirects, bad payloads, reflected credentials, and oversized bodies are suppressed", async (t) => { + const cases = [ + ["redirect", () => new Response(null, { status: 302, headers: { Location: CEO_REPORT_URL } }), FAILURE_LINE], + ["authorization failure", () => new Response("unauthorized", { status: 401 }), ACCESS_BLOCKED_LINE], + ["authorization forbidden", () => new Response("forbidden", { status: 403 }), ACCESS_BLOCKED_LINE], + ["report unavailable", () => new Response("unavailable", { status: 503 }), REPORT_UNAVAILABLE_LINE], + ["other server failure", () => new Response("failed", { status: 500 }), FAILURE_LINE], + ["invalid JSON", () => successfulResponse("not json"), FAILURE_LINE], + ["schema drift", () => successfulResponse(JSON.stringify({ ...fixture, report_contract_version: "9" })), FAILURE_LINE], + [ + "advertised oversized body", + () => new Response("{}", { + status: 200, + headers: { "Content-Length": String(MAX_RESPONSE_BYTES + 1) }, + }), + FAILURE_LINE, + ], + [ + "streamed oversized body", + () => new Response(new ReadableStream({ + start(controller) { + controller.enqueue(new Uint8Array(MAX_RESPONSE_BYTES)); + controller.enqueue(new Uint8Array(1)); + controller.close(); + }, + }), { status: 200 }), + FAILURE_LINE, + ], + ]; + + for (const [name, responseFactory, expectedFailureLine] of cases) { + await t.test(name, async () => { + const secret = VALID_REPORT_TOKEN; + const stdout = writer(); + const stderr = writer(); + let fetchCalls = 0; + const exitCode = await runDiagnostic({ + environment: { [REPORT_TOKEN_ENV]: secret }, + stdout, + stderr, + fetchImpl: async () => { + fetchCalls += 1; + return responseFactory(); + }, + }); + + assert.equal(exitCode, 1); + assert.equal(fetchCalls, 1); + assert.equal(stdout.text, ""); + assert.equal(stderr.text, expectedFailureLine); + assert.equal(stderr.text.includes(secret), false); + }); + } + + await t.test("schema-valid response containing the credential", async () => { + const secret = fixture.report_id; + const stdout = writer(); + const stderr = writer(); + const exitCode = await runDiagnostic({ + environment: { [REPORT_TOKEN_ENV]: secret }, + stdout, + stderr, + fetchImpl: async () => successfulResponse(), + }); + + assert.equal(exitCode, 1); + assert.equal(stdout.text, ""); + assert.equal(stderr.text, FAILURE_LINE); + assert.equal(stderr.text.includes(secret), false); + }); +}); + +test("the 15-second abort path is single-shot and emits only the static failure", async () => { + const secret = SECOND_VALID_REPORT_TOKEN; + const stdout = writer(); + const stderr = writer(); + let fetchCalls = 0; + let scheduledDelay = null; + let cleared = false; + + const exitCode = await runDiagnostic({ + environment: { [REPORT_TOKEN_ENV]: secret }, + stdout, + stderr, + fetchImpl: async (_url, { signal }) => { + fetchCalls += 1; + return await new Promise((_resolve, reject) => { + signal.addEventListener("abort", () => reject(new Error(`do not print ${secret}`)), { once: true }); + }); + }, + scheduleTimeout(callback, delay) { + scheduledDelay = delay; + queueMicrotask(callback); + return Symbol("timeout"); + }, + cancelTimeout() { + cleared = true; + }, + }); + + assert.equal(exitCode, 1); + assert.equal(fetchCalls, 1); + assert.equal(scheduledDelay, REQUEST_TIMEOUT_MS); + assert.equal(cleared, true); + assert.equal(stdout.text, ""); + assert.equal(stderr.text, FAILURE_LINE); + assert.equal(stderr.text.includes(secret), false); +});