Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -99,3 +99,8 @@ jobs:
# without failing the job, so main stays green until the next change.
continue-on-error: ${{ github.event_name == 'schedule' }}
run: npm audit --audit-level=low
- name: Report baseline freshness
# Offline and deterministic: reads graph.json and the date, fetches nothing, and writes a
# table of baselines that may have a newer release to the job summary. It never fails.
if: github.event_name == 'schedule'
run: npm run check:freshness
35 changes: 24 additions & 11 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,17 +62,18 @@ link.
A node is an indicator or a lever. Its fields are defined in `src/lib/schema.ts` and enforced by
`src/lib/validate.ts`.

| Field | Rule |
| -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `id` | Lowercase words joined by single underscores, unique. |
| `category` | One of `economic`, `fiscal`, `social`, `institutional`, `policy`. |
| `valueType` | `observed` is a measured series. `projected` is a forecast, never shown as an observation. `index` is an abstract 0 to 100 lever. |
| `baseline` | A number, or `null`. An `index` node must be `null`. |
| `range` | An editorial display scale that comfortably holds the baseline. It is not data. Explain your choice in the pull request. |
| `asOf` | The period the baseline describes: `YYYY`, `YYYY-MM`, `YYYY-MM-DD`, `YYYY-Qn` or `FYYYYY`. Required when there is a baseline. |
| `verification` | `primary`, `secondary` or `pending` for observed and projected nodes. `null` for an `index` node. |
| `sourceUrl`, `sourceDetail`, `retrievedDate` | Required for `primary` and `secondary`. For `pending`, `sourceUrl` and `retrievedDate` must be `null`. |
| `terminal` | Why nothing is modeled downstream, required exactly when no edge leaves the node. Otherwise `null`. |
| Field | Rule |
| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | Lowercase words joined by single underscores, unique. |
| `category` | One of `economic`, `fiscal`, `social`, `institutional`, `policy`. |
| `valueType` | `observed` is a measured series. `projected` is a forecast, never shown as an observation. `index` is an abstract 0 to 100 lever. |
| `baseline` | A number, or `null`. An `index` node must be `null`. |
| `range` | An editorial display scale that comfortably holds the baseline. It is not data. Explain your choice in the pull request. |
| `asOf` | The period the baseline describes: `YYYY`, `YYYY-MM`, `YYYY-MM-DD`, `YYYY-Qn` or `FYYYYY`. Required when there is a baseline. |
| `verification` | `primary`, `secondary` or `pending` for observed and projected nodes. `null` for an `index` node. |
| `sourceUrl`, `sourceDetail`, `retrievedDate` | Required for `primary` and `secondary`. For `pending`, `sourceUrl` and `retrievedDate` must be `null`. |
| `cadence` | How often the publisher releases a new figure: `daily`, `weekly`, `monthly`, `quarterly`, `annual` or `irregular`. `null` for `index` nodes. |
| `terminal` | Why nothing is modeled downstream, required exactly when no edge leaves the node. Otherwise `null`. |

How to verify a starting value:

Expand All @@ -89,6 +90,18 @@ How to verify a starting value:
- Do not store a derived number, such as a growth rate you computed from two readings, as though it
had been fetched. `debt_growth_rate` is empty for that reason.

## Refreshing a baseline

`npm run check:freshness` lists the baselines that may have a newer release, judged from each
node's `cadence` and the end of its period. It reads no network and does not fail. The weekly CI run
writes the same table to its job summary. A flag is a prompt to read the source, not a finding that
the value is wrong, and a date that looks current does not catch a revision to the same period.

To refresh a node, open the publisher's page, read the new figure, and update `baseline`, `asOf`,
`sourceUrl`, `sourceDetail` and `retrievedDate` together in one change, with the release date and
any revision noted in `sourceDetail`. Update the matching row of the table in README.md. Never
change the value alone, and never take it from a search result or summary.

## Adding an edge

- `id` is `from__to`, using the two node ids, and both nodes must exist. No self loops, and no two
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,10 @@ and not an observation.
| Trust in media | 28% | Sep 2025 | [Gallup, Trust in Media](https://news.gallup.com/poll/695762/trust-media-new-low.aspx) |
| Reported hate crimes | 10,606 reported incidents | 2025 | [FBI, 2025 Reported Crimes in the Nation](https://www.fbi.gov/news/press-releases/fbi-releases-2025-reported-crimes-in-the-nation-statistics) |

Each node records how often its figure is published. `npm run check:freshness`, which also runs
weekly in CI, lists the baselines that may have a newer release. It is a reminder to read the
source, not a check of any value. [CONTRIBUTING.md](CONTRIBUTING.md) describes how to refresh one.

Two nodes have no baseline. `worker_bargaining_power` is an abstract 0-to-100 lever.
`debt_growth_rate` is a rate that would be derived from two Treasury readings a year apart, and a
derived number is computed where it is shown and not stored as though it had been fetched.
Expand Down
4 changes: 4 additions & 0 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,10 @@ whether those assumptions are true of the economy.
Automated rules catch only part of WCAG, so the axe-core scan is
not an accessibility certification.

`npm run check:freshness` reports baselines that may have a newer release. It is offline and
deterministic, runs on the weekly schedule in CI, writes to the job summary and never fails the
run. It flags likely staleness from each node's cadence; it does not check any value.

`npm run check:sources` produces a best-effort external source reachability report. It needs
network access and is not part of `verify` or CI. A 2xx or 3xx response counts as reachable. The
report does not establish source validity, accuracy, or currency; read the source itself.
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
"preview": "vite preview",
"test:e2e": "npm run build && npm run test:math && node scripts/block-e2e.mjs",
"check:sources": "node scripts/check-sources.mjs",
"check:freshness": "node scripts/check-freshness.mjs",
"test:math": "node scripts/model-audit.mjs",
"check:attribution": "python3 scripts/check-attribution.py"
},
Expand Down
73 changes: 73 additions & 0 deletions scripts/check-freshness.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
/**
* Freshness report for stored baselines. Offline and deterministic: it reads graph.json and a date,
* and fetches nothing, so a publisher's site being down can never fail it.
*
* npm run check:freshness report as of today (UTC)
* npm run check:freshness -- --today 2026-09-27 report as of a fixed date
* npm run check:freshness -- --strict exit 1 when any baseline is flagged
*
* A flag means a newer release has probably been published, judged from the node's cadence and the
* end of its period (src/lib/freshness.ts). It does not mean the stored value is wrong. Refresh a
* flagged node by reading the publisher's page and updating baseline, asOf, sourceUrl, sourceDetail
* and retrievedDate together, as CONTRIBUTING.md describes. The weekly CI run writes this report to
* the job summary and never fails on it.
*/
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import process from "node:process";
import { fileURLToPath, pathToFileURL } from "node:url";
import ts from "typescript";

const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
const args = process.argv.slice(2);
const strict = args.includes("--strict");
const todayIndex = args.indexOf("--today");
const today = todayIndex >= 0 ? args[todayIndex + 1] : new Date().toISOString().slice(0, 10);
// Round-trip the date, so an impossible one such as 2026-02-30 is refused, not rolled over.
const parsed = new Date(`${today ?? ""}T00:00:00Z`);
if (
!/^\d{4}-\d{2}-\d{2}$/.test(today ?? "") ||
Number.isNaN(parsed.getTime()) ||
parsed.toISOString().slice(0, 10) !== today
) {
console.error("--today needs a date as YYYY-MM-DD");
process.exit(2);
}

const source = await fs.readFile(path.join(root, "src/lib/freshness.ts"), "utf8");
const temporary = await fs.mkdtemp(path.join(os.tmpdir(), "usl-freshness-"));
let freshness;
try {
const file = path.join(temporary, "freshness.mjs");
await fs.writeFile(
file,
ts.transpileModule(source, {
compilerOptions: { target: ts.ScriptTarget.ES2023, module: ts.ModuleKind.ESNext },
}).outputText,
);
({ freshness } = await import(pathToFileURL(file).href));
} finally {
await fs.rm(temporary, { recursive: true, force: true });
}

const graph = JSON.parse(await fs.readFile(path.join(root, "src/data/graph.json"), "utf8"));
const rows = graph.nodes.map((node) => ({ node, result: freshness(node, today) }));
const flagged = rows.filter((row) => row.result.status === "check");
const lines = [
`## Baseline freshness as of ${today}`,
"",
`${String(flagged.length)} of ${String(rows.length)} baselines may have a newer release. A flag is a prompt to read the source, not a finding that the value is wrong.`,
"",
"| Indicator | As of | Cadence | Days since period end | Status |",
"| --- | --- | --- | --- | --- |",
...rows.map(
({ node, result }) =>
`| ${node.id} | ${node.asOf ?? "none"} | ${node.cadence ?? "none"} | ${result.ageDays === null ? "n/a" : String(result.ageDays)} | ${result.status === "check" ? "check source" : result.status === "current" ? "current" : "not applicable"} |`,
),
"",
];
const text = lines.join("\n");
console.log(text);
if (process.env.GITHUB_STEP_SUMMARY) await fs.appendFile(process.env.GITHUB_STEP_SUMMARY, text);
if (strict && flagged.length > 0) process.exit(1);
Loading
Loading