[WesIngwersen]

Closes #1157. Stacked on #1162. It is held for a combined overnight rebuild, and nothing has been rebuilt with this change yet.

What was wrong

_load_usa_summary_sut read 2017–2022 from Supply_Tables_2017-2022_Summary.xlsx (and the matching Use workbook). Only 2023–2024 came from the 1997–2024 workbook. Gross output (UGO305-A, BEA_DATA_VERSION = "2025Q2") is on the newer release for every year. As a result, the 2018–2022 summary controls disagreed with the gross output they are balanced against. In 2022, the absolute revisions to commodity output between the two workbooks add up to $1.32T across the 73 commodity rows.

The change

• _load_usa_summary_sut reads USA_SUMMARY_SUT_MAPPING_1997_2024 for every year. The two workbooks agree exactly on 2017, so 2017 does not move.
• USA_SUMMARY_SUT_MAPPING_2017_2022 is no longer read by any loader. It stays only to name the superseded workbook.
• Comments in matrix_mappings.py and nowcast_intermediate.py that described the old year split are updated. nowcast_intermediate already read the 1997–2024 workbook for every year, so its numbers do not change.

The annual-update check

A new module, extract/iot/bea_annual_update.py, checks that gross output and the summary supply-use tables come from the same BEA annual update. nowcast_sut_assembly.assemble() runs the check first and raises if they differ. That means the old pin would have stopped the build.

The check does not rely on file names, GCS folders or release dates. BEA_DATA_VERSION is a BEA Data Archive label, and the supply-use workbooks carry no release stamp. Instead, each source is tied to its update by its content: an annual update released in September of year Y publishes data through Y − 1.

source	last year published	annual update
gross output UGO305-A, 2025Q2 (what the build uses)	2024	2025
gross output UGO305-A, 2026Q2	2025	2026
summary supply-use, 1997–2024 workbooks (what the build now uses)	2024	2025
summary supply-use, 2017–2022 workbooks (what the build used for 2017–2022)	2022	2023

NEWEST_BEA_ANNUAL_UPDATE = 2026 records the 30 September 2026 release. Today both inputs are on the 2025 update, so the check passes and logs a warning that 2026 is newer. Moving to the 2026 update would mean a 2026Q2 gross output pin and the 1997–2025 summary workbooks, changed together. #1167, stacked on this PR, makes that move and adds the components-of-value-added archive to the check.

Unit tests cover the rule, a match, a mismatch, and the warning (extract/iot/__tests__/test_bea_annual_update.py). They need no data.

What moves

Everything that reads the summary supply-use tables for 2018–2022:

• Nowcast steps:
  • trade-margin give-up (nowcast_trade_margins);
  • the imports column (nowcast_import_conditioning);
  • final demand (nowcast_fd_conditioning);
  • subsidies;
  • taxes on production (nowcast_va_taxes);
  • import adjustments (trade/madj.py).
• Flow-by-activity tables: BEA_Summary_Supply and BEA_Summary_Use_SUT, and the methods built on them: Detail_Supply_Mix_* and Detail_Use_SUT.
• Outside the nowcast: CAP_HAP_{Nonpoint,Nonroad,Onroad}_2020 read the 2020 summary Supply table. Revisions to 2020 commodity output total $246bn in absolute terms.

Before the rebuild

These cached parquets are named by commit, not by their inputs. The 2018–2022 cached BEA_Summary_Supply and BEA_Summary_Use_SUT tables, and the flow-by-sector methods built on them, must be cleared, or they will keep serving the old workbook's numbers.

Acceptance, after the rebuild

• The 2022 trade-margin net (Step 5: the trade margin column stops netting to zero under the balance (T15, $34bn in 2022) #809) on this build. Today it is −$34.0bn, 0.75% of gross margin, against 0.03–0.22% in the other years. If it falls to about 0.1%, close Step 5: the trade margin column stops netting to zero under the balance (T15, $34bn in 2022) #809 as resolved here.
• Any control read from the summary tables that still uses a different release is listed with a reason.

Checks run locally

black --check, ruff check and mypy are clean on the changed files. The new unit tests pass. The live check against the real workbooks returns 2025 for both sources. The suite is left to CI.

🤖 Generated with Claude Code