[WesIngwersen]

Closes #1161.

What was wrong

In the nowcast MUT, each commodity's output in the Make table should equal its row total in the producer-price Use table. Before this change the two disagreed by $189bn in 2017, growing to $1.46T in 2022 and $1.44T in 2024 (summed across commodities, ignoring sign). BEA's published 2017 tables close the same test to $1.1bn.

Step 6 (mut_from_balanced, the purchaser-to-producer conversion) introduced it in three places:

1. Margins placed by the 2017 rates alone. Each good's stripped margin and each trade or transport commodity's booked margin came from 2017 rates applied to the year's purchaser values, never tied to the balanced Supply table's TRADE and TRANS columns.
2. Make's tax wedge built from the pre-balance tax rows. Step 5 moves the Use table's T00TOP/T00SUB rows and the Supply table's TOP/SUB columns. In 2022, T00TOP moved by $88bn net. Make still added the pre-balance rows, so Make's industry totals disagreed with the Use table's value added too.
3. Make spread each industry's tax over its whole output mix. Housing subsidies, utility taxes and dealers' sales tax (which landed on their repair services) went to the wrong commodities.

Published 2017 settles the open question in the issue about taxes. Each good's stripped margin in BEA's tables is its Supply TRADE + TRANS plus the trade-level share of its TOP (sales and fuel taxes ride inside the trade margin), to $1.1bn across all goods. So the tax-inclusive margin is correct. The error was in the amounts, not the convention, and #823 is not needed for this fix.

The change

• nowcast_va_taxes: t00top_cells and t00sub_cells return the tax rows on the Make axes (industry × commodity produced). The existing rows are now their row sums, unchanged. The --check still reports T00TOP r = 0.947, T00SUB exact. The trade-level leg now lands only on the wholesale and retail commodities a trade industry produces.
• supply_to_make.make_from_sut: optional wedge_cells places the wedge from those cells instead of the output-mix spread. Against BEA's published 2017 Make (--check), matching cells rise from 160,788 to 161,047 of 161,604, and the $18.3bn housing residual is gone.
• nowcast_mut:
  • balanced_make_wedge fits those cells to the balanced Use tax rows (Make's industry totals then equal value added) and to the balanced Supply tax columns.
  • _reconcile_placement fits the margin placement so each good gives up exactly its Supply margin and each margin commodity books what its Make output requires. Each (good, margin commodity) pair keeps its buyer split.
  • Two new exact gates:
    • each good's stripped margin equals its Supply margin, per family;
    • the commodity identity residual on every non-margin commodity equals the Step 5 tax disagreement on it, and nothing else.
  • The remaining residual is written to the sidecar (commodity_identity_residual).

Results: Make output minus producer-price Use row, summed across commodities ignoring sign ($bn)

	2017	2018	2019	2020	2021	2022	2023	2024
before	189	378	457	891	1,009	1,460	1,380	1,439
after	2.9	5.7	14.9	37.6	93.7	175.3	22.7	24.9
commodities off by more than 1%	0	0	2	3	9	21	0	0

All eight years build and pass every gate (in memory, from the local balanced tables v0.5.0_a166975; 2017 from v0.5.0_011f41f).

What remains is Step 5's, and the gates prove it. On every commodity other than the 24 trade and transport commodities, the leftover equals exactly the amount by which the balanced Use tax rows and the balanced Supply tax columns disagree for that commodity's producers. Step 6 cannot place a subsidy on a commodity when the industry that makes it carries a different subsidy. The 2021–22 leaders:

2022	$bn
531ORE other real estate (Supply SUB moved from −1.6 to −28.7 by the balance; industry row did not follow)	+26.9
531HST tenant-occupied housing	−17.4
5241XX insurance carriers	+16.0
541100 legal services	+10.8

The trade and transport commodities carry the margin columns' failure to net to zero (#809): $17.6bn in 2022, $0.3–7.2bn in other years.

Industry identity. Make's industry output now equals each producer-price Use column plus value added, except where the balanced SUT's own industry identity is off (S00600 $3.5–3.9bn, real estate $0.2–1.6bn). Those match the balanced SUT to the $0.1M. On main, Make also carried the pre-balance taxes, which put the trade industries off by $1.4–3.0bn on top.

Trade-off to review

Against BEA's published 2017 before-redefinition tables, totals get closer but individual Use cells get a little further:

sum of absolute differences vs published 2017 ($bn)	main	this PR
Make cells	171.1	144.9
commodity output	133.6	72.1
Use row totals	111.0	73.0
Use cells	1,297.5	1,359.2
…margin-commodity rows	59.0	81.3

The 2017 rates are BEA's own 2017 placement, so unreconciled cells sit as close to the published table as they can. Requiring Use rows to equal Make output moves the trade commodities' rows onto our balanced Supply table, whose trade give-up by wholesale and retail commodity is $162bn away from BEA's in 2017 (on goods rows it is $22bn of $3.3T). That is a Supply-table difference, not Step 6's to hide.

Not in this PR

• Rebuilding and storing the MUT vintages, and switching nowcast_mut_vintage in the configs. After merge, run python -m bedrock.transform.iot.nowcast_mut for 2017–2024 with upload.
• The xfail on test_compare_output_from_make_and_use. It reads the stored MUT, so it can come off once the new vintage is stored and configured.
• Step 5 tax consistency. The balance should move each industry's T00TOP/T00SUB with the TOP/SUB of the commodities it makes. That is the follow-up for the 2021–22 residual.

Checks run locally

• ruff check, ruff format --check, and mypy clean on the three files.
• nowcast_va_taxes --check passes and supply_to_make --check passes.
• All eight years built through the new gates.
• The suite is left to CI.

🤖 Generated with Claude Code

[bl-young]

Looks good to approve as a Step 6 builder fix. The wedge fit, margin reconcile, and identity gates match the #1161 diagnosis; CI impact stays deferred until a stored MUT + config pin.

One thing to confirm: after _reconcile_placement, Producers' Value is PUR - stripped, and scaling margins up to Supply give-up can push individual buyer cells negative. Those would flow into producer-price Use and then A. Have you seen negative PRO / producer Use cells on the 2017-2024 builds? If yes, how large and on which commodities; if not, worth a quick scan or a light gate so we notice if a later vintage introduces them.

Also: the 2021-22 leftover is owned as Step 5 tax row/column disagreement (plus #809 on margin commodities). Is there already a follow-up filed or planned to move industry T00TOP/T00SUB with commodity TOP/SUB in the balance, or should we open one before this stacks further?

[WesIngwersen]

Thanks. Both answered below.

Negative producer-value cells: yes, a few, and they are new with this PR. I scanned the stored Margins tables of the newest build, v1.0.0_5514c3b (this PR plus the rest of the stack), for cells with purchasers' value ≥ 0 and producers' value < 0, the ones margin stripping turned negative.

	2017	2018	2019	2020	2021	2022	2023	2024
industry-buyer cells	3	17	29	55	35	75	69	84
their producers' value, $M	−0.1	−6.7	−11.0	−348.8	−61.0	−183.1	−209.5	−700.6

On v0.6.0_5d40d42, before this PR, 2024 had none. These are the cells that reach A. The total is at most about 0.003% of intermediate use, but individual cells are not small:

• 325411 medicinal and botanical manufacturing: 2024 only, −$526M over 12 buyers. The largest is 621100 physicians' offices, −$380M on $2.8bn purchasers' value, where the wholesale margin ($3.1bn) exceeds the purchase.
• 315000 apparel and 316000 leather: many small cells, −$409M and −$380M over all years, with retail margins above purchasers' value.
• 324110 refineries to 325110 and 325190, 2020 only: −$70M, again a retail margin on an industrial purchase.

The largest negative cells overall are expected and are not counted above: change in inventories (F03000) and the margin commodities' own rows, where margins net to zero (for example F03000 × 486000 pipeline, −$7.8bn in 2021). These were there before this PR at about the same size (2024: −$797M before, −$751M now).

I'd propose a follow-up rather than changing this PR: a gate in Step 6 that reports industry-buyer cells with producers' value below zero, and a clip that moves the excess margin to the commodity's other buyers.

Step 5 tax rows: that's #1163, filed on board 36 when #1162 went up: move industry T00TOP/T00SUB with the commodity TOP/SUB columns in the balance. Nothing in the stack depends on it.

[WesIngwersen]

Filed as #1177 (board 36): a Step 6 gate and clip for industry-buyer cells with negative producers' value.