Skip to content

feat(allocation): draft crosswalk-only E_USA_ES snapshot export - #1153

Merged
bl-young merged 2 commits into
by-rename-derivefrom
by-export-E
Oct 29, 2026
Merged

bl-young merged 2 commits into
by-rename-derivefrom
by-export-E

Conversation

@bl-young

@bl-young bl-young commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

cc:
Related: #1134

Stack: step after #1152 (by-rename-derive).

What changed? Why?

Draft fill-in for source-resolved USA E snapshots: derive_E_usa_emissions_sources / derive_E_usa_es_long consume the hand-written analysis crosswalk (fbs_to_ceda_emissions_source.csv) and a vendored 123-row ceda EmissionsSource catalog, after HFC speciation on the retained-activity Cornerstone GHG FBS. Nested-split shares and GHG method YAML breakouts are intentionally not applied -- lumped soils / NEU / aviation / res-com lines often stay bundled or unmatched and are omitted from E_USA_ES. E_USA_GAS is unchanged. Analysis README points at this draft path; nested-split recipes in analysis/emissions_source_crosswalk/ remain review-only.

Impact

Next generate_snapshots writes non-stub E_USA_ES / E_USA_ES_LONG with incomplete source coverage; do not publish CEDA-bound bags until ceda readers switch off gas-shaped E_USA_ES. Does not close #1134.

Testing

Unit tests for catalog load, crosswalk load, and specificity tagging.

uv run pytest bedrock/transform/allocation/__tests__/test_emissions_source_crosswalk.py -v
uv run black --check .
uv run ruff check .

Review focus

Heads-up

Hold CEDA year-series / product snapshot cuts until coordinated E reader switch.

Replace stub-zero E_USA_ES / empty E_USA_ES_LONG with a draft derive that
tags the retained-activity GHG FBS via the analysis crosswalk CSV (plus HFC
speciation). Nested splits and method YAML breakouts are not wired yet;
soils/NEU/aviation/res-com coverage stays incomplete.
@bl-young
bl-young added this pull request to stack #1154 October 28, 2026 18:56
@bl-young
bl-young requested a review from MoLi7 October 28, 2026 19:07
@bl-young

Copy link
Copy Markdown
Member Author

@MoLi7 a more complete, but not yet finished, wiring to the E parquets. I will be doing updated ingestion in CEDA for https://github.com/cornerstone-data/ceda/issues/470 and https://github.com/cornerstone-data/ceda/issues/582.

That way when the time series first draft runs for snapshots they will be much closer to the final outcome

@bl-young

bl-young commented Oct 29, 2026 •

Copy link
Copy Markdown
Member Author

Draft year-series snapshot cut (throwaway; draft E_USA_ES).

Branch tip: 235509a6c9630a147bc4c26249b8f6012429a6d4 (by-export-E)

Year GCS prefix GHA
2017 gs://cornerstone-default/snapshots/235509a6c9630a147bc4c26249b8f6012429a6d4_2026_usa_cornerstone_v1_0_2017/ https://github.com/cornerstone-data/bedrock/actions/runs/40796457650
2018 gs://cornerstone-default/snapshots/235509a6c9630a147bc4c26249b8f6012429a6d4_2026_usa_cornerstone_v1_0_2018/ https://github.com/cornerstone-data/bedrock/actions/runs/40796463103
2019 gs://cornerstone-default/snapshots/235509a6c9630a147bc4c26249b8f6012429a6d4_2026_usa_cornerstone_v1_0_2019/ https://github.com/cornerstone-data/bedrock/actions/runs/40796467442
2020 gs://cornerstone-default/snapshots/235509a6c9630a147bc4c26249b8f6012429a6d4_2026_usa_cornerstone_v1_0_2020/ https://github.com/cornerstone-data/bedrock/actions/runs/40796472233
2021 gs://cornerstone-default/snapshots/235509a6c9630a147bc4c26249b8f6012429a6d4_2026_usa_cornerstone_v1_0_2021/ https://github.com/cornerstone-data/bedrock/actions/runs/40796477659
2022 gs://cornerstone-default/snapshots/235509a6c9630a147bc4c26249b8f6012429a6d4_2026_usa_cornerstone_v1_0_2022/ https://github.com/cornerstone-data/bedrock/actions/runs/40796482848
2023 gs://cornerstone-default/snapshots/235509a6c9630a147bc4c26249b8f6012429a6d4_2026_usa_cornerstone_v1_0_2023/ https://github.com/cornerstone-data/bedrock/actions/runs/40796487469
2024 gs://cornerstone-default/snapshots/235509a6c9630a147bc4c26249b8f6012429a6d4_2026_usa_cornerstone_v1_0_2024/ https://github.com/cornerstone-data/bedrock/actions/runs/40796492771

Trust the run log Snapshot prefix: line; Slack success prints the bare SHA path.

@MoLi7 MoLi7 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

draft LGTM

@bl-young
bl-young merged commit f0de644 into main Oct 29, 2026
29 checks passed
@bl-young
bl-young deleted the by-export-E branch October 29, 2026 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Export E by emission source on Cornerstone codes

2 participants