feat: observed multi-cropping baseline from MIRCA-OS - #31
Merged
koen-vg merged 5 commits intoJul 22, 2026
Conversation
Anchor multiple cropping to the observed 2020 baseline derived from MIRCA-OS v2. A config-independent Snakemake checkpoint (derive_mirca_multicropping) attributes MIRCA's extra-cycle harvested area (annual harvested minus AEI-capped footprint) to crop-sequence combinations on the 5-arcmin grid, discovering the combination set itself; the static multiple_cropping config section now only supplements the derived set, and combinations referencing unmodeled crops are dropped at resolution time (workflow/scripts/multi_cropping_combinations.py). crop_production_multi links are anchored via baseline_area_mha with p_nom_max = max(GAEZ potential, anchor), and share the crop stability budget, the bounded cost-calibration mechanism (new per-(combination, country) multi_crop_cost.csv artefact extracted directly from the multi links' own band duals), the crop growth cap (cycle-multiplicity weighted) and validation-mode pinning with single-crop links. Harvested cycles carried by multi links are reconciled out of the single-crop FAOSTAT baselines (with country-level redistribution) so each harvested cycle is counted exactly once, and the land-correction generator absorbs only the residual. Multi-cropping is no longer disabled under the land deviation penalty or use_actual_production. The GAEZ sequence_feasible gate is removed: GAEZ attainable season lengths overshoot the farmed cycle and rejected nearly all observed double-cropping. Feasibility evidence is MIRCA observation plus the GAEZ multiple-cropping zone; GAEZ supplies suitability, yields and water requirements. MIRCA-OS grids are fetched from the HydroShare BagIt zip via HTTP-range partial extraction (32 MB / 284 MB / 1.5 GB members instead of the ~10 GB bag); the nested RAR5 archives are unpacked with bsdtar (new libarchive dependency). Extracted from the water-aware branch, without the intra-year water periods and MIRCA crop-calendar retiming that belong to the upcoming water supply restructure.
… breakdown crop_production_multi links carry bounded_subsidy/bounded_penalty rates like single-crop links, but the objective-breakdown extractor only recovered those linopy-level auxiliary terms for crop_production, grassland_production and animal_production. The unaccounted subsidy made extracted costs exceed the model objective by ~5.5%, tripping the identity check in analyze_model.
add_multi_cropping_links wired crop outputs, water, fertilizer, residues and soil-N2O but never emission:ch4, so every wetland-rice cycle carried by a multi-cropping bundle emitted no methane. The gap predates this branch but was dormant: multi links had no baseline anchor, were left out of the crop growth cap, and were disabled entirely under actual-production mode and under an active land deviation penalty. Anchoring them removes all three protections, and the single-crop baseline reconciliation moves the same harvested cycles off the links that do carry CH4 -- on the test config, 89 Mha of rice cycles, about 11.5 Mt CH4/yr, silently vanished. It also opened a free-abatement channel: under a GHG price, shifting rice onto a double-rice link cost nothing and the growth cap credits the cycles either way. A bundle running m rice cycles now carries m times the per-hectare emission factor (rainfed cycles scaled as on the single-crop path), so rice methane is invariant to whether a hectare is represented as a single-crop link or a cycle of a bundle. Verified on the built test model: 134.47 kt CH4/Mha per irrigated rice cycle on both carriers. Also: ship multi_crop_cost.csv in the gbd-anchored calibration set (its absence broke DAG resolution for every health-enabled config), raise instead of silently skipping when a built link's combination is absent from the effective set (its cycles would be double-counted), and pass the static multiple_cropping section as a build_model param again so editing it retriggers the build (dropping the unused solve-time deviation_penalty param at the same time).
The cost calibration emits per-(combination, country) corrections to multi_crop_cost.csv, but the wrapper's artefact and cost-target lists never learned about it, so a 'tools/calibrate cost' run left the file untouched and the staleness check ignored it.
Member
Author
|
Known limitation worth recording: the calibration provenance stamp snapshots That is tolerable because the derived set is deterministic given the pinned MIRCA-OS release and the derivation script, both version-controlled, but it does mean the stamp no longer fully characterises the multi-cropping structure. Closing it properly would mean hashing |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Anchors multiple cropping to an observed baseline derived from MIRCA-OS v2, so established crop sequences are represented explicitly instead of treating every extra cropping cycle as unconstrained greenfield potential.
What changes
Curated combinations and configuration
The supported MIRCA-OS crop sequences now live in a small, reviewed YAML catalog under
data/curated/. The catalog documents how its eight sequences were selected from the static MIRCA-OS crop-calendar data, so the workflow no longer needs a dynamic Snakemake checkpoint or runtime combination discovery.multiple_croppinghas deliberately simple semantics:nulldisables it.Only combinations whose crops are active in the current config are materialized.
baseline_yearselects the closest available MIRCA-OS snapshot (2010, 2015, or 2020), with the earlier year winning ties.Data workflow and validation
MIRCA-OS archives are downloaded once per archive and extracted in one pass, replacing per-member remote archive probing. Baseline derivation is config-specific and keeps irrigated and rainfed attribution separate while enforcing shared crop, footprint, and extra-cycle budgets.
The derivation validates CRS, shape, affine transform, coordinates, crop coverage, and non-negative finite raster values before attribution. The affine comparison tolerates only sub-pixel metadata rounding observed in the Cassava source raster; actual shifts, rotations, resolution changes, and CRS mismatches fail early.
Model accounting
Only complete crop-cycle sequences become
crop_production_multilinks. Links carry explicit JSON cycle metadata plus crop, country, region, resource-class, and water-supply columns, so downstream code filters metadata instead of parsing component names.Multi-crop anchors are reconciled against joint FAOSTAT country/crop/water budgets. Attribution can relocate within the same country, combination, and water system when needed, while incomplete sequences stay on single-crop links. The accounting fails rather than silently dropping harvested area, and tests assert exact cycle conservation.
Multi-crop links participate consistently in stability bounds, growth caps, validation pinning, cost calibration, and objective accounting. Wetland-rice cycles carry the same per-cycle methane emissions as single-crop production, including repeated rice cycles.
Calibration
Both tracked calibration sets were regenerated as part of this PR:
defaultgbd-anchoredAll feed, food-waste, food-demand, cost, multi-crop-cost, and stability artifacts were recalibrated. Calibration intermediates now use distinct per-source/per-step namespaces, preventing one calibration step from invalidating another step's staleness check. Both provenance stamps match their consuming configs, and all 24 tracked calibration files are nonempty with finite numeric data.
Data
MIRCA-OS v2 (Kebede et al. 2025, Scientific Data 12:208, CC BY 4.0) is fetched automatically. Derived rasters remain workflow outputs under the configured processing root; the curated sequence catalog is the only new source-controlled MIRCA configuration artifact.
Verification
Other fixes
The PR also makes crop-cost calibration config-specific, tracks
multi_crop_cost.csvin every calibration set, validates CropGrids crop coverage, preserves baseline-year configuration throughout the workflow, fixes documentation inconsistencies, and removes ambiguous Unicode from touched source text.