docs: rewrite Ch26 for disslucc

Author: profsergiocosta

docs/part5/ch26_lucc.ipynb

@@ -28,12 +28,7 @@
     "\n",
     "*Part V — Domain Modeling: Land Use & Coastal Systems*\n",
     "\n",
-    "Implemented by the [`disslucc-continuous`](https://github.com/DisSModel/disslucc-continuous) and [`disslucc-discrete`](https://github.com/DisSModel/disslucc-discrete) packages.\n",
-    "\n",
-    "<div class=\"admonition warning\">\n",
-    "<p class=\"admonition-title\">Watch out</p>\n",
-    "<p>This chapter is a draft, deliberately written ahead of the underlying DisSLUCC packages settling — student work on both is still in progress and the API is likely to shift. Treat this as a base to revise once that work stabilizes, not a final reference; the CLI commands shown were not re-verified end to end against the currently installed packages.</p>\n",
-    "</div>"
+    "Implemented by the [`disslucc`](https://github.com/DisSModel/disslucc) package."
    ]
   },
   {
@@ -45,10 +40,10 @@
     "\n",
     "By the end of this chapter you will be able to:\n",
     "\n",
-    "- Explain what DisSLUCC means as a family, not a single package\n",
+    "- Explain why `disslucc` implements both continuous and discrete allocation in one package, on one raster substrate\n",
     "- Choose between continuous and discrete allocation for a given research question\n",
     "- Describe the Demand/Potential/Allocation loop every LUCC model in the ecosystem shares\n",
-    "- Know how each package validates itself against its TerraME/LUCCME predecessor"
+    "- Know how each allocation style validates itself against its TerraME/LUCCME predecessor, and run it yourself via script or CLI"
    ]
   },
   {
@@ -77,16 +72,16 @@
    "metadata": {},
    "id": "ddc11f3a",
    "source": [
-    "## What DisSLUCC Is\n",
+    "## What `disslucc` Is\n",
     "\n",
-    "**DisSLUCC** isn't one package — it's the name for two libraries that implement spatially explicit land-use and cover change modeling on top of `dissmodel`, mirroring two allocation philosophies LUCCME itself historically supported:\n",
+    "`disslucc` implements spatially explicit land-use and cover change modeling on top of `dissmodel`, raster-only, covering two allocation philosophies LUCCME itself historically supported — not as two packages, but as two components within one:\n",
     "\n",
-    "| Approach | Package | Style | Unit of allocation |\n",
-    "|---|---|---|---|\n",
-    "| Continuous | `disslucc-continuous` | LUCCME-like | area/percentage per cell |\n",
-    "| Discrete | `disslucc-discrete` | CLUE-S-like | one land use per cell |\n",
+    "| Approach | Style | Unit of allocation | Potential | Allocation |\n",
+    "|---|---|---|---|---|\n",
+    "| Continuous | LUCCME-like | area/percentage per cell | `PotentialLinearRegression` | `AllocationClueLike` |\n",
+    "| Discrete | CLUE-S-like | one land use per cell | `PotentialDLogisticRegression` | `AllocationDClueSLike` |\n",
     "\n",
-    "Both depend on `dissmodel` as an ordinary package dependency, following the same additive philosophy Chapter 33 described for every satellite package in the ecosystem — neither modifies the core."
+    "Both import from the same `disslucc` package and depend on `dissmodel` as an ordinary package dependency, following the same additive philosophy Chapter 33 described for every satellite package in the ecosystem — neither modifies the core. There are two entry points into the same underlying components: a **script-first** path (build the models directly in Python — no TOML, no provenance tracking, the fastest way to experiment) and an **Executor** path (`disslucc.executors`, wraps the same math in `ModelExecutor`/`ExperimentRecord` for automatic provenance, and is what gets registered in [`dissmodel-configs`](https://github.com/DisSModel/dissmodel-configs) to run on `dissmodel-platform`). Both produce identical results; which one you reach for depends on whether you need the provenance record."
    ]
   },
   {
@@ -110,65 +105,84 @@
    "metadata": {},
    "id": "8435fe0f",
    "source": [
-    "## Continuous Allocation: disslucc-continuous\n",
+    "## Continuous Allocation\n",
+    "\n",
+    "`PotentialLinearRegression` + `AllocationClueLike` answer \"how much does this cell's land use change\" — the right choice whenever a cell can legitimately hold more than one land use at once (a partially-deforested cell, a partially-urbanized one). Potential is one linear regression per land-use type, each with its own intercept and driving-factor coefficients (`RegressionSpec`); allocation distributes demand across cells according to that potential map, subject to per-class minimum/maximum bounds (`AllocationSpec`).\n",
     "\n",
-    "`disslucc-continuous` answers \"how much does this cell's land use change\" — the right choice whenever a cell can legitimately hold more than one land use at once (a partially-deforested cell, a partially-urbanized one). Potential comes from `PotentialLinearRegression`, one regression per land-use type, each with its own intercept and driving-factor coefficients; allocation comes from `AllocationClueLike`, distributing demand across cells according to that potential map, subject to per-class minimum/maximum bounds.\n",
+    "Script-first — no `ModelExecutor`, no TOML, the fastest way to run this:\n",
     "\n",
     "```python\n",
-    "from disslucc_continuous import (\n",
-    "    DemandPreComputedValues, load_demand_csv,\n",
-    "    PotentialLinearRegression, RegressionSpec,\n",
-    "    AllocationClueLike, AllocationSpec,\n",
-    ")\n",
+    "from dissmodel.core import Environment\n",
+    "from disslucc import DemandInline, PotentialLinearRegression, AllocationClueLike\n",
+    "from disslucc.schemas import RegressionSpec, AllocationSpec\n",
     "\n",
-    "demand = DemandPreComputedValues(\n",
-    "    annual_demand=load_demand_csv(\"demand.csv\", [\"forest\", \"dev\", \"other\"]),\n",
-    "    land_use_types=[\"forest\", \"dev\", \"other\"],\n",
-    ")\n",
+    "env = Environment(end_time=7)\n",
+    "\n",
+    "demand = DemandInline(values=demand_matrix, land_use_types=LAND_USE_TYPES)\n",
     "\n",
     "potential = PotentialLinearRegression(\n",
-    "    gdf=gdf,\n",
-    "    land_use_types=[\"forest\", \"dev\", \"other\"],\n",
-    "    land_use_no_data=\"other\",\n",
+    "    backend=backend,\n",
+    "    demand=demand,\n",
+    "    land_use_types=LAND_USE_TYPES,\n",
     "    potential_data=[[\n",
-    "        RegressionSpec(const=0.74, betas={\"dist_roads\": -0.22, \"protected\": 0.18}),\n",
-    "        RegressionSpec(const=0.27, betas={\"dist_roads\": -9.9e-7}),\n",
-    "        RegressionSpec(const=0.0),\n",
+    "        RegressionSpec(const=-0.2, betas={\"slope\": 0.4}),                     # forest\n",
+    "        RegressionSpec(const=0.4, betas={\"dist_road\": -0.1, \"slope\": -0.5}),  # agriculture\n",
+    "        RegressionSpec(const=0.3, betas={\"dist_road\": -0.6, \"slope\": -0.3}),  # urban\n",
     "    ]],\n",
     ")\n",
+    "\n",
+    "allocation = AllocationClueLike(\n",
+    "    backend=backend, demand=demand, potential=potential,\n",
+    "    land_use_types=LAND_USE_TYPES,\n",
+    "    static={\"forest\": 0, \"agriculture\": -1, \"urban\": -1},\n",
+    "    complementar_lu=\"forest\", cell_area=1.0, max_difference=5.0,\n",
+    "    allocation_data=[AllocationSpec(static=0), AllocationSpec(static=-1), AllocationSpec(static=-1)],\n",
+    ")\n",
+    "\n",
+    "env.run()\n",
     "```\n",
     "\n",
-    "Validation follows the same benchmark pattern Chapter 32 already described for the ecosystem generally: a dedicated benchmark executor runs vector and raster substrates side by side and checks both against a TerraME/LUCCME reference dataset, asserting mean absolute error and root-mean-square error stay under a configurable tolerance — never trusting a production run without that check passing first."
+    "**Validated against real TerraME/LUCCME output**, not synthetic data: Lab1 (Amazon deforestation, csAC region, 6,574 cells, 6 steps) reproduces the reference at **MAE = 0.0036** (RMSE 0.0062, max error 0.027), well within the 0.01 tolerance the original benchmark used to call something \"equivalent to TerraME\". The Pontius & Millones decomposition attributes 90% of that residual to quantity disagreement and 10% to allocation disagreement — the spatial pattern matches almost perfectly; the small gap is in total allocated quantity, not position. The residual itself traces to the original LuccME script's own convergence tolerance (`maxDifference = 1643` against a 2014 demand of 21,607 — a 7.6% band the reference itself doesn't close). Runs at 44.0 ms/step."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "id": "1aad283a",
    "source": [
-    "## Discrete Allocation: disslucc-discrete\n",
+    "## Discrete Allocation\n",
+    "\n",
+    "`PotentialDLogisticRegression` + `AllocationDClueSLike` answer a different question: \"which single land use dominates this cell\" — a CLUE-S-style allocation, right whenever ground-truth is itself categorical (a classified land-cover map, not a fractional-cover raster). Potential comes from a logistic regression predicting *which class* a cell most likely belongs to, not *how much* of a fractional quantity it holds (`LogisticRegressionSpec`, adding an `elasticity` term over the continuous case); allocation runs through a competition-based CLUE-S loop governed by a transition matrix (`[region][from][to]`, which pairs are even reachable).\n",
     "\n",
-    "`disslucc-discrete` answers a different question: \"which single land use dominates this cell\" — a CLUE-S-style allocation, right whenever ground-truth is itself categorical (a classified land-cover map, not a fractional-cover raster). Potential here comes from `PotentialDLogisticRegression` instead of a linear one — a logistic regression predicts *which class* a cell most likely belongs to, not *how much* of a fractional quantity it holds — and allocation runs through a competition-based `AllocationDClueSLike`. Configuration lives entirely in a TOML file rather than inline Python — land-use types, per-class regression coefficients, elasticities, and the allowed transition matrix:\n",
+    "The Executor path is what registers with `dissmodel-configs`/`dissmodel-platform`, and — as of `dissmodel` 0.6.4 — also runs locally via its CLI, configuration in a TOML file rather than inline Python:\n",
     "\n",
     "```toml\n",
     "[model]\n",
-    "land_use_types = [\"forest\", \"dev\", \"other\"]\n",
-    "region_attr    = \"region\"\n",
+    "land_use_types    = [\"f\", \"d\", \"o\"]\n",
+    "transition_matrix = [[[1, 1, 0], [0, 1, 0], [0, 0, 1]]]  # irreversible deforestation\n",
+    "\n",
+    "[model.parameters]\n",
+    "n_steps = 6\n",
     "\n",
-    "[[model.potential]]\n",
-    "const      = -2.34\n",
+    "[[model.potential_data]]\n",
+    "lu         = \"f\"\n",
+    "const      = -2.34187976925989\n",
     "elasticity = 0.0\n",
-    "[model.potential.betas]\n",
-    "soil_decl  = -0.03\n",
-    "dist_road  = 3.10\n",
-    "\n",
-    "[model.allocation]\n",
-    "max_difference   = 10.0\n",
-    "max_iteration    = 1000\n",
-    "factor_iteration = 0.0001\n",
+    "  [model.potential_data.betas]\n",
+    "  dist_br = 3.10319957497883\n",
     "```\n",
     "\n",
-    "`disslucc-discrete`'s validation reaches the strongest bar anywhere in this book: not a tolerance band, but **exact cell-level parity** against a TerraME/LuccME reference dataset — 100% accuracy, Cohen's κ = 1.0, F1 = 1.0, checked automatically in continuous integration. That exactness is possible specifically *because* the output is categorical — a cell either matches its reference class or it doesn't, with no \"close enough\" in between the way a continuous percentage would have."
+    "```bash\n",
+    "python -m disslucc.executors.discrete run \\\n",
+    "  --toml examples/dissmodel-configs/lucc_discrete.toml \\\n",
+    "  --input data/input/cs_moju.zip \\\n",
+    "  --param demand_csv=data/input/demand_moju.csv \\\n",
+    "  --output outputs/result.tif\n",
+    "```\n",
+    "\n",
+    "**Validated against real TerraME/LUCCME output**: Lab15 (deforestation, Moju region, 5,914 cells, 6 steps) reaches **exact cell-for-cell agreement** — zero quantity disagreement, zero allocation disagreement (Pontius & Millones decomposition), 100% accuracy, F1 = 1.0. Runs at 10.3 ms/step.\n",
+    "\n",
+    "**Discriminance caveat, inherited from the original benchmark and still true here:** the Lab15 scenario is nearly non-discriminative — a trivial static ranking by `(prob_d - prob_f)`, with no CLUE-S, no iteration, no time steps, already reproduces the same cell-by-cell output. So this result confirms the logistic regression coefficients were transcribed correctly; it does *not* by itself prove the CLUE-S allocation algorithm (iteration/convergence via `factor_iteration`) is faithful in a scenario where competition between classes actually matters. A shipped discriminance test makes this explicit rather than letting the strong headline number stand unqualified; a dynamic-covariate scenario that would actually exercise the competition logic is planned."
    ]
   },
   {
@@ -178,15 +192,16 @@
    "source": [
     "## Choosing Between Continuous and Discrete\n",
     "\n",
-    "Both packages implement the identical Demand/Potential/Allocation loop with different algorithms at each step — the decision between them is about your data, not architecture:\n",
+    "Both components implement the identical Demand/Potential/Allocation loop with different algorithms at each step — the decision between them is about your data, not architecture:\n",
     "\n",
-    "| | `disslucc-continuous` | `disslucc-discrete` |\n",
+    "| | Continuous | Discrete |\n",
     "|---|---|---|\n",
     "| Potential | linear regression | logistic regression |\n",
     "| Allocation | CLUE-like, per-class bounds | competition-based CLUE-S |\n",
     "| Validation | MAE/RMSE tolerance vs. reference | exact cell-level parity vs. reference |\n",
+    "| Speed (validated benchmark) | 44.0 ms/step | 10.3 ms/step |\n",
     "\n",
-    "The fastest way to decide: look at your calibration and validation data first. If it's expressed as a class label per cell, `disslucc-discrete` is the model that can be checked against it exactly. If it's expressed as an area or percentage per cell, `disslucc-continuous` is the only one that represents it without lossy discretization forced on it first."
+    "The fastest way to decide: look at your calibration and validation data first. If it's expressed as a class label per cell, discrete allocation is what can be checked against it exactly. If it's expressed as an area or percentage per cell, continuous allocation is the only one that represents it without lossy discretization forced on it first."
    ]
   },
   {
@@ -208,9 +223,9 @@
    "source": [
     "## Exercises\n",
     "\n",
-    "1. **Match the package to the question.** For each research question, name which package fits and why: (a) \"how does the percentage of forest cover in each cell change over the next 20 years,\" (b) \"which cells convert from forest to pasture by 2030.\"\n",
-    "2. **Why two different regressions?** `PotentialLinearRegression` and `PotentialDLogisticRegression` both take driving-factor coefficients (`betas`) per land-use type. From the model names alone, explain why one needs a linear regression and the other a logistic one — what is each one actually trying to predict?\n",
-    "3. **Tolerance vs. exact parity.** Explain, in your own words, why a tolerance-based check makes sense for `disslucc-continuous`'s validation but not for `disslucc-discrete`'s.\n",
+    "1. **Match the component to the question.** For each research question, name which allocation style fits and why: (a) \"how does the percentage of forest cover in each cell change over the next 20 years,\" (b) \"which cells convert from forest to pasture by 2030.\"\n",
+    "2. **Why two different regressions?** `PotentialLinearRegression` and `PotentialDLogisticRegression` both take driving-factor coefficients (`betas`) per land-use type. From the class names alone, explain why one needs a linear regression and the other a logistic one — what is each one actually trying to predict?\n",
+    "3. **Tolerance vs. exact parity.** Explain, in your own words, why a tolerance-based check makes sense for the continuous allocation's validation but not for the discrete one's.\n",
     "4. **Multiscale comparison, by hand.** Sketch, conceptually, how you'd compute a 3×3-window agreement score between two categorical land-use grids of the same shape — what would you compare within each window, and how would you turn that into a single agreement number for the whole grid?"
    ]
   },
@@ -233,10 +248,11 @@
     "\n",
     "### Key concepts introduced\n",
     "\n",
-    "- DisSLUCC as two packages, not one, sharing the identical Demand/Potential/Allocation loop LUCCME originally established\n",
-    "- `disslucc-continuous`: fractional, per-cell land-use change, validated by MAE/RMSE tolerance against a TerraME/LUCCME reference\n",
-    "- `disslucc-discrete`: categorical, one-class-per-cell allocation, validated by exact cell-level parity — the strongest equivalence claim in the ecosystem, possible because the output is categorical\n",
-    "- Choosing between them by looking at your own calibration/validation data's type first, not by architectural preference\n",
+    "- `disslucc`: one raster-only package implementing both continuous and discrete LUCC allocation, sharing the identical Demand/Potential/Allocation loop LUCCME originally established\n",
+    "- **Continuous**: fractional, per-cell land-use change, validated by MAE/RMSE tolerance against a real TerraME/LUCCME reference (MAE 0.0036, 44.0 ms/step)\n",
+    "- **Discrete**: categorical, one-class-per-cell allocation, validated by exact cell-level parity — the strongest equivalence claim in the ecosystem, possible because the output is categorical (10.3 ms/step) — with a discriminance test making explicit that this validates coefficient transcription, not full allocation-algorithm fidelity\n",
+    "- Two entry points into the same components: script-first (fastest to experiment) and `ModelExecutor`-based (automatic provenance, TOML registration, CLI)\n",
+    "- Choosing between allocation styles by looking at your own calibration/validation data's type first, not by architectural preference\n",
     "- Engineering validation (matching a TerraME reference) versus scientific validation (calibration/validation split against real-world data), and multiscale comparison as an alternative to an overly strict pixel-for-pixel match\n",
     "\n",
     "Chapter 27 stays in domain-application territory but moves from land far inland to the coastline itself — a coupled flood and mangrove-migration model, run on both substrates and validated two different ways at once."
@@ -252,8 +268,9 @@
     "- Verburg, P. H. et al. (2002). \"Modeling the spatial dynamics of regional land use: the CLUE-S model.\" *Environmental Management*, 30(3), 391-405\n",
     "- Verburg, P. H. et al. (2006). \"Downscaling of land use change scenarios to assess the dynamics of European landscapes.\" *Agriculture, Ecosystems & Environment*, 114(1), 39-56\n",
     "- Costanza, R. (1989). \"Model goodness of fit: a multiple resolution procedure.\" *Ecological Modelling*, 47(3-4), 199-215 — the multiscale comparison method this chapter's validation section draws on\n",
+    "- Pontius Jr., R. G. & Millones, M. (2011). \"Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment.\" *International Journal of Remote Sensing*, 32(15), 4407-4429 — the decomposition `disslucc` uses in place of Cohen's kappa\n",
     "- LuccME documentation (INPE): <http://www.dpi.inpe.br/luccme/>\n",
-    "- disslucc-continuous and disslucc-discrete on GitHub: <https://github.com/DisSModel>"
+    "- `disslucc` on GitHub: <https://github.com/DisSModel/disslucc>"
    ]
   }
  ]