diff --git a/.github/ISSUE_TEMPLATE/onboarding_task.md b/.github/ISSUE_TEMPLATE/onboarding_task.md index f53aa06..34bc37f 100644 --- a/.github/ISSUE_TEMPLATE/onboarding_task.md +++ b/.github/ISSUE_TEMPLATE/onboarding_task.md @@ -18,7 +18,7 @@ assignees: "" 2. Follow the setup in `README.md` (`pip install -e ".[examples,dev]"`) and activate the virtual environment. 3. Make the specified adjustments. -4. Run `pytest tests/ -v` -- expect `14 passed, 2 xfailed` unmodified, +4. Run `pytest tests/ -v` -- expect `19 passed, 2 xfailed` unmodified, unless your task changes that on purpose. 5. Run `mypy src/disslucc` and fix any new warnings it reports. diff --git a/CHANGELOG.md b/CHANGELOG.md index 2bb84e0..9989028 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,14 @@ All notable changes to `disslucc` are documented here. +## [Unreleased] + +### Changed +- `benchmark/goldens/` updated to terrame-docker v0.1.1: same results (CSVs within + 1e-12), with `terrame.log` and `manifest.json` in English; `benchmark/README.md` + translated to English. +- `CONTRIBUTING.md` and the onboarding issue template expect `19 passed, 2 xfailed`. + ## [0.4.0] -- 2026-09-23 Reference results now come from diff --git a/CLAUDE.md b/CLAUDE.md index fca2743..12951dd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -57,7 +57,7 @@ scripts, then three more `examples/*.py` files found later) -- grep for **Data provenance is not "whatever GitHub script has a matching name."** The reference results live in `benchmark/goldens/`, a copy of the goldens generated in [LambdaGeo/terrame-docker](https://github.com/LambdaGeo/terrame-docker) -v0.1.0, which keeps the generating scripts, the original TerraME outputs, the +v0.1.1, which keeps the generating scripts, the original TerraME outputs, the generator and goldens for all 21 LuccME labs. This repository keeps only the goldens its tests use (`lab01`, `lab15`: the LuccME package's labs; `lab01_md1643`, `lab15_md10`); add one together with the component and test diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9ed07cb..f687622 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -92,7 +92,7 @@ pip install -e ".[examples,dev]" pytest tests/ -v ``` -Expect `14 passed, 2 xfailed` on a clean checkout (the two `xfail` cases are +Expect `19 passed, 2 xfailed` on a clean checkout (the two `xfail` cases are documented, known limitations of the Lab15 discriminance test -- see `tests/test_benchmark_discriminance_lab15.py`). If your change makes that number change unexpectedly, treat it as a regression until proven otherwise. @@ -118,7 +118,7 @@ have their own conventions: -- the numbers they produce are cited in `dissmodel`'s JOSS paper. - **Reference results** (`benchmark/goldens/`) are a copy of the goldens this repository's tests use, generated in [LambdaGeo/terrame-docker](https://github.com/LambdaGeo/terrame-docker) - v0.1.0 (which has goldens for all 21 LuccME labs; copy one here only with + v0.1.1 (which has goldens for all 21 LuccME labs; copy one here only with the component and test that use it), which also keeps the original LuccME "Model Configurator" scripts, the original TerraME outputs and the generator. Never edit a golden by hand: diff --git a/benchmark/README.md b/benchmark/README.md index d09b9d8..d42a2e2 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -1,52 +1,52 @@ -# Goldens do TerraME/LuccME +# TerraME/LuccME goldens -`goldens/` é uma cópia dos goldens gerados em -[LambdaGeo/terrame-docker](https://github.com/LambdaGeo/terrame-docker) -**v0.1.0** (`benchmark/goldens/`), com TerraME 2.0.1 + LuccME `6244dd4`. É contra eles que o -disslucc é validado, algoritmo por algoritmo. +`goldens/` is a copy of goldens generated in +[LambdaGeo/terrame-docker](https://github.com/LambdaGeo/terrame-docker) **v0.1.1** +(`benchmark/goldens/`), with TerraME 2.0.1 + LuccME `6244dd4`. disslucc is validated +against them, one algorithm at a time. -Scripts, saídas originais do TerraME e o gerador ficam **só** no terrame-docker; aqui -fica só o resultado, para os testes rodarem sem Docker. +The scripts, the original TerraME outputs and the generator live **only** in +terrame-docker; this repository keeps only the results, so the tests run without Docker. -## O que há em `goldens//` +## Contents of `goldens//` -| Arquivo | Conteúdo | +| File | Contents | |---|---| -| `.csv.gz` | estado de cada célula ao fim de cada ano: `year,id,col,row`, `_out` e `_pot`, 12 casas decimais | -| `terrame.log` | saída do TerraME (demanda, área alocada, iterações por ano) | -| `manifest.json` | script de origem e SHA-256, versões, anos, colunas, iterações por ano (`iterations_per_year`) e a verificação cruzada | +| `.csv.gz` | state of every cell at the end of every year: `year,id,col,row`, `_out` and `_pot`, 12 decimal places | +| `terrame.log` | TerraME output (demand, allocated area, iterations per year) | +| `manifest.json` | source script and SHA-256, versions, years, columns, iterations per year (`iterations_per_year`) and the cross-check | -`col`/`row` alinham com `data/input/csAC.zip` (`col`, `row`) e `cs_moju.zip` (`col`, `lin`). +`col`/`row` match `data/input/csAC.zip` (`col`, `row`) and `cs_moju.zip` (`col`, `lin`). -## Quais são +## Which ones -Só os goldens que os testes deste repositório usam. Os outros (os 21 labs do pacote -LuccME) ficam no terrame-docker e entram aqui quando o componente correspondente for -implementado, junto com o teste que os usa. +Only the goldens this repository's tests use. The others (the 21 labs of the LuccME +package) stay in terrame-docker and come here when the corresponding component is +implemented, together with the test that uses them. -| Golden | Componentes | Usado em | +| Golden | Components | Used in | |---|---|---| | `lab01` | PreComputedValues + CLinearRegression + CClueLike (`maxDifference` 5000) | `test_goldens_per_year.py` | -| `lab01_md1643` | idem, `maxDifference` 1643: itera 8–26 vezes por ano | `test_goldens_per_year.py`, `test_validation_lab1.py`, discriminância | +| `lab01_md1643` | same, `maxDifference` 1643: iterates up to 26 times per year | `test_goldens_per_year.py`, `test_validation_lab1.py`, discriminance | | `lab15` | PreComputedValues + DLogisticRegression + DClueSLike (`maxDifference` 300) | `test_goldens_per_year.py` | -| `lab15_md10` | idem, `maxDifference` 10: itera 56–67 vezes por ano | `test_goldens_per_year.py`, `test_validation_lab15.py`, discriminância | +| `lab15_md10` | same, `maxDifference` 10: iterates 56–67 times per year | `test_goldens_per_year.py`, `test_validation_lab15.py`, discriminance | -O último ano de `lab01_md1643` e de `lab15_md10` é a antiga referência deste repositório +The last year of `lab01_md1643` and of `lab15_md10` is this repository's former reference (`benchmark/data/*.zip`). -## Antes de usar como prova +## Before using them as evidence -- Nos labs do pacote a alocação aceita a primeira passada em todos os anos; só as - variantes `_md` testam o laço de convergência. -- Nos labs com `CClueLike`, o TerraME nunca executa `correctCellChange` (typo - `regionregionAloc`). O disslucc executa por padrão; compare com - `cell_correction=False`. Ver `docs/validation.md`. -- Compare com tolerância (1e-9 no arquivo; os testes usam MAE < 1e-6), nunca pelo - SHA-256: de uma geração para outra, algumas células mudam na 12ª casa decimal. +- In the package labs the allocation is accepted at the first pass every year; only the + `_md` variants test the convergence loop. +- In the labs with `CClueLike`, TerraME never runs `correctCellChange` (a + `regionregionAloc` typo). disslucc runs it by default; compare with + `cell_correction=False`. See `docs/validation.md`. +- Compare with a tolerance (1e-9 in the file; the tests use MAE < 1e-6), never by + SHA-256: from one generation to the next, a few cells change in the 12th decimal place. -## Adicionar ou atualizar +## Adding or updating -Copie a pasta do golden de `benchmark/goldens//` do terrame-docker, na mesma versão -(`v0.1.0`), para `benchmark/goldens//` aqui, no mesmo commit do teste que passa a -usá-lo. Nunca edite um golden à mão; para regenerar, use `benchmark/generate.sh ` -no terrame-docker. +Copy the golden's folder from terrame-docker's `benchmark/goldens//`, at the same +version (`v0.1.1`), to `benchmark/goldens//` here, in the same commit as the test +that starts using it. Never edit a golden by hand; to regenerate one, run +`benchmark/generate.sh ` in terrame-docker. diff --git a/benchmark/goldens/lab01/manifest.json b/benchmark/goldens/lab01/manifest.json index fa91aea..ff43be7 100644 --- a/benchmark/goldens/lab01/manifest.json +++ b/benchmark/goldens/lab01/manifest.json @@ -12,7 +12,7 @@ }, "generator": { "script": "benchmark/harness.lua", - "sha256": "be7c575428c574da799f7f745da7766a05f6b878df08b12bcd0bab882e346581" + "sha256": "5f7504f11442d83981565676635ffd996034f721a5ad08ec0cab082d2e948a97" }, "status": "ok", "years": [ @@ -38,7 +38,7 @@ "max_abs_diff": { "Lab01_2014.dbf:d_out": 5.000028169277471e-13 }, - "note": "último ano do CSV comparado com os .dbf da pasta de saída: a saída do próprio script e, nas referências, a saída original do TerraME" + "note": "last year of the CSV compared with the .dbf files of the output folder: the script's own output and, for references, the original TerraME output" }, "iterations_per_year": { "2008": 0, @@ -49,5 +49,5 @@ "2013": 0, "2014": 0 }, - "iterations_note": "contínuo: 'Number of iterations' do LuccME; discreto: maior n de 'Iteration -> n' (0 = aceito na 1ª passada)" + "iterations_note": "continuous: LuccME's 'Number of iterations'; discrete: largest n in 'Iteration -> n' (0 = first pass accepted)" } diff --git a/benchmark/goldens/lab01/terrame.log b/benchmark/goldens/lab01/terrame.log index cfe123d..5eb3a04 100644 --- a/benchmark/goldens/lab01/terrame.log +++ b/benchmark/goldens/lab01/terrame.log @@ -14,7 +14,7 @@ d Area: 19982 Difference: 0 outros Area: 6489 Difference: 0 Demand allocated correctly in 2008. Number of iterations: 0 Maximum error: 2.7950358344242e-05 -[harness] lab01: Lab01, anos 2008..2014, colunas f_out,d_out,outros_out,f_pot,d_pot,outros_pot +[harness] lab01: Lab01, years 2008..2014, columns f_out,d_out,outros_out,f_pot,d_pot,outros_pot Executing Demand component Executing Potential component @@ -83,8 +83,8 @@ outros Area: 6489 Difference: 0 Demand allocated correctly in 2014. Number of iterations: 0 Maximum error: 3353.6276049931 Saving Lab01_2014. -Elapsed time: 00:00:04 hh:mm:ss +Elapsed time: 00:00:03 hh:mm:ss End of Simulation -[harness] lab01: 5 arquivos da saída original em /work/out/lab01 -[harness] lab01: CSV gravado em /work/out/lab01 +[harness] lab01: 5 files of the lab's own output in /work/out/lab01 +[harness] lab01: CSV written to /work/out/lab01 diff --git a/benchmark/goldens/lab01_md1643/lab01_md1643.csv.gz b/benchmark/goldens/lab01_md1643/lab01_md1643.csv.gz index 2d25df9..b674723 100644 Binary files a/benchmark/goldens/lab01_md1643/lab01_md1643.csv.gz and b/benchmark/goldens/lab01_md1643/lab01_md1643.csv.gz differ diff --git a/benchmark/goldens/lab01_md1643/manifest.json b/benchmark/goldens/lab01_md1643/manifest.json index d8d42db..bd39472 100644 --- a/benchmark/goldens/lab01_md1643/manifest.json +++ b/benchmark/goldens/lab01_md1643/manifest.json @@ -18,7 +18,7 @@ }, "generator": { "script": "benchmark/harness.lua", - "sha256": "be7c575428c574da799f7f745da7766a05f6b878df08b12bcd0bab882e346581" + "sha256": "5f7504f11442d83981565676635ffd996034f721a5ad08ec0cab082d2e948a97" }, "status": "ok", "years": [ @@ -37,7 +37,7 @@ "file": { "name": "lab01_md1643.csv.gz", "rows": 46018, - "sha256": "c0a46b524be5e482921c7fd10239168b3f472636dd762b18df34056793803c46" + "sha256": "979d63d14ebed6040ee3000b91939abdb050924be36385503cd1d74a2744759f" }, "crosscheck_vs_original_output": { "tolerance": 1e-09, @@ -45,7 +45,7 @@ "Lab1_2014.dbf:d_out": 4.999993474807951e-13, "original_Lab1_2014.dbf:d_out": 4.999993474807951e-13 }, - "note": "último ano do CSV comparado com os .dbf da pasta de saída: a saída do próprio script e, nas referências, a saída original do TerraME" + "note": "last year of the CSV compared with the .dbf files of the output folder: the script's own output and, for references, the original TerraME output" }, "iterations_per_year": { "2008": 0, @@ -56,5 +56,5 @@ "2013": 17, "2014": 17 }, - "iterations_note": "contínuo: 'Number of iterations' do LuccME; discreto: maior n de 'Iteration -> n' (0 = aceito na 1ª passada)" + "iterations_note": "continuous: LuccME's 'Number of iterations'; discrete: largest n in 'Iteration -> n' (0 = first pass accepted)" } diff --git a/benchmark/goldens/lab01_md1643/terrame.log b/benchmark/goldens/lab01_md1643/terrame.log index 9e079eb..5711e5c 100644 --- a/benchmark/goldens/lab01_md1643/terrame.log +++ b/benchmark/goldens/lab01_md1643/terrame.log @@ -14,7 +14,7 @@ d Area: 19982 Difference: 0 outros Area: 6489 Difference: 0 Demand allocated correctly in 2008. Number of iterations: 0 Maximum error: 2.7950358344242e-05 -[harness] lab01_md1643: Lab1, anos 2008..2014, colunas f_out,d_out,outros_out,f_pot,d_pot,outros_pot +[harness] lab01_md1643: Lab1, years 2008..2014, columns f_out,d_out,outros_out,f_pot,d_pot,outros_pot Executing Demand component Executing Potential component @@ -519,4 +519,4 @@ outros Area: 6489 Difference: 0 Demand allocated correctly in 2014. Number of iterations: 17 Maximum error: 1638.220954259 Saving Lab1_2014. -[harness] lab01_md1643: CSV gravado em /work/out/lab01_md1643 +[harness] lab01_md1643: CSV written to /work/out/lab01_md1643 diff --git a/benchmark/goldens/lab15/manifest.json b/benchmark/goldens/lab15/manifest.json index 4d7c79c..7502457 100644 --- a/benchmark/goldens/lab15/manifest.json +++ b/benchmark/goldens/lab15/manifest.json @@ -12,7 +12,7 @@ }, "generator": { "script": "benchmark/harness.lua", - "sha256": "be7c575428c574da799f7f745da7766a05f6b878df08b12bcd0bab882e346581" + "sha256": "5f7504f11442d83981565676635ffd996034f721a5ad08ec0cab082d2e948a97" }, "status": "ok", "years": [ @@ -38,7 +38,7 @@ "max_abs_diff": { "Lab15_2004.dbf:d_out": 0.0 }, - "note": "último ano do CSV comparado com os .dbf da pasta de saída: a saída do próprio script e, nas referências, a saída original do TerraME" + "note": "last year of the CSV compared with the .dbf files of the output folder: the script's own output and, for references, the original TerraME output" }, "iterations_per_year": { "1999": 0, @@ -48,5 +48,5 @@ "2003": 0, "2004": 0 }, - "iterations_note": "contínuo: 'Number of iterations' do LuccME; discreto: maior n de 'Iteration -> n' (0 = aceito na 1ª passada)" + "iterations_note": "continuous: LuccME's 'Number of iterations'; discrete: largest n in 'Iteration -> n' (0 = first pass accepted)" } diff --git a/benchmark/goldens/lab15/terrame.log b/benchmark/goldens/lab15/terrame.log index 7687750..1851d4f 100644 --- a/benchmark/goldens/lab15/terrame.log +++ b/benchmark/goldens/lab15/terrame.log @@ -24,7 +24,7 @@ d -> 205 demand -> 205 difference -> 0 o -> 3 demand -> 3 difference -> 0 Demand allocated correctly in this time: 1999 -[harness] lab15: Lab15, anos 1999..2004, colunas f_out,d_out,o_out,f_pot,d_pot,o_pot +[harness] lab15: Lab15, years 1999..2004, columns f_out,d_out,o_out,f_pot,d_pot,o_pot Executing Demand component Executing Potential component @@ -132,8 +132,8 @@ o -> 3 demand -> 3 difference -> 0 Demand allocated correctly in this time: 2004 Saving Lab15_2004. -Elapsed time: 00:00:03 hh:mm:ss +Elapsed time: 00:00:02 hh:mm:ss End of Simulation -[harness] lab15: 5 arquivos da saída original em /work/out/lab15 -[harness] lab15: CSV gravado em /work/out/lab15 +[harness] lab15: 5 files of the lab's own output in /work/out/lab15 +[harness] lab15: CSV written to /work/out/lab15 diff --git a/benchmark/goldens/lab15_md10/manifest.json b/benchmark/goldens/lab15_md10/manifest.json index ed4b4ef..584499a 100644 --- a/benchmark/goldens/lab15_md10/manifest.json +++ b/benchmark/goldens/lab15_md10/manifest.json @@ -18,7 +18,7 @@ }, "generator": { "script": "benchmark/harness.lua", - "sha256": "be7c575428c574da799f7f745da7766a05f6b878df08b12bcd0bab882e346581" + "sha256": "5f7504f11442d83981565676635ffd996034f721a5ad08ec0cab082d2e948a97" }, "status": "ok", "years": [ @@ -45,7 +45,7 @@ "Lab6_2004.dbf:d_out": 0.0, "original_Lab15_2004.dbf:d_out": 0.0 }, - "note": "último ano do CSV comparado com os .dbf da pasta de saída: a saída do próprio script e, nas referências, a saída original do TerraME" + "note": "last year of the CSV compared with the .dbf files of the output folder: the script's own output and, for references, the original TerraME output" }, "iterations_per_year": { "1999": 0, @@ -55,5 +55,5 @@ "2003": 61, "2004": 61 }, - "iterations_note": "contínuo: 'Number of iterations' do LuccME; discreto: maior n de 'Iteration -> n' (0 = aceito na 1ª passada)" + "iterations_note": "continuous: LuccME's 'Number of iterations'; discrete: largest n in 'Iteration -> n' (0 = first pass accepted)" } diff --git a/benchmark/goldens/lab15_md10/terrame.log b/benchmark/goldens/lab15_md10/terrame.log index 3f79c0e..e147415 100644 --- a/benchmark/goldens/lab15_md10/terrame.log +++ b/benchmark/goldens/lab15_md10/terrame.log @@ -24,7 +24,7 @@ d -> 205 demand -> 205 difference -> 0 o -> 3 demand -> 3 difference -> 0 Demand allocated correctly in this time: 1999 -[harness] lab15_md10: Lab6, anos 1999..2004, colunas f_out,d_out,o_out,f_pot,d_pot,o_pot +[harness] lab15_md10: Lab6, years 1999..2004, columns f_out,d_out,o_out,f_pot,d_pot,o_pot Executing Demand component Executing Potential component @@ -1637,7 +1637,7 @@ o -> 3 demand -> 3 difference -> 0 Demand allocated correctly in this time: 2004 Saving Lab6_2004. -Elapsed time: 00:00:11 hh:mm:ss +Elapsed time: 00:00:07 hh:mm:ss End of Simulation -[harness] lab15_md10: CSV gravado em /work/out/lab15_md10 +[harness] lab15_md10: CSV written to /work/out/lab15_md10 diff --git a/docs/validation.md b/docs/validation.md index c7c7367..132da50 100644 --- a/docs/validation.md +++ b/docs/validation.md @@ -40,7 +40,7 @@ allocated quantity, not position. Reproduce: `examples/run_lab1_validation.py`. Provenance: the original LuccME scripts that generated this reference -(`Lab1`, `maxDifference=1643`) are vendored at +(`Lab1`, `maxDifference=1643`) are kept in terrame-docker's `benchmark/references/lab01_md1643/` (`lab1_main.lua` + `lab1_submodel.lua`, with the original TerraME output zip). These are **not** the same scripts as `terrame/luccme`'s public