Translate the codebase to English

README.md

@@ -62,8 +62,8 @@ RasterMap(
     backend=backend,
     band="state",
     color_map={0: "#ffffff", 1: "#2f8f6e"},
-    labels={0: "morta", 1: "viva"},
-    title=f"Padrões clássicos sobre fronteiras de bloco",
+    labels={0: "dead", 1: "alive"},
+    title="Classic patterns over block boundaries",
 )
 env.run()
 
@@ -191,8 +191,8 @@ is what tends to expose it.
 
 ## Two input paths: GeoTIFF/VRT or Zarr
 
-`geotiff_io.py`/`mosaic_io.py` (TIFF/VRT, via `rasterio`) and
-`zarr_io.py` (via `zarr`) are **interchangeable** input paths — both
+`disk/io/geotiff.py` (TIFF/VRT, via `rasterio`) and
+`disk/io/zarr.py` (via `zarr`) are **interchangeable** input paths — both
 populate the same `MemmapRasterWorkspace`, block by block, without
 materializing the full array in RAM. Swap the loader; the rest of the
 pipeline (halo, disk, models) doesn't change.
@@ -250,7 +250,7 @@ never imported by `haloexec`'s runtime code).
 
 ## Loading GeoTIFF straight to disk
 
-`geotiff_io.py` (`load_geotiff_into_workspace`) loads a real GeoTIFF
+`disk/io/geotiff.py` (`load_geotiff_into_workspace`) loads a real GeoTIFF
 block by block directly into a `MemmapRasterWorkspace`, via
 `rasterio.windows.Window` — never materializing a whole band in RAM.
 It uses the `band_spec` convention already established by
@@ -281,11 +281,10 @@ routing, watershed delineation — where a cell's value can, in
 principle, depend on the entire domain, not just its immediate
 neighbors.
 
-Generalized from a domain-specific student prototype (tidal
-connectivity via `scipy.ndimage.binary_propagation`), though that
-prototype's specific rule is not part of this primitive — only the
-orchestration pattern was extracted: a small halo plus repeated global
-sweeps until no block changes, instead of one large halo. Each sweep
+The primitive holds only the orchestration pattern — the rule itself
+(e.g. tidal connectivity via `scipy.ndimage.binary_propagation`) is
+supplied by the caller: a small halo plus repeated global sweeps until
+no block changes, instead of one large halo. Each sweep
 writes its result **immediately** back (Gauss-Seidel, via
 `write_block_core_in_place` — no ping-pong), so a block processed
 later in the same sweep already sees the update from a block processed
@@ -346,16 +345,13 @@ seeds).
 
 ## Disk layer (grids larger than RAM)
 
-`disk_backend.py` (`MemmapRasterWorkspace`) and `disk_sync_model.py`
+`disk/workspace.py` (`MemmapRasterWorkspace`) and `disk/sync_model.py`
 (`DiskChunkedSyncRasterModel`) generalize the same domain decomposition
 for grids that don't fit in memory, using `np.memmap` with
 double-buffering and checkpointing. **No dependency on dissmodel** in
-`disk_backend.py` — reusable by any framework.
+`disk/workspace.py` — reusable by any framework.
 
-Extracted and generalized from a student prototype (a domain-specific
-preprocessing pipeline) that already had correct memmap+double-buffer
-+checkpoint mechanics, but tied to domain-specific state names. Here
-arrays are named generically (a `name -> dtype` dict), with no coupling
+Arrays are named generically (a `name -> dtype` dict), with no coupling
 to any particular domain's variable names.
 
 ```python
@@ -425,7 +421,7 @@ The cost is low — a typical workspace has ~1500 blocks, so the index
 is on the order of 1500 bits, versus the ~371 MB a per-cell mask would
 cost. `read_block_*` for an absent block would return the declared
 `nodata` without touching disk. The point of attention is
-`disk_sync_model.py`, which today writes every block unconditionally
+`disk/sync_model.py`, which today writes every block unconditionally
 in `write_block_core` — the change would need to decide whether a
 fully-nodata block should be written back at all.
 
@@ -438,7 +434,7 @@ evidence that the axis order stored on disk isn't guaranteed.
 Reproduced with real `xarray`, writing exactly the way disscube's
 `VariableWriter` writes (`da.to_dataset(...).to_zarr(...)`): a
 **square** array with `(x, y)` axes instead of `(y, x)` has the
-**same shape** in both cases — the shape check in `zarr_io.py` didn't
+**same shape** in both cases — the shape check in `disk/io/zarr.py` didn't
 catch the inversion. Without a fix, this corrupted row/column
 **silently**, with no error at all.
 

docs/README.md

@@ -125,10 +125,10 @@ from haloexec import (
     DiskChunkedRasterCellularAutomaton,
 )
 
-# DiskChunkedRasterCellularAutomaton é um mixin cooperativo: precisa vir
-# primeiro no MRO, com uma classe real do dissmodel (aqui,
-# RasterCellularAutomaton) como segunda base — do contrário setup()
-# não tem para onde delegar via super() e a classe falha ao instanciar.
+# DiskChunkedRasterCellularAutomaton is a cooperative mixin: it must come
+# first in the MRO, with a real dissmodel class (here,
+# RasterCellularAutomaton) as the second base — otherwise setup() has
+# nowhere to delegate through super() and the class fails to instantiate.
 class LargeScaleGameOfLife(DiskChunkedRasterCellularAutomaton, RasterCellularAutomaton):
     def rule(self, arrays: dict[str, np.ndarray]) -> dict[str, np.ndarray]:
         state = arrays["state"]

docs/api_reference.md

@@ -41,7 +41,7 @@ This document provides a comprehensive, exhaustive reference for all public clas
 
 ## `haloexec.engine`
 
-Primitivas de decomposição de domínio sem dependências externas.
+Domain decomposition primitives with no external dependencies.
 
 ### `Block`
 ```python
@@ -94,7 +94,7 @@ Resolves the external ghost cell fill value for a given array.
 
 ## `haloexec.disk.workspace`
 
-Gerenciador de arrays bidimensionais em disco baseados em `np.memmap` com suporte a double-buffering e checkpoints.
+Manager of two-dimensional on-disk arrays based on `np.memmap`, with double-buffering and checkpoints.
 
 ### `HaloWindow`
 ```python

docs/tutorials_and_recipes.md

@@ -72,9 +72,9 @@ from dissmodel.core import Environment
 from dissmodel.geo.raster.cellular_automaton import RasterCellularAutomaton
 from haloexec import MemmapRasterWorkspace, DiskChunkedRasterCellularAutomaton
 
-# Mixin cooperativo: precisa de RasterCellularAutomaton como segunda
-# base (MRO) — herança simples aqui falha em runtime (setup() não tem
-# para onde delegar via super()).
+# Cooperative mixin: needs RasterCellularAutomaton as the second base
+# (MRO) — single inheritance here fails at runtime (setup() has nowhere
+# to delegate through super()).
 class MassiveGameOfLife(DiskChunkedRasterCellularAutomaton, RasterCellularAutomaton):
     def rule(self, arrays: dict[str, np.ndarray]) -> dict[str, np.ndarray]:
         state = arrays["state"]

examples/gol/gol_patterns_haloexec.py

@@ -1,23 +1,23 @@
 """
-Game of Life com padrões clássicos + haloexec + RasterMap
-============================================================
-Posiciona padrões conhecidos (glider, blinker, beacon, toad, block,
-pulsar) DELIBERADAMENTE sobre as fronteiras de bloco, para que seja
-fácil verificar visualmente se o halo está sincronizando corretamente
-(um padrão que atravessa a borda de um bloco e continua se comportando
-como deveria é a evidência visual mais direta).
-
-Requisitos
-----------
+Game of Life with classic patterns + haloexec + RasterMap
+=========================================================
+Places well-known patterns (glider, blinker, beacon, toad, block,
+pulsar) DELIBERATELY over block boundaries, so it is easy to check
+visually whether the halo synchronizes correctly (a pattern that
+crosses a block edge and keeps behaving as it should is the most direct
+visual evidence).
+
+Requirements
+------------
     pip install dissmodel
-    pip install -e /caminho/para/dissmodel-ca
-    pip install -e /caminho/para/haloexec
+    pip install -e /path/to/dissmodel-ca
+    pip install -e /path/to/haloexec
 
-Uso
----
-    python gol_patterns_haloexec.py
+Usage
+-----
+    python examples/gol/gol_patterns_haloexec.py
 
-Sem display interativo, os PNGs caem em ./raster_map_frames/.
+Without an interactive display, the PNGs go to ./raster_map_frames/.
 """
 from __future__ import annotations
 
@@ -31,7 +31,7 @@
 
 
 # ---------------------------------------------------------------------------
-# Modelo: GameOfLife com halo
+# Model: GameOfLife with a halo
 # ---------------------------------------------------------------------------
 class GameOfLifeHalo(HaloChunkedRasterCellularAutomaton):
     def setup(
@@ -59,7 +59,7 @@ def rule(self, arrays: dict) -> dict:
 
 
 def place(grid: np.ndarray, pattern: list[list[int]], top: int, left: int) -> None:
-    """Escreve um padrão na grade a partir de (top, left)."""
+    """Write a pattern into the grid starting at (top, left)."""
     arr = np.array(pattern)
     h, w = arr.shape
     grid[top:top + h, left:left + w] = arr
@@ -75,19 +75,19 @@ def place(grid: np.ndarray, pattern: list[list[int]], top: int, left: int) -> No
 
 grid = np.zeros((ROWS, COLS), dtype=np.int8)
 
-# Posicionados DE PROPÓSITO sobre fronteiras de bloco (múltiplos de 10)
-# — o pior caso para testar a sincronização do halo.
-# CORRIGIDO: beacon movido de col 25 -> col 33 (na versão original,
-# beacon em (9,25) 4x4 colidia com pulsar em (4,20) 13x13 -- a área
-# do beacon [9:13, 25:29] cai inteira dentro da área do pulsar
-# [4:17, 20:33]. place() sobrescreve por atribuição direta, então os
-# dois padrões ficariam corrompidos silenciosamente, sem erro nenhum.
-place(grid, PATTERNS["glider"], 8, 8)     # atravessa o cruzamento (10,10) na diagonal
-place(grid, PATTERNS["blinker"], 20, 5)   # atravessa a borda horizontal em r=20
-place(grid, PATTERNS["beacon"], 9, 33)    # atravessa a borda horizontal em r=10
-place(grid, PATTERNS["toad"], 29, 15)     # atravessa a borda horizontal em r=30
-place(grid, PATTERNS["block"], 19, 19)    # atravessa o cruzamento de 4 blocos (10,20)+(10,20)
-place(grid, PATTERNS["pulsar"], 4, 20)    # oscilador maior, período 3
+# Placed ON PURPOSE over block boundaries (multiples of 10)
+# — the worst case for testing halo synchronization.
+# FIXED: beacon moved from col 25 -> col 33 (in the original version,
+# the 4x4 beacon at (9,25) collided with the 13x13 pulsar at (4,20) --
+# the beacon's area [9:13, 25:29] falls entirely inside the pulsar's
+# area [4:17, 20:33]. place() overwrites by direct assignment, so both
+# patterns would have been silently corrupted, with no error at all.
+place(grid, PATTERNS["glider"], 8, 8)     # crosses the (10,10) corner diagonally
+place(grid, PATTERNS["blinker"], 20, 5)   # crosses the horizontal edge at r=20
+place(grid, PATTERNS["beacon"], 9, 33)    # crosses the horizontal edge at r=10
+place(grid, PATTERNS["toad"], 29, 15)     # crosses the horizontal edge at r=30
+place(grid, PATTERNS["block"], 19, 19)    # sits on the corner where 4 blocks meet (20,20)
+place(grid, PATTERNS["pulsar"], 4, 20)    # larger oscillator, period 3
 
 backend = raster_grid(rows=ROWS, cols=COLS, attrs={"state": grid})
 
@@ -101,14 +101,14 @@ def place(grid: np.ndarray, pattern: list[list[int]], top: int, left: int) -> No
 )
 
 # ---------------------------------------------------------------------------
-# Visualização
+# Visualization
 # ---------------------------------------------------------------------------
 RasterMap(
     backend=backend,
     band="state",
     color_map={0: "#ffffff", 1: "#2f8f6e"},
-    labels={0: "morta", 1: "viva"},
-    title=f"Padrões clássicos sobre fronteiras de bloco (halo={HALO})",
+    labels={0: "dead", 1: "alive"},
+    title=f"Classic patterns over block boundaries (halo={HALO})",
 )
 
 # ---------------------------------------------------------------------------

examples/gol/scale_30m.py

@@ -0,0 +1,151 @@
+"""
+Scale test: ~30 million pixels, writing the TIFF in windows (never
+materializing the whole grid in RAM), loading it block by block, and
+running dissmodel_ca's real GameOfLife through
+DiskChunkedRasterCellularAutomaton (haloexec) -- the same scale
+challenge a real coastal model faces.
+
+Compares against a monolithic reference AT THE SAME SCALE (30M uint8
+cells fit in RAM as a single array, ~30 MB -- what must not happen is
+materializing while WRITING/LOADING the large file, which is what this
+script actually tests).
+"""
+
+import time
+from pathlib import Path
+
+import numpy as np
+import rasterio
+from dissmodel.core import Environment
+from dissmodel.geo import raster_grid
+from dissmodel_ca.models.game_of_life_raster import GameOfLife
+from rasterio.transform import from_origin
+from rasterio.windows import Window
+
+from haloexec import DiskChunkedRasterCellularAutomaton, MemmapRasterWorkspace, load_geotiff_into_workspace
+
+
+class GameOfLifeHalo(DiskChunkedRasterCellularAutomaton, GameOfLife):
+    pass
+
+
+def memory_mb() -> dict:
+    """RssAnon (real heap) from /proc/self/status -- the metric that
+    proves materialization, not VmRSS/ru_maxrss (which include the page
+    cache of mapped files, always high with memmap without meaning a
+    problem)."""
+    values = {}
+    with open("/proc/self/status") as f:
+        for line in f:
+            for key in ("VmRSS", "RssAnon", "RssFile"):
+                if line.startswith(key + ":"):
+                    values[key] = int(line.split()[1]) / 1024
+    return values
+
+
+def write_tiff_in_windows(path: Path, height: int, width: int, density: float,
+                          seed: int, block: int = 512) -> None:
+    """Write the GeoTIFF block by block through rasterio.windows.Window
+    -- never allocates the whole (height, width) grid in RAM at once.
+    Deterministic RNG per block position (reproducible)."""
+    transform = from_origin(500_000.0, 9_700_000.0, 30.0, 30.0)
+    master_rng = np.random.default_rng(seed)
+
+    with rasterio.open(
+        str(path), "w", driver="GTiff", height=height, width=width,
+        count=1, dtype="uint8", crs="EPSG:31984", transform=transform,
+        tiled=True, blockxsize=block, blockysize=block, compress="lzw",
+    ) as dst:
+        for r0 in range(0, height, block):
+            r1 = min(r0 + block, height)
+            for c0 in range(0, width, block):
+                c1 = min(c0 + block, width)
+                block_seed = int(master_rng.integers(0, 2**31 - 1)) ^ (r0 * 92821 + c0)
+                rng = np.random.default_rng(block_seed & 0xFFFFFFFF)
+                data = (rng.random((r1 - r0, c1 - c0)) < density).astype("uint8")
+                dst.write(data, 1, window=Window(c0, r0, c1 - c0, r1 - r0))
+
+
+def main():
+    # ~30 million pixels
+    HEIGHT, WIDTH = 5480, 5480  # 30,030,400 cells
+    DENSITY = 0.35
+    SEED = 42
+    GENERATIONS = 5
+    BLOCK = 256
+    HALO = 1
+
+    tmp = Path("/tmp/haloexec_scale_30m")
+    tmp.mkdir(exist_ok=True)
+    tif_path = tmp / "initial_state.tif"
+
+    print(f"Grid: {HEIGHT}x{WIDTH} = {HEIGHT*WIDTH:,} cells "
+          f"(~{HEIGHT*WIDTH/1024**2:.1f} MB per uint8 array)")
+
+    # ── 1. write the TIFF in windows ────────────────────────────────
+    rss_before = memory_mb()
+    t0 = time.time()
+    write_tiff_in_windows(tif_path, HEIGHT, WIDTH, DENSITY, SEED, block=BLOCK)
+    t_write = time.time() - t0
+    rss_after_write = memory_mb()
+    print(f"\n[1/3] TIFF written in {t_write:.1f}s -- "
+          f"RssAnon={rss_after_write['RssAnon']:.1f}MB "
+          f"(delta since start: {rss_after_write['RssAnon']-rss_before['RssAnon']:.1f}MB)")
+
+    # ── 2. load block by block into the workspace ───────────────────
+    t0 = time.time()
+    ws = MemmapRasterWorkspace.create(
+        root=tmp / "workspace", shape=(HEIGHT, WIDTH),
+        arrays={"state": np.uint8}, block_h=BLOCK, block_w=BLOCK, halo=HALO,
+    )
+    load_geotiff_into_workspace(ws, tif_path, [("state", "uint8", 0)])
+    t_load = time.time() - t0
+    rss_after_load = memory_mb()
+    print(f"[2/3] Loaded in {t_load:.1f}s -- "
+          f"RssAnon={rss_after_load['RssAnon']:.1f}MB "
+          f"(delta since writing: {rss_after_load['RssAnon']-rss_after_write['RssAnon']:.1f}MB)")
+
+    # ── 3. run GameOfLife on disk+halo, through dissmodel ───────────
+    t0 = time.time()
+    env = Environment(start_time=1, end_time=GENERATIONS)
+    GameOfLifeHalo(workspace=ws, halo=HALO, boundary_value=0)
+    env.run()
+    ws.flush()
+    t_run = time.time() - t0
+    rss_after_run = memory_mb()
+    print(f"[3/3] {GENERATIONS} generations in {t_run:.1f}s "
+          f"({t_run/GENERATIONS*1000:.0f}ms/generation) -- "
+          f"RssAnon={rss_after_run['RssAnon']:.1f}MB "
+          f"(delta since loading: {rss_after_run['RssAnon']-rss_after_load['RssAnon']:.1f}MB)")
+
+    disk_result = ws.snapshot("state")
+
+    print("\n=== memory summary ===")
+    print(f"size of one full array: {HEIGHT*WIDTH/1024**2:.1f} MB")
+    print(f"final RssAnon: {rss_after_run['RssAnon']:.1f} MB "
+          f"(ratio to 1 array: {rss_after_run['RssAnon']/(HEIGHT*WIDTH/1024**2):.2f}x)")
+    print(f"final RssFile: {rss_after_run['RssFile']:.1f} MB (page cache, not materialization)")
+
+    # ── 4. equivalence against a monolithic reference AT THE SAME SCALE ──
+    # 30M uint8 cells fit in RAM as a SINGLE array (~30 MB) -- what must
+    # not happen is materializing while WRITING and LOADING the file,
+    # which was shown above through RssAnon.
+    print("\n[extra] building a monolithic reference at the same scale for the equivalence proof...")
+    with rasterio.open(str(tif_path)) as ds:
+        state0 = ds.read(1)  # here we DO materialize, on purpose, only for the golden reference
+    backend_mono = raster_grid(rows=HEIGHT, cols=WIDTH, attrs={"state": state0.copy()})
+    env_mono = Environment(start_time=1, end_time=GENERATIONS)
+    GameOfLife(backend=backend_mono)
+    t0 = time.time()
+    env_mono.run()
+    t_mono = time.time() - t0
+    golden = backend_mono.arrays["state"].copy()
+
+    n_diff = int(np.sum(golden != disk_result))
+    print(f"monolithic run took {t_mono:.1f}s ({t_mono/GENERATIONS*1000:.0f}ms/generation)")
+    print(f"\ndisk-vs-monolithic differences: {n_diff} of {HEIGHT*WIDTH:,} cells")
+    print(f"IDENTICAL? {n_diff == 0}")
+
+
+if __name__ == "__main__":
+    main()