Flattening CLUBB's grid: allocations by any name, bounds by component, generics by arity

[chenyueqi]

Stacked on #25. What the flat plan, the gates and the emitter needed to take type(grid) :: gr and its neighbours through, found on the CLUBB extension's tier-1 units.

Flattening

• Allocations: allocate( gr%zm(ngrdcol,gr%nzm), gr%zt(...), … ) — one statement, many components, the object named by the setup routine's dummy rather than this.
• Bounds: an axis sized by a component of the same or another object (gr%nzm) is spelled by that component's flat name and the component is carried as an input; a dummy declared thvm(ngrdcol, gr%nzt) the same; an allocation sized by the allocating routine's dummy (1:nz) by the planned subprogram's dummy of that name; a bound from one is its upper bound alone.
• ngrdcol is declared once when the subprogram already takes it.
• Generics: a call through a generic is followed into the specifics its arity fits; the recursion guard is a stack, not a memo.
• Private module state (error_code % clubb_debug_level) is left to the module, under left_to_module.
• An intent(out) object's written components are read off the object the translation returned.

Read/write scope: a sibling's generics are procedures to it; a sibling's buffer-OUT actuals are read; a stub under control flow (if (stats%l_sample) then / call stats_update / end if) is a stub block.

Two silent translation defects, both fixed with executed tests

• A DO loop's index read after the loop: Fortran leaves it one step past the end on completion, for leaves the last value. lscale_width_vert_avg integrated one level short. A loop whose index a later statement reads before anything redefines it gets a for … else with the completion value; an EXIT (break) skips it, as Fortran keeps the exit value.
• ep = Rd / Rv over two real parameters was folded with integer division (the old "no real literal means integer arithmetic" rule) and came out zero; every unit reading ep1/ep2 computed NaN. The resolver records each constant's declared base type and the fold types names by it.

Bit parity: SUM accumulates in element order (_f_vsum beside _f_vdot); np.sum's pairwise rounding drifted 12 ULP in vertical_integral.

Gate: a subprogram the operator declared ungated is not sampled; a translated bare error stop (SystemExit) is a failed comparison, not the end of the run.

Tier-2 replay (recorded runs through dump-replay), found on CLUBB's advance_* solvers replayed on bomex recordings:

• The dump parser took only numbers: a logical T/F header (l_implemented) was no value and a zero-extent component (scalar tracers under sclr_dim = 0) vanished with its empty body. Both are values.
• The recorder wrote what the objects received; the plain OUT/INOUT dummies (wp2, wp3, mono_flux_limiter's low_lev_effect) are outputs too.
• Two specifics of one arity (tridiag_solve with its optional rcond and the multiple-rhs one without) were picked by count; the ranks of the bare-name actuals decide.
• A component name on an OUT actual (pdf_params%chi_1) was counted a read of a variable of that name — the documented disagreement inherited from the pipeline, resolved the tidier way now that a translation showed it (test flipped, deliberately).
• A call to a procedure of a stubbed module that no statement stub answers (lapack_band_solvex) deferred its whole block, if ( method == lapack ) included, so the candidate raised on the LU path the run takes. It raises on its own line.
• max(2, edsclr_dim) sizing a local is a bound Python can spell; a comma in a bound is legal inside max/min alone.
• An optional OUT handed on (rcond = rcond) carries the caller's presence (want_rcond=want_rcond, not True) — xm_wpxp_solve took the LAPACK diagnostic path on every call — and the read/write gate counts that hand-on as a read of the optional on both sides.
• A module-state object handed whole to a companion function (sponge_layer_damping's profile into sponge_damp_xm) is followed under the callee's dummy, so the plan carries the components the callee reads; a derived-type state variable is no longer listed as "left to the module".
• A local parameter initialized by a scalar times an array constructor is refused at emission #26: 100._core_rknd * (/ ... /) as a local parameter — each constructor rendered where it stands, the rest through the token pass, elementwise (saturation; pdf_closure's hybrid-PDF path).
• A subroutine with no OUT argument returns zero values, not one (finalize_tau_sponge_damp_api); a declared-ungated subprogram is not compared on a recording either, and the reason is reported.

Merged with main (2026-09-04, after PRs #29 and #30 landed there): main's comparison loop is kept as is — a translated ERROR STOP or an IndexError is a draw redrawn — with this branch's recorded-sample path, zero-value return and declared-ungated skip beside it; constant folding carries main's parameter aliases together with this branch's intrinsic calls; the appended tests of both sides are kept. Merged bottom-up through #22 and #25 so each PR stays mergeable.

Held to (self-check 2026-09-04 before tier 3): full suite and mypy green; conformance/ --plugin-set recast-clubb green; the corpus re-run on this branch regresses no unit (numfor's genrand 7→3 disagreeing blocks, slsqp 12→8, csplines/fitpack/slsqp_core as before); tools/emit_diff.py against the translator goes from 34 differing subprograms at this branch's base to 48, every added row an intended class — _f_vsum for SUM, want_x=want_x where an optional OUT is handed on (wvsat's qsat*: the translator's True is the defect fixed here), declared-initializer locals, and define_swept_areas where the translator drops the reshape around a constructor and this branch keeps it (#26). CLUBB tier 0 stays 12/14; tier 1 is 7/11 bit-exact with gr flattened (advance_helper_module: 25 subprograms, 7,770 points); tier 2 (summaries/tier2.json in clubb-jax): 10 of 12 units bit-exact on 20–40 recorded calls each from bomex and cgils_s6 under the hybrid PDF — advance_wp2_wp3, advance_xm_wpxp, advance_xp2_xpyp, advance_xp3, advance_windm_edsclrm, clip_explicit, mono_flux_limiter, mixing_length (incl. diagnose_lscale_from_tau), pdf_closure, new_hybrid_pdf_main — plus the sampled plain routines of clip_explicit, mixing_length and pdf_closure. sponge_layer_damping compares nothing (initializer's below-layer levels are undefined on both sides, declared; finalizer has no output; the sponge_damp_* functions take gr and have no flat plan); new_pdf_main has no recording (the new PDF needs LES input fields).

Tier 3 (2026-09-04): advance_clubb_core — the whole CLUBB time step, 135 dummies flattened to 476 arguments over 7 derived dummies and the 14 sponge state objects — is bit-exact on 40 recorded steps (20 bomex, 20 cgils_s6 under the hybrid PDF): 453,700 compared values, no rewrites to notarize. Two more fixes got it there: a probe spans blank and comment lines inside a continued call (cpp leaves them where an #ifdef stood), and a generic function reference is ranked by the specific it dispatches to (a nested sat_mixrat_liq_api( ..., thlm2T_in_K_api( ... ) ) deferred a block, and a deferred block took the subprogram out of the gate).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Cne4hrGgYqhTMzVgzJtXYd

2026-09-05 (late): optional INOUT/OUT dummies; whole step in JAX

• a282641 / 74e3e59: a flat kernel takes optional INOUT dummies (value, default None) and optional OUT dummies (their want_<name> sentinel) as parameters and returns them after the plan's outputs; the adapter's wrapper drops them (optional_returns on the flat record); the discard name is never a carry. Found on the whole step (advance_xm_wpxp's clipping counter handed to xm_wpxp_clipping_and_stats by keyword never moved); the recorded advance_xm_wpxp JAX port had regressed to 160/200 integer mismatches since c4e8c82 and is bit-exact again.
• Self-check at 5876e1e before that: suite, mypy, ruff, conformance green; corpus one improvement (897 blocks, 56 deferred), no regressions; tiers 0-2 as recorded.
• Whole step (advance_clubb_core, 40 bomex+cgils steps, 453,700 points) in JAX: within the 32-ULP gate once XLA's algebraic simplifier is off (--xla_disable_hlo_passes=algsimp, set by the case runner beside optimization level 0). algsimp's rewrites of double arithmetic leave a jitted kernel 1-2 ULP from its eager evaluation, and the step turns 2 ULP in the mixing length's tau into a different vpwp through xm_wpxp's near-singular penta solve (measured by nudging tau in the exact eager step). With the flag the ten recorded units are 8 bit-exact, 2 within the gate; the whole step has 453,524 points bit-exact, max 53 ULP, max_rel 3.2e-15.

2026-09-05 (night): a local derived object is its components

• 134aa0e: the anchor's tmp = _make_T() declares a local object whose components are the locals tmp__<comp> (None until a callee creates them); an OUT object the anchor takes back whole binds through the assignment's target; an unallocated component handed to the callee that allocates it goes through _f_buffer as zeros of the callee's declared extents in the caller's terms. Found on CLUBB's e3sm_maint32 configuration (l_call_pdf_closure_twice makes pdf_closure_driver_zm live, which builds its own implicit_coefs_terms); with it the whole step lowers fully under that configuration too.
• Two more cases (arm, dycoms2_rf02_nd) and bomex under CLUBB's default flags needed no engine change (NumPy bit-exact, JAX within the gate). Under e3sm_maint32 the JAX step is within 2.5e-15 relative but fails the 32-ULP dominant gate at 128 ULP from the step cloud forms: XLA's erf/exp against libm's in the cloud-fraction component, amplified by the trapezoidal rule; nudging libm's erf/exp by 1 ULP in the exact NumPy step moves rcm by 5.7e6 ULP, so this is conditioning, recorded in the case repo.

2026-09-06: the tolerance gate measures conditioning

• b8dd6e9: when the dominant tier fails with the relative tier inside its gate on a recording, the candidate's own bit-exact anchor is rerun on the worst recorded samples with every transcendental's result moved one ULP (up, down, seeded coin; recast.verify.conditioning) and compared to the recording by the same dominance rule. Inside that spread: TOLERANCED with the evidence on the verdict (metrics["conditioning"]); past it: FAILED with the same evidence; conditioning: off disables it. On CLUBB's e3sm_maint32 configuration the whole step's 128 ULP sits against a 17,320,256 ULP spread of the anchor.