Add Powell's method as a second fit-refinement method (complement to Simplex)

[wshlavacek]

Summary

PyBNF's refine = 1 post-fit polish currently offers exactly one local method — Nelder–Mead Simplex (_refine_best_fit in pybnf/pybnf.py → SimplexAlgorithm). Add Powell's method as a second, user-selectable refiner that complements (does not replace) Simplex.

Why Powell (and why these constraints)

The refiner's role is local polish from an already-good point found by the global search. PyBNF objectives are typically black-box (simulator outputs, no analytic gradient) and often noisy (SSA / replicate-averaged), which rules out gradient local-optimizers (BFGS/L-BFGS) unless finite-differenced — fragile under simulation noise. Powell's method is derivative-free (conjugate-direction) like Nelder–Mead, fits the same black-box/noisy regime, and frequently converges better on smooth-ish objectives. Both are available via scipy.optimize.minimize(method='Powell'|'Nelder-Mead').

(Levenberg–Marquardt was considered as a third option — highest value for least-squares objectives — but it is gradient-based and objective-restricted to sum-of-squares (chi_sq/sos); meaningless for kl/neg_bin/likelihood objfuncs. Out of scope here; revisit separately if requested.)

Design notes (from the #399 grill, 2026-06-04)

• This is the second refiner, which is the event that clears ADR-0009's ≥2-user bar for a general cross-method-dependency mechanism. M2.1 Stage (c): narrow each fit_type's effective config to its own keys (per-method narrowing, ADR-0006 #1) #399 deliberately keeps the refine→simplex reach as an explicit one-off (ADR-0006 Use scipy.stats functions for defining distributions #5) BUT routes it through a single _REFINER_SCHEMA / _refine_pulls_in(d) seam in config.py so adding a refiner is a localized change, not a rewrite.
• Likely surface: a refine_method = sim | powell config key (default sim, backward-compatible). The selected refiner's config schema is what _build_config conditionally pulls in (generalizing today's hardwired SimplexConfig overlay to "the chosen refiner's schema") — at which point the registry cross-dependency mechanism (ADR-0006 considered-option (b)) may finally be worth building.
• Keep .conf backward compatibility: refine = 1 with no refine_method ⇒ Simplex, byte-identical to today.

Scope / acceptance (rough — to be planned in its own session)

• refine_method config key + grammar/schema; default sim.
• A Powell refiner path in _refine_best_fit (likely a thin scipy.optimize.minimize(method='Powell') driver respecting bounds / parameter scales).
• Generalize the _REFINER_SCHEMA seam from M2.1 Stage (c): narrow each fit_type's effective config to its own keys (per-method narrowing, ADR-0006 #1) #399 to dispatch on refine_method.
• End-to-end test: refine_method = powell on a non-sim fit over an AnalyticalModel target reaches the mode (mirrors the Simplex refine test).
• Docs: refinement section lists both methods + when to prefer each.

Depends on #399 (the refiner seam). Separate session. Any new runtime dep → add to .github/actions/setup-pybnf (scipy is already a dep).

🤖 Filed with Claude Code

[wshlavacek]

Addressed in commit 27aff93 (master). Scope was expanded (per discussion) from "Powell as a second refiner" to two new native, derivative-free optimizers, giving PyBNF three black-box refiners: Simplex, Powell, and CMA-ES.

How it was addressed

Both new optimizers are implemented natively (no scipy/cma dependency) as Algorithm subclasses that plug in through start_run/got_result only — there are still zero run() overrides, so ADR-0007's single shared run loop is preserved. They keep all search state as plain numpy/float/list (no generator, no thread) so Algorithm.backup's pickle.dump works and backup/resume behave like every other method.

• Powell (pybnf/algorithms/optimizers/powell.py): conjugate-direction (Numerical Recipes §10.7) with a parabolic line search; the two ±powell_step probes are evaluated concurrently. Exact on a locally quadratic objective.
• CMA-ES (pybnf/algorithms/optimizers/cmaes.py): (μ/μ_w, λ)-CMA-ES (Hansen 2001). Population-based and generation-synchronized like Differential Evolution, so the whole generation evaluates in parallel; λ = population_size.
• Shared StartPointOptimizer base (local_base.py): start-point resolution + u↔PSet conversion in sampling space (log parameters adapt geometrically). Simplex keeps its byte-identical start-point parsing.

Acceptance checklist

• refine_method config key + grammar/schema; default sim (backward-compatible).
• Powell (and CMA-ES) refiner path in _refine_best_fit — registry-driven dispatch.
• Generalized the _REFINER_SCHEMA seam to dispatch on refine_method: it is now Configuration._refiner_schema(d), a registry-keyed lookup off a register_fit_type(..., refiner=True) flag. Adding a future refiner needs no config.py/pybnf.py edit.
• End-to-end tests: parametrized slow test runs refine_method = powell|cmaes over a non-self (de) fit on an AnalyticalModel target, reaching the mode and never worsening the best; plus fast standalone mode-recovery tests and a pickle round-trip test.
• Docs: a Refinement section listing all three methods with when-to-prefer guidance, plus Powell and CMA-ES algorithm sections (docs/algorithms.rst) and key reference (docs/config_keys.rst).

Design note (the cross-dependency mechanism this issue flagged)

We deliberately did not build the general registry cross-dependency framework. The three refiners are a single dependency edge (refine → the chosen refiner's whole schema) parameterized by refine_method, not N distinct cross-dependencies — so the right-sized generalization is the constant→lookup the #399 seam was built for. A genuinely different cross-fit_type reach is what would justify the framework. Rationale recorded in ADR-0015.

Verification

ruff clean; fast suite (1259 + 45 benchmark + 11 optimizer-integration) green; slow tier (7, incl. the new refine end-to-end nets) green; pre-push bngsim suite passed. No new runtime dependency (numpy only), so no CI/setup-pybnf change. Backward compatibility verified via the golden-config net: refine = 1 with no refine_method still runs Simplex, byte-identical apart from the new refine_method = sim default key.

Depended on #399 (the refiner seam), which is complete.