aa: opt-in trust region + adaptive regularization

[bodono]

Summary

Adds a single new aa_init parameter, trust_factor, that turns on two coupled mechanisms for slow-contraction maps (ADMM/DRS). trust_factor = INFINITY (the new "no cap" sentinel) is the default and keeps the existing behavior — no test changes for callers who don't opt in.

When trust_factor is finite and positive (recommended value: ~10):

1. Trust region — each solve rejects the step if ||D γ||_2 > trust_factor · ||g||_2. This catches the failure mode where γ passes the LS solve and weight-norm cap, but the resulting iterate displacement is much larger than the current residual — i.e. the LS basis is pointing somewhere unhelpful. Each step marginally satisfies the safeguard's monotone test, but the iterate drifts laterally rather than approaching the fixed point.

2. Adaptive regularization — replaces ε·||S||_F·||Y||_F with a self-tuning r:

   • Initial value: 1.0 (so initial γ ≈ 0, i.e. AA ≈ DRS).
   • On each safeguard accept: r *= 0.9 (earn trust, let AA do more).
   • On each rejection (safeguard, trust, or in-solve): r *= 10 (back off toward DRS).
   • Clamped to [1e-12, 1e30].

The two mechanisms feed each other: the trust region detects bad steps, adaptive r damps γ in response, the next solve produces smaller γ and the trust check usually passes. The equilibrium r settles at the value appropriate for the problem.

Same INFINITY-as-sentinel convention now applies to max_weight_norm as well — previously an isfinite() check rejected it; the new validation uses isnan() || ≤ 0, so callers can write INFINITY for both fields to mean "no cap." The 34 existing tests pass through identically because they don't opt in.

Why this exists

Diagnosed on the SCS Maros-Meszaros / Netlib QP/LP benchmark. With the best AA settings at default aa_init parameters, ~30 problems hit max_iters for type-II while type-I solved them. The mechanism:

• Near convergence on slow-contraction maps, ||Y||_F and ||S||_F shrink and r = ε·||S||·||Y|| underflows to ~1e-15.
• γ blows up from the resulting LS solve (~||γ||₂ = 700 observed on PRIMALC1).
• The safeguard's monotone test ||g_new|| ≤ ||g_old|| is too weak to catch this — a wild step can satisfy it by a hair while the iterate drifts laterally.
• The result is "creep": many barely-improving steps that don't approach the fixed point.

The trust region directly catches the "step too big" condition, and adaptive r provides the feedback so AA self-tunes per-problem rather than picking a magic regularization at init time.

Empirical results (SCS benchmark)

On the type-II failure set (33 problems where type-II hits max_iters at default settings, while type-I solves):

Config	Type-I solved	Type-II solved	Regressions on working sets
Master (pre-PR #53)	33/33	0/33	—
PR #53 (sym		S
This PR (trust_factor=10)	30/33	30/33	none on type-I or type-II

Both types now solve essentially all the problems the other can solve, at parity. The 3 still-failing problems (stair, capri, share2b) also hit max_iters with plain DRS — they're DRS-level failures, not AA failures.

Test plan

• make test passes all 34 existing tests
• New validation tests added for trust_factor (NaN/0/negative rejected; INFINITY and positive finite values accepted)
• max_weight_norm=INFINITY now accepted (previously rejected — see flipped assertion in test_nonfinite_scalar_options_rejected)
• SCS benchmark on Maros-Meszaros + Netlib: 30/33 recovery on both types, 0 regressions

Breaking changes

• aa_init signature now takes trust_factor between max_weight_norm and ir_max_steps. Callers in this repo (tests, gd, bench, Python binding) all updated to pass INFINITY (= old behavior). Downstream callers (e.g. SCS) need a one-line update to pass INFINITY for the new parameter, or a finite positive value to opt in.

🤖 Generated with Claude Code