[Non-Record] QAT Dead-Code Analysis + 7 Novel Technique Sweep (1xH100)

[wfproc]

Summary

Non-record research contribution on 1xH100:

• QAT dead-code bug: confirmed torch.compile constant-folds Late QAT in Record: 11L Partial RoPE + LN Scale + EMA + XSA4 (val_bpb: 1.1248) #315-derived code (present in SOTA Record: LeakyReLU² + Legal Score-First TTT + Parallel Muon — val_bpb 1.1194 (3-seed mean) #549). Tested three fix approaches, found a working tensor-scale STE method. Fixing it actually made the int6 gap worse, suggesting WD+EMA may already handle what QAT was supposed to do.
• 7 untried techniques swept: Muon-VS, Deep Delta Learning, Thinking Deeper recurrence, anti-layer removal, NS step reduction, spectral SVD compression, wider models. All negative.
• Key finding: at 83ms/step, each 1ms of overhead costs about 0.007 bpb. Only zero-overhead changes (like activation slope) survive at this step budget.

Also includes working implementations of prune-then-quantize and anti-layer diagnostic as env var toggles.

Full details in the README.