feat(routing): soonest-reset credential selector (MASTRA-765)

[ericlitman]

Replaces #2, which GitHub closed when its base branch (claude/MASTRA-768-e2e-hermetic) was deleted at the merge of #1. The CI run on the tests-only commit is recorded on #2: S1, S2, S3 and S5 fail on stock as intended (run 37140121133).

Linear: MASTRA-765 (tests from MASTRA-768, U4)

Adds routing.strategy: soonest-reset. Each new session goes to the credential whose weekly quota window resets soonest, among credentials that still have quota. Existing sessions follow upstream session affinity: they move only on cooldown or binding expiry. For one shared weekly limit, this order wastes the least quota. With per-model weekly limits it is a heuristic, and those limits stay out of scope here.

The hermetic e2e suite it extends merged in #1.

Commits

1. 06eff342 tests only: hermetic selector scenarios S1, S2, S3 and S5, which must fail on stock, plus S7 and S8 as regression tests. The mock gains POST /_mock/script, which sets one credential's quota headers or 429s. Each scenario owns its own credentials. The hermetic config turns on soonest-reset, session affinity and save-cooldown-status.
2. 143cc73c, 6c0fccaa, 7f35e809: zsprackett/CLIProxyAPI faa4ccff (soonest-reset strategy), 01ad0021 (persist observed quota with the cooldown state) and 67f5749d (Claude thread_not_found as a request fault). They are cherry-picked with -x, and Zac Sprackett stays the author. conductor_cooldown.go merged cleanly: upstream's v8.0.10..v8.0.13 change to it only adds unauthorized-credential handling. 4d7b2e84 (model-scoped weekly) was not taken, because it is per-model scope only.
3. ee450e85: our selector rework.
   • Windows by length. A Codex window is weekly when its length is at least 10080 minutes, whichever position carries it, because Codex Pro reports its weekly window as primary. Claude windows are 5h and 7d.
   • Exhaustion gate. Skip a credential while an observed window is exhausted and its reset is ahead. If every candidate is gated, the ordering alone decides.
   • Unknown resets. An unknown Claude reset sorts first, and a passed one rolls forward by whole weeks. An unknown or passed Codex reset sorts last.
   • Deletions. The scheduler fast-path additions and the isBuiltInSelector entry are deleted. With session affinity on, picks always take the legacy path.
4. fce6a86b: Go unit tests for S1, S2, S3, S5 and S10 (a real restart through the file store); a pick through session affinity; and a race test.
5. Review fixes from Astra (gpt-6-astra, high), each a subtraction:
   • d595299a: save every observation, not once a minute.
   • 1d90a949: keep a Codex window with no Window-Minutes.
   • 1e43486d: no age cutoff on saved observations.
   • a714c907: the save-cooldown-status comment in config.example.yaml no longer mentions the removed throttle.

Size against base 1aa79f5: non-test Go +254/−7 (selector 152, persistence 84); Go tests +620; e2e +215/−12.

Acceptance criteria from MASTRA-765

• "Go unit tests for S1, S2, S3, S5 and S10 … pass with go test -race."
  • Locally: go test -race -count=2 ./sdk/cliproxy/auth/ passes on 1e43486 (a714c90 changes only a YAML comment).
  • Race evidence covers this package only. Three upstream tests in sdk/cliproxy fail intermittently under -race -count=6 on stock v8.0.13 as well as on this branch: TestHandleAuthUpdates_SameRevisionWaitDoesNotWaitForOtherAuthInBatch and TestEndToEndAuthFile{Replacement,Patch}_RestoresModelsInV1ModelsWithoutRestart.
  • go test ./... passes, and CI runs it.
• "The MASTRA-768 hermetic selector tests fail on the tests-only commit and pass at the PR head."
  • On stock (06eff34): S1, S2, S3 and S5 fail with ASSERTION_FAILED. For example, S1 expected e2e-s1-b and got e2e-s1-a. S7, S8 and the 9 protocol tests pass.
  • At the head (a714c90, clean tree): 15 of 15 pass.
  • The CI run on 06eff34 for this PR is the recorded failing run.
• "A bound session follows upstream affinity; new sessions follow the ranking." Hermetic S7 and the Go affinity test both cover it.
• "No token refresh, token-endpoint call or credential-file write." None was added. The persistence writes cooldown-state files, not credentials.
• "The added non-test line count is in the PR." It is above.

Review

• Plan: reviewed by Astra at xhigh. The disposition is on MASTRA-765.
• Code: reviewed by Astra at high, which returned four findings. Three were fixed above.
• Rejected: removing thread_not_found as a request fault. Upstream already classifies OpenAI's equivalent, previous_response_not_found, the same way. Without it, a 404 cools down a healthy account and pulls it out of the drain order.
• Mine: I read every diff and re-ran the suites myself. Removing the picked == nil guard is safe, because both candidate paths return an error instead of an empty list.

Known limitation

A Codex window started outside this proxy stays invisible until the proxy sends that account a request. MASTRA-766 measures how often that happens before anyone adds usage polling.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith with what you need. Autofix is disabled.}

[linear-code]

MASTRA-765 Finish the soonest-weekly-reset credential selector on CLIProxyAPI v8.0.13 (from zsprackett's fork)

Context

Eric chose option 1 on MASTRA-763 (2026-10-03): drain subscription windows inside CLIProxyAPI with a soonest-weekly-reset selector, starting from zsprackett/CLIProxyAPI. Upstream will not take quota-aware routing (router-for-me/CLIProxyAPI#4715). We carry the patch in a private repo, mobilyze-llc/CLIProxyAPI, made by pushing upstream history and tags into a new private repository. GitHub allows no private fork of a public repo.

Policy: send each new session to the account whose weekly window resets soonest, among accounts that still have quota. For one shared weekly limit, that order never wastes more quota than any other. With per-model weekly limits it is a heuristic, not an optimum.

The plan was reviewed by Astra (gpt-6-astra, xhigh) on 2026-10-03, and I verified every finding I accepted against the source. This body reflects that review.

Starting point: zsprackett commits from 2026-10-02, MIT:

• faa4ccff: soonest-reset strategy (sdk/cliproxy/auth/selector_soonest_reset.go, 118 lines).
• 4d7b2e84: model-scoped weekly windows.
• 01ad0021: persist observed quota with cooldown state.
• 67f5749d: Claude thread_not_found as a request fault.

Upstream selector.go, service_config.go and config_basic.go are unchanged from v8.0.10 to v8.0.13. Only conductor_cooldown.go changed (+55/-20).

Scope

1. U1. Create the private repo at tag v8.0.13 (d7914af), with an upstream remote for later rebases. CI runs go build ./... and go test ./... and must be green on unmodified upstream.
2. U5, in one PR. The first commit carries the failing selector tests from MASTRA-768 (U4); its CI run is the evidence that they fail on stock. Later commits:
   1. Cherry-pick the four commits with authorship, and hand-merge conductor_cooldown.go.
   2. Windows by length. Codex: weekly when window-minutes >= 10080. Claude: 5h and 7d.
   3. Exhaustion gate. Skip a credential while an observed window is exhausted (Codex used-percent >= 100 or limit reached; Claude utilization >= 1 or status rejected) and that window's reset is in the future. If every credential is gated, ignore the gate and keep the ordering: upstream 429 handling and cooldown decide.
   4. Unknown resets.
      • Claude: an unknown weekly reset sorts first. Its weekly window runs on a fixed schedule, so one real request learns it and the ordering corrects itself. Roll a passed Claude reset forward by whole weeks.
      • Codex: an unknown or passed reset sorts last. That window is usually unstarted, and starting it early can waste it.
   5. Deletions. Remove zsprackett's unreachable scheduler.go additions and the isBuiltInSelector entry. Add a test proving routing reaches the selector through session affinity.
   6. Read quota signals in a way go test -race accepts.

Not in scope

• Per-model weekly limits (Claude seven_day_opus and similar). No observed header establishes them. They stay with upstream's per-model cooldown after a 429.
• Active usage polling. It stays out until measured.

Known limitation. A Codex window started outside the proxy is invisible until the proxy sends that account a request. Outside starts come from Eric's interactive CLIs and the factory's native routes, which use the same accounts. MASTRA-766 measures how often selection sees an unknown reset before anything is added.

Acceptance criteria

• Go unit tests for S1 (order), S2 (soonest small remainder before fresh-but-late), S3 (5-hour gate), S5 (unknown handling per provider) and S10 (persisted observations). Each passes with go test -race ./sdk/cliproxy/..., and upstream's tests pass too.
• The MASTRA-768 hermetic selector tests fail on the tests-only commit and pass at the PR head.
• Stickiness. With routing.strategy: soonest-reset and session-affinity: true, a bound session follows upstream affinity: it moves only on cooldown or binding expiry. New sessions follow the ranking.
• No token handling added. No token refresh, token-endpoint call or credential-file write.
• Size recorded. The added non-test line count is in the PR.
• Contribution back. Once the tests above pass, open one PR against zsprackett/CLIProxyAPI's main carrying only the selector changes and their Go tests: no e2e suite, no CI or MOBILYZE.md changes, nothing about studio2. Eric approved this on 2026-10-03. Do not wait on a merge.

Source refs

• Parent: MASTRA-763 (findings, audit and decision comments).
• Blocks MASTRA-766. Related: MASTRA-768 (tests), MASTRA-767 (factory routes).
• Not in mastra-pilot: CODING_STANDARDS.md:21.

Review in Linear

[ericlitman]

Review gate: pstack:interrogate, run because Unfret is not installed here. Reviewers: Opus at max and Sol at max. Grok was a dropout. The verdict is recorded on MASTRA-765. Fix-round commits:

• 91d1e912: keep quota observations across a credential replacement. An OAuth token refresh reloads the auth file through the watcher, and Manager.Update dropped Quota.Signals. The Go test fails without the fix, as does the service-path test.
• 83abd29f: delete S3's wall-clock reset transition. Go's "claude 5h reset passed" case covers it.
• 2db0cd3b: the hermetic Claude Code test now sends the four Claude Code detection signals and asserts the passthrough path, with no cloaking system injected. That assertion fails without the signals.
• 83a9afb9: MOBILYZE.md lists only the taken commits, and config.example.yaml pairs soonest-reset with session affinity.

Correction to commit d595299's message: upstream does not rewrite the auth file on every result. FileTokenStore.Save skips unchanged JSON (sdk/auth/filestore.go:131). Saving every observation rewrites the .cds files on each observed response, about 1.2 ms for 6 credentials. That cost is accepted at our volume.