Skip to content

fix(tui): smart auth-profile auto-switch selection + auto-resume interrupted turn#406

Merged
andrei-hasna merged 3 commits into
mainfrom
fix/authswitch-smart-select-restart
Jul 25, 2026
Merged

fix(tui): smart auth-profile auto-switch selection + auto-resume interrupted turn#406
andrei-hasna merged 3 commits into
mainfrom
fix/authswitch-smart-select-restart

Conversation

@andrei-hasna

Copy link
Copy Markdown
Contributor

Problem

On weekly/5h rate-limit exhaustion, codewith's auth-profile auto-switch had two gaps:

  1. Not smart when data was thin / reset info was lost. Cached per-profile usage snapshots dropped the server-reported reset time (display_rate_limit_window hardcoded resets_at: None), so exhausted alternates looked "reset-unknown". When no alternate had available usage the user got a vague "no alternate profile with available usage is known" with no reset info.
  2. Switched but never resumed. On the automatic weekly-exhaustion path, when no usage reset was available the agent switched to another profile but the interrupted turn was silently dropped — the failed in-flight turn lives in usage_self_heal.last_submitted_turn, not the input queue, and the SwitchAuthProfile { resume_queued_input: true } handler only drains the queue. The user had to re-type go.

(The healthiest-profile selection itself already worked when per-profile snapshots were fresh — see existing exhausted_limit_auto_switches_to_profile_with_most_fresh_usage.)

Changes

  • Preserve reset time in cached snapshots. Added resets_at_unix to RateLimitWindowDisplay and carried it through the broker so exhausted profiles report their real retry_at, are ranked/skipped by reset, and can surface the earliest reset.
  • Graceful no-eligible handling. The broker now returns a UsageProfileSwitchOutcome::{Switch, NoEligibleProfile { earliest_reset_at }}; when every alternate is also rate-limited the agent does not switch onto an exhausted profile and shows a clear message including the earliest reset time.
  • Auto-resume the interrupted turn. On a successful fallback auto-switch, the failed turn is re-queued at the front of the input queue (without sending), so the pending SwitchAuthProfile resume re-runs it on the new profile after the profile override is applied — correct ordering, exactly once. Loop/idempotency guards (last_auth_profile_auto_switch_trigger, cooldown keyed on reset) are unchanged.

Tests (Rust unit tests)

  • core: highest_available_picks_best_remaining_and_skips_exhausted, all_exhausted_snapshots_report_earliest_reset_gracefully.
  • tui: all_alternate_profiles_exhausted_reports_earliest_reset_without_switching (graceful message + earliest reset, no switch), automatic_weekly_switch_requeues_failed_turn_for_resume_on_new_profile (switch re-queues the failed turn and resumes it exactly once on the new profile).

Build / verify

Heavy builds are offloaded to the AWS testbox. Crates to build/test: codex-core and codex-tui (cargo test -p codex-core -p codex-tui). cargo fmt --check passes locally.

🤖 Generated with Claude Code

@andrei-hasna
andrei-hasna force-pushed the fix/authswitch-smart-select-restart branch 3 times, most recently from 4f37742 to b444752 Compare July 24, 2026 19:38
…rrupted turn

On rate-limit exhaustion the agent now:
- resumes the interrupted turn on the newly selected profile instead of
  switching silently and dropping the failed request (re-queues the failed
  turn so the SwitchAuthProfile resume drains it on the new profile);
- preserves each cached per-profile snapshot's server reset time so exhausted
  profiles are ranked by their real reset and skipped until they reset;
- when every alternate profile is also rate-limited, reports the earliest
  reset time instead of switching onto an exhausted profile.

Adds unit tests: best-remaining selection skips exhausted profiles;
all-exhausted surfaces the earliest reset; the automatic weekly fallback
re-queues and resumes the failed turn on the new profile.
- Reorder the UsageProfileSwitchOutcome import so `cargo fmt --check`
  passes for codex-tui/codex-core (also picks up incidental fmt on an
  adjacent test assertion).
- Guard the "every alternate profile is also rate-limited" info message
  behind a per-exhaustion-window trigger key (last_no_eligible_auth_profile_trigger),
  mirroring the existing auto-switch trigger dedupe, so it is emitted once
  per window instead of on every rate-limit poll.
- Add a regression test asserting the notice fires exactly once across
  repeated polls of the same exhausted window.
Fold the new auth-profile-switch requeue path into the existing
resume_after_usage_limit_reset body as requeue_failed_turn_at_front, so
both callers use one implementation and only differ in whether they send
the queued turn immediately.
@andrei-hasna
andrei-hasna force-pushed the fix/authswitch-smart-select-restart branch from b444752 to 56d6a17 Compare July 25, 2026 04:51
@andrei-hasna

Copy link
Copy Markdown
Contributor Author

Rebased onto main + Blacksmith testbox gate green

Rebase: rebased fix/authswitch-smart-select-restart from base 00ba6c1f6 onto current origin/main (54333c9e0, includes #395 and #411) and force-pushed (--force-with-lease). New head: 56d6a1733.
The rebase was verified content-neutral: diff(old_base…old_tip) vs diff(new_base…new_tip) are byte-identical apart from one hunk-header line offset in chatwidget.rs. Scope unchanged: 14 files, +385/−20.

Build gate — Blacksmith testbox (blacksmith-16vcpu-ubuntu-2404):

  • run 30144792388conclusion: success (head 56d6a1733, warm_target=false)
  • build_command:
    cd codex-rs && cargo clippy --tests --locked -- -D warnings \
      && cargo nextest run --no-fail-fast -p codex-core -p codex-tui \
         -E 'not binary_id(codex-core::all) and not test(shell_snapshot::tests::try_new)'
    
  • Result: clippy clean under -D warnings; 5704 tests run, 5704 passed, 9 skipped.
  • The codex-core::all integration binary is excluded because it is already red on main in this runner image (run 30121948648 on main: ~40 pre-existing suite::rmcp_client / suite::shell_snapshot / sandbox failures) — unrelated to this PR.

All five new tests pass on the rebased tree:

  • codex-core usage_profile_health::tests::highest_available_picks_best_remaining_and_skips_exhausted
  • codex-core usage_profile_health::tests::all_exhausted_snapshots_report_earliest_reset_gracefully
  • codex-tui chatwidget::tests::status_and_layout::all_alternate_profiles_exhausted_reports_earliest_reset_without_switching
  • codex-tui chatwidget::tests::status_and_layout::no_eligible_profile_message_is_not_repeated_on_every_poll
  • codex-tui chatwidget::tests::usage_limit_reset_tests::automatic::automatic_weekly_switch_requeues_failed_turn_for_resume_on_new_profile

Adversarial self-review notes (no blocking findings):

  • Ordering is safe: send_auth_profile_auto_switch only enqueues the SwitchAuthProfile app event, and requeue_failed_turn_at_front() runs synchronously before the event is handled; submit_auth_profile_switch submits the OverrideTurnContext op before calling maybe_send_next_queued_input, so the re-queued turn always runs on the new profile. last_submitted_turn.take() makes the requeue exactly-once.
  • Preserving resets_at in display_rate_limit_window does not change which profiles are selectable — usage_health_for_snapshots classifies Healthy/Exhausted from used_percent, and resets_at only populates Exhausted { retry_at }. So this is a reporting/ranking enrichment, not a behavioural change to selection.
  • requeue_failed_turn_at_front is a faithful extraction of the existing resume_after_usage_limit_reset body (same pending_retry = None / consecutive_retries = 0 / front-push of message + history record); the only difference is that the send is left to the caller.
  • Every RateLimitWindowDisplay literal in the crate is updated for the new resets_at_unix field (compile-enforced; verified by grep).
  • Minor nit, not fixed: when the auto-switch profile list has zero alternates the new message reads "every alternate profile is also rate-limited or unavailable" — vacuously true, but slightly off. Left alone to keep the diff scoped.

Staged secrets scan of the full diff: 0 findings.

@andrei-hasna
andrei-hasna merged commit b71a05e into main Jul 25, 2026
28 checks passed
@andrei-hasna
andrei-hasna deleted the fix/authswitch-smart-select-restart branch July 25, 2026 05:15
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 25, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant