Skip to content

fix(executor): upstream_empty 不冷却账号 / do not cool accounts down for upstream_empty - #288

Merged
caigee-cmd merged 1 commit into
caigee-cmd:mainfrom
BrianBoyCN:fix/upstream-empty-no-cooldown
Oct 9, 2026
Merged

caigee-cmd merged 1 commit into
caigee-cmd:mainfrom
BrianBoyCN:fix/upstream-empty-no-cooldown

Conversation

@BrianBoyCN

Copy link
Copy Markdown
Contributor

中文

问题现象

#286 合入后的实测中发现:当上游空流率高企时(一次实验中 6 发有 4 发空流,约 40-60%),qoder 账号出现频繁冷却,账号池被饿死,请求在长时间排队后才成功。

根因

upstream_empty(#286 引入的转发前守卫)目前按普通 unavailable 错误分类,走 Classify 的 default 分支:

  • 默认 15 秒冷却(ParseRetryAfterHint(body, retryAfter, 15*time.Second))
  • Pool.MarkClassified 对同类错误的重复失败还会指数退避(15s → 更长)

但空流只说明 Qoder 网关本次没有产出内容,账号本身完全健康——凭据、配额、限流状态都正常,同一账号下一次请求就可能成功。把"上游瞬时故障"按"账号故障"冷却,在空流率高时会把整个账号池冻结。

修复

internal/executor/error.go 的 ClassifyError 中对 Code == "upstream_empty" 特例处理:

其他 unavailable 错误(连接被重置等真实账号层故障)保持原有 15 秒冷却不变。Pool.MarkClassified 中 Cooldown=0 天然不设置 DownUntil,无需改池逻辑。

测试

新增 TestClassifyErrorUpstreamEmptyNoCooldown:

  • upstream_empty → Failover=true、Cooldown=0、RetryAfter=0
  • 对照组:普通 unavailable 错误保留默认冷却(确认只有特例豁免)

go test ./...、go test -race、go vet ./...、gofmt 全部通过。


English

Problem

Live testing after #286: when the upstream empty-stream rate spiked (4/6 requests in one sample, ~40-60%), qoder accounts were cooled down repeatedly and the pool starved — requests queued behind long cooldowns even though the accounts themselves were healthy.

Root cause

upstream_empty (the pre-relay guard from #286) is classified as a generic unavailable error, which applies the default 15s cooldown plus exponential backoff on repeat failures. But a heartbeat-only upstream stream says nothing about the account's health: credentials, quota, and rate-limit state are all fine, and the same account may succeed on the very next request. Cooling accounts for a transient upstream fault freezes the whole pool when the empty-stream rate is high.

Fix

Special-case Code == "upstream_empty" in ClassifyError: keep Failover = true (another account serves the turn) but set Cooldown = 0 so the account stays in the rotation. Generic unavailable errors keep their 15s cooldown. Pool.MarkClassified already treats Cooldown=0 as no DownUntil, so no pool changes needed.

Tests

New TestClassifyErrorUpstreamEmptyNoCooldown covers both the exemption and a control case (generic unavailable keeps its cooldown). go test ./..., -race, go vet, gofmt all pass.

A heartbeat-only upstream stream (upstream_empty, introduced as a
pre-relay guard in caigee-cmd#286) says nothing about the account's health:
credential, quota, and rate-limit state are all fine, and the same
account may succeed on the very next request. Classifying it as a
generic unavailable failure applies the default 15s cooldown with
exponential backoff on repeats, so whenever the upstream empty-stream
rate spikes the pool starves: every account ends up cooled down and
requests queue behind unrelated timeouts. Observed live while the
upstream empty-stream rate was around 40-60%.

Special-case Code=upstream_empty in ClassifyError: keep Failover=true
(another account serves the turn) but set Cooldown=0 so the account
stays in the rotation. Generic unavailable errors keep their cooldown.
@caigee-cmd
caigee-cmd merged commit cdc01ab into caigee-cmd:main Oct 9, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants