Repository navigation
feat(routing): keep quota-exhausted accounts on free-model routes - #298
Conversation
Qoder prices some catalog models at zero — they carry `limited_time_free` or a zero `price_factor`, and the console already renders them with a free badge. A request for such a model spends no credits, so an account whose paid quota is exhausted can still serve it. `routeBaseMatches` filtered every quota-exceeded account out unconditionally, so those accounts lost every route, including the free ones. That is why a Qoder account with `total: 0` sat at `quota_exhausted` while the free models it can still serve — qwen3.8-flash among them, which records 0 credits across every logged request — had no account at all. Carry the free-model list alongside the cached catalog: `CatalogIDsFromInfos` already flattens `ModelInfo` into ids and dropped the `Free` flag on the way, so add `FreeCatalogIDsFromInfos` and record it on the pool item next to `Models`. The exemption is scoped to the requested model, so the same account stays filtered out of paid routes. Models the catalog has not priced are not assumed free — unknown pricing keeps the account out, rather than gambling a request on it. `RemoveModel` now prunes the free list too, so a model the account stopped serving cannot keep exempting it.
Qoder's model catalog publishes a friendly public ID alongside a distinct gateway key. The worker resolved the public ID through the per-account catalog before sending a chat, but the in-process COSY client filled `model_config.key` with whatever the caller asked for. On an account whose gateway routes only the mapped key, that request was never answered: the gateway held it open and the client waited out its own timeout with no error and no bytes in return. The pool records the public-ID-to-gateway-key mapping from the catalog, and the chat request carries the mapped key. An account whose catalog reports no distinct key keeps its public spelling.
|
补充一个 commit — 中文修的是什么Qoder 的模型目录给出一个易读的公网 ID 和一个不同的网关键。Worker 发送聊天前会用账号目录把公网 ID 解析一遍；但进程内 COSY 客户端此前直接把调用方传入的名字填进 路由池现在记录「公网 ID → 网关键」的映射（ 怎么定位的两条链路各加一处临时打印，输出即将发出的 HTTP 请求（URL、全部 header、解码后的 body；密钥类 header 掩码成长度+md5）。同一账号上对比两条链路，33 个 body 字段里只有一个不同——
验证
关于两个 commit它们是同一条链路的前后两步，不是两个独立修复：前一个 commit 让请求能轮到额度耗尽的账号，后一个让这个账号发出去的请求能被网关认。只合其中一个，这类账号上的免费模型仍然不通——所以两个 commit 放在这个 PR 里，而不是另开一个。 diff 中不含 token、凭据、抓包内容、账号标识或主机信息。 EnglishWhat it fixesQoder's catalog publishes a friendly public ID alongside a distinct gateway key. The worker resolves the public ID through the per-account catalog before sending a chat; the in-process COSY client filled The pool now records the public-ID-to-gateway-key mapping ( How it was foundBoth chains were instrumented to print the exact request about to go out (URL, headers, decoded body; secret headers masked to length+md5). Across the two chains on the same account, one field of 33 differed —
Verification
On the two commitsThey are consecutive steps of one path rather than two independent fixes. The first commit lets a request reach a quota-exhausted account; the second makes that account's request recognizable to the gateway. Merging only one of them leaves free models unusable on such an account — which is why both live here rather than in a separate PR. No tokens, credentials, captured traffic, account identifiers, or host details in the diff. |
caigee-cmd
left a comment
caigee-cmd
left a comment
There was a problem hiding this comment.
审查结论：Approved，有 1 个必修项 + 1 个流程建议
整体质量很高。free-model 豁免的设计我特别认可两点：
- 豁免按请求模型限定（
itemServesFreeModel(item, q.PublicModel)），且routeMatches仍要求itemHasModel通过，所以耗尽账号不会被放进付费路由。测试TestPickRouteExemptionDoesNotLeakToPaidModels也覆盖了这一点。 - 未报价不假定免费。
Free只在limited_time_free或price_factor<=0时为真，且routeModel("auto") == ""，所以auto请求永远不会被豁免 —— 定价未知时保持过滤，这是正确的 fail-closed 方向。
MergeModelsWithPricing 的 canonicalize + dedup、RemoveModel 对 FreeModels 的同步清理都做对了。
必修：Upsert 没有 carry-forward FreeModels
internal/executor/pool.go
Upsert 在 item.Models == nil 时把 Models / ModelsAt 从前一个条目带过来，但 FreeModels 没有对应处理。运行时的三条 Upsert 调用（manager_process.go 的进程注册/刷新/重启）都不带 pricing，所以每次运行时 Upsert 都会把免费名单清空，账号随即从免费路由上掉下去，直到下一次 catalog 刷新才恢复。
这段时间里该账号的免费模型（不花积分、上游会正常应答的请求）无账号可用 —— 是白丢的可用路由，且依赖刷新时机自愈。
建议在 Models 的 carry-forward 旁边补上（我已本地验证，go test ./... / go vet / gofmt 全过）：
if item.Models == nil {
item.Models = p.items[i].Models
item.ModelsAt = p.items[i].ModelsAt
}
// FreeModels is pricing metadata, not request state. A runtime
// Upsert (process registration, refresh) carries no pricing, so
// without this carry-forward the free list is wiped and the
// account drops off free-model routes until the next catalog
// refresh repopulates it.
if item.FreeModels == nil {
item.FreeModels = p.items[i].FreeModels
}
if item.ProvenModels == nil {
item.ProvenModels = p.items[i].ProvenModels
}
配一个回归测试（internal/executor/free_model_quota_test.go），锁住 carry-forward 本身和"显式传入时仍可覆盖"两条：
// A runtime Upsert — process registration or refresh — carries no pricing, so
// FreeModels must survive it the same way Models does. Losing the free list
// would drop an exhausted account off its free-model routes until the next
// catalog refresh, a route that works and costs no credits.
func TestUpsertKeepsFreeModelsWhenPricingUnsupplied(t *testing.T) {
pool := NewPool(nil, nil)
pool.Upsert(Item{
ID: "acc",
Provider: "qoder",
Region: "global",
Models: []string{"qwen3.8-flash", "ultimate"},
FreeModels: []string{"qwen3.8-flash"},
Quota: &QuotaSnapshot{Exceeded: true, Total: 0},
})
// The runtime path re-registers the account without catalog or pricing.
pool.Upsert(Item{ID: "acc", Provider: "qoder", Region: "global"})
item, ok := pool.ByID("acc")
if !ok {
t.Fatal("account missing from pool")
}
if len(item.FreeModels) != 1 || CanonicalModelID(item.FreeModels[0]) != CanonicalModelID("qwen3.8-flash") {
t.Fatalf("FreeModels = %v after Upsert, want [qwen3.8-flash]", item.FreeModels)
}
if len(item.Models) != 2 {
t.Fatalf("Models = %v after Upsert, want the two cached entries", item.Models)
}
if _, ok := pool.PickRoute(RouteQuery{PublicModel: "qwen3.8-flash"}); !ok {
t.Fatal("free model: PickRoute returned no account after Upsert, want the exhausted one")
}
if _, ok := pool.PickRoute(RouteQuery{PublicModel: "ultimate"}); ok {
t.Fatal("paid model: PickRoute returned the exhausted account, want none")
}
}
// An explicit free list still overrides the carried-forward one, so a catalog
// refresh that reprices a model to zero clears the stale entry.
func TestUpsertReplacesFreeModelsWhenSupplied(t *testing.T) {
pool := NewPool(nil, nil)
pool.Upsert(Item{
ID: "acc",
Provider: "qoder",
Region: "global",
Models: []string{"was-free", "now-free"},
FreeModels: []string{"was-free"},
})
pool.Upsert(Item{
ID: "acc",
Provider: "qoder",
Region: "global",
Models: []string{"was-free", "now-free"},
FreeModels: []string{"now-free"},
})
item, ok := pool.ByID("acc")
if !ok {
t.Fatal("account missing from pool")
}
if len(item.FreeModels) != 1 || CanonicalModelID(item.FreeModels[0]) != CanonicalModelID("now-free") {
t.Fatalf("FreeModels = %v, want [now-free]", item.FreeModels)
}
}流程：这个 PR 打包了两个不相关的功能
PR 标题是 free-model 豁免，但 diff 里还包含一整套 Qoder native model 映射：
Item.NativeModels/Pool.MergeNativeModelsqoder.NativeCatalogIDsFromInfos（读 catalog 的mapped_key）internal/executor/native_model_test.go、internal/providers/qoder/native_model_test.go- changelog fragment
qoder-native-model-mapping.md
这是两个独立的修复，建议拆成两个 PR。我看了 native mapping 那部分，它本身是自洽的（self-mapping 会被跳过、canonical form 折叠、MergeNativeModels 用 map 语义逐 key 合并而非替换、RemoveModel 会同步 delete），测试也够，质量没问题 —— 只是不该和 free-model 挤在一个 PR 里，否则回滚和 bisect 时会互相牵动。
如果你们不介意打包，请在 PR 描述里说明它包含两部分，至少让 review 的人知道自己在看什么。
其他
- 本地验证：
go build ./...、go test ./...（全量通过）、go vet ./...、gofmt -l（仅internal/store/migrations.go，为 main 既有基线）、release-notes.py validate（exit 0）。打上补丁后又跑了-race覆盖executor/runtime/providers/qoder，全过。 - 这个分支的 CI 是
action_required，需要人工批准 fork 的 workflow run 才会跑。合并前麻烦点一下。
补丁落地后我可以直接合。
… Upsert Upsert already carries Models forward when the caller supplies none, so a runtime re-registration — process start, health refresh, restart — does not drop the cached catalog. FreeModels and NativeModels were added afterwards and missed that carry-forward, so every runtime Upsert wiped both: the free list leaves an exhausted account off its free-model routes until the next catalog refresh, and the mapping sends the public model ID upstream again, which is the silent hang the mapping exists to prevent. Carry both forward on the same terms as Models, and cover each with a regression test: one locks that an Upsert without pricing keeps the carried list and that an explicit list still replaces it.
|
已按审查意见补上，第三个 commit 中文必修项已修： 同一个补丁还带上了 关于打包两个功能：认可这个意见，PR 描述里已经明确写出包含两部分并说明为什么放在同一个 PR 里（只合其中一个，免费模型在这类账号上仍然不通）。如果你仍然倾向拆成两个 PR，我可以把 native mapping 拆出去重开一个，并在描述里声明依赖 #298。 验证： EnglishThe must-fix is in: The same patch also carries On bundling two features: agreed, and the description now states plainly that the PR contains both and why they belong together (merging only one leaves free models unusable on such an account). If you would still rather have two PRs, I can split the native mapping into a new one that declares the dependency on #298. Verification: |
caigee-cmd
left a comment
caigee-cmd
left a comment
There was a problem hiding this comment.
The FreeModels carry-forward is in and correct — verified locally:
go build ./... OK
go vet ./... OK
go test ./... OK — 24 packages, 0 failures
gofmt -l . only internal/store/migrations.go (pre-existing on main)
release-notes validate exit 0
Both tests land exactly where they should: TestUpsertKeepsFreeModelsWhenPricingUnsupplied proves the free list survives a runtime Upsert, and TestUpsertReplacesFreeModelsWhenSupplied proves an explicit pricing payload still replaces it rather than merging — so the exemption can't go stale after a catalog re-prices a model. That's the pair I wanted; the second one is the one that makes the first safe.
Splitting the changelog into two fragments was the right call — the free-model exemption and the Qoder native model mapping are independent features that happen to share a PR, and separate fragments mean they can be described and released independently. Both read well.
One last note for the record, not a blocker: this PR still carries two features (routing exemption + Qoder model-key mapping). They're now cleanly separable in review, the tests cover both, and both are user-visible, so I'm not asking you to split a green PR over it. But next time, opening the routing change and the Qoder key-mapping change as separate PRs would let each get its own focused review — the model-key one in particular is the kind of change that wants live-account acceptance against a gateway-routed account, which is awkward to do while it's riding along with a routing change.
Approving and merging.
2 participants
中文
改动概要
limited_time_free标签或price_factor为零），请求这些模型不消耗积分。但routeBaseMatches无条件排除所有额度耗尽的账号，于是额度为 0 的账号连自己的免费模型路由也一起丢掉——表现是账号停在quota_exhausted，而它本来还能服务的免费模型没有任何账号可用。FreeCatalogIDsFromInfos→Item.FreeModels），额度过滤对命中该名单的请求予以豁免。上游未报价的模型不视为免费；豁免按请求模型限定，同一账号请求付费模型时照旧被过滤；RemoveModel同步清理免费名单。model_config.key。在只按网关键路由的账号上，这个请求不会得到任何响应——网关把它挂着，客户端一直等到自己的超时，既没有错误也没有数据。路由池现在记录目录给出的「公网 ID → 网关键」映射（Item.NativeModels），聊天请求携带映射后的键；目录未给出不同键的账号保持公网 ID 原样。测试计划
go test ./internal/executor/ -run "ExhaustedAccount|ExemptionDoesNotLeak|RemoveModelDropsFree"— 新增 4 个用例：免费模型仍被路由、付费模型仍被过滤、价格未知不假定为免费、RemoveModel 清理免费名单go test ./internal/executor/ ./internal/providers/qoder/ ./internal/runtime/— 全部通过，含新增的 5 个映射用例go vet对涉及包无告警changelog/unreleased/free-model-quota-exemption.md、changelog/unreleased/qoder-native-model-mapping.mdEnglish
Summary
limited_time_freetag or a zeroprice_factor); requests for them spend no credits.routeBaseMatchesfiltered every quota-exceeded account unconditionally, so an exhausted account lost even its free-model routes — it sat atquota_exhaustedwhile the free models it can still serve had no account at all.FreeCatalogIDsFromInfos→Item.FreeModels), and the quota filter exempts a request whose model is on that list. Models the catalog has not priced are not assumed free; the exemption is scoped per model, so exhausted accounts stay out of paid routes;RemoveModelprunes the free list too.model_config.keywith whatever the caller asked for. On an account whose gateway routes only the mapped key, that request was never answered: the gateway held it open and the client waited out its own timeout with no error and no bytes in return. The pool now records the public-ID-to-gateway-key mapping (Item.NativeModels), and the chat request carries the mapped key; an account whose catalog reports no distinct key keeps its public spelling.Test plan
go test ./internal/executor/ -run "ExhaustedAccount|ExemptionDoesNotLeak|RemoveModelDropsFree"— 4 new cases: free model still routed, paid model still filtered, pricing-unknown not assumed free, RemoveModel prunes the free listgo test ./internal/executor/ ./internal/providers/qoder/ ./internal/runtime/— all pass, including 5 new mapping casesgo vetclean on the touched packageschangelog/unreleased/free-model-quota-exemption.mdandchangelog/unreleased/qoder-native-model-mapping.md