diff --git a/AGENTS.md b/AGENTS.md index 8ca2b25..8d3bf21 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -132,7 +132,53 @@ Two built-in identities: **netgate / netGate** for SSO-issued keys (the "Get a n The load-bearing consequence: the provider id **is** CoDev's authorship marker for codex/opencode/codev-code. It gates `detectConfiguredTools` (whose configs `codevhub model` rewrites), `isCodevAuthored` (restore's delete-vs-`kept-live` decision), and `readAgentConfig`'s base_url readback. Since it's no longer a compile-time constant, `codevProviderIds()` returns the candidate set — the id saved in `~/.codev-hub/auth.json`, then `netgate`, `ai-gateway`, and the pre-rename `aigateway` — and `firstNestedKey` resolves the live entry against it. Nothing writes `aigateway` any more; it stays recognized so installs predating the rename keep working, and they converge on the new id at the next config rewrite. Detection is deliberately *saved-id-first*: without auth.json a custom-id config is unattributable and restore keeps it rather than deleting it, matching the module's standing rule that a config we can't attribute is one we don't delete. -`Credentials`/`ApiKeyCreds` carry `providerId`/`providerName` (persisted as `provider_id`/`provider_name`), and `resolveProvider` supplies the netGate default when they're absent. **`saveApiKey` writes the whole api-key block, so an omitted provider pair clears it** — every re-save site (`ModelApp`'s two re-auth branches and its model switch, `refresh.ts#ensureFreshGatewayKey`, `SetupApp`'s model-choice) must thread it through, or a manually-named provider silently reverts to netGate on the next model switch or launch-time key refresh. +`Credentials`/`ApiKeyCreds` carry `providerId`/`providerName` (persisted as `provider_id`/`provider_name`), and `resolveProvider` supplies the netGate default when they're absent. **`saveApiKey` writes the whole api-key block, so an omitted provider pair clears it** — every re-save site (`ModelApp`'s two re-auth branches and its model switch, `refresh.ts#ensureFreshGatewayKey`, `SetupApp`'s model-choice) must thread it through, or a manually-named provider silently reverts to netGate on the next model switch or launch-time key refresh. `logout()` is the same hazard by a different route: it rebuilds the surviving file field-by-field rather than deleting SSO keys from it, so anything not listed in its `preserved` object is dropped. The provider pair was missing there and had to be added back — a field added to `AuthFileContents` is not automatically a field that survives sign-out. + +## Context windows and auto-compaction + +Every agent CoDev configures has to be *told* the window of the model it's talking to. The gateway serves custom models none of them recognize, and each guesses differently when unconfigured: Codex assumes a 272K fallback, OpenCode assumes context `0` (which disables compaction outright), Continue falls back to a generic default. `src/lib/model-limits.ts` is the single source of truth; the four writers in `configure.ts` translate it into each agent's own knob and hold no window constants of their own. The flat `GATEWAY_CONTEXT_WINDOW` / `GATEWAY_COMPACT_*` constants that used to live in `const.ts` are gone — they encoded the assumption that every gateway model shares one 196608-token window, which stopped being true once the gateway served both a 1M-token and a 200K-token model. + +`ModelLimits` is `{ context, trigger, output? }`: the true window, the absolute token count where auto-compaction should fire, and an optional output cap. **`trigger` is explicit rather than a percentage** — the gap between window and trigger is a per-model judgement call, not a constant. `limitsFor(modelId)` resolves **remote → table → `DEFAULT_LIMITS`**, where remote is the gateway's own numbers cached in auth.json and `DEFAULT_LIMITS` (200K/160K) covers anything unrecognized. `MiniMax/MiniMax-M2.7` is deliberately *absent* from the table: the default already describes it, and an entry that merely restates the default is one more thing to keep in sync. + +Each agent takes a different shape, and the differences are the whole reason this module exists: + +- **Claude Code** — `CLAUDE_CODE_AUTO_COMPACT_WINDOW` + `CLAUDE_AUTOCOMPACT_PCT_OVERRIDE`, via `claudeWindow()` / `claudeCompactPct()`. The one agent that will **not** accept an arbitrary window; see below. +- **Codex** — `model_context_window` + `model_auto_compact_token_limit` (an absolute count). Also single-model, also exact. +- **Continue** — per-model `defaultCompletionOptions.contextLength` / `maxTokens`. Continue has no compaction of its own; it prunes history to fit `contextLength`, so the window is all it needs and there is no trigger to express. +- **OpenCode / CoDev Code** — the hard one, below. + +**`limit.input` is what makes OpenCode's trigger per-model, and it is not optional.** The decompiled threshold (identical in both the `opencode` and `codev` binaries) is: + +```js +const reserved = cfg.compaction?.reserved ?? Math.min(20000, maxOutputTokens(model)); +return model.limit.input + ? Math.max(0, model.limit.input - reserved) // reserved IS used + : Math.max(0, ctx - maxOutputTokens(model)); // reserved is DISCARDED +``` + +Two consequences. First, **`compaction.reserved` is dead unless `limit.input` is present** — CoDev wrote `{context, output}` for a while and the configured reserve did nothing; the real trigger was `context − maxOutputTokens`, ~36K tokens earlier than intended. Second, `reserved` is a single **top-level** value with no per-model variant in the config schema, so it alone cannot put a 1M model and a 200K model on different triggers: sized for the big one it drives the small one's trigger negative, sized for the small one the big one fires at ~96%. + +`declaredInput()` resolves this by solving `input − reserved = trigger`, i.e. `input = trigger + reserved`, per model, against one global reserve. `limit.context` therefore stays the **true** window — the TUI's "% context used" gauge divides by it, so understating it there would misreport every session. The result is clamped to `context`: `trigger + reserved` above the real window would overstate the budget and let a session run past the model's ceiling before compacting, and clamping can only move a trigger earlier, never later. + +**Claude Code cannot be told a window larger than 200000, and three separate ceilings enforce it.** All three were read out of the shipped binary (2.1.220); none is documented. + +1. `nc()` resolves the window as `Math.min(nativeWindow, envValue)`, so `CLAUDE_CODE_AUTO_COMPACT_WINDOW` can only ever **shrink** it. For a model Claude Code doesn't recognize — every gateway model — `w37()` falls through to `_Z_ = 200000`. A 1M-token model is a 200K-token model to Claude Code, and there is no way around it: `CLAUDE_CODE_MAX_CONTEXT_TOKENS` is read only when `DISABLE_COMPACT` is set, which turns compaction off. +2. `Rzq = Math.min(T − round(T × precomputeBufferFraction), qB6(T, opts))` with `precomputeBufferFraction` defaulting to `0.2`, so the trigger is capped at **80% of the effective window**. `CLAUDE_AUTOCOMPACT_PCT_OVERRIDE` above 80 is inert — the `Math.min` discards it. Hence `CLAUDE_MAX_COMPACT_PCT`. +3. **Pinning a window *below* 200000 disables auto-compaction outright.** Setting the variable makes `nc()` report `source: "env"`, which puts `aiK` on the branch reading `if (window < 200000) return false`. Omitting it leaves `source: "auto"`, which skips that gate and resolves to the same 200000 anyway. So `claudeWindow()` returns `null` below the ceiling and the writer omits the variable — the pre-existing `196608` was tripping exactly this. + +The percentage is therefore taken against the **clamped** window, not the model's true one (`800000/1000000` = 80 is a coincidence; `800000/200000` = 400 is what the raw ratio would give), and bounded to `[1, 80]` — Claude Code's own guard is `K > 0 && K <= 100`, so a 0 would be ignored silently. + +Net effect: Claude Code compacts at `0.8 × (200000 − min(modelMaxOutput, 20000))`, i.e. **~144–160K regardless of the model**. `S$H` (the model's max output) is not statically resolvable in the binary, so the exact point inside that range is unverified. Claude Code is the one agent where CoDev's per-model windows genuinely cannot take effect — don't "fix" it by raising the numbers. + +`CLAUDE_AUTOCOMPACT_PCT_OVERRIDE` is also read into a field named **`testPctOverride`**. It is honored on the production path, but the name says test hook: treat it as unsupported and expect it to disappear. + +**Verify OpenCode-family behavior against the shipped binary, not the published schema.** `https://opencode.ai/config.json` documents `reserved` only as "token buffer for compaction" and says nothing about the `limit.input` branch that decides whether it is read at all. The threshold function is greppable in the binary (`grep -aob "cfg.compaction?.reserved"`, then read the surrounding bytes). + +The remote source is wired but currently inert: `backend.ts#fetchModelWindows` reads LiteLLM's `/model_group/info` (at the gateway **root**, next to `/key/info`, not under `/v1`) and keeps entries with a numeric `max_input_tokens`. The live gateway reports `null` for every model, so it returns `{}` and the static table carries everything — the moment an admin populates the field, that model becomes gateway-driven with no CoDev release. Unlike `fetchModels`, it **never throws**: a window is an optimization over a sane default, and install must not break because a metadata endpoint 404s on some other gateway build. `ModelSelect` refreshes it fire-and-forget alongside the model list, so it can never delay or fail the picker. + +The cache is its **own top-level `model_limits` block** in auth.json with its own `saveModelLimits`/`loadModelLimits`, deliberately *not* a field on the api-key block — see the `saveApiKey` hazard above; a field there would be cleared by every re-save site that didn't thread it through. `limitsFor` memoizes the read once per process (configure* runs once per selected agent and once per model in the OpenCode map), so tests that write the cache must call `resetModelLimitsCache()`. + +One test-hygiene note: `ModelSelect` now fetches on mount, so **every test that renders it must stub `fetchModelWindows`**. Left unmocked, the `baseUrl` cases issue real HTTPS requests, and a non-empty result writes to the developer's actual `~/.codev-hub/auth.json` — `tests/components/ModelSelect.test.tsx` stubs neither `$HOME` nor the network on its own. ## Config refresh and upload self-healing diff --git a/src/components/ModelSelect.tsx b/src/components/ModelSelect.tsx index e211758..dc5800d 100644 --- a/src/components/ModelSelect.tsx +++ b/src/components/ModelSelect.tsx @@ -1,7 +1,13 @@ import { Box, Text, useInput } from "ink"; import Spinner from "ink-spinner"; import { useEffect, useRef, useState } from "react"; -import { fetchModels, isInvalidKeyError } from "@/lib/backend.js"; +import { saveModelLimits } from "@/lib/auth.js"; +import { + fetchModels, + fetchModelWindows, + isInvalidKeyError, +} from "@/lib/backend.js"; +import { limitsFromWindow, resetModelLimitsCache } from "@/lib/model-limits.js"; interface ModelSelectProps { apiKey: string; @@ -58,6 +64,22 @@ export function ModelSelect({ errorReported.current = false; setPhase("loading"); setError(null); + // Refresh the cached per-model windows alongside the list. Fire-and-forget + // and deliberately not awaited: fetchModelWindows never rejects, an empty + // result is the norm (the gateway leaves max_input_tokens unset), and + // lib/model-limits.ts falls back to its static table either way — so + // nothing here should delay the picker or fail the flow. The cache reset + // makes a fresh map visible to configure*, which runs later in the flow. + fetchModelWindows(apiKey, baseUrl).then((windows) => { + const limits = Object.fromEntries( + Object.entries(windows).map(([id, w]) => [ + id, + limitsFromWindow(w.context, w.output), + ]), + ); + saveModelLimits(limits); + resetModelLimitsCache(); + }); fetchModels(apiKey, baseUrl) .then((ids) => { if (cancelled) return; diff --git a/src/lib/auth.ts b/src/lib/auth.ts index 7abecdb..d8f3a5b 100644 --- a/src/lib/auth.ts +++ b/src/lib/auth.ts @@ -16,6 +16,9 @@ import open from "open"; import { fetchCodevConfig } from "@/lib/backend.js"; import { LOGIN_SUCCESS_URL, SSO_URL } from "@/lib/const.js"; import { logDebug, logError, loggedFetch, logWarn } from "@/lib/log.js"; +// Type-only: lib/model-limits.ts imports loadModelLimits from here, so a value +// import would close a runtime cycle. `import type` is erased at compile time. +import type { ModelLimits } from "@/lib/model-limits.js"; const CLIENT_ID = atob("bGl0ZWxsbS10ZXN0"); const REVOKE_TIMEOUT_MS = 3_000; @@ -99,6 +102,12 @@ interface AuthFileContents { supabase_url?: string; supabase_anon_key?: string; gateway_url?: string; + // Per-model context windows as reported by the gateway, cached at the + // model-choice step. Its own block rather than a field on the api-key block + // above, precisely because saveApiKey rewrites that block wholesale — a + // field there would be cleared by every re-save site that didn't thread it + // through. Absent ⇒ lib/model-limits.ts falls back to its static table. + model_limits?: Record; // SkillHub session cookie (`skill-hub-session=…`), captured by // `codevhub login --admin` for local ADMIN/SUPERADMIN accounts that can't use // SSO. SSO users don't have one — skillhubFetch falls back to a Bearer token. @@ -233,6 +242,19 @@ export function saveCodevConfig(config: CodevConfig): void { }); } +// Cache the gateway's per-model windows. Skips the write entirely for an empty +// map so a gateway that reports nothing (every max_input_tokens null, which is +// the case today) doesn't churn auth.json on every model-choice step. +export function saveModelLimits(limits: Record): void { + if (Object.keys(limits).length === 0) return; + const existing = readAuthFile() ?? {}; + writeAuthFile({ ...existing, model_limits: limits }); +} + +export function loadModelLimits(): Record | null { + return readAuthFile()?.model_limits ?? null; +} + export function loadApiKey(): ApiKeyCreds | null { const raw = readAuthFile(); if (!raw?.api_key) return null; @@ -287,10 +309,18 @@ export async function logout(): Promise { api_key: raw.api_key, base_url: raw.base_url, model: raw.model, + // The provider pair belongs to the api-key block above and must travel + // with it. Dropping it here silently re-labels a manually-named + // provider as the netGate default on the next config write — the same + // failure saveApiKey's whole-block rewrite is documented to cause, + // reached by a different route. + provider_id: raw.provider_id, + provider_name: raw.provider_name, supabase_url: raw.supabase_url, supabase_anon_key: raw.supabase_anon_key, gateway_url: raw.gateway_url, skillhub_cookie: raw.skillhub_cookie, + model_limits: raw.model_limits, }; const hasAnything = Object.values(preserved).some((v) => v !== undefined); if (hasAnything) { diff --git a/src/lib/backend.ts b/src/lib/backend.ts index ee52289..f456243 100644 --- a/src/lib/backend.ts +++ b/src/lib/backend.ts @@ -185,6 +185,78 @@ export async function fetchModels( return ids; } +// LiteLLM's aggregated per-model-name view. Lives at the gateway root next to +// /key/info, not under /v1, so it reuses the root-stripping join rather than +// gatewayV1Url. +function modelGroupInfoUrl(baseUrl?: string): string { + const base = baseUrl ?? AI_GATEWAY_URL(); + const stripped = base.replace(/\/?v1\/?$/, ""); + const trailing = stripped.endsWith("/") ? stripped : `${stripped}/`; + return `${trailing}model_group/info`; +} + +interface ModelGroupInfo { + model_group?: string; + max_input_tokens?: number | null; + max_output_tokens?: number | null; +} + +// A window as the gateway reports it. Deliberately NOT lib/model-limits.ts's +// ModelLimits: that module reads auth.json, which imports this one, and taking +// its type as a value import would close a require cycle. Callers convert with +// limitsFromWindow. +export interface ModelWindow { + context: number; + output?: number; +} + +// Per-model context windows, straight from the gateway. Entries whose +// max_input_tokens the gateway hasn't been told are skipped, which today is all +// of them — the field is nullable in LiteLLM and only populated when an admin +// sets it on the model. So this returns {} on the current deployment and the +// static table in lib/model-limits.ts carries every model; the moment an admin +// fills the field in, that model becomes gateway-driven with no CoDev release. +// +// Unlike fetchModels, this NEVER throws and never fail-stops the caller: a +// window is an optimization over a sane default, and install must not break +// because a metadata endpoint 404s on some other gateway build. +export async function fetchModelWindows( + apiKey: string, + baseUrl?: string, +): Promise> { + try { + const res = await loggedFetch( + "gateway.model-limits", + modelGroupInfoUrl(baseUrl), + { + method: "GET", + headers: { + accept: "application/json", + Authorization: `Bearer ${apiKey}`, + }, + signal: AbortSignal.timeout(MODELS_TIMEOUT_MS), + }, + ); + if (!res.ok) return {}; + const data = (await res.json()) as { data?: ModelGroupInfo[] }; + const out: Record = {}; + for (const entry of data.data ?? []) { + const id = entry.model_group; + const context = entry.max_input_tokens; + if (!id || typeof context !== "number" || context <= 0) continue; + const output = + typeof entry.max_output_tokens === "number" && + entry.max_output_tokens > 0 + ? entry.max_output_tokens + : undefined; + out[id] = { context, ...(output ? { output } : {}) }; + } + return out; + } catch { + return {}; + } +} + // Confirms the configured key can actually RUN the chosen model through the // gateway. validateApiKey (/key/info) and fetchModels (/v1/models) only prove // the key exists and that models are listable — neither proves inference is diff --git a/src/lib/configure.ts b/src/lib/configure.ts index e1f8914..f84f3dd 100644 --- a/src/lib/configure.ts +++ b/src/lib/configure.ts @@ -12,16 +12,16 @@ import { homedir } from "node:os"; import { dirname, join } from "node:path"; import TOML from "@iarna/toml"; import { type ParseError, parse } from "jsonc-parser"; -import { - AI_GATEWAY_OPENAI_URL, - AI_GATEWAY_URL, - GATEWAY_COMPACT_PCT, - GATEWAY_COMPACT_RESERVED, - GATEWAY_COMPACT_TRIGGER, - GATEWAY_CONTEXT_WINDOW, - GATEWAY_MAX_OUTPUT_TOKENS, -} from "@/lib/const.js"; +import { AI_GATEWAY_OPENAI_URL, AI_GATEWAY_URL } from "@/lib/const.js"; import { logInfo } from "@/lib/log.js"; +import { + COMPACT_RESERVED, + claudeCompactPct, + claudeWindow, + declaredInput, + limitsFor, + outputTokens, +} from "@/lib/model-limits.js"; import { codevProviderIds, resolveProvider } from "@/lib/provider.js"; import type { Agent } from "@/providers/types.js"; @@ -155,6 +155,10 @@ const OPENCODE_K = { image: atob("aW1hZ2U="), limit: atob("bGltaXQ="), context: atob("Y29udGV4dA=="), + // Same literal as `input` above, which names the modalities key. Kept as its + // own entry because the two are unrelated: this one is the compaction + // budget, that one is a media type list. + inputLimit: atob("aW5wdXQ="), output: atob("b3V0cHV0"), compaction: atob("Y29tcGFjdGlvbg=="), auto: atob("YXV0bw=="), @@ -297,6 +301,9 @@ const CONTINUE_K = { model: atob("bW9kZWw="), apiBase: atob("YXBpQmFzZQ=="), apiKey: atob("YXBpS2V5"), + defaultCompletionOptions: atob("ZGVmYXVsdENvbXBsZXRpb25PcHRpb25z"), + contextLength: atob("Y29udGV4dExlbmd0aA=="), + maxTokens: atob("bWF4VG9rZW5z"), // The config title is `CoDev ()`, so only the prefix is // stable — that prefix is what isCodevContinueConfig matches on. configNamePrefix: atob("Q29EZXYgKA=="), @@ -660,6 +667,17 @@ export function configureClaudeCode(creds: Credentials): ConfigureResult[] { ? normalizeClaudeBaseUrl(creds.baseUrl) : AI_GATEWAY_URL(); const model = requireModel(creds); + // Claude Code pins one model (ANTHROPIC_MODEL, just below), so its window + // and trigger are simply that model's — no reconciling across a model list + // the way the OpenCode family needs. + // + // It is also the one agent that will not accept an arbitrary window: it + // clamps to what it believes the model's native window is (200000 for + // anything it doesn't recognize, i.e. all of them) and caps the trigger at + // 80% of that. claudeWindow/claudeCompactPct encode both ceilings, and the + // window is omitted entirely when pinning it would disable compaction. + const limits = limitsFor(model); + const claudeCompactWindow = claudeWindow(limits); writeJson(sourcePath, { [CLAUDE_K.schema]: CLAUDE_SCHEMA_URL, @@ -671,9 +689,11 @@ export function configureClaudeCode(creds: Credentials): ConfigureResult[] { [CLAUDE_K.sonnet]: model, [CLAUDE_K.haiku]: model, [CLAUDE_K.agentTeams]: "1", - // Env-var values are strings; the shared window/percentage are numeric. - [CLAUDE_K.autoCompactWindow]: String(GATEWAY_CONTEXT_WINDOW), - [CLAUDE_K.autoCompactPct]: String(GATEWAY_COMPACT_PCT), + // Env-var values are strings; the window/percentage are numeric. + ...(claudeCompactWindow !== null + ? { [CLAUDE_K.autoCompactWindow]: String(claudeCompactWindow) } + : {}), + [CLAUDE_K.autoCompactPct]: String(claudeCompactPct(limits)), }, }); @@ -820,15 +840,18 @@ export function configureCodex(creds: Credentials): ConfigureResult[] { : AI_GATEWAY_OPENAI_URL(); const model = requireModel(creds); const provider = resolveProvider(creds); + // Single-model, like Claude Code — `model` above is what Codex will run. + const limits = limitsFor(model); writeToml(sourcePath, { [CODEX_K.model]: model, [CODEX_K.modelProvider]: provider.id, - // The gateway model isn't in Codex's catalog, so Codex would otherwise - // assume a 272K fallback window — larger than the real 196608 ceiling. - // Pin the true window and compact at ~85% of it, mirroring Claude Code. - [CODEX_K.modelContextWindow]: GATEWAY_CONTEXT_WINDOW, - [CODEX_K.autoCompactTokenLimit]: GATEWAY_COMPACT_TRIGGER, + // The gateway models aren't in Codex's catalog, so Codex would otherwise + // assume a 272K fallback window for every one of them — too large for a + // 200K model, too small for the 1M one. Pin the real window, and give + // the trigger as the absolute token count Codex expects. + [CODEX_K.modelContextWindow]: limits.context, + [CODEX_K.autoCompactTokenLimit]: limits.trigger, [CODEX_K.modelProviders]: { [provider.id]: { [CODEX_K.name]: provider.name, @@ -864,6 +887,7 @@ export function configureContinue(creds: Credentials): ConfigureResult[] { lines.push(`${CONTINUE_K.schema}: ${yamlScalar(CONTINUE_K.schemaValue)}`); lines.push(`${CONTINUE_K.models}:`); for (const id of allModels) { + const limits = limitsFor(id); lines.push(` - ${CONTINUE_K.name}: ${yamlScalar(id)}`); lines.push( ` ${CONTINUE_K.provider}: ${yamlScalar(CONTINUE_K.providerValue)}`, @@ -871,6 +895,14 @@ export function configureContinue(creds: Credentials): ConfigureResult[] { lines.push(` ${CONTINUE_K.model}: ${yamlScalar(id)}`); lines.push(` ${CONTINUE_K.apiBase}: ${yamlScalar(baseUrl)}`); lines.push(` ${CONTINUE_K.apiKey}: ${yamlScalar(creds.apiKey)}`); + // Continue has no compaction of its own — it prunes history to fit + // `contextLength`, so the window is all it needs, and there's no trigger + // to express. Without it Continue falls back to a generic default for an + // unrecognized model, which is the same mis-sizing the other three + // agents get. Per-model, since Continue switches models in-IDE. + lines.push(` ${CONTINUE_K.defaultCompletionOptions}:`); + lines.push(` ${CONTINUE_K.contextLength}: ${limits.context}`); + lines.push(` ${CONTINUE_K.maxTokens}: ${outputTokens(limits)}`); } writeText(sourcePath, `${lines.join("\n")}\n`); @@ -939,30 +971,43 @@ function configureOpenCodeKind( // A custom-provider model with no `limit` defaults to context 0, which both // mis-sizes the window and disables OpenCode's auto-compaction entirely. - // Declare the gateway's real window so compaction works; `output` is required + // Declare each model's real window so compaction works; `output` is required // whenever a `limit` object is present. // + // `input` is what makes the trigger per-model. OpenCode computes it as + // `limit.input − compaction.reserved`, and falls back to + // `limit.context − maxOutputTokens` when `input` is absent — a branch that + // ignores `reserved` entirely. Since `reserved` is a single top-level value + // shared by every model, `input` (= trigger + reserved, see declaredInput) + // is the only way to land a 1M-token and a 200K-token model on their own + // triggers from one config. `context` stays the true window, which is what + // the TUI's "% context used" gauge divides by. + // // Image input defaults to off for custom-provider models, which makes // OpenCode strip attached images before the request and the model reply // that it can't see them. Declare image support so attachments pass // through; for a text-only model the gateway/model then decides (reject or // ignore) instead of the client silently dropping the image. const modelsMap = Object.fromEntries( - allModels.map((id) => [ - id, - { - [OPENCODE_K.name]: id, - [OPENCODE_K.attachment]: true, - [OPENCODE_K.modalities]: { - [OPENCODE_K.input]: [OPENCODE_K.text, OPENCODE_K.image], - [OPENCODE_K.output]: [OPENCODE_K.text], + allModels.map((id) => { + const limits = limitsFor(id); + return [ + id, + { + [OPENCODE_K.name]: id, + [OPENCODE_K.attachment]: true, + [OPENCODE_K.modalities]: { + [OPENCODE_K.input]: [OPENCODE_K.text, OPENCODE_K.image], + [OPENCODE_K.output]: [OPENCODE_K.text], + }, + [OPENCODE_K.limit]: { + [OPENCODE_K.context]: limits.context, + [OPENCODE_K.inputLimit]: declaredInput(limits), + [OPENCODE_K.output]: outputTokens(limits), + }, }, - [OPENCODE_K.limit]: { - [OPENCODE_K.context]: GATEWAY_CONTEXT_WINDOW, - [OPENCODE_K.output]: GATEWAY_MAX_OUTPUT_TOKENS, - }, - }, - ]), + ]; + }), ); writeJson(sourcePath, { @@ -982,12 +1027,12 @@ function configureOpenCodeKind( // ahead of config providers — and, within a provider, to the server's // own model sort, neither of which reads this map's order. This is why // `codevhub model` steers only Claude Code and Codex directly. - // OpenCode has no percentage trigger; it compacts at `context − reserved`. - // Reserve the headroom that lands the trigger at ~85% of the window, to - // match Claude Code and Codex. + // One global reserve for every model in the map — OpenCode's schema has no + // per-model compaction block. Each model's own trigger comes from its + // `limit.input` above, which is sized against exactly this number. [OPENCODE_K.compaction]: { [OPENCODE_K.auto]: true, - [OPENCODE_K.reserved]: GATEWAY_COMPACT_RESERVED, + [OPENCODE_K.reserved]: COMPACT_RESERVED, }, [OPENCODE_K.provider]: { [provider.id]: { diff --git a/src/lib/const.ts b/src/lib/const.ts index 206ecca..0d4a71f 100644 --- a/src/lib/const.ts +++ b/src/lib/const.ts @@ -12,24 +12,13 @@ export const SKILLHUB_URL = `${BASE_URL}/netmindhub`; // binaries) from its public/ dir under the site's /codev base path. export const CODE_DOWNLOADS_URL = `${BASE_URL}/codev/docs/code/downloads`; -export const FALLBACK_MODEL = atob("TWluaU1heC9NaW5pTWF4LU0yLjc="); +export const FALLBACK_MODEL = atob("TWluaU1heC9NaW5pTWF4LU0z"); -// The self-hosted gateway model has a 196608-token window. Each agent is told -// to treat that as its effective window and to fire auto-compaction at ~85% of -// it (≈167K), keeping compaction well below the hard limit. -export const GATEWAY_CONTEXT_WINDOW = 196608; -export const GATEWAY_COMPACT_PCT = 85; -// Compaction trigger and reserve, derived from the window and percentage above. -// Codex's `model_auto_compact_token_limit` is an absolute token threshold (≈167K); -// OpenCode has no percentage knob — it compacts at `context − reserved`, so the -// reserve is the headroom that lands the trigger at the same ~85% point. -export const GATEWAY_COMPACT_TRIGGER = Math.round( - GATEWAY_CONTEXT_WINDOW * (GATEWAY_COMPACT_PCT / 100), -); -export const GATEWAY_COMPACT_RESERVED = - GATEWAY_CONTEXT_WINDOW - GATEWAY_COMPACT_TRIGGER; -// Max output tokens advertised to OpenCode (required whenever `limit` is set). -export const GATEWAY_MAX_OUTPUT_TOKENS = 65536; +// Context windows and auto-compaction triggers are per-model and live in +// lib/model-limits.ts. They used to be the flat GATEWAY_CONTEXT_WINDOW / +// GATEWAY_COMPACT_* constants here, which assumed every gateway model shared +// one 196608-token window — untrue once the gateway served both a 1M-token and +// a 200K-token model. export const VERSION: string = pkg.version; diff --git a/src/lib/model-limits.ts b/src/lib/model-limits.ts new file mode 100644 index 0000000..db9264b --- /dev/null +++ b/src/lib/model-limits.ts @@ -0,0 +1,161 @@ +import { loadModelLimits } from "@/lib/auth.js"; + +// Per-model context windows and auto-compaction triggers. +// +// Every agent CoDev configures needs to be told the window of the model it's +// talking to — the gateway serves custom models none of them recognize, so +// each would otherwise guess (Codex assumes 272K, OpenCode assumes 0, which +// disables compaction outright). This module is the single source of truth for +// those numbers; the four writers in lib/configure.ts translate them into each +// agent's own knob and hold no window constants of their own. + +export interface ModelLimits { + // The model's true window. Written verbatim wherever an agent wants "how big + // is this model" — including OpenCode's `limit.context`, which drives the + // TUI's "% context used" gauge, so understating it here would misreport + // every session. + context: number; + // Where auto-compaction should fire. Deliberately explicit rather than a + // percentage of `context`: the gap between the two is a judgement call per + // model, not a constant. + trigger: number; + // Max output tokens. Absent ⇒ DEFAULT_OUTPUT_TOKENS. + output?: number; +} + +// Max output tokens advertised to agents that require one alongside a window. +export const DEFAULT_OUTPUT_TOKENS = 65536; + +// OpenCode's `compaction.reserved` — a single global token buffer, with no +// per-model variant in its config schema. See declaredInput() for how a shared +// reserve still yields exact per-model triggers. +export const COMPACT_RESERVED = 40000; + +// Percentage used to derive a trigger from a window we didn't pick ourselves, +// i.e. one reported by the gateway. Table entries state their trigger outright. +export const DEFAULT_COMPACT_PCT = 80; + +// Unrecognized models are treated as 200K-class. Chosen over the older 196608 +// default because it matches the smaller of the two models actually served, +// and because guessing low is the safe direction: too small a window wastes +// capacity, too large overruns the model and 400s mid-session. +export const DEFAULT_LIMITS: ModelLimits = { context: 200000, trigger: 160000 }; + +// Known gateway models. Keyed by the exact id `/v1/models` reports, which is +// what lands in every agent config. +// +// MiniMax/MiniMax-M2.7 is deliberately absent: DEFAULT_LIMITS already describes +// it correctly, and an entry that merely restates the default is one more thing +// to keep in sync. +const TABLE: Record = { + "MiniMax/MiniMax-M3": { context: 1000000, trigger: 800000 }, + "zai-org/GLM-4.7-cc": { context: 200000, trigger: 160000 }, +}; + +// Windows reported by the gateway, cached in auth.json by the model-choice +// step. Read once per process: configure* runs several times per command (one +// call per selected agent, and once per model in the OpenCode models map) and +// none of them can change the file mid-run. +let remoteCache: Record | null | undefined; + +function remoteLimits(): Record | null { + if (remoteCache === undefined) remoteCache = loadModelLimits(); + return remoteCache; +} + +// Test seam: lets a test install (or clear) the remote map without writing +// auth.json and without the once-per-process cache leaking across cases. +export function resetModelLimitsCache(): void { + remoteCache = undefined; +} + +// Resolve one model's limits. Precedence is remote → table → default: the +// gateway is authoritative about its own models when it says anything at all, +// and the table exists to carry the models it currently reports nothing for. +export function limitsFor(modelId: string): ModelLimits { + const remote = remoteLimits()?.[modelId]; + if (remote) return remote; + return TABLE[modelId] ?? DEFAULT_LIMITS; +} + +// Turn a gateway-reported window into full limits. Exported for backend.ts, +// which has the raw max_input_tokens/max_output_tokens and no opinion about +// where the trigger belongs. +export function limitsFromWindow( + context: number, + output?: number, +): ModelLimits { + return { + context, + trigger: Math.round((context * DEFAULT_COMPACT_PCT) / 100), + ...(output ? { output } : {}), + }; +} + +// Claude Code takes a window plus a percentage, so the trigger is expressed as +// a share of the window. Integer percent, so a trigger that isn't a whole +// percentage of its window lands within half a percent of the intent. +export function compactPct(limits: ModelLimits): number { + return Math.round((limits.trigger / limits.context) * 100); +} + +// Claude Code's native window for a model it doesn't recognize — which is every +// model the gateway serves. From its `nc()`: the window is +// `Math.min(nativeWindow, envValue)`, so CLAUDE_CODE_AUTO_COMPACT_WINDOW can +// only ever SHRINK the window, never raise it. A 1M-token model is therefore a +// 200000-token model as far as Claude Code is concerned, and there is no way to +// tell it otherwise (CLAUDE_CODE_MAX_CONTEXT_TOKENS is read only when +// DISABLE_COMPACT is set, which turns compaction off). +export const CLAUDE_MAX_WINDOW = 200000; + +// Claude Code caps its own trigger at 80% of the effective window: +// `Rzq = Math.min(T − round(T × precomputeBufferFraction), qB6(T, opts))`, with +// precomputeBufferFraction defaulting to 0.2. A percentage above this is inert +// — the Math.min discards it — so 80 is the highest reachable trigger, not a +// preference. +export const CLAUDE_MAX_COMPACT_PCT = 80; + +// The value for CLAUDE_CODE_AUTO_COMPACT_WINDOW, or null to omit the variable. +// +// Below CLAUDE_MAX_WINDOW the variable is actively harmful. Setting it makes +// `nc()` report `source: "env"`, which puts `aiK` on the branch that reads +// `if (window < 200000) return false` — auto-compaction stops firing at all. +// Omitting it leaves the source as "auto", which skips that gate and already +// resolves to the same 200000 window. So we pin only when the pin is a no-op +// against Claude Code's own default, and stay out of the way otherwise. +export function claudeWindow(limits: ModelLimits): number | null { + return limits.context >= CLAUDE_MAX_WINDOW ? CLAUDE_MAX_WINDOW : null; +} + +// The trigger percentage, taken against the window Claude Code will actually +// use rather than the model's true one — for a 1M model those differ by 5x, and +// a percentage of the true window would ask for a trigger beyond the clamped +// ceiling. Bounded by CLAUDE_MAX_COMPACT_PCT above and 1 below (0 or a negative +// value fails Claude Code's `K > 0` guard and would be ignored outright). +export function claudeCompactPct(limits: ModelLimits): number { + const window = claudeWindow(limits) ?? limits.context; + const pct = Math.round((limits.trigger / window) * 100); + return Math.min(Math.max(pct, 1), CLAUDE_MAX_COMPACT_PCT); +} + +// OpenCode's trigger is `limit.input − compaction.reserved`, falling back to +// `limit.context − maxOutputTokens` when `limit.input` is absent — in which +// case `reserved` is computed and then discarded. So `input` is what makes the +// reserve authoritative, and it is the only per-model lever over a trigger that +// otherwise shares one global reserve across every model in the config. +// +// Solving `input − reserved = trigger` gives `input = trigger + reserved`, which +// lands each model's trigger exactly on target regardless of what the others +// need. `context` stays truthful. +// +// Clamped to the true window: `trigger + reserved` above `context` would +// overstate the budget and let a session run past the model's real ceiling +// before compacting. Clamping can only move a trigger earlier, never later, so +// a bad table entry degrades into early compaction rather than 400s. +export function declaredInput(limits: ModelLimits): number { + return Math.min(limits.trigger + COMPACT_RESERVED, limits.context); +} + +export function outputTokens(limits: ModelLimits): number { + return limits.output ?? DEFAULT_OUTPUT_TOKENS; +} diff --git a/tests/ConfigApp.test.tsx b/tests/ConfigApp.test.tsx index a10ec4a..ac582af 100644 --- a/tests/ConfigApp.test.tsx +++ b/tests/ConfigApp.test.tsx @@ -32,6 +32,8 @@ import * as shims from "@/lib/shims.js"; // already covered by InstallApp.test.tsx against the same component. function stubModels() { + // See InstallApp.test.tsx — ModelSelect refreshes cached model windows too. + vi.spyOn(backend, "fetchModelWindows").mockResolvedValue({}); return vi .spyOn(backend, "fetchModels") .mockResolvedValue(["m-alpha", "m-beta"]); diff --git a/tests/InstallApp.test.tsx b/tests/InstallApp.test.tsx index adf12b9..285ab3c 100644 --- a/tests/InstallApp.test.tsx +++ b/tests/InstallApp.test.tsx @@ -26,6 +26,10 @@ import { } from "@/lib/vscode-settings.js"; function stubModels() { + // ModelSelect also refreshes the cached per-model windows. Unmocked it would + // issue a real request and, on a non-empty result, write to the developer's + // real ~/.codev-hub/auth.json. + vi.spyOn(backend, "fetchModelWindows").mockResolvedValue({}); return vi .spyOn(backend, "fetchModels") .mockResolvedValue(["m-alpha", "m-beta"]); diff --git a/tests/ModelApp.test.tsx b/tests/ModelApp.test.tsx index 79c97ca..7455193 100644 --- a/tests/ModelApp.test.tsx +++ b/tests/ModelApp.test.tsx @@ -17,6 +17,9 @@ beforeEach(() => { // homedir() reads USERPROFILE on Windows, HOME on POSIX. Stub both so tests // hit the temp home on every platform. vi.stubEnv("USERPROFILE", tempHome); + // ModelSelect refreshes the cached per-model windows alongside the model + // list; unmocked it would issue a real request. + vi.spyOn(backend, "fetchModelWindows").mockResolvedValue({}); }); afterEach(() => { diff --git a/tests/components/ModelSelect.test.tsx b/tests/components/ModelSelect.test.tsx index 9dbd33a..fef4807 100644 --- a/tests/components/ModelSelect.test.tsx +++ b/tests/components/ModelSelect.test.tsx @@ -1,8 +1,18 @@ import { cleanup, render } from "ink-testing-library"; -import { afterEach, describe, expect, test, vi } from "vitest"; +import { afterEach, beforeEach, describe, expect, test, vi } from "vitest"; import { ModelSelect } from "@/components/ModelSelect.js"; +import * as auth from "@/lib/auth.js"; import * as backend from "@/lib/backend.js"; +beforeEach(() => { + // The component refreshes the cached per-model windows alongside the model + // list. Left unmocked these tests would issue a real HTTPS request (the + // baseUrl case below points at a domain that doesn't exist) and could write + // to the developer's real ~/.codev-hub/auth.json, which no test stubs here. + vi.spyOn(backend, "fetchModelWindows").mockResolvedValue({}); + vi.spyOn(auth, "saveModelLimits").mockImplementation(() => {}); +}); + afterEach(() => { cleanup(); vi.restoreAllMocks(); @@ -97,6 +107,43 @@ describe("ModelSelect", () => { expect(spy).toHaveBeenCalledWith("sk-x", "https://my-gw.example.com/v1"); }); + test("caches the gateway's per-model windows alongside the list", async () => { + vi.spyOn(backend, "fetchModels").mockResolvedValue(["alpha"]); + vi.spyOn(backend, "fetchModelWindows").mockResolvedValue({ + alpha: { context: 500000 }, + }); + const saved = vi + .spyOn(auth, "saveModelLimits") + .mockImplementation(() => {}); + + render( + {}} + onError={() => {}} + />, + ); + await tick(50); + // The window is stored with a trigger derived at 80%, which is what + // configure* consumes — a bare window would not be usable. + expect(saved).toHaveBeenCalledWith({ + alpha: { context: 500000, trigger: 400000 }, + }); + }); + + // The window refresh is an optimization over a static table, so it must + // never gate or fail the picker. + test("still renders the list when the window refresh fails", async () => { + vi.spyOn(backend, "fetchModels").mockResolvedValue(["alpha", "beta"]); + vi.spyOn(backend, "fetchModelWindows").mockResolvedValue({}); + const { lastFrame } = render( + {}} onError={() => {}} />, + ); + await tick(50); + expect(lastFrame() ?? "").toContain("alpha"); + }); + test("readOnly ignores Enter even after the list is ready", async () => { vi.spyOn(backend, "fetchModels").mockResolvedValue(["alpha"]); const onSelect = vi.fn(); diff --git a/tests/lib/auth.test.ts b/tests/lib/auth.test.ts index 7b00bce..351ff11 100644 --- a/tests/lib/auth.test.ts +++ b/tests/lib/auth.test.ts @@ -18,6 +18,7 @@ import { clearSkillhubCookie, loadApiKey, loadAuth, + loadModelLimits, loadSkillhubCookie, login, logout, @@ -245,6 +246,48 @@ describe("logout", () => { }); }); + // The provider pair identifies a manually-named provider and is what every + // config writer keys off. Losing it here reverts the user to the netGate + // default on the next model switch or launch-time key refresh, silently. + test("preserves the provider pair alongside the key it belongs to", async () => { + const dir = join(tempDir, ".codev-hub"); + mkdirSync(dir, { recursive: true }); + writeFileSync( + join(dir, "auth.json"), + JSON.stringify({ + ...VALID_AUTH, + api_key: "sk-keep-me", + model: "m1", + provider_id: "my-corp-gw", + provider_name: "My Corp GW", + }), + ); + expect(await logout()).toBe(true); + expect(loadApiKey()).toEqual({ + apiKey: "sk-keep-me", + baseUrl: undefined, + model: "m1", + providerId: "my-corp-gw", + providerName: "My Corp GW", + }); + }); + + test("preserves cached per-model windows", async () => { + const dir = join(tempDir, ".codev-hub"); + mkdirSync(dir, { recursive: true }); + writeFileSync( + join(dir, "auth.json"), + JSON.stringify({ + ...VALID_AUTH, + model_limits: { "a/model": { context: 500000, trigger: 400000 } }, + }), + ); + expect(await logout()).toBe(true); + expect(loadModelLimits()).toEqual({ + "a/model": { context: 500000, trigger: 400000 }, + }); + }); + test("returns false when only api_key is present (already logged out)", async () => { const dir = join(tempDir, ".codev-hub"); mkdirSync(dir, { recursive: true }); diff --git a/tests/lib/backend.test.ts b/tests/lib/backend.test.ts index 03dc2e9..02dcc1d 100644 --- a/tests/lib/backend.test.ts +++ b/tests/lib/backend.test.ts @@ -7,6 +7,7 @@ import { fetchApiKey, fetchCodevConfig, fetchModels, + fetchModelWindows, isInvalidKeyError, smokeTestModel, validateApiKey, @@ -192,6 +193,86 @@ describe("validateApiKey", () => { }); }); +describe("fetchModelWindows", () => { + test("reads windows from the gateway root's /model_group/info", async () => { + const fetchSpy = vi.spyOn(globalThis, "fetch").mockResolvedValue( + jsonResponse(200, { + data: [ + { + model_group: "big/model", + max_input_tokens: 1000000, + max_output_tokens: 32768, + }, + ], + }), + ); + await expect(fetchModelWindows("sk-test")).resolves.toEqual({ + "big/model": { context: 1000000, output: 32768 }, + }); + + const [url, init] = fetchSpy.mock.calls[0] as [ + string, + { method?: string; headers?: Record }, + ]; + // Sibling of /key/info at the gateway root — NOT under /v1. + expect(url).toBe(`${AI_GATEWAY_URL()}/model_group/info`); + expect(init.method).toBe("GET"); + expect(init.headers?.Authorization).toBe("Bearer sk-test"); + }); + + // The live gateway reports null for every model today, which is precisely + // why the static table in lib/model-limits.ts exists. + test("skips entries whose window the gateway hasn't been told", async () => { + vi.spyOn(globalThis, "fetch").mockResolvedValue( + jsonResponse(200, { + data: [ + { model_group: "a", max_input_tokens: null, max_output_tokens: null }, + { model_group: "b", max_input_tokens: 0 }, + { model_group: "c" }, + { max_input_tokens: 4096 }, + { model_group: "d", max_input_tokens: 8192 }, + ], + }), + ); + await expect(fetchModelWindows("sk-x")).resolves.toEqual({ + d: { context: 8192 }, + }); + }); + + test("omits output when the gateway reports no cap", async () => { + vi.spyOn(globalThis, "fetch").mockResolvedValue( + jsonResponse(200, { + data: [ + { model_group: "a", max_input_tokens: 8192, max_output_tokens: null }, + ], + }), + ); + const out = await fetchModelWindows("sk-x"); + expect(out.a).not.toHaveProperty("output"); + }); + + // Unlike fetchModels, this must never fail-stop a caller: a missing window + // degrades to the default, a thrown error would break install. + test("resolves to {} rather than throwing on a non-2xx", async () => { + vi.spyOn(globalThis, "fetch").mockResolvedValue( + jsonResponse(404, { error: "not found" }), + ); + await expect(fetchModelWindows("sk-x")).resolves.toEqual({}); + }); + + test("resolves to {} rather than throwing on a network error", async () => { + vi.spyOn(globalThis, "fetch").mockRejectedValue(new Error("ECONNREFUSED")); + await expect(fetchModelWindows("sk-x")).resolves.toEqual({}); + }); + + test("resolves to {} rather than throwing on an unparseable body", async () => { + vi.spyOn(globalThis, "fetch").mockResolvedValue( + new Response("proxy blocked", { status: 200 }), + ); + await expect(fetchModelWindows("sk-x")).resolves.toEqual({}); + }); +}); + describe("fetchModels", () => { test("returns the list of model ids from the gateway /v1/models", async () => { const fetchSpy = vi.spyOn(globalThis, "fetch").mockResolvedValue( diff --git a/tests/lib/configure.test.ts b/tests/lib/configure.test.ts index 5449d12..b82131a 100644 --- a/tests/lib/configure.test.ts +++ b/tests/lib/configure.test.ts @@ -286,11 +286,33 @@ describe("configureClaudeCode", () => { ANTHROPIC_DEFAULT_SONNET_MODEL: "chosen-model", ANTHROPIC_DEFAULT_HAIKU_MODEL: "chosen-model", CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS: "1", - CLAUDE_CODE_AUTO_COMPACT_WINDOW: "196608", - CLAUDE_AUTOCOMPACT_PCT_OVERRIDE: "85", + // "chosen-model" isn't in the model table, so it takes the 200K + // default — which is also Claude Code's own ceiling — at 80%. + CLAUDE_CODE_AUTO_COMPACT_WINDOW: "200000", + CLAUDE_AUTOCOMPACT_PCT_OVERRIDE: "80", }); }); + // Claude Code clamps the window env var to the model's native window (200000 + // for anything it doesn't recognize), so a 1M model cannot be described to + // it honestly — writing 1000000 would be silently reduced. See + // claudeWindow/claudeCompactPct. + test("pins Claude Code's own 200K ceiling even for a 1M model", async () => { + const { configureClaudeCode } = await import("@/lib/configure.js"); + const read = () => + JSON.parse( + readFileSync(join(tempDir, ".claude", "settings.json"), "utf-8"), + ).env; + + configureClaudeCode({ apiKey: "sk", model: "MiniMax/MiniMax-M3" }); + expect(read().CLAUDE_CODE_AUTO_COMPACT_WINDOW).toBe("200000"); + expect(read().CLAUDE_AUTOCOMPACT_PCT_OVERRIDE).toBe("80"); + + configureClaudeCode({ apiKey: "sk", model: "zai-org/GLM-4.7-cc" }); + expect(read().CLAUDE_CODE_AUTO_COMPACT_WINDOW).toBe("200000"); + expect(read().CLAUDE_AUTOCOMPACT_PCT_OVERRIDE).toBe("80"); + }); + test("does not touch ~/.claude.json on its own (handled by resetClaudeAuth earlier in the install flow)", async () => { const { configureClaudeCode } = await import("@/lib/configure.js"); configureClaudeCode({ apiKey: "sk-abc", model: "m" }); @@ -405,11 +427,13 @@ describe("configureOpenCode", () => { // No top-level `model`: OpenCode and CoDev Code switch models in-CLI, and // a pin would outrank that selection on every launch. expect(config.model).toBeUndefined(); - // Declares the gateway window so OpenCode sizes context correctly and its + // Declares the model's window so OpenCode sizes context correctly and its // auto-compaction fires (a model with no `limit` defaults to context 0, // which disables compaction). `output` is required alongside `context`. + // `input` is the compaction budget — see the declaredInput tests. expect(config.provider.netgate.models["chosen-model"].limit).toEqual({ - context: 196608, + context: 200000, + input: 200000, output: 65536, }); // Declares image input so OpenCode doesn't strip attached images @@ -422,9 +446,34 @@ describe("configureOpenCode", () => { input: ["text", "image"], output: ["text"], }); - // Reserve lands the compaction trigger at ~85% of the window (196608 − - // 29491 ≈ 167K), matching Claude Code and Codex. - expect(config.compaction).toEqual({ auto: true, reserved: 29491 }); + // One global reserve; each model's trigger comes from its own + // `limit.input` minus this. + expect(config.compaction).toEqual({ auto: true, reserved: 40000 }); + }); + + // The reason `limit.input` is written at all. OpenCode's trigger is + // `input − reserved`, and `reserved` is a single top-level value, so `input` + // is the only per-model lever — without it a 1M model and a 200K model in + // one config cannot both land on their intended triggers. + test("gives each model in one config its own compaction trigger", async () => { + const { configureOpenCode } = await import("@/lib/configure.js"); + configureOpenCode({ + apiKey: "sk-xyz", + model: "MiniMax/MiniMax-M3", + models: ["MiniMax/MiniMax-M3", "zai-org/GLM-4.7-cc"], + }); + + const filePath = join(tempDir, ".config", "opencode", "opencode.json"); + const config = JSON.parse(readFileSync(filePath, "utf-8")); + const map = config.provider.netgate.models; + const reserved = config.compaction.reserved; + + // True windows, so the TUI's "% context used" gauge stays honest. + expect(map["MiniMax/MiniMax-M3"].limit.context).toBe(1000000); + expect(map["zai-org/GLM-4.7-cc"].limit.context).toBe(200000); + // ...and each fires where it should, off one shared reserve. + expect(map["MiniMax/MiniMax-M3"].limit.input - reserved).toBe(800000); + expect(map["zai-org/GLM-4.7-cc"].limit.input - reserved).toBe(160000); }); test("writes every fetched model into the provider's models map", async () => { @@ -441,7 +490,11 @@ describe("configureOpenCode", () => { expect(Object.keys(map).sort()).toEqual(["model-a", "model-b", "model-c"]); for (const id of ["model-a", "model-b", "model-c"]) { expect(map[id].name).toBe(id); - expect(map[id].limit).toEqual({ context: 196608, output: 65536 }); + expect(map[id].limit).toEqual({ + context: 200000, + input: 200000, + output: 65536, + }); expect(map[id].attachment).toBe(true); expect(map[id].modalities).toEqual({ input: ["text", "image"], @@ -609,7 +662,8 @@ describe("configureCodevCode", () => { // The fork shares OpenCode's window/compaction handling, so the same // limit + compaction blocks must land in its config. expect(config.provider.netgate.models["chosen-model"].limit).toEqual({ - context: 196608, + context: 200000, + input: 200000, output: 65536, }); // Same image-input declaration as OpenCode (shared writer). @@ -620,7 +674,7 @@ describe("configureCodevCode", () => { input: ["text", "image"], output: ["text"], }); - expect(config.compaction).toEqual({ auto: true, reserved: 29491 }); + expect(config.compaction).toEqual({ auto: true, reserved: 40000 }); }); test("does not touch ~/.config/opencode/opencode.json (fork-only install)", async () => { @@ -862,13 +916,26 @@ describe("configureCodex", () => { ); }); - test("pins the gateway window and compaction trigger (Codex would otherwise assume a larger fallback window)", async () => { + test("pins the chosen model's window and compaction trigger (Codex would otherwise assume a 272K fallback window)", async () => { const { configureCodex } = await import("@/lib/configure.js"); configureCodex({ apiKey: "sk-codex", model: "m" }); const config = readCodexToml(); - expect(config.model_context_window).toBe(196608); - expect(config.model_auto_compact_token_limit).toBe(167117); // ≈85% of the window + // "m" is unknown, so it takes the 200K default. + expect(config.model_context_window).toBe(200000); + expect(config.model_auto_compact_token_limit).toBe(160000); + }); + + test("pins each model's own window, not a shared constant", async () => { + const { configureCodex } = await import("@/lib/configure.js"); + + configureCodex({ apiKey: "sk-codex", model: "MiniMax/MiniMax-M3" }); + expect(readCodexToml().model_context_window).toBe(1000000); + expect(readCodexToml().model_auto_compact_token_limit).toBe(800000); + + configureCodex({ apiKey: "sk-codex", model: "zai-org/GLM-4.7-cc" }); + expect(readCodexToml().model_context_window).toBe(200000); + expect(readCodexToml().model_auto_compact_token_limit).toBe(160000); }); test("does not touch ~/.claude.json (Codex-only install)", async () => { @@ -997,6 +1064,23 @@ describe("configureContinue", () => { expect(raw).toContain(`model: "chosen-model"`); }); + // Continue has no compaction of its own; it prunes history to fit + // contextLength, so the window is the whole of what it needs from us. + test("declares each model's own window", async () => { + const { configureContinue } = await import("@/lib/configure.js"); + configureContinue({ + apiKey: "sk", + model: "MiniMax/MiniMax-M3", + models: ["MiniMax/MiniMax-M3", "zai-org/GLM-4.7-cc"], + }); + + const raw = readContinueYaml(); + expect(raw).toContain("defaultCompletionOptions:"); + expect(raw).toContain("contextLength: 1000000"); + expect(raw).toContain("contextLength: 200000"); + expect(raw.match(/^\s*maxTokens: 65536$/gm)?.length).toBe(2); + }); + test("emits one model entry per fetched model", async () => { const { configureContinue } = await import("@/lib/configure.js"); configureContinue({ diff --git a/tests/lib/const.test.ts b/tests/lib/const.test.ts index 307b79d..14acdd6 100644 --- a/tests/lib/const.test.ts +++ b/tests/lib/const.test.ts @@ -8,10 +8,6 @@ import { ANALYSIS_BACKEND_ANON_KEY, ANALYSIS_BACKEND_URL, FALLBACK_MODEL, - GATEWAY_COMPACT_RESERVED, - GATEWAY_COMPACT_TRIGGER, - GATEWAY_CONTEXT_WINDOW, - GATEWAY_MAX_OUTPUT_TOKENS, MIN_NODE_STRING, nodeVersionMeets, parseNodeVersion, @@ -114,20 +110,7 @@ describe("gateway URL accessors", () => { describe("FALLBACK_MODEL", () => { test("decodes to the expected model id", () => { - expect(FALLBACK_MODEL).toBe("MiniMax/MiniMax-M2.7"); - }); -}); - -describe("gateway compaction constants", () => { - test("window and percentage are the gateway's real values", () => { - expect(GATEWAY_CONTEXT_WINDOW).toBe(196608); - expect(GATEWAY_MAX_OUTPUT_TOKENS).toBe(65536); - }); - - test("trigger is ~85% of the window and reserve is the remaining headroom", () => { - expect(GATEWAY_COMPACT_TRIGGER).toBe(167117); - expect(GATEWAY_COMPACT_RESERVED).toBe(GATEWAY_CONTEXT_WINDOW - 167117); - expect(GATEWAY_COMPACT_RESERVED).toBe(29491); + expect(FALLBACK_MODEL).toBe("MiniMax/MiniMax-M3"); }); }); diff --git a/tests/lib/model-limits.test.ts b/tests/lib/model-limits.test.ts new file mode 100644 index 0000000..1729f25 --- /dev/null +++ b/tests/lib/model-limits.test.ts @@ -0,0 +1,213 @@ +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, test, vi } from "vitest"; +import { + CLAUDE_MAX_COMPACT_PCT, + COMPACT_RESERVED, + claudeCompactPct, + claudeWindow, + compactPct, + DEFAULT_LIMITS, + DEFAULT_OUTPUT_TOKENS, + declaredInput, + limitsFor, + limitsFromWindow, + type ModelLimits, + outputTokens, + resetModelLimitsCache, +} from "@/lib/model-limits.js"; + +let tempDir: string; + +function writeCachedLimits(limits: Record): void { + const dir = join(tempDir, ".codev-hub"); + mkdirSync(dir, { recursive: true }); + writeFileSync( + join(dir, "auth.json"), + JSON.stringify({ model_limits: limits }), + ); + // The remote map is read once per process; drop the memo so this write is + // the one the next limitsFor() call sees. + resetModelLimitsCache(); +} + +beforeEach(() => { + tempDir = mkdtempSync(join(tmpdir(), "codev-model-limits-")); + vi.stubEnv("HOME", tempDir); + vi.stubEnv("USERPROFILE", tempDir); + resetModelLimitsCache(); +}); + +afterEach(() => { + vi.unstubAllEnvs(); + resetModelLimitsCache(); + rmSync(tempDir, { recursive: true, force: true }); +}); + +describe("limitsFor", () => { + test("returns the table entry for a known model", () => { + expect(limitsFor("MiniMax/MiniMax-M3")).toEqual({ + context: 1000000, + trigger: 800000, + }); + expect(limitsFor("zai-org/GLM-4.7-cc")).toEqual({ + context: 200000, + trigger: 160000, + }); + }); + + test("falls back to the 200K default for an unrecognized model", () => { + expect(limitsFor("some/model-nobody-has-heard-of")).toEqual(DEFAULT_LIMITS); + }); + + test("MiniMax-M2.7 is covered by the default rather than its own entry", () => { + // Intentionally absent from the table — the default already describes it. + expect(limitsFor("MiniMax/MiniMax-M2.7")).toEqual({ + context: 200000, + trigger: 160000, + }); + }); + + test("a gateway-reported window outranks the table", () => { + writeCachedLimits({ + "MiniMax/MiniMax-M3": { context: 524288, trigger: 419430 }, + }); + expect(limitsFor("MiniMax/MiniMax-M3")).toEqual({ + context: 524288, + trigger: 419430, + }); + }); + + test("a model the gateway says nothing about still gets the table entry", () => { + writeCachedLimits({ "other/model": { context: 12345, trigger: 9876 } }); + expect(limitsFor("zai-org/GLM-4.7-cc")).toEqual({ + context: 200000, + trigger: 160000, + }); + }); + + test("an absent cache file is not an error", () => { + expect(limitsFor("MiniMax/MiniMax-M3")).toEqual({ + context: 1000000, + trigger: 800000, + }); + }); +}); + +describe("limitsFromWindow", () => { + test("derives the trigger at 80% of a gateway-reported window", () => { + expect(limitsFromWindow(1000000)).toEqual({ + context: 1000000, + trigger: 800000, + }); + }); + + test("carries an output cap through when the gateway reports one", () => { + expect(limitsFromWindow(200000, 8192)).toEqual({ + context: 200000, + trigger: 160000, + output: 8192, + }); + }); + + test("omits output entirely when the gateway reports none", () => { + expect(limitsFromWindow(200000)).not.toHaveProperty("output"); + }); +}); + +describe("compactPct", () => { + test("expresses the trigger as a whole percentage of the window", () => { + expect(compactPct({ context: 1000000, trigger: 800000 })).toBe(80); + expect(compactPct({ context: 200000, trigger: 160000 })).toBe(80); + }); + + test("rounds a trigger that isn't a whole percentage of its window", () => { + expect(compactPct({ context: 196608, trigger: 167117 })).toBe(85); + }); +}); + +// Claude Code will not accept an arbitrary window. Its `nc()` resolves the +// window as `Math.min(nativeWindow, envValue)` — 200000 for a model it doesn't +// recognize — and `Rzq` caps the trigger at 80% of that via +// precomputeBufferFraction. Both were verified against the shipped binary. +describe("claudeWindow / claudeCompactPct", () => { + test("a 1M model is pinned at Claude Code's 200K ceiling, not its true window", () => { + const m3 = limitsFor("MiniMax/MiniMax-M3"); + expect(m3.context).toBe(1000000); + // Writing 1000000 would be silently clamped to 200000 anyway. + expect(claudeWindow(m3)).toBe(200000); + }); + + test("the percentage is taken against the clamped window, not the true one", () => { + // 800000/1000000 would be 80, but 800000/200000 is 400 — the raw ratio is + // meaningless once the window is clamped, so it must be bounded. + expect(claudeCompactPct(limitsFor("MiniMax/MiniMax-M3"))).toBe(80); + expect(claudeCompactPct(limitsFor("zai-org/GLM-4.7-cc"))).toBe(80); + }); + + test("never exceeds 80%, which Claude Code's Rzq discards anything above", () => { + expect(claudeCompactPct({ context: 200000, trigger: 195000 })).toBe( + CLAUDE_MAX_COMPACT_PCT, + ); + }); + + test("a below-target trigger is still honored", () => { + // The ceiling is a cap, not a fixed value: asking to compact earlier works. + expect(claudeCompactPct({ context: 200000, trigger: 100000 })).toBe(50); + }); + + test("never returns 0, which Claude Code's `K > 0` guard would ignore", () => { + expect(claudeCompactPct({ context: 1000000, trigger: 100 })).toBe(1); + }); + + // The trap: with source "env", Claude Code's `aiK` takes the branch that + // reads `if (window < 200000) return false` — pinning a smaller window turns + // auto-compaction OFF rather than tightening it. Omitting the variable + // leaves source "auto", which skips that gate. + test("omits the window for a model below the ceiling rather than disabling compaction", () => { + expect(claudeWindow({ context: 128000, trigger: 100000 })).toBeNull(); + // The percentage still applies, against the model's own window. + expect(claudeCompactPct({ context: 128000, trigger: 100000 })).toBe(78); + }); + + test("pins exactly at the boundary", () => { + expect(claudeWindow({ context: 200000, trigger: 160000 })).toBe(200000); + expect(claudeWindow({ context: 199999, trigger: 160000 })).toBeNull(); + }); +}); + +describe("declaredInput", () => { + // The whole point of limit.input: OpenCode's trigger is + // `input − reserved` with ONE global reserve, so input is the only + // per-model lever. These are the numbers that must land on target. + test("lands each model's trigger exactly on target under one shared reserve", () => { + const m3 = limitsFor("MiniMax/MiniMax-M3"); + const glm = limitsFor("zai-org/GLM-4.7-cc"); + expect(declaredInput(m3) - COMPACT_RESERVED).toBe(800000); + expect(declaredInput(glm) - COMPACT_RESERVED).toBe(160000); + }); + + test("clamps to the true window so the budget is never overstated", () => { + // trigger + reserved (195000) exceeds the window (180000): declaring it + // would let a session run past the model's real ceiling before + // compacting, so the window wins and the trigger only moves earlier. + const tight: ModelLimits = { context: 180000, trigger: 155000 }; + expect(declaredInput(tight)).toBe(180000); + expect(declaredInput(tight) - COMPACT_RESERVED).toBeLessThan(155000); + }); +}); + +describe("outputTokens", () => { + test("defaults when the model declares no output cap", () => { + expect(outputTokens({ context: 200000, trigger: 160000 })).toBe( + DEFAULT_OUTPUT_TOKENS, + ); + }); + + test("prefers the model's own cap", () => { + expect( + outputTokens({ context: 200000, trigger: 160000, output: 8192 }), + ).toBe(8192); + }); +});