Repository navigation
No warning when llama.cpp parallel slots leave too little context per request (HTTP 500 context exceeded) #641
Description
Bug
A custom llama.cpp model can fail every request with HTTP 500 "Context size has been exceeded" while Codeman gives no warning.
Steps to reproduce
- Add a custom llama.cpp (llama-swap) endpoint with the default parallel slots (4 on the build measured) and a ~131K context pool.
- Run a Claude Code session that starts parallel subagents.
- Concurrent ~40K-token requests all fail with HTTP 500; with
--parallel 1the same requests succeed.
Cause
llama-server's parallel slots share one context pool, so each slot gets only n_ctx / slots. Claude Code sends concurrent requests (main session plus each subagent, each ~40K tokens of system prompt and tools). The existing 40K floor check only looks at total context, so it never warns.
Status
Fixed, with tests, on branch feature/llamacpp-slot-aware-context in my fork. Not submitted as a PR yet: I'm at the open-PR limit and the fix is queued behind my current open PRs (#567, #592, #616, #617, #618). I'll open the PR as soon as a slot frees, or sooner if you'd like to lift the limit or take it directly from the branch.
Activity
Ark0N commentedon Oct 18, 2026
Thanks @opticon454, the measurement makes the cause clear: llama-server splits one context pool across its parallel slots, so the 40k floor check on the total says nothing about what one request gets once Claude Code runs a main session and subagents at the same time. A warning for that is welcome, so yes, please open the PR.
One thing to settle in it, since I had a look at
feature/llamacpp-slot-aware-context: for llama-swap endpoints the rule is that context is parsed from/running'scmdand never trusted from/props(CLAUDE.md, Custom Model Endpoint Profiles, and architecture-invariants#custom-model-endpoint-profiles). The branch readsn_ctxandtotal_slotsfrom/props. I can see why fortotal_slots, since without an explicit--parallelonly the server knows what it chose, but please keep/running'scmdas the source of truth for the context size, treat a/propsvalue only as a hint that can raise the warning and never lower a floor or skip a confirmation, and say so in the docs where that rule is written. Also keep it a warning:confirmedSwapandconfirmedContextstay separate flags.On the open-PR limit: drafts do not count toward it, so a draft is fine until a slot frees.
Ark0N commentedon Oct 22, 2026
Fixed by #658, which shipped in 1.44.0 (squashed as 33d697a). Discovery now records each llama.cpp model's parallel slot count, and a Claude launch warns when the context split across those slots leaves less than 40K tokens per request, naming the fix (
--parallel 1, or a context large enough for every slot). Thanks again @opticon454 for the report with the cause already worked out.