Conversation
Motivation:
llama.cpp's llama-server exposes a --reasoning [on|off|auto] flag that
toggles reasoning/thinking mode in the chat template, but Docker Model
Runner rejects it because it is missing from LlamaCppAllowedFlags. The
map already allows the related --reasoning-format and --reasoning-budget
flags. Without --reasoning, the only way to pin a hybrid thinking model
non-thinking is --chat-template-kwargs '{"enable_thinking": false}',
which current llama.cpp builds flag as deprecated in favor of
--reasoning on / --reasoning off.
Approach:
Add "--reasoning" to the LlamaCppAllowedFlags map in
pkg/inference/runtime_flags_allowlist.go, following the same enum-flag
pattern already used for --pooling and --reasoning-format (no argument
value is validated beyond the existing path-safety and flag-injection
checks, which apply uniformly to all allowed flags). Added a
"reasoning" category to the table-driven allowlist category test and a
new scenario case in the runtime flag validation test.
Validation:
- go build ./pkg/inference/...
- go vet ./pkg/inference/...
- gofmt -l on the changed files (no output)
- golangci-lint run ./pkg/inference/... (0 issues)
- go test ./pkg/inference/... and go test -race ./pkg/inference/... (all pass)
- Confirmed the new test is a genuine regression test: with only the
allowlist map entry reverted, the new "llama.cpp: reasoning flag
allowed" case fails with `runtime flag "--reasoning" is not allowed
for backend "llama.cpp"`; restoring the entry makes it pass.
Report: docker#1064
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Assisted-by: claude-sonnet-5 (via Claude Code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation:
llama.cpp's llama-server exposes a --reasoning [on|off|auto] flag that
toggles reasoning/thinking mode in the chat template, but Docker Model
Runner rejects it because it is missing from LlamaCppAllowedFlags. The
map already allows the related --reasoning-format and --reasoning-budget
flags. Without --reasoning, the only way to pin a hybrid thinking model
non-thinking is --chat-template-kwargs '{"enable_thinking": false}',
which current llama.cpp builds flag as deprecated in favor of
--reasoning on / --reasoning off.
Approach:
Add "--reasoning" to the LlamaCppAllowedFlags map in
pkg/inference/runtime_flags_allowlist.go, following the same enum-flag
pattern already used for --pooling and --reasoning-format (no argument
value is validated beyond the existing path-safety and flag-injection
checks, which apply uniformly to all allowed flags). Added a
"reasoning" category to the table-driven allowlist category test and a
new scenario case in the runtime flag validation test.
Validation:
allowlist map entry reverted, the new "llama.cpp: reasoning flag
allowed" case fails with
runtime flag "--reasoning" is not allowed for backend "llama.cpp"; restoring the entry makes it pass.Report: #1064
Signed-off-by: Pujitha Paladugu 10557236+pujitha24@users.noreply.github.com
Assisted-by: claude-sonnet-5 (via Claude Code)
Fixes #1064