Skip to content

Allow llama.cpp --reasoning flag in runtime flag allowlist - #1065

Open
pujitha24 wants to merge 1 commit into
docker:mainfrom
pujitha24:auto/issue-1064
Open

pujitha24 wants to merge 1 commit into
docker:mainfrom
pujitha24:auto/issue-1064

Conversation

@pujitha24

Copy link
Copy Markdown

Motivation:
llama.cpp's llama-server exposes a --reasoning [on|off|auto] flag that
toggles reasoning/thinking mode in the chat template, but Docker Model
Runner rejects it because it is missing from LlamaCppAllowedFlags. The
map already allows the related --reasoning-format and --reasoning-budget
flags. Without --reasoning, the only way to pin a hybrid thinking model
non-thinking is --chat-template-kwargs '{"enable_thinking": false}',
which current llama.cpp builds flag as deprecated in favor of
--reasoning on / --reasoning off.

Approach:
Add "--reasoning" to the LlamaCppAllowedFlags map in
pkg/inference/runtime_flags_allowlist.go, following the same enum-flag
pattern already used for --pooling and --reasoning-format (no argument
value is validated beyond the existing path-safety and flag-injection
checks, which apply uniformly to all allowed flags). Added a
"reasoning" category to the table-driven allowlist category test and a
new scenario case in the runtime flag validation test.

Validation:

  • go build ./pkg/inference/...
  • go vet ./pkg/inference/...
  • gofmt -l on the changed files (no output)
  • golangci-lint run ./pkg/inference/... (0 issues)
  • go test ./pkg/inference/... and go test -race ./pkg/inference/... (all pass)
  • Confirmed the new test is a genuine regression test: with only the
    allowlist map entry reverted, the new "llama.cpp: reasoning flag
    allowed" case fails with runtime flag "--reasoning" is not allowed for backend "llama.cpp"; restoring the entry makes it pass.

Report: #1064
Signed-off-by: Pujitha Paladugu 10557236+pujitha24@users.noreply.github.com
Assisted-by: claude-sonnet-5 (via Claude Code)

Fixes #1064

Motivation:
llama.cpp's llama-server exposes a --reasoning [on|off|auto] flag that
toggles reasoning/thinking mode in the chat template, but Docker Model
Runner rejects it because it is missing from LlamaCppAllowedFlags. The
map already allows the related --reasoning-format and --reasoning-budget
flags. Without --reasoning, the only way to pin a hybrid thinking model
non-thinking is --chat-template-kwargs '{"enable_thinking": false}',
which current llama.cpp builds flag as deprecated in favor of
--reasoning on / --reasoning off.

Approach:
Add "--reasoning" to the LlamaCppAllowedFlags map in
pkg/inference/runtime_flags_allowlist.go, following the same enum-flag
pattern already used for --pooling and --reasoning-format (no argument
value is validated beyond the existing path-safety and flag-injection
checks, which apply uniformly to all allowed flags). Added a
"reasoning" category to the table-driven allowlist category test and a
new scenario case in the runtime flag validation test.

Validation:
- go build ./pkg/inference/...
- go vet ./pkg/inference/...
- gofmt -l on the changed files (no output)
- golangci-lint run ./pkg/inference/... (0 issues)
- go test ./pkg/inference/... and go test -race ./pkg/inference/... (all pass)
- Confirmed the new test is a genuine regression test: with only the
  allowlist map entry reverted, the new "llama.cpp: reasoning flag
  allowed" case fails with `runtime flag "--reasoning" is not allowed
  for backend "llama.cpp"`; restoring the entry makes it pass.

Report: docker#1064
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Assisted-by: claude-sonnet-5 (via Claude Code)

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

llama.cpp backend: allow --reasoning on|off in runtime flag allowlist

1 participant