Skip to content

Fix RawRepresentationFactory settings being overwritten in AsChatClient - #1442

Merged
martindevans merged 4 commits into
SciSharp:masterfrom
Laurianti:fix/chat-client-raw-representation
Sep 26, 2026
Merged

martindevans merged 4 commits into
SciSharp:masterfrom
Laurianti:fix/chat-client-raw-representation

Conversation

@Laurianti

Copy link
Copy Markdown
Contributor

Settings passed through ChatOptions.RawRepresentationFactory were partly overwritten by AsChatClient:

  • MaxTokens was reset to 256 when MaxOutputTokens wasn't set
  • Temperature, TopP and TopK were reset to the defaults
  • Seed was replaced with a random one
  • A sampling pipeline other than DefaultSamplingPipeline was replaced

These now fall back to the raw settings, the same way the other sampling settings already did. Values set on ChatOptions still take priority.

Added LLamaExecutorChatClientTests, which don't need a model.

Copilot AI lite review requested due to automatic review settings September 25, 2026 00:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Preserve raw MaxTokens, including the documented unlimited value -1.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Medium severity

Open (1)
What changed in this PR

This PR updates AsChatClient to preserve settings from RawRepresentationFactory while honoring explicit ChatOptions values.

Changes:

  • Preserves raw token and sampling settings.
  • Retains custom sampling pipelines.
  • Adds model-independent regression tests.

A moderate issue remains: raw MaxTokens = -1 is still overwritten with 256 when MaxOutputTokens is unset.

File Description
LLama/​Extensions/​LLamaExecutorExtensions.cs Updates inference parameter merging.
LLama.Unittest/​LLamaExecutorChatClientTests.cs Adds coverage for raw settings and option precedence.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread LLama/Extensions/LLamaExecutorExtensions.cs Outdated
Comment thread LLama/Extensions/LLamaExecutorExtensions.cs
@martindevans

Copy link
Copy Markdown
Member

Thanks for fixing this. Looks good to merge once the tests are completed :D

@martindevans
martindevans self-requested a review September 26, 2026 00:08
@martindevans
martindevans merged commit 4476237 into SciSharp:master Sep 26, 2026
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants