Skip to content

Constrained tool-call decoding: validate (local vLLM) + wire #868

@anandgupta42

Description

@anandgupta42

Context

provider/constrained.ts (grammar/JSON-Schema constrained decoding for local providers) was split out of #858 so the validated tool-retrieval lever could land clean. Code + unit tests live on branch feat/constrained-decoding.

Work

  1. Validate: run a local vLLM guided-decoding A/B (constrained ON vs OFF) on a tool-heavy task set; confirm it improves tool-call parse-rate / reduces malformed calls at no quality cost.
  2. Wire: re-add the flag-gated (ALTIMATE_CONSTRAINED_TOOLCALLS, local-providers-only, toolChoice: "required") block into session/llm.ts (marker-wrapped).
  3. Open the PR once it actually does something measurable.

Do NOT open a PR with the flag wired-but-unvalidated or present-but-unwired (the no-op-flag anti-pattern). Branch: feat/constrained-decoding.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions