Skip to content

Add best-effort tool-calling/template auto-detection #38

@michalharakal

Description

@michalharakal

Summary

Use metadata to infer likely chat template and possible tool-calling support, without treating it as authoritative.

Scope

  • Inspect GGUF metadata such as tokenizer.chat_template when available
  • Inspect tokenizer/config-side chat template hints where available
  • Add best-effort inference logic for tool-calling compatibility
  • Support explicit override when metadata is absent or wrong

Acceptance criteria

  • Auto-detection can suggest a likely template/provider
  • Explicit configuration still wins over auto-detection
  • Missing metadata does not break existing behavior

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions