Skip to content

feat: prefer provider model metadata - #54

Open
josephcy95 wants to merge 3 commits into
yuhp:mainfrom
josephcy95:main
Open

feat: prefer provider model metadata#54
josephcy95 wants to merge 3 commits into
yuhp:mainfrom
josephcy95:main

Conversation

@josephcy95

@josephcy95 josephcy95 commented Jul 27, 2026

Copy link
Copy Markdown

Summary

  • use provider-reported context windows, reasoning levels, defaults, and input modalities
  • enrich matching models from /v1/models?client_version= when available
  • preserve exact model IDs and keep existing metadata sources as fallback

Why

Custom providers and distributors can expose limits that differ from public model metadata, such as deployments through AWS or Azure. Using provider-reported metadata makes discovery reflect the actual upstream deployment.

Testing

  • npm test -- --run (90 tests)
  • npm run typecheck

@yuhp yuhp left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution. The goal of preferring provider-reported deployment metadata is useful, especially for CLIProxyAPI / CPA deployments.

However, this needs to be provider-specific and explicit before we can merge it. /v1/models?client_version= is not part of the OpenAI-compatible API contract. It is CLIProxyAPI's Codex-client model catalog path, and its response can depend on the specific client version (for example client_version=pi). Please do not request it automatically after every generic /v1/models discovery.

Please revise this as an explicit CPA/CLIProxyAPI metadata path, with a documented configuration opt-in. The implementation should:

  • use a named provider-specific format, e.g. modelInfoFormat: "cliproxyapi", rather than applying CPA behavior to generic providers
  • require an explicit configured metadata endpoint, including a deliberate client_version value when needed
  • parse only the documented CPA catalog response for that format
  • retain the normal /v1/models request as the generic discovery path
  • add docs and tests for the opt-in configuration and for a provider that does not enable it

Please also remove the unrelated limit normalization changes from this PR:

  • missing output metadata must remain unset; it must not become limit.output: 0
  • do not rewrite every existing provider.<id>.models entry through ensureModelLimitOutput()

0 means a real zero-token limit, while an omitted field means the provider did not report a limit. That distinction is important and is consistent with the plugin's existing no-guessed-defaults behavior.

Once the change is scoped to the explicit CLIProxyAPI contract and the limit regressions are removed, I'm happy to review it again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants