feat: prefer provider model metadata - #54
Conversation
yuhp
left a comment
There was a problem hiding this comment.
Thanks for the contribution. The goal of preferring provider-reported deployment metadata is useful, especially for CLIProxyAPI / CPA deployments.
However, this needs to be provider-specific and explicit before we can merge it. /v1/models?client_version= is not part of the OpenAI-compatible API contract. It is CLIProxyAPI's Codex-client model catalog path, and its response can depend on the specific client version (for example client_version=pi). Please do not request it automatically after every generic /v1/models discovery.
Please revise this as an explicit CPA/CLIProxyAPI metadata path, with a documented configuration opt-in. The implementation should:
- use a named provider-specific format, e.g.
modelInfoFormat: "cliproxyapi", rather than applying CPA behavior to generic providers - require an explicit configured metadata endpoint, including a deliberate
client_versionvalue when needed - parse only the documented CPA catalog response for that format
- retain the normal
/v1/modelsrequest as the generic discovery path - add docs and tests for the opt-in configuration and for a provider that does not enable it
Please also remove the unrelated limit normalization changes from this PR:
- missing output metadata must remain unset; it must not become
limit.output: 0 - do not rewrite every existing
provider.<id>.modelsentry throughensureModelLimitOutput()
0 means a real zero-token limit, while an omitted field means the provider did not report a limit. That distinction is important and is consistent with the plugin's existing no-guessed-defaults behavior.
Once the change is scoped to the explicit CLIProxyAPI contract and the limit regressions are removed, I'm happy to review it again.
Summary
/v1/models?client_version=when availableWhy
Custom providers and distributors can expose limits that differ from public model metadata, such as deployments through AWS or Azure. Using provider-reported metadata makes discovery reflect the actual upstream deployment.
Testing
npm test -- --run(90 tests)npm run typecheck