diff --git a/docs/advanced/model-metadata.mdx b/docs/advanced/model-metadata.mdx new file mode 100644 index 000000000..d21b0b1d9 --- /dev/null +++ b/docs/advanced/model-metadata.mdx @@ -0,0 +1,69 @@ +--- +title: "Model metadata" +description: "Where a model's pricing, context window, capabilities, and category come from, and which source wins when they disagree." +icon: "layers" +keywords: ["model metadata", "pricing", "context window", "capabilities", "model catalog", "ai-model-list", "enrichment"] +--- + +Every model in the catalog carries metadata — pricing, context window, max output +tokens, capabilities, and modes (which decide whether it shows up as a chat, +embeddings, image, or audio model). GoModel assembles it from five sources, +strongest first: + +```mermaid +flowchart LR + A[Pricing overrides] -->|pricing fields only| B[config.yaml metadata] + B -->|declared fields win| C[ai-model-list catalog] + C -->|when it knows the model| D[Provider discovery] + D -.->|modes still missing| E[ID heuristic] +``` + +## The sources + +1. **Pricing overrides** — set per model in the dashboard's **Models** page. + The top layer for pricing fields only; unset price types keep inheriting. + See [Cost tracking](/features/cost-tracking). +2. **`config.yaml` metadata** — `providers..models` entries can attach + `metadata` (`pricing`, `context_window`, `modes`, `capabilities`, …). Declared + fields win field-by-field over everything below; omitted fields inherit. This + is the escape hatch for local models: declaring `modes: [embedding]` also + derives the model's category. +3. **The model catalog** — the + [`ai-model-list`](https://github.com/ENTERPILOT/ai-model-list) registry, + fetched from `MODEL_LIST_URL` (default: the registry's `models.min.json` on + GitHub) at startup and on every catalog refresh. It supplies the rich + defaults — pricing, context windows, capabilities, modes — for most hosted + models, matching IDs directly, through aliases, and with release-date + suffixes stripped. Wrong or missing data is best fixed by contributing to the + registry; use an override for an immediate fix. +4. **Provider discovery** — some providers report capabilities in their own + model listings, and GoModel keeps them for the models it discovers there: + Gemini's `supportedGenerationMethods`, Cohere's per-model `endpoints` and + context length, OpenRouter's architecture modalities and context length, and + Ollama's `/api/show` capabilities. Models declared via configured model + lists skip this step. +5. **ID heuristic** — a last-resort name check for models that end up with no + modes at all (typical for llama.cpp and LM Studio): IDs containing `embed` or + matching well-known embedding families (`bge`, `e5`, `gte`, `minilm`) become + embedding models, IDs containing `rerank` become reranking models. Namespaced + IDs are matched by their final path segment. When unsure, it claims nothing. + +## What metadata affects + +- **Pricing** drives [cost tracking](/features/cost-tracking), budgets, and + `cost` load-balancing. Each priced field remembers its source, so the + dashboard can show where a rate came from. +- **Modes and categories** drive dashboard grouping and failover suggestions + only — routing never blocks on them, so `/v1/embeddings` reaches any model + the provider serves. +- **Context window and capabilities** are advertised on `GET /v1/models` for + clients that pick models dynamically. + +## Offline behavior + +If the catalog fetch fails or the deployment is air-gapped, the gateway runs +normally — only the catalog-supplied defaults (including catalog pricing) are +missing. Pricing overrides, `config.yaml` metadata, provider discovery signals, +and the ID heuristic still apply. Mirror `MODEL_LIST_URL` internally or declare +metadata in `config.yaml`; see [Production guide](/guides/production) for +details. diff --git a/docs/docs.json b/docs/docs.json index 2e18bc59b..3b12f3ede 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -114,6 +114,7 @@ "pages": [ "advanced/configuration", "advanced/config-yaml", + "advanced/model-metadata", "advanced/cli", "advanced/api-endpoints", "advanced/resilience", diff --git a/docs/features/cost-tracking.mdx b/docs/features/cost-tracking.mdx index ff7f5d813..20ccad775 100644 --- a/docs/features/cost-tracking.mdx +++ b/docs/features/cost-tracking.mdx @@ -39,6 +39,8 @@ Pricing for a model is resolved in priority order: The catalog supplies the default pricing for most models. If a model's price looks wrong or a rate is missing (for example, a cached-input rate), check it against [`ai-model-list`](https://github.com/ENTERPILOT/ai-model-list) and contribute a correction there, or set an override for an immediate fix. +Pricing is one part of a wider metadata pipeline that also resolves context windows, capabilities, and model categories — see [Model metadata](/advanced/model-metadata) for the full source chain. + ## Override pricing Override pricing when the catalog price is wrong, missing, or differs from your negotiated rate. Open the **Models** page, find the model, and open its **Pricing override** editor. Set one or more price types (input, output, cached input, and so on) in USD. Saved fields override catalog and `config.yaml` pricing for that selector; unset fields keep inheriting. diff --git a/docs/providers/overview.mdx b/docs/providers/overview.mdx index fdac634b2..82c51547b 100644 --- a/docs/providers/overview.mdx +++ b/docs/providers/overview.mdx @@ -139,6 +139,20 @@ support, not every individual model capability exposed by an upstream provider. `/models` is unavailable or empty. Set `CONFIGURED_PROVIDER_MODELS_MODE=allowlist` to expose only configured models for providers that define a list, skipping their upstream `/models` calls. +- **Model categories (chat vs embeddings vs audio)** — a model's category comes + from its `modes`, resolved from four sources in precedence order: the remote + model registry, operator metadata declared under `providers..models` in + `config.yaml` (e.g. `modes: [embedding]`), capability signals in the + provider's own model listing (Gemini `supportedGenerationMethods`, Cohere + `endpoints`, OpenRouter architecture modalities, Ollama `/api/show` + capabilities), and finally a conservative name check for models still + unclassified: IDs containing `embed` or matching well-known embedding + families (`bge`, `e5`, `gte`, `minilm`) are categorized as embedding models, + and IDs containing `rerank` as reranking models — namespaced IDs like + `org/model` are checked by their final path segment. Declare operator + metadata only when a model stays unclassified after all of this. Categories + affect dashboard grouping and failover suggestions only; `/v1/embeddings` + routes to any model the provider serves regardless of category. - **vLLM** — set `VLLM_API_KEY` only if the upstream server was started with `--api-key`. - **llm-d** — `LLMD_BASE_URL` is required. `LLMD_API_KEY` is optional and is diff --git a/internal/modeldata/infer.go b/internal/modeldata/infer.go new file mode 100644 index 000000000..699d45d6e --- /dev/null +++ b/internal/modeldata/infer.go @@ -0,0 +1,56 @@ +package modeldata + +import "strings" + +// embeddingFamilyTokens are model-family names that identify embedding models +// without containing the substring "embed" (bge-m3, e5-large-v2, gte-large, +// all-minilm). Matched as whole delimited tokens only, so IDs like +// "gemma-3n-e4b" or "bge2000-chat" are not misclassified. +var embeddingFamilyTokens = map[string]struct{}{ + "bge": {}, + "e5": {}, + "gte": {}, + "minilm": {}, +} + +// InferModesFromID guesses a model's modes from its ID alone. It is a +// last-resort fallback for models absent from the remote model registry — +// typically local models served by llama.cpp, LM Studio, Ollama, or vLLM, +// whose IDs (often GGUF file names or user-chosen aliases) the registry can +// never enumerate. Without a mode, such a model is never categorized as an +// embedding model anywhere in the gateway even though calling it works fine. +// +// The heuristic is deliberately conservative: it only claims the modes it is +// confident about and returns nil otherwise, so unknown models keep the +// "no metadata" state rather than being mislabeled. Real registry entries and +// operator-declared metadata always take precedence over this inference. +func InferModesFromID(modelID string) []string { + id := strings.ToLower(strings.TrimSpace(modelID)) + // Namespaced IDs (hf repo paths, "org/model") classify by the final segment. + if idx := strings.LastIndex(id, "/"); idx >= 0 { + id = id[idx+1:] + } + if id == "" { + return nil + } + if strings.Contains(id, "rerank") { + return []string{"rerank"} + } + if strings.Contains(id, "embed") { + return []string{"embedding"} + } + for _, token := range strings.FieldsFunc(id, isModelIDDelimiter) { + if _, ok := embeddingFamilyTokens[token]; ok { + return []string{"embedding"} + } + } + return nil +} + +func isModelIDDelimiter(r rune) bool { + switch r { + case '-', '_', '.', ':', '@', ' ': + return true + } + return false +} diff --git a/internal/modeldata/infer_test.go b/internal/modeldata/infer_test.go new file mode 100644 index 000000000..c1407bf02 --- /dev/null +++ b/internal/modeldata/infer_test.go @@ -0,0 +1,56 @@ +package modeldata + +import ( + "testing" +) + +func TestInferModesFromID(t *testing.T) { + tests := []struct { + id string + want string // single expected mode, or "" for no inference + }{ + // "embed" substring — the common local-model spellings. + {"nomic-embed-text", "embedding"}, + {"nomic-embed-text-v1.5.Q8_0.gguf", "embedding"}, + {"text-embedding-nomic-embed-text-v1.5@q8_0", "embedding"}, + {"mxbai-embed-large", "embedding"}, + {"snowflake-arctic-embed", "embedding"}, + {"embeddinggemma", "embedding"}, + {"qwen3-embedding-0.6b", "embedding"}, + {"text-embedding-3-small", "embedding"}, + {"granite-embedding:278m", "embedding"}, + // Family tokens without "embed" in the name. + {"bge-m3", "embedding"}, + {"bge-large-en-v1.5", "embedding"}, + {"e5-large-v2", "embedding"}, + {"gte-large", "embedding"}, + {"all-minilm", "embedding"}, + {"all-MiniLM-L6-v2", "embedding"}, + // Namespaced IDs classify by the final path segment. + {"BAAI/bge-m3", "embedding"}, + {"intfloat/e5-mistral-7b-instruct", "embedding"}, + // Rerankers. + {"bge-reranker-v2-m3", "rerank"}, + {"jina-reranker-v2", "rerank"}, + // Token matching must not fire on lookalike substrings. + {"gemma-3n-e4b", ""}, + {"bge2000-chat", ""}, + {"gte", "embedding"}, // bare family name still counts + // Ordinary chat models stay uninferred. + {"gpt-4o", ""}, + {"llama-3.1-8b-instruct", ""}, + {"qwen2.5-coder:7b", ""}, + {"", ""}, + {" ", ""}, + {"org/", ""}, + } + for _, tt := range tests { + got := InferModesFromID(tt.id) + switch { + case tt.want == "" && len(got) != 0: + t.Errorf("InferModesFromID(%q) = %v, want none", tt.id, got) + case tt.want != "" && (len(got) != 1 || got[0] != tt.want): + t.Errorf("InferModesFromID(%q) = %v, want [%s]", tt.id, got, tt.want) + } + } +} diff --git a/internal/modeldata/merge.go b/internal/modeldata/merge.go index b0209c85d..991f1aa5c 100644 --- a/internal/modeldata/merge.go +++ b/internal/modeldata/merge.go @@ -20,7 +20,9 @@ func MergeMetadata(base, override *core.ModelMetadata) *core.ModelMetadata { return base.Clone() } if base == nil { - return override.Clone() + merged := override.Clone() + deriveCategoriesFromModes(merged, override) + return merged } merged := base.Clone() @@ -40,6 +42,7 @@ func MergeMetadata(base, override *core.ModelMetadata) *core.ModelMetadata { if len(override.Categories) > 0 { merged.Categories = append([]core.ModelCategory(nil), override.Categories...) } + deriveCategoriesFromModes(merged, override) if len(override.Tags) > 0 { merged.Tags = append([]string(nil), override.Tags...) } @@ -80,6 +83,17 @@ func MergeMetadata(base, override *core.ModelMetadata) *core.ModelMetadata { return merged } +// deriveCategoriesFromModes keeps Categories consistent when an override +// declares Modes without Categories. Categories are derived data (the +// dashboard's category filter and failover suggestions read them), so an +// operator writing `modes: [embedding]` must end up with the embedding +// category rather than the base's stale categories or none at all. +func deriveCategoriesFromModes(merged, override *core.ModelMetadata) { + if len(override.Modes) > 0 && len(override.Categories) == 0 { + merged.Categories = core.CategoriesForModes(merged.Modes) + } +} + func clonePricingSources(in map[string]string) map[string]string { if len(in) == 0 { return nil diff --git a/internal/modeldata/merge_categories_test.go b/internal/modeldata/merge_categories_test.go new file mode 100644 index 000000000..b43efad5d --- /dev/null +++ b/internal/modeldata/merge_categories_test.go @@ -0,0 +1,50 @@ +package modeldata + +import ( + "testing" + + "github.com/enterpilot/gomodel/internal/core" +) + +// Categories are derived data: an operator declaring modes in config must get +// the matching categories without knowing about the internal categories field. +func TestMergeMetadata_DerivesCategoriesFromOverrideModes(t *testing.T) { + t.Run("nil base", func(t *testing.T) { + merged := MergeMetadata(nil, &core.ModelMetadata{Modes: []string{"embedding"}}) + if len(merged.Categories) != 1 || merged.Categories[0] != core.CategoryEmbedding { + t.Errorf("Categories = %v, want [embedding]", merged.Categories) + } + }) + + t.Run("replaces stale base categories", func(t *testing.T) { + base := &core.ModelMetadata{ + Modes: []string{"chat"}, + Categories: []core.ModelCategory{core.CategoryTextGeneration}, + } + merged := MergeMetadata(base, &core.ModelMetadata{Modes: []string{"embedding"}}) + if len(merged.Modes) != 1 || merged.Modes[0] != "embedding" { + t.Errorf("Modes = %v, want [embedding]", merged.Modes) + } + if len(merged.Categories) != 1 || merged.Categories[0] != core.CategoryEmbedding { + t.Errorf("Categories = %v, want [embedding]", merged.Categories) + } + }) + + t.Run("explicit override categories win", func(t *testing.T) { + merged := MergeMetadata(nil, &core.ModelMetadata{ + Modes: []string{"embedding"}, + Categories: []core.ModelCategory{core.CategoryUtility}, + }) + if len(merged.Categories) != 1 || merged.Categories[0] != core.CategoryUtility { + t.Errorf("Categories = %v, want [utility]", merged.Categories) + } + }) + + t.Run("no modes leaves base categories alone", func(t *testing.T) { + base := &core.ModelMetadata{Categories: []core.ModelCategory{core.CategoryTextGeneration}} + merged := MergeMetadata(base, &core.ModelMetadata{DisplayName: "X"}) + if len(merged.Categories) != 1 || merged.Categories[0] != core.CategoryTextGeneration { + t.Errorf("Categories = %v, want [text_generation]", merged.Categories) + } + }) +} diff --git a/internal/providers/cohere/cohere.go b/internal/providers/cohere/cohere.go index 0d520b51c..5fc22f956 100644 --- a/internal/providers/cohere/cohere.go +++ b/internal/providers/cohere/cohere.go @@ -90,9 +90,17 @@ func (p *Provider) ListModels(ctx context.Context) (*core.ModelsResponse, error) continue } var metadata *core.ModelMetadata - if model.ContextLength > 0 { - contextWindow := int(model.ContextLength) - metadata = &core.ModelMetadata{ContextWindow: &contextWindow} + modes := modesFromEndpoints(model.Endpoints) + if model.ContextLength > 0 || len(modes) > 0 { + metadata = &core.ModelMetadata{} + if model.ContextLength > 0 { + contextWindow := int(model.ContextLength) + metadata.ContextWindow = &contextWindow + } + if len(modes) > 0 { + metadata.Modes = modes + metadata.Categories = core.CategoriesForModes(modes) + } } models = append(models, core.Model{ ID: model.Name, @@ -104,6 +112,30 @@ func (p *Provider) ListModels(ctx context.Context) (*core.ModelsResponse, error) return &core.ModelsResponse{Object: "list", Data: models}, nil } +// modesFromEndpoints maps Cohere's per-model endpoints list onto gateway mode +// strings so models are classified from discovery even when the remote model +// registry lacks an entry. Endpoints without a gateway surface are skipped; +// registry enrichment and operator config still override this stamp. +func modesFromEndpoints(endpoints []string) []string { + modes := make([]string, 0, len(endpoints)) + for _, endpoint := range endpoints { + switch strings.ToLower(strings.TrimSpace(endpoint)) { + case "chat": + modes = append(modes, "chat") + case "embed": + modes = append(modes, "embedding") + case "rerank": + modes = append(modes, "rerank") + case "transcriptions": + modes = append(modes, "audio_transcription") + } + } + if len(modes) == 0 { + return nil + } + return modes +} + func supportedModel(model modelInfo) bool { if len(model.Endpoints) == 0 { return true diff --git a/internal/providers/cohere/cohere_test.go b/internal/providers/cohere/cohere_test.go index e12991d6f..a5fd22432 100644 --- a/internal/providers/cohere/cohere_test.go +++ b/internal/providers/cohere/cohere_test.go @@ -585,6 +585,28 @@ func TestListModelsFiltersUnsupportedEndpointsAndRotatesKeys(t *testing.T) { *resp.Data[0].Metadata.ContextWindow != 128000 { t.Fatalf("model metadata = %#v", resp.Data[0].Metadata) } + wantModes := map[string][]string{ + "command-a": {"chat"}, + "embed-v4.0": {"embedding"}, + "cohere-transcribe-03-2026": {"audio_transcription"}, + "legacy-unknown": nil, + } + for _, model := range resp.Data { + want := wantModes[model.ID] + var got []string + if model.Metadata != nil { + got = model.Metadata.Modes + } + if len(got) != len(want) || (len(want) > 0 && got[0] != want[0]) { + t.Errorf("%s Modes = %v, want %v", model.ID, got, want) + } + if len(want) > 0 { + cats := core.CategoriesForModes(want) + if model.Metadata == nil || len(model.Metadata.Categories) != len(cats) || model.Metadata.Categories[0] != cats[0] { + t.Errorf("%s Categories = %+v, want %v", model.ID, model.Metadata, cats) + } + } + } } if len(headers) != 2 || headers[0] != "Bearer first" || headers[1] != "Bearer second" { t.Fatalf("Authorization headers = %#v", headers) diff --git a/internal/providers/gemini/gemini.go b/internal/providers/gemini/gemini.go index d4fc24670..dcd96a9cd 100644 --- a/internal/providers/gemini/gemini.go +++ b/internal/providers/gemini/gemini.go @@ -720,6 +720,28 @@ func geminiModelSupportedMethods(modelID string, methods []string) (supportsGene slices.Contains(methods, "embedContent") } +// geminiDiscoveredMetadata stamps modes/categories from the native listing's +// supportedGenerationMethods so embedding models are classified even when the +// remote model registry has no entry (new or preview IDs). Registry enrichment +// replaces this metadata whenever it does have an entry, and operator config +// merges on top, so the discovery stamp is only the lowest-precedence signal. +func geminiDiscoveredMetadata(supportsGenerate, supportsEmbed bool) *core.ModelMetadata { + modes := make([]string, 0, 2) + if supportsGenerate { + modes = append(modes, "chat") + } + if supportsEmbed { + modes = append(modes, "embedding") + } + if len(modes) == 0 { + return nil + } + return &core.ModelMetadata{ + Modes: modes, + Categories: core.CategoriesForModes(modes), + } +} + // ListModels retrieves the list of available models from Gemini func (p *Provider) ListModels(ctx context.Context) (*core.ModelsResponse, error) { if err := p.ready(); err != nil { @@ -770,10 +792,11 @@ func (p *Provider) ListModels(ctx context.Context) (*core.ModelsResponse, error) isOpenAICompatModel := isGeminiExposedModel(modelID) if (supportsGenerate || supportsEmbed) && isOpenAICompatModel { models = append(models, core.Model{ - ID: modelID, - Object: "model", - OwnedBy: "google", - Created: now, + ID: modelID, + Object: "model", + OwnedBy: "google", + Created: now, + Metadata: geminiDiscoveredMetadata(supportsGenerate, supportsEmbed), }) } } diff --git a/internal/providers/gemini/gemini_test.go b/internal/providers/gemini/gemini_test.go index 080aa32bb..66dbe9497 100644 --- a/internal/providers/gemini/gemini_test.go +++ b/internal/providers/gemini/gemini_test.go @@ -629,6 +629,61 @@ func TestSetBaseURLDerivesModelsURL(t *testing.T) { } } +// ListModels must stamp modes/categories from supportedGenerationMethods so +// embedding models are classified even when the remote model registry has no +// entry for them (new or preview IDs). +func TestListModels_StampsDiscoveredModes(t *testing.T) { + t.Setenv(useNativeAPIEnvVar, "true") + + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(`{ + "models": [{ + "name": "models/gemini-2.5-flash", + "supportedGenerationMethods": ["generateContent", "streamGenerateContent"] + }, { + "name": "models/text-embedding-004", + "supportedGenerationMethods": ["embedContent"] + }] + }`)) + })) + defer server.Close() + + provider := NewWithHTTPClient("test-api-key", nil, llmclient.Hooks{}) + provider.SetBaseURL(server.URL + "/v1beta/openai") + + resp, err := provider.ListModels(context.Background()) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + byID := make(map[string]core.Model, len(resp.Data)) + for _, m := range resp.Data { + byID[m.ID] = m + } + + chat, ok := byID["gemini-2.5-flash"] + if !ok || chat.Metadata == nil { + t.Fatalf("gemini-2.5-flash missing or has no metadata: %+v", resp.Data) + } + if len(chat.Metadata.Modes) != 1 || chat.Metadata.Modes[0] != "chat" { + t.Errorf("chat Modes = %v, want [chat]", chat.Metadata.Modes) + } + if len(chat.Metadata.Categories) != 1 || chat.Metadata.Categories[0] != core.CategoryTextGeneration { + t.Errorf("chat Categories = %v, want [text_generation]", chat.Metadata.Categories) + } + + embed, ok := byID["text-embedding-004"] + if !ok || embed.Metadata == nil { + t.Fatalf("text-embedding-004 missing or has no metadata: %+v", resp.Data) + } + if len(embed.Metadata.Modes) != 1 || embed.Metadata.Modes[0] != "embedding" { + t.Errorf("embed Modes = %v, want [embedding]", embed.Metadata.Modes) + } + if len(embed.Metadata.Categories) != 1 || embed.Metadata.Categories[0] != core.CategoryEmbedding { + t.Errorf("embed Categories = %v, want [embedding]", embed.Metadata.Categories) + } +} + func TestVertexNativeChatUsesOAuthAuthorization(t *testing.T) { t.Setenv(useNativeAPIEnvVar, "true") diff --git a/internal/providers/ollama/ollama.go b/internal/providers/ollama/ollama.go index c33157485..db99369b3 100644 --- a/internal/providers/ollama/ollama.go +++ b/internal/providers/ollama/ollama.go @@ -8,6 +8,7 @@ import ( "net/http" "reflect" "strings" + "sync" "time" "github.com/goccy/go-json" @@ -44,6 +45,41 @@ type Provider struct { compat *openai.CompatibleProvider nativeClient *llmclient.Client keys *providers.Keyring // Optional; Ollama accepts a bearer token but does not require one + // modeCache remembers /api/show-derived modes per model name (including + // confirmed-empty results) so repeated listings don't re-probe upstream. + modeCache sync.Map // string → []string +} + +// discoveredModes maps a model's native /api/show capabilities onto gateway +// mode strings. Errors are swallowed and not cached, so a transient failure +// retries on the next listing while a server without the capabilities field +// (older Ollama, or an OpenAI-compatible impostor) caches an empty result. +func (p *Provider) discoveredModes(ctx context.Context, model string) []string { + if cached, ok := p.modeCache.Load(model); ok { + return cached.([]string) + } + var show struct { + Capabilities []string `json:"capabilities"` + } + err := p.nativeClient.Do(ctx, llmclient.Request{ + Method: http.MethodPost, + Endpoint: "/api/show", + Body: map[string]string{"model": model}, + }, &show) + if err != nil { + return nil + } + modes := make([]string, 0, len(show.Capabilities)) + for _, capability := range show.Capabilities { + switch strings.ToLower(strings.TrimSpace(capability)) { + case "completion": + modes = append(modes, "chat") + case "embedding": + modes = append(modes, "embedding") + } + } + p.modeCache.Store(model, modes) + return modes } // New creates a new Ollama provider. @@ -130,9 +166,29 @@ func (p *Provider) StreamChatCompletion(ctx context.Context, req *core.ChatReque return p.compat.StreamChatCompletion(ctx, req) } -// ListModels retrieves the list of available models from Ollama +// ListModels retrieves the list of available models from Ollama, stamping +// modes/categories from each model's native /api/show capabilities so local +// embedding models are classified without a remote-registry entry. Capability +// lookups are best-effort (older Ollama versions lack the field; a failed call +// just leaves the model unstamped for the ID heuristic) and cached per model +// name, so steady-state listings cost no extra requests. func (p *Provider) ListModels(ctx context.Context) (*core.ModelsResponse, error) { - return p.compat.ListModels(ctx) + resp, err := p.compat.ListModels(ctx) + if err != nil || resp == nil { + return resp, err + } + for i := range resp.Data { + if resp.Data[i].Metadata != nil { + continue + } + if modes := p.discoveredModes(ctx, resp.Data[i].ID); len(modes) > 0 { + resp.Data[i].Metadata = &core.ModelMetadata{ + Modes: modes, + Categories: core.CategoriesForModes(modes), + } + } + } + return resp, nil } // Responses sends a Responses API request to Ollama (converted to chat format) diff --git a/internal/providers/ollama/ollama_test.go b/internal/providers/ollama/ollama_test.go index 490e5ad78..f75e913d7 100644 --- a/internal/providers/ollama/ollama_test.go +++ b/internal/providers/ollama/ollama_test.go @@ -381,6 +381,12 @@ func TestListModels(t *testing.T) { for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Best-effort capability probe issued per listed model. + if r.URL.Path == "/api/show" && r.Method == http.MethodPost { + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(`{"capabilities":["completion"]}`)) + return + } // Verify request method and path if r.Method != http.MethodGet { t.Errorf("Method = %q, want %q", r.Method, http.MethodGet) @@ -415,6 +421,76 @@ func TestListModels(t *testing.T) { } } +// ListModels must stamp modes from native /api/show capabilities (embedding +// models get classified without a remote-registry entry), cache the results so +// repeat listings don't re-probe, and leave models unstamped when the probe +// fails so the ID heuristic can still apply. +func TestListModels_StampsShowCapabilities(t *testing.T) { + showCalls := map[string]int{} + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/api/show" { + var req struct { + Model string `json:"model"` + } + _ = json.NewDecoder(r.Body).Decode(&req) + showCalls[req.Model]++ + switch req.Model { + case "nomic-embed-text": + _, _ = w.Write([]byte(`{"capabilities":["embedding"]}`)) + case "llama3.2": + _, _ = w.Write([]byte(`{"capabilities":["completion","tools"]}`)) + default: + w.WriteHeader(http.StatusInternalServerError) + } + return + } + _, _ = w.Write([]byte(`{"object":"list","data":[ + {"id":"llama3.2","object":"model","owned_by":"library"}, + {"id":"nomic-embed-text","object":"model","owned_by":"library"}, + {"id":"mystery-model","object":"model","owned_by":"library"} + ]}`)) + })) + defer server.Close() + + provider := NewWithHTTPClient("", nil, llmclient.Hooks{}) + provider.SetBaseURL(server.URL) + + resp, err := provider.ListModels(context.Background()) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + byID := map[string]core.Model{} + for _, m := range resp.Data { + byID[m.ID] = m + } + + embed := byID["nomic-embed-text"] + if embed.Metadata == nil || len(embed.Metadata.Modes) != 1 || embed.Metadata.Modes[0] != "embedding" { + t.Errorf("nomic-embed-text metadata = %+v, want embedding modes", embed.Metadata) + } + if embed.Metadata == nil || len(embed.Metadata.Categories) != 1 || embed.Metadata.Categories[0] != core.CategoryEmbedding { + t.Errorf("nomic-embed-text categories = %+v, want [embedding]", embed.Metadata) + } + chat := byID["llama3.2"] + if chat.Metadata == nil || len(chat.Metadata.Modes) != 1 || chat.Metadata.Modes[0] != "chat" { + t.Errorf("llama3.2 metadata = %+v, want chat modes (tools capability skipped)", chat.Metadata) + } + if byID["mystery-model"].Metadata != nil { + t.Errorf("mystery-model metadata = %+v, want nil after failed probe", byID["mystery-model"].Metadata) + } + + // Second listing: successes served from cache, the failure re-probed. + if _, err := provider.ListModels(context.Background()); err != nil { + t.Fatalf("unexpected error on second listing: %v", err) + } + if showCalls["nomic-embed-text"] != 1 || showCalls["llama3.2"] != 1 { + t.Errorf("show calls = %v, want cached results for successful probes", showCalls) + } + if showCalls["mystery-model"] != 2 { + t.Errorf("mystery-model show calls = %d, want re-probe after failure", showCalls["mystery-model"]) + } +} + func TestChatCompletionWithContext(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // Simulate a slow response diff --git a/internal/providers/openrouter/openrouter.go b/internal/providers/openrouter/openrouter.go index c9437ee6a..c592b64d4 100644 --- a/internal/providers/openrouter/openrouter.go +++ b/internal/providers/openrouter/openrouter.go @@ -1,6 +1,7 @@ package openrouter import ( + "context" "net/http" "os" "strings" @@ -75,6 +76,114 @@ func (p *Provider) mutateRequest(req *llmclient.Request) { } } +// openrouterModel is the subset of OpenRouter's /models entry the gateway +// reads beyond the OpenAI-compatible shape: per-model architecture modalities +// and context length, which the generic listing parser would drop. +type openrouterModel struct { + ID string `json:"id"` + Created int64 `json:"created"` + ContextLength int `json:"context_length"` + Architecture struct { + InputModalities []string `json:"input_modalities"` + OutputModalities []string `json:"output_modalities"` + } `json:"architecture"` +} + +// ListModels parses OpenRouter's native models listing so architecture +// modalities and context length survive into model metadata. OpenRouter's +// catalog is far larger than the remote model registry, so discovery-time +// classification keeps its long tail categorized; registry enrichment and +// operator config still override the stamp. +// +// output_modalities=all is required: the endpoint defaults to text-output +// models only, which would hide OpenRouter's embedding models from the +// catalog. Models whose every modality maps outside the gateway's OpenRouter +// surface (rerank-only, video-only, speech/transcription-only) are skipped so +// the catalog never advertises a model that can only fail. +func (p *Provider) ListModels(ctx context.Context) (*core.ModelsResponse, error) { + var upstream struct { + Data []openrouterModel `json:"data"` + } + if err := p.Do(ctx, llmclient.Request{ + Method: http.MethodGet, + Endpoint: "/models?output_modalities=all", + }, &upstream); err != nil { + return nil, err + } + models := make([]core.Model, 0, len(upstream.Data)) + for _, m := range upstream.Data { + id := strings.TrimSpace(m.ID) + if id == "" || !openrouterServable(m) { + continue + } + models = append(models, core.Model{ + ID: id, + Object: "model", + OwnedBy: "openrouter", + Created: m.Created, + Metadata: openrouterMetadata(m), + }) + } + return &core.ModelsResponse{Object: "list", Data: models}, nil +} + +// servableOpenRouterModalities are output modalities the gateway can reach on +// OpenRouter: text and image generation flow through chat completions, and +// embeddings through /embeddings. A model listing none of these (rerank-only, +// video, speech, transcription) has no working endpoint here. +var servableOpenRouterModalities = map[string]struct{}{ + "text": {}, + "image": {}, + "embeddings": {}, +} + +func openrouterServable(m openrouterModel) bool { + // Missing architecture info means no signal, not proof of unservability; + // keep the model rather than hiding it. + if len(m.Architecture.OutputModalities) == 0 { + return true + } + for _, modality := range m.Architecture.OutputModalities { + if _, ok := servableOpenRouterModalities[strings.ToLower(strings.TrimSpace(modality))]; ok { + return true + } + } + return false +} + +// openrouterMetadata maps output modalities onto gateway modes. Only the +// unambiguous mappings are claimed; anything else is left for the registry or +// ID inference. +func openrouterMetadata(m openrouterModel) *core.ModelMetadata { + modes := make([]string, 0, 2) + for _, modality := range m.Architecture.OutputModalities { + switch strings.ToLower(strings.TrimSpace(modality)) { + case "text": + modes = append(modes, "chat") + case "image": + modes = append(modes, "image_generation") + case "embeddings": + modes = append(modes, "embedding") + // "rerank" is deliberately not mapped: the gateway has no rerank + // surface on OpenRouter, and the rerank mode would sort the model + // into the Embeddings category despite being unreachable here. + } + } + if len(modes) == 0 && m.ContextLength <= 0 { + return nil + } + meta := &core.ModelMetadata{} + if len(modes) > 0 { + meta.Modes = modes + meta.Categories = core.CategoriesForModes(modes) + } + if m.ContextLength > 0 { + contextWindow := m.ContextLength + meta.ContextWindow = &contextWindow + } + return meta +} + func setHeaders(req *http.Request, apiKey string) { providers.SetAuthHeaders(req, apiKey, providers.AuthHeaderConfig{ AuthScheme: "Bearer ", diff --git a/internal/providers/openrouter/openrouter_test.go b/internal/providers/openrouter/openrouter_test.go index d1f7feb4a..e70e7e1be 100644 --- a/internal/providers/openrouter/openrouter_test.go +++ b/internal/providers/openrouter/openrouter_test.go @@ -2,6 +2,7 @@ package openrouter import ( "context" + "errors" "io" "net/http" "net/http/httptest" @@ -12,6 +13,107 @@ import ( "github.com/enterpilot/gomodel/internal/llmclient" ) +// ListModels must keep OpenRouter's architecture modalities and context +// length, mapping output modalities onto gateway modes so the catalog's long +// tail is categorized without remote-registry entries. +func TestListModels_StampsArchitectureModalities(t *testing.T) { + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/models" { + t.Errorf("Path = %q, want /models", r.URL.Path) + } + // The endpoint defaults to text-output models; without this parameter + // embedding models would never enter the catalog. + if got := r.URL.Query().Get("output_modalities"); got != "all" { + t.Errorf("output_modalities = %q, want all", got) + } + _, _ = w.Write([]byte(`{"data":[ + {"id":"openai/gpt-4o-mini","created":1721260800,"context_length":128000, + "architecture":{"input_modalities":["text","image"],"output_modalities":["text"]}}, + {"id":"google/gemini-3-pro-image","created":1721260800, + "architecture":{"input_modalities":["text"],"output_modalities":["image"]}}, + {"id":"voyageai/voyage-4-lite","created":1721260800, + "architecture":{"input_modalities":["text"],"output_modalities":["embeddings"]}}, + {"id":"cohere/rerank-only","created":1721260800, + "architecture":{"input_modalities":["text"],"output_modalities":["rerank"]}}, + {"id":"acme/video-only","created":1721260800, + "architecture":{"input_modalities":["text"],"output_modalities":["video"]}}, + {"id":"mystery/no-architecture","created":1721260800} + ]}`)) + })) + defer server.Close() + + provider := NewWithHTTPClient("test-api-key", server.Client(), llmclient.Hooks{}) + provider.SetBaseURL(server.URL) + + resp, err := provider.ListModels(context.Background()) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if len(resp.Data) != 4 { + t.Fatalf("len(Data) = %d, want 4 (rerank-only and video-only skipped): %+v", len(resp.Data), resp.Data) + } + byID := map[string]core.Model{} + for _, m := range resp.Data { + byID[m.ID] = m + } + + chat := byID["openai/gpt-4o-mini"] + if chat.Metadata == nil || len(chat.Metadata.Modes) != 1 || chat.Metadata.Modes[0] != "chat" { + t.Errorf("gpt-4o-mini metadata = %+v, want chat modes", chat.Metadata) + } + if chat.Metadata == nil || chat.Metadata.ContextWindow == nil || *chat.Metadata.ContextWindow != 128000 { + t.Errorf("gpt-4o-mini context window = %+v, want 128000", chat.Metadata) + } + image := byID["google/gemini-3-pro-image"] + if image.Metadata == nil || len(image.Metadata.Modes) != 1 || image.Metadata.Modes[0] != "image_generation" { + t.Errorf("image model metadata = %+v, want image_generation modes", image.Metadata) + } + if image.Metadata == nil || len(image.Metadata.Categories) != 1 || image.Metadata.Categories[0] != core.CategoryImage { + t.Errorf("image model categories = %+v, want [image]", image.Metadata) + } + embed := byID["voyageai/voyage-4-lite"] + if embed.Metadata == nil || len(embed.Metadata.Modes) != 1 || embed.Metadata.Modes[0] != "embedding" { + t.Errorf("voyage-4-lite metadata = %+v, want embedding modes", embed.Metadata) + } + if embed.Metadata == nil || len(embed.Metadata.Categories) != 1 || embed.Metadata.Categories[0] != core.CategoryEmbedding { + t.Errorf("voyage-4-lite categories = %+v, want [embedding]", embed.Metadata) + } + if _, ok := byID["cohere/rerank-only"]; ok { + t.Error("rerank-only model must be skipped: no gateway surface reaches it on OpenRouter") + } + if _, ok := byID["acme/video-only"]; ok { + t.Error("video-only model must be skipped: no gateway surface reaches it on OpenRouter") + } + noArch, ok := byID["mystery/no-architecture"] + if !ok { + t.Fatal("no-architecture model must be retained: missing signal is not proof of unservability") + } + if noArch.Metadata != nil { + t.Errorf("no-architecture metadata = %+v, want nil", noArch.Metadata) + } +} + +// A failed upstream listing must propagate as an error, not an empty catalog. +func TestListModels_UpstreamErrorPropagates(t *testing.T) { + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(http.StatusInternalServerError) + _, _ = w.Write([]byte(`{"error":{"message":"upstream exploded"}}`)) + })) + defer server.Close() + + provider := NewWithHTTPClient("test-api-key", server.Client(), llmclient.Hooks{}) + provider.SetBaseURL(server.URL) + + resp, err := provider.ListModels(context.Background()) + if err == nil { + t.Fatalf("expected error, got response: %+v", resp) + } + var gatewayErr *core.GatewayError + if !errors.As(err, &gatewayErr) { + t.Fatalf("error type = %T, want *core.GatewayError: %v", err, err) + } +} + func TestChatCompletion_AddsDefaultAttributionHeaders(t *testing.T) { var gotReferer string var gotTitle string diff --git a/internal/providers/registry_cache.go b/internal/providers/registry_cache.go index 2eea38ab4..930434686 100644 --- a/internal/providers/registry_cache.go +++ b/internal/providers/registry_cache.go @@ -128,6 +128,7 @@ func (r *ModelRegistry) LoadFromCache(ctx context.Context) (int, error) { } configOverrides := r.snapshotConfigOverrides() metadataStats.Enriched += applyConfigMetadataOverrides(configOverrides, newModelsByProvider, nil) + metadataStats.Enriched += applyInferredModelMetadata(newModelsByProvider, nil) r.mu.Lock() r.models = newModels diff --git a/internal/providers/registry_inferred_metadata_test.go b/internal/providers/registry_inferred_metadata_test.go new file mode 100644 index 000000000..7cde81ba4 --- /dev/null +++ b/internal/providers/registry_inferred_metadata_test.go @@ -0,0 +1,156 @@ +package providers + +import ( + "context" + "testing" + + "github.com/enterpilot/gomodel/internal/core" + "github.com/enterpilot/gomodel/internal/modeldata" +) + +// TestInitialize_InfersEmbeddingModesForUnknownModels verifies the last-resort +// ID heuristic: a local model absent from the remote model registry (the +// llama.cpp / LM Studio / Ollama case) whose ID clearly names an embedding +// model is categorized as an embedding model, so the dashboard's Embeddings +// filter and category counts see it. Registry data and operator overrides +// always win over the inference. +func TestInitialize_InfersEmbeddingModesForUnknownModels(t *testing.T) { + registry := NewModelRegistry() + + local := ®istryMockProvider{ + name: "provider-lagash", + modelsResponse: &core.ModelsResponse{ + Object: "list", + Data: []core.Model{ + {ID: "nomic-embed-text-v1.5.Q8_0.gguf", Object: "model", OwnedBy: "llamacpp"}, + {ID: "bge-m3", Object: "model", OwnedBy: "llamacpp"}, + {ID: "llama-3.1-8b-instruct", Object: "model", OwnedBy: "llamacpp"}, + }, + }, + } + registry.RegisterProviderWithNameAndType(local, "lagash", "openai") + + if err := registry.Initialize(context.Background()); err != nil { + t.Fatalf("Initialize: %v", err) + } + + for _, id := range []string{"nomic-embed-text-v1.5.Q8_0.gguf", "bge-m3"} { + info := registry.GetModel("lagash/" + id) + if info == nil || info.Model.Metadata == nil { + t.Fatalf("expected %s to have inferred metadata", id) + } + meta := info.Model.Metadata + if len(meta.Modes) != 1 || meta.Modes[0] != "embedding" { + t.Errorf("%s Modes = %v, want [embedding]", id, meta.Modes) + } + if len(meta.Categories) != 1 || meta.Categories[0] != core.CategoryEmbedding { + t.Errorf("%s Categories = %v, want [embedding]", id, meta.Categories) + } + } + + if info := registry.GetModel("lagash/llama-3.1-8b-instruct"); info == nil { + t.Fatal("expected llama-3.1-8b-instruct to be registered") + } else if info.Model.Metadata != nil { + t.Errorf("llama-3.1-8b-instruct metadata = %+v, want nil (no inference)", info.Model.Metadata) + } + + embeddings := registry.ListModelsWithProviderByCategory(core.CategoryEmbedding) + found := map[string]bool{} + for _, m := range embeddings { + found[m.Model.ID] = true + } + if !found["nomic-embed-text-v1.5.Q8_0.gguf"] || !found["bge-m3"] { + t.Errorf("embedding category listing = %v, want both local embedding models", found) + } + if found["llama-3.1-8b-instruct"] { + t.Error("chat model must not appear in the embedding category") + } +} + +// TestApplyInferredModelMetadata_ReplacementsProtocol exercises the published- +// map path (EnrichModels), where entries must be replaced rather than mutated +// in place so concurrent readers keep a stable view. Covers both a fresh entry +// and one already replaced by an earlier enrichment step (reverse-chain case). +func TestApplyInferredModelMetadata_ReplacementsProtocol(t *testing.T) { + fresh := &ModelInfo{Model: core.Model{ID: "nomic-embed-text"}, ProviderName: "eridu"} + orig := &ModelInfo{Model: core.Model{ID: "bge-m3"}, ProviderName: "eridu"} + // Simulate a prior pass (registry enrichment) having already replaced orig + // with a clone that still lacks modes/categories. + priorClone := *orig + prior := &priorClone + chat := &ModelInfo{Model: core.Model{ID: "some-chat-model"}, ProviderName: "eridu"} + + providerModels := map[string]*ModelInfo{ + "nomic-embed-text": fresh, + "bge-m3": prior, + "some-chat-model": chat, + } + replacements := map[*ModelInfo]*ModelInfo{orig: prior} + + applied := applyInferredModelMetadata(map[string]map[string]*ModelInfo{"eridu": providerModels}, replacements) + if applied != 2 { + t.Fatalf("applied = %d, want 2", applied) + } + + // Original pointers must be untouched; new entries carry the metadata. + if fresh.Model.Metadata != nil || prior.Model.Metadata != nil { + t.Error("published ModelInfo values were mutated in place") + } + for _, id := range []string{"nomic-embed-text", "bge-m3"} { + next := providerModels[id] + if next.Model.Metadata == nil || len(next.Model.Metadata.Modes) != 1 || next.Model.Metadata.Modes[0] != "embedding" { + t.Errorf("%s replacement metadata = %+v, want embedding modes", id, next.Model.Metadata) + } + } + if got := replacements[fresh]; got != providerModels["nomic-embed-text"] { + t.Error("fresh entry not recorded in replacements") + } + // The chain must point from the ORIGINAL pre-enrichment pointer, not the + // intermediate clone, so callers fixing up r.models find their entry. + if got := replacements[orig]; got != providerModels["bge-m3"] { + t.Error("replacement chain broken: orig does not map to the final entry") + } + if chat.Model.Metadata != nil || providerModels["some-chat-model"] != chat { + t.Error("non-inferable model must be left untouched") + } +} + +// TestEnrichModels_RegistryDataWinsOverInference verifies that when the remote +// model list later supplies real metadata for a model the heuristic had +// classified, the registry data replaces the inferred modes. +func TestEnrichModels_RegistryDataWinsOverInference(t *testing.T) { + registry := NewModelRegistry() + + local := ®istryMockProvider{ + name: "provider-umma", + modelsResponse: &core.ModelsResponse{ + Object: "list", + Data: []core.Model{ + // "gte-large" would be inferred as embedding; the model list + // below deliberately declares it as chat to prove precedence. + {ID: "gte-large", Object: "model", OwnedBy: "test"}, + }, + }, + } + registry.RegisterProviderWithNameAndType(local, "umma", "openai") + + if err := registry.Initialize(context.Background()); err != nil { + t.Fatalf("Initialize: %v", err) + } + + raw := []byte(`{"version":1,"updated_at":"2025-01-01T00:00:00Z","providers":{},"models":{"gte-large":{"modes":["chat"]}},"provider_models":{}}`) + list, err := modeldata.Parse(raw) + if err != nil { + t.Fatalf("Parse: %v", err) + } + registry.SetModelList(list, raw) + registry.EnrichModels() + + info := registry.GetModel("umma/gte-large") + if info == nil || info.Model.Metadata == nil { + t.Fatal("expected gte-large to have metadata") + } + if len(info.Model.Metadata.Modes) != 1 || info.Model.Metadata.Modes[0] != "chat" { + t.Errorf("Modes = %v, want [chat] from model list", info.Model.Metadata.Modes) + } +} diff --git a/internal/providers/registry_init.go b/internal/providers/registry_init.go index 4934f154e..570eeae95 100644 --- a/internal/providers/registry_init.go +++ b/internal/providers/registry_init.go @@ -388,6 +388,7 @@ func (r *ModelRegistry) enrichFetchedProviderModelMaps( metadataStats = enrichProviderModelMaps(list, providerTypes, modelsByProvider, nil) } metadataStats.Enriched += applyConfigMetadataOverrides(configOverrides, modelsByProvider, nil) + metadataStats.Enriched += applyInferredModelMetadata(modelsByProvider, nil) return metadataStats } diff --git a/internal/providers/registry_metadata.go b/internal/providers/registry_metadata.go index 83d0da8d3..5cfd730e1 100644 --- a/internal/providers/registry_metadata.go +++ b/internal/providers/registry_metadata.go @@ -40,9 +40,6 @@ func (r *ModelRegistry) enrichModelsLocked() metadataEnrichmentStats { if len(r.models) == 0 { return metadataEnrichmentStats{} } - if r.modelList == nil && len(r.configMetadataOverrides) == 0 { - return metadataEnrichmentStats{} - } providerTypes := make(map[core.Provider]string, len(r.providerTypes)) maps.Copy(providerTypes, r.providerTypes) @@ -53,6 +50,7 @@ func (r *ModelRegistry) enrichModelsLocked() metadataEnrichmentStats { stats = enrichProviderModelMaps(r.modelList, providerTypes, r.modelsByProvider, replacements) } stats.Enriched += applyConfigMetadataOverrides(r.configMetadataOverrides, r.modelsByProvider, replacements) + stats.Enriched += applyInferredModelMetadata(r.modelsByProvider, replacements) for modelID, info := range r.models { if replacement, ok := replacements[info]; ok { r.models[modelID] = replacement @@ -362,6 +360,64 @@ func applyConfigMetadataOverrides( return applied } +// applyInferredModelMetadata is the last enrichment step: models that ended up +// with no modes and no categories — neither the remote model registry nor +// operator config knows them, the common case for local llama.cpp/LM Studio/ +// Ollama models — get modes inferred from their ID so obvious embedding models +// are categorized as such instead of falling out of every category listing. +// Runs after enrichProviderModelMaps and applyConfigMetadataOverrides so both +// real sources always win; uses the same replacements protocol (nil for fresh, +// unpublished maps). Returns the number of models that received inferred modes. +func applyInferredModelMetadata( + modelsByProvider map[string]map[string]*ModelInfo, + replacements map[*ModelInfo]*ModelInfo, +) int { + var reverse map[*ModelInfo]*ModelInfo + if replacements != nil { + reverse = make(map[*ModelInfo]*ModelInfo, len(replacements)) + for orig, repl := range replacements { + reverse[repl] = orig + } + } + applied := 0 + for _, providerModels := range modelsByProvider { + for modelID, current := range providerModels { + meta := current.Model.Metadata + if meta != nil && (len(meta.Modes) > 0 || len(meta.Categories) > 0) { + continue + } + modes := modeldata.InferModesFromID(modelID) + if len(modes) == 0 { + continue + } + merged := meta.Clone() + if merged == nil { + merged = &core.ModelMetadata{} + } + merged.Modes = modes + merged.Categories = core.CategoriesForModes(modes) + if replacements == nil { + current.Model.Metadata = merged + applied++ + continue + } + cloned := *current + cloned.Model.Metadata = merged + next := &cloned + providerModels[modelID] = next + if orig, hasOrig := reverse[current]; hasOrig { + replacements[orig] = next + reverse[next] = orig + } else { + replacements[current] = next + reverse[next] = current + } + applied++ + } + } + return applied +} + func enrichProviderModelMaps( list *modeldata.ModelList, providerTypes map[core.Provider]string,