feat(models): classify local embedding models without registry metadata

docs/advanced/model-metadata.mdx

@@ -0,0 +1,69 @@
+---
+title: "Model metadata"
+description: "Where a model's pricing, context window, capabilities, and category come from, and which source wins when they disagree."
+icon: "layers"
+keywords: ["model metadata", "pricing", "context window", "capabilities", "model catalog", "ai-model-list", "enrichment"]
+---
+
+Every model in the catalog carries metadata — pricing, context window, max output
+tokens, capabilities, and modes (which decide whether it shows up as a chat,
+embeddings, image, or audio model). GoModel assembles it from five sources,
+strongest first:
+
+```mermaid
+flowchart LR
+    A[Pricing overrides] -->|pricing fields only| B[config.yaml metadata]
+    B -->|declared fields win| C[ai-model-list catalog]
+    C -->|when it knows the model| D[Provider discovery]
+    D -.->|modes still missing| E[ID heuristic]
+```
+
+## The sources
+
+1. **Pricing overrides** — set per model in the dashboard's **Models** page.
+   The top layer for pricing fields only; unset price types keep inheriting.
+   See [Cost tracking](/features/cost-tracking).
+2. **`config.yaml` metadata** — `providers.<name>.models` entries can attach
+   `metadata` (`pricing`, `context_window`, `modes`, `capabilities`, …). Declared
+   fields win field-by-field over everything below; omitted fields inherit. This
+   is the escape hatch for local models: declaring `modes: [embedding]` also
+   derives the model's category.
+3. **The model catalog** — the
+   [`ai-model-list`](https://github.com/ENTERPILOT/ai-model-list) registry,
+   fetched from `MODEL_LIST_URL` (default: the registry's `models.min.json` on
+   GitHub) at startup and on every catalog refresh. It supplies the rich
+   defaults — pricing, context windows, capabilities, modes — for most hosted
+   models, matching IDs directly, through aliases, and with release-date
+   suffixes stripped. Wrong or missing data is best fixed by contributing to the
+   registry; use an override for an immediate fix.
+4. **Provider discovery** — some providers report capabilities in their own
+   model listings, and GoModel keeps them for the models it discovers there:
+   Gemini's `supportedGenerationMethods`, Cohere's per-model `endpoints` and
+   context length, OpenRouter's architecture modalities and context length, and
+   Ollama's `/api/show` capabilities. Models declared via configured model
+   lists skip this step.
+5. **ID heuristic** — a last-resort name check for models that end up with no
+   modes at all (typical for llama.cpp and LM Studio): IDs containing `embed` or
+   matching well-known embedding families (`bge`, `e5`, `gte`, `minilm`) become
+   embedding models, IDs containing `rerank` become reranking models. Namespaced
+   IDs are matched by their final path segment. When unsure, it claims nothing.
+
+## What metadata affects
+
+- **Pricing** drives [cost tracking](/features/cost-tracking), budgets, and
+  `cost` load-balancing. Each priced field remembers its source, so the
+  dashboard can show where a rate came from.
+- **Modes and categories** drive dashboard grouping and failover suggestions
+  only — routing never blocks on them, so `/v1/embeddings` reaches any model
+  the provider serves.
+- **Context window and capabilities** are advertised on `GET /v1/models` for
+  clients that pick models dynamically.
+
+## Offline behavior
+
+If the catalog fetch fails or the deployment is air-gapped, the gateway runs
+normally — only the catalog-supplied defaults (including catalog pricing) are
+missing. Pricing overrides, `config.yaml` metadata, provider discovery signals,
+and the ID heuristic still apply. Mirror `MODEL_LIST_URL` internally or declare
+metadata in `config.yaml`; see [Production guide](/guides/production) for
+details.

docs/docs.json

@@ -114,6 +114,7 @@
                         "pages": [
                             "advanced/configuration",
                             "advanced/config-yaml",
+                            "advanced/model-metadata",
                             "advanced/cli",
                             "advanced/api-endpoints",
                             "advanced/resilience",

docs/features/cost-tracking.mdx

@@ -39,6 +39,8 @@ Pricing for a model is resolved in priority order:
 
 The catalog supplies the default pricing for most models. If a model's price looks wrong or a rate is missing (for example, a cached-input rate), check it against [`ai-model-list`](https://github.com/ENTERPILOT/ai-model-list) and contribute a correction there, or set an override for an immediate fix.
 
+Pricing is one part of a wider metadata pipeline that also resolves context windows, capabilities, and model categories — see [Model metadata](/advanced/model-metadata) for the full source chain.
+
 ## Override pricing
 
 Override pricing when the catalog price is wrong, missing, or differs from your negotiated rate. Open the **Models** page, find the model, and open its **Pricing override** editor. Set one or more price types (input, output, cached input, and so on) in USD. Saved fields override catalog and `config.yaml` pricing for that selector; unset fields keep inheriting.

docs/providers/overview.mdx

@@ -139,6 +139,20 @@ support, not every individual model capability exposed by an upstream provider.
   `/models` is unavailable or empty. Set
   `CONFIGURED_PROVIDER_MODELS_MODE=allowlist` to expose only configured models
   for providers that define a list, skipping their upstream `/models` calls.
+- **Model categories (chat vs embeddings vs audio)** — a model's category comes
+  from its `modes`, resolved from four sources in precedence order: the remote
+  model registry, operator metadata declared under `providers.<name>.models` in
+  `config.yaml` (e.g. `modes: [embedding]`), capability signals in the
+  provider's own model listing (Gemini `supportedGenerationMethods`, Cohere
+  `endpoints`, OpenRouter architecture modalities, Ollama `/api/show`
+  capabilities), and finally a conservative name check for models still
+  unclassified: IDs containing `embed` or matching well-known embedding
+  families (`bge`, `e5`, `gte`, `minilm`) are categorized as embedding models,
+  and IDs containing `rerank` as reranking models — namespaced IDs like
+  `org/model` are checked by their final path segment. Declare operator
+  metadata only when a model stays unclassified after all of this. Categories
+  affect dashboard grouping and failover suggestions only; `/v1/embeddings`
+  routes to any model the provider serves regardless of category.
 - **vLLM** — set `VLLM_API_KEY` only if the upstream server was started with
   `--api-key`.
 - **llm-d** — `LLMD_BASE_URL` is required. `LLMD_API_KEY` is optional and is

internal/modeldata/infer.go

@@ -0,0 +1,56 @@
+package modeldata
+
+import "strings"
+
+// embeddingFamilyTokens are model-family names that identify embedding models
+// without containing the substring "embed" (bge-m3, e5-large-v2, gte-large,
+// all-minilm). Matched as whole delimited tokens only, so IDs like
+// "gemma-3n-e4b" or "bge2000-chat" are not misclassified.
+var embeddingFamilyTokens = map[string]struct{}{
+	"bge":    {},
+	"e5":     {},
+	"gte":    {},
+	"minilm": {},
+}
+
+// InferModesFromID guesses a model's modes from its ID alone. It is a
+// last-resort fallback for models absent from the remote model registry —
+// typically local models served by llama.cpp, LM Studio, Ollama, or vLLM,
+// whose IDs (often GGUF file names or user-chosen aliases) the registry can
+// never enumerate. Without a mode, such a model is never categorized as an
+// embedding model anywhere in the gateway even though calling it works fine.
+//
+// The heuristic is deliberately conservative: it only claims the modes it is
+// confident about and returns nil otherwise, so unknown models keep the
+// "no metadata" state rather than being mislabeled. Real registry entries and
+// operator-declared metadata always take precedence over this inference.
+func InferModesFromID(modelID string) []string {
+	id := strings.ToLower(strings.TrimSpace(modelID))
+	// Namespaced IDs (hf repo paths, "org/model") classify by the final segment.
+	if idx := strings.LastIndex(id, "/"); idx >= 0 {
+		id = id[idx+1:]
+	}
+	if id == "" {
+		return nil
+	}
+	if strings.Contains(id, "rerank") {
+		return []string{"rerank"}
+	}
+	if strings.Contains(id, "embed") {
+		return []string{"embedding"}
+	}
+	for _, token := range strings.FieldsFunc(id, isModelIDDelimiter) {
+		if _, ok := embeddingFamilyTokens[token]; ok {
+			return []string{"embedding"}
+		}
+	}
+	return nil
+}
+
+func isModelIDDelimiter(r rune) bool {
+	switch r {
+	case '-', '_', '.', ':', '@', ' ':
+		return true
+	}
+	return false
+}

internal/modeldata/infer_test.go

@@ -0,0 +1,56 @@
+package modeldata
+
+import (
+	"testing"
+)
+
+func TestInferModesFromID(t *testing.T) {
+	tests := []struct {
+		id   string
+		want string // single expected mode, or "" for no inference
+	}{
+		// "embed" substring — the common local-model spellings.
+		{"nomic-embed-text", "embedding"},
+		{"nomic-embed-text-v1.5.Q8_0.gguf", "embedding"},
+		{"text-embedding-nomic-embed-text-v1.5@q8_0", "embedding"},
+		{"mxbai-embed-large", "embedding"},
+		{"snowflake-arctic-embed", "embedding"},
+		{"embeddinggemma", "embedding"},
+		{"qwen3-embedding-0.6b", "embedding"},
+		{"text-embedding-3-small", "embedding"},
+		{"granite-embedding:278m", "embedding"},
+		// Family tokens without "embed" in the name.
+		{"bge-m3", "embedding"},
+		{"bge-large-en-v1.5", "embedding"},
+		{"e5-large-v2", "embedding"},
+		{"gte-large", "embedding"},
+		{"all-minilm", "embedding"},
+		{"all-MiniLM-L6-v2", "embedding"},
+		// Namespaced IDs classify by the final path segment.
+		{"BAAI/bge-m3", "embedding"},
+		{"intfloat/e5-mistral-7b-instruct", "embedding"},
+		// Rerankers.
+		{"bge-reranker-v2-m3", "rerank"},
+		{"jina-reranker-v2", "rerank"},
+		// Token matching must not fire on lookalike substrings.
+		{"gemma-3n-e4b", ""},
+		{"bge2000-chat", ""},
+		{"gte", "embedding"}, // bare family name still counts
+		// Ordinary chat models stay uninferred.
+		{"gpt-4o", ""},
+		{"llama-3.1-8b-instruct", ""},
+		{"qwen2.5-coder:7b", ""},
+		{"", ""},
+		{"  ", ""},
+		{"org/", ""},
+	}
+	for _, tt := range tests {
+		got := InferModesFromID(tt.id)
+		switch {
+		case tt.want == "" && len(got) != 0:
+			t.Errorf("InferModesFromID(%q) = %v, want none", tt.id, got)
+		case tt.want != "" && (len(got) != 1 || got[0] != tt.want):
+			t.Errorf("InferModesFromID(%q) = %v, want [%s]", tt.id, got, tt.want)
+		}
+	}
+}

internal/modeldata/merge.go

@@ -20,7 +20,9 @@ func MergeMetadata(base, override *core.ModelMetadata) *core.ModelMetadata {
 		return base.Clone()
 	}
 	if base == nil {
-		return override.Clone()
+		merged := override.Clone()
+		deriveCategoriesFromModes(merged, override)
+		return merged
 	}
 
 	merged := base.Clone()
@@ -40,6 +42,7 @@ func MergeMetadata(base, override *core.ModelMetadata) *core.ModelMetadata {
 	if len(override.Categories) > 0 {
 		merged.Categories = append([]core.ModelCategory(nil), override.Categories...)
 	}
+	deriveCategoriesFromModes(merged, override)
 	if len(override.Tags) > 0 {
 		merged.Tags = append([]string(nil), override.Tags...)
 	}
@@ -80,6 +83,17 @@ func MergeMetadata(base, override *core.ModelMetadata) *core.ModelMetadata {
 	return merged
 }
 
+// deriveCategoriesFromModes keeps Categories consistent when an override
+// declares Modes without Categories. Categories are derived data (the
+// dashboard's category filter and failover suggestions read them), so an
+// operator writing `modes: [embedding]` must end up with the embedding
+// category rather than the base's stale categories or none at all.
+func deriveCategoriesFromModes(merged, override *core.ModelMetadata) {
+	if len(override.Modes) > 0 && len(override.Categories) == 0 {
+		merged.Categories = core.CategoriesForModes(merged.Modes)
+	}
+}
+
 func clonePricingSources(in map[string]string) map[string]string {
 	if len(in) == 0 {
 		return nil

internal/modeldata/merge_categories_test.go

@@ -0,0 +1,50 @@
+package modeldata
+
+import (
+	"testing"
+
+	"github.com/enterpilot/gomodel/internal/core"
+)
+
+// Categories are derived data: an operator declaring modes in config must get
+// the matching categories without knowing about the internal categories field.
+func TestMergeMetadata_DerivesCategoriesFromOverrideModes(t *testing.T) {
+	t.Run("nil base", func(t *testing.T) {
+		merged := MergeMetadata(nil, &core.ModelMetadata{Modes: []string{"embedding"}})
+		if len(merged.Categories) != 1 || merged.Categories[0] != core.CategoryEmbedding {
+			t.Errorf("Categories = %v, want [embedding]", merged.Categories)
+		}
+	})
+
+	t.Run("replaces stale base categories", func(t *testing.T) {
+		base := &core.ModelMetadata{
+			Modes:      []string{"chat"},
+			Categories: []core.ModelCategory{core.CategoryTextGeneration},
+		}
+		merged := MergeMetadata(base, &core.ModelMetadata{Modes: []string{"embedding"}})
+		if len(merged.Modes) != 1 || merged.Modes[0] != "embedding" {
+			t.Errorf("Modes = %v, want [embedding]", merged.Modes)
+		}
+		if len(merged.Categories) != 1 || merged.Categories[0] != core.CategoryEmbedding {
+			t.Errorf("Categories = %v, want [embedding]", merged.Categories)
+		}
+	})
+
+	t.Run("explicit override categories win", func(t *testing.T) {
+		merged := MergeMetadata(nil, &core.ModelMetadata{
+			Modes:      []string{"embedding"},
+			Categories: []core.ModelCategory{core.CategoryUtility},
+		})
+		if len(merged.Categories) != 1 || merged.Categories[0] != core.CategoryUtility {
+			t.Errorf("Categories = %v, want [utility]", merged.Categories)
+		}
+	})
+
+	t.Run("no modes leaves base categories alone", func(t *testing.T) {
+		base := &core.ModelMetadata{Categories: []core.ModelCategory{core.CategoryTextGeneration}}
+		merged := MergeMetadata(base, &core.ModelMetadata{DisplayName: "X"})
+		if len(merged.Categories) != 1 || merged.Categories[0] != core.CategoryTextGeneration {
+			t.Errorf("Categories = %v, want [text_generation]", merged.Categories)
+		}
+	})
+}

internal/providers/cohere/cohere.go

@@ -90,9 +90,17 @@ func (p *Provider) ListModels(ctx context.Context) (*core.ModelsResponse, error)
 			continue
 		}
 		var metadata *core.ModelMetadata
-		if model.ContextLength > 0 {
-			contextWindow := int(model.ContextLength)
-			metadata = &core.ModelMetadata{ContextWindow: &contextWindow}
+		modes := modesFromEndpoints(model.Endpoints)
+		if model.ContextLength > 0 || len(modes) > 0 {
+			metadata = &core.ModelMetadata{}
+			if model.ContextLength > 0 {
+				contextWindow := int(model.ContextLength)
+				metadata.ContextWindow = &contextWindow
+			}
+			if len(modes) > 0 {
+				metadata.Modes = modes
+				metadata.Categories = core.CategoriesForModes(modes)
+			}
 		}
 		models = append(models, core.Model{
 			ID:       model.Name,
@@ -104,6 +112,30 @@ func (p *Provider) ListModels(ctx context.Context) (*core.ModelsResponse, error)
 	return &core.ModelsResponse{Object: "list", Data: models}, nil
 }
 
+// modesFromEndpoints maps Cohere's per-model endpoints list onto gateway mode
+// strings so models are classified from discovery even when the remote model
+// registry lacks an entry. Endpoints without a gateway surface are skipped;
+// registry enrichment and operator config still override this stamp.
+func modesFromEndpoints(endpoints []string) []string {
+	modes := make([]string, 0, len(endpoints))
+	for _, endpoint := range endpoints {
+		switch strings.ToLower(strings.TrimSpace(endpoint)) {
+		case "chat":
+			modes = append(modes, "chat")
+		case "embed":
+			modes = append(modes, "embedding")
+		case "rerank":
+			modes = append(modes, "rerank")
+		case "transcriptions":
+			modes = append(modes, "audio_transcription")
+		}
+	}
+	if len(modes) == 0 {
+		return nil
+	}
+	return modes
+}
+
 func supportedModel(model modelInfo) bool {
 	if len(model.Endpoints) == 0 {
 		return true

internal/providers/cohere/cohere_test.go

@@ -585,6 +585,28 @@ func TestListModelsFiltersUnsupportedEndpointsAndRotatesKeys(t *testing.T) {
 			*resp.Data[0].Metadata.ContextWindow != 128000 {
 			t.Fatalf("model metadata = %#v", resp.Data[0].Metadata)
 		}
+		wantModes := map[string][]string{
+			"command-a":                 {"chat"},
+			"embed-v4.0":                {"embedding"},
+			"cohere-transcribe-03-2026": {"audio_transcription"},
+			"legacy-unknown":            nil,
+		}
+		for _, model := range resp.Data {
+			want := wantModes[model.ID]
+			var got []string
+			if model.Metadata != nil {
+				got = model.Metadata.Modes
+			}
+			if len(got) != len(want) || (len(want) > 0 && got[0] != want[0]) {
+				t.Errorf("%s Modes = %v, want %v", model.ID, got, want)
+			}
+			if len(want) > 0 {
+				cats := core.CategoriesForModes(want)
+				if model.Metadata == nil || len(model.Metadata.Categories) != len(cats) || model.Metadata.Categories[0] != cats[0] {
+					t.Errorf("%s Categories = %+v, want %v", model.ID, model.Metadata, cats)
+				}
+			}
+		}
 	}
 	if len(headers) != 2 || headers[0] != "Bearer first" || headers[1] != "Bearer second" {
 		t.Fatalf("Authorization headers = %#v", headers)