feat(llamacpp): surface model context window and modalities at /v1/models

docs/advanced/model-metadata.mdx

@@ -13,8 +13,8 @@ strongest first:
 ```mermaid
 flowchart LR
     A[Pricing overrides] -->|pricing fields only| B[config.yaml metadata]
-    B -->|declared fields win| C[ai-model-list catalog]
-    C -->|when it knows the model| D[Provider discovery]
+    B -->|declared fields win| C[Provider discovery]
+    C -->|fields the provider reports| D[ai-model-list catalog]
     D -.->|modes still missing| E[ID heuristic]
 ```
 
@@ -28,20 +28,25 @@ flowchart LR
    fields win field-by-field over everything below; omitted fields inherit. This
    is the escape hatch for local models: declaring `modes: [embedding]` also
    derives the model's category.
-3. **The model catalog** — the
+3. **Provider discovery** — some providers report metadata in their own model
+   listings, and what a provider says about its own deployment wins over the
+   catalog field by field: Gemini's `supportedGenerationMethods`, Cohere's
+   per-model `endpoints` and context length, OpenRouter's architecture
+   modalities and context length, Chutes' context length, max output,
+   capabilities, and pricing, Ollama's `/api/show` capabilities, and llama.cpp's
+   context window and modalities (see [llama.cpp](/providers/llamacpp)). This
+   matters most for self-hosted servers, where the running process is the only
+   source that can know the real context window. Models declared via configured
+   model lists skip this step.
+4. **The model catalog** — the
    [`ai-model-list`](https://github.com/ENTERPILOT/ai-model-list) registry,
    fetched from `MODEL_LIST_URL` (default: the registry's `models.min.json` on
    GitHub) at startup and on every catalog refresh. It supplies the rich
    defaults — pricing, context windows, capabilities, modes — for most hosted
    models, matching IDs directly, through aliases, and with release-date
-   suffixes stripped. Wrong or missing data is best fixed by contributing to the
-   registry; use an override for an immediate fix.
-4. **Provider discovery** — some providers report capabilities in their own
-   model listings, and GoModel keeps them for the models it discovers there:
-   Gemini's `supportedGenerationMethods`, Cohere's per-model `endpoints` and
-   context length, OpenRouter's architecture modalities and context length, and
-   Ollama's `/api/show` capabilities. Models declared via configured model
-   lists skip this step.
+   suffixes stripped, and it fills in every field the provider above did not
+   report. Wrong or missing data is best fixed by contributing to the registry;
+   use an override for an immediate fix.
 5. **ID heuristic** — a last-resort name check for models that end up with no
    modes at all (typical for llama.cpp and LM Studio): IDs containing `embed` or
    matching well-known embedding families (`bge`, `e5`, `gte`, `minilm`) become

docs/providers/llamacpp.mdx

@@ -75,17 +75,41 @@ curl -s http://localhost:8080/v1/embeddings \
   -d '{"model": "llamacpp/nomic-embed-text-v1.5", "input": "hello"}'
 ```
 
+## Model metadata
+
+Local GGUFs are not in the upstream model catalog, so GoModel reads what it can
+from llama-server itself and reports it on `GET /v1/models`:
+
+- **Context window** — the per-slot `n_ctx` llama-server reports for the model,
+  which is the context it is *running* with (`--ctx-size`, divided across
+  `--parallel` slots) rather than the one the GGUF was trained for. Recent
+  builds report it as `meta.n_ctx` in the listing itself; older ones are asked
+  for it via `/props`. If neither answers — LM Studio, or a proxy that hides
+  `/props` — the model's trained `n_ctx_train` is used instead, which is only an
+  upper bound and may exceed what the server will accept.
+- **Modalities** — a multimodal server (started with `--mmproj`) reports
+  `vision`, `video`, and `audio` on `/props`; the supported ones become model
+  capabilities.
+
+`/props` describes the one model the server has loaded, so it is only consulted
+for a single-entry listing. In [router
+mode](https://github.com/ggml-org/llama.cpp/tree/master/tools/server), where one
+process serves several models, each model carries its own `meta.n_ctx` and needs
+no `/props` at all.
+
+Both are defaults, not decisions: anything you declare under the provider's
+model metadata wins — see [Model metadata](/advanced/model-metadata).
+
 ## Model classification
 
-llama-server's `/v1/models` listing carries no capability metadata, so GoModel
-classifies its models by ID: names containing `embed` or matching well-known
-embedding families (`bge`, `e5`, `gte`, `minilm`) are categorized as embedding
-models, and names containing `rerank` as reranking models — namespaced IDs are
-checked by their final path segment. For anything that stays unclassified,
-declare `modes` (and context window or pricing) under the provider's model
-metadata — see [Model metadata](/advanced/model-metadata). Categories only
-affect dashboard grouping and failover suggestions; `/v1/embeddings` routes to
-any model the provider serves regardless of category.
+llama-server reports nothing that separates a chat model from an embedding one,
+so GoModel classifies its models by ID: names containing `embed` or matching
+well-known embedding families (`bge`, `e5`, `gte`, `minilm`) are categorized as
+embedding models, and names containing `rerank` as reranking models — namespaced
+IDs are checked by their final path segment. For anything that stays
+unclassified, declare `modes` under the provider's model metadata. Categories
+only affect dashboard grouping and failover suggestions; `/v1/embeddings` routes
+to any model the provider serves regardless of category.
 
 ## Beyond chat and embeddings
 

internal/modeldata/enricher.go

@@ -14,6 +14,11 @@ type ModelInfoAccessor interface {
 	GetProviderType(modelID string) string
 	// SetMetadata sets the metadata for a model ID.
 	SetMetadata(modelID string, meta *core.ModelMetadata)
+	// DiscoveredMetadata returns the metadata the provider itself reported for
+	// a model, or nil when it reported none. It must keep returning the same
+	// value across enrichment passes: Enrich merges onto it rather than onto
+	// its own previous result, which is what keeps repeated passes idempotent.
+	DiscoveredMetadata(modelID string) *core.ModelMetadata
 }
 
 // EnrichStats summarizes one metadata enrichment pass.
@@ -22,8 +27,15 @@ type EnrichStats struct {
 	Total    int
 }
 
-// Enrich iterates all models accessible via the accessor and attaches resolved
-// metadata from the model list. Models not found in the list are left unchanged.
+// Enrich iterates all models accessible via the accessor and merges resolved
+// catalog metadata into each one. Models the catalog does not know are left
+// unchanged.
+//
+// The catalog is the base and the provider's own report the override, field by
+// field: a running provider describes its actual deployment (a local server's
+// real context window, an API's live capability flags) better than a static
+// registry can, while the catalog still supplies everything the provider never
+// reports — display names, pricing, rankings.
 func Enrich(accessor ModelInfoAccessor, list *ModelList) EnrichStats {
 	if list == nil || accessor == nil {
 		return EnrichStats{}
@@ -34,11 +46,17 @@ func Enrich(accessor ModelInfoAccessor, list *ModelList) EnrichStats {
 
 	for _, modelID := range ids {
 		providerType := accessor.GetProviderType(modelID)
-		meta := Resolve(list, providerType, modelID)
-		if meta != nil {
-			accessor.SetMetadata(modelID, meta)
-			enriched++
+		discovered := accessor.DiscoveredMetadata(modelID)
+		catalog := Resolve(list, providerType, modelID)
+		if catalog == nil {
+			// The catalog does not know this model. Fall back to the provider's
+			// own report rather than leaving whatever a previous pass merged in:
+			// a refresh that drops an entry must drop its fields too.
+			accessor.SetMetadata(modelID, discovered.Clone())
+			continue
 		}
+		accessor.SetMetadata(modelID, MergeMetadata(catalog, discovered))
+		enriched++
 	}
 
 	return EnrichStats{

internal/modeldata/enricher_test.go

@@ -11,12 +11,14 @@ type mockAccessor struct {
 	ids           []string
 	providerTypes map[string]string
 	metadata      map[string]*core.ModelMetadata
+	discovered    map[string]*core.ModelMetadata
 }
 
 func newMockAccessor(models map[string]string) *mockAccessor {
 	a := &mockAccessor{
 		providerTypes: models,
 		metadata:      make(map[string]*core.ModelMetadata),
+		discovered:    make(map[string]*core.ModelMetadata),
 	}
 	for id := range models {
 		a.ids = append(a.ids, id)
@@ -30,6 +32,10 @@ func (a *mockAccessor) SetMetadata(modelID string, meta *core.ModelMetadata) {
 	a.metadata[modelID] = meta
 }
 
+func (a *mockAccessor) DiscoveredMetadata(modelID string) *core.ModelMetadata {
+	return a.discovered[modelID]
+}
+
 func TestEnrich_MatchedAndUnmatched(t *testing.T) {
 	list := &ModelList{
 		Models: map[string]ModelEntry{
@@ -64,14 +70,13 @@ func TestEnrich_MatchedAndUnmatched(t *testing.T) {
 		}
 	}
 
-	// unknown-model should NOT be enriched
-	if _, ok := accessor.metadata["unknown-model"]; ok {
-		t.Error("expected unknown-model to NOT be enriched")
+	// Models the catalog does not know keep only what their provider reported,
+	// which here is nothing.
+	if meta := accessor.metadata["unknown-model"]; meta != nil {
+		t.Errorf("expected unknown-model to carry no catalog metadata, got %+v", meta)
 	}
-
-	// custom-finetune should NOT be enriched
-	if _, ok := accessor.metadata["custom-finetune"]; ok {
-		t.Error("expected custom-finetune to NOT be enriched")
+	if meta := accessor.metadata["custom-finetune"]; meta != nil {
+		t.Errorf("expected custom-finetune to carry no catalog metadata, got %+v", meta)
 	}
 }
 
@@ -165,3 +170,91 @@ func TestEnrich_ProviderModelOverride(t *testing.T) {
 		t.Errorf("ContextWindow = %d, want 64000 (azure override)", *meta.ContextWindow)
 	}
 }
+
+func TestEnrich_ProviderDiscoveryWinsFieldWiseOverCatalog(t *testing.T) {
+	accessor := newMockAccessor(map[string]string{"gemma-3-4b-it": "llamacpp"})
+	// What a local server reported about the model it is actually running.
+	accessor.discovered["gemma-3-4b-it"] = &core.ModelMetadata{
+		ContextWindow: new(4096),
+		Capabilities:  map[string]bool{"vision": true},
+	}
+	// A catalog entry that knows the bare ID, with no llamacpp entry at all.
+	list := &ModelList{Models: map[string]ModelEntry{
+		"gemma-3-4b-it": {
+			DisplayName:   "Gemma 3 4B IT",
+			ContextWindow: new(131072),
+			Capabilities:  map[string]bool{"tools": true},
+		},
+	}}
+
+	Enrich(accessor, list)
+
+	got := accessor.metadata["gemma-3-4b-it"]
+	if got == nil {
+		t.Fatal("metadata = nil, want merged metadata")
+	}
+	if got.ContextWindow == nil || *got.ContextWindow != 4096 {
+		t.Fatalf("context window = %v, want the running server's 4096", got.ContextWindow)
+	}
+	if !got.Capabilities["vision"] {
+		t.Fatal("discovered capability vision was dropped")
+	}
+	// Fields the provider never reports still come from the catalog.
+	if !got.Capabilities["tools"] {
+		t.Fatal("catalog capability tools was dropped")
+	}
+	if got.DisplayName != "Gemma 3 4B IT" {
+		t.Fatalf("display name = %q, want the catalog's", got.DisplayName)
+	}
+}
+
+func TestEnrich_RepeatedPassesTrackCatalogUpdates(t *testing.T) {
+	accessor := newMockAccessor(map[string]string{"gpt-4o": "openai"})
+	list := &ModelList{Models: map[string]ModelEntry{
+		"gpt-4o": {DisplayName: "GPT-4o", ContextWindow: new(128000)},
+	}}
+
+	Enrich(accessor, list)
+	if got := accessor.metadata["gpt-4o"]; got.ContextWindow == nil || *got.ContextWindow != 128000 {
+		t.Fatalf("first pass context window = %v, want 128000", got.ContextWindow)
+	}
+
+	// A later catalog refresh corrects the value. Because Enrich merges onto the
+	// provider's pristine report rather than onto its own previous output, the
+	// new value must win instead of being pinned by the stale one.
+	list.Models["gpt-4o"] = ModelEntry{DisplayName: "GPT-4o", ContextWindow: new(200000)}
+	Enrich(accessor, list)
+
+	if got := accessor.metadata["gpt-4o"]; got.ContextWindow == nil || *got.ContextWindow != 200000 {
+		t.Fatalf("second pass context window = %v, want the refreshed 200000", got.ContextWindow)
+	}
+}
+
+func TestEnrich_DropsCatalogFieldsWhenEntryDisappears(t *testing.T) {
+	accessor := newMockAccessor(map[string]string{"gemma-3-4b-it": "llamacpp"})
+	accessor.discovered["gemma-3-4b-it"] = &core.ModelMetadata{ContextWindow: new(4096)}
+	list := &ModelList{Models: map[string]ModelEntry{
+		"gemma-3-4b-it": {DisplayName: "Gemma 3 4B IT", ContextWindow: new(131072)},
+	}}
+
+	Enrich(accessor, list)
+	if got := accessor.metadata["gemma-3-4b-it"]; got.DisplayName != "Gemma 3 4B IT" {
+		t.Fatalf("first pass display name = %q, want the catalog's", got.DisplayName)
+	}
+
+	// The catalog drops the entry on a later refresh; its fields must go with
+	// it, leaving only what the provider itself reported.
+	delete(list.Models, "gemma-3-4b-it")
+	Enrich(accessor, list)
+
+	got := accessor.metadata["gemma-3-4b-it"]
+	if got == nil {
+		t.Fatal("metadata = nil, want the provider's own report to survive")
+	}
+	if got.DisplayName != "" {
+		t.Fatalf("display name = %q, want it dropped with the catalog entry", got.DisplayName)
+	}
+	if got.ContextWindow == nil || *got.ContextWindow != 4096 {
+		t.Fatalf("context window = %v, want the provider's 4096", got.ContextWindow)
+	}
+}

internal/providers/configured_models.go

@@ -160,12 +160,7 @@ func modelInfoMapFromResponse(resp *core.ModelsResponse, provider core.Provider,
 			continue
 		}
 		model.ID = modelID
-		out[modelID] = &ModelInfo{
-			Model:        model,
-			Provider:     provider,
-			ProviderName: providerName,
-			ProviderType: providerType,
-		}
+		out[modelID] = newModelInfo(model, provider, providerName, providerType)
 	}
 	return out
 }

internal/providers/llamacpp/llamacpp.go

@@ -33,6 +33,12 @@ var Registration = providers.Registration{
 type Provider struct {
 	compatible *openai.CompatibleProvider
 	rootClient *llmclient.Client
+	// propsClient issues the optional /props metadata call. It is deliberately
+	// separate from rootClient: /props is best-effort enrichment, so it must not
+	// spend the retry budget or trip the circuit breaker that native passthrough
+	// routes share. A server that fails /props with a retryable status would
+	// otherwise take /health, /rerank and /tokenize down with it.
+	propsClient *llmclient.Client
 }
 
 var (
@@ -63,9 +69,23 @@ func New(cfg providers.ProviderConfig, opts providers.ProviderOptions) core.Prov
 		}, func(req *http.Request) {
 			setHeaders(req, keys.NextForContext(req.Context()))
 		}),
+		propsClient: newPropsClient(baseURL, opts.Hooks, func(req *http.Request) {
+			setHeaders(req, keys.NextForContext(req.Context()))
+		}),
 	}
 }
 
+// newPropsClient builds the client used for optional /props enrichment: no
+// retries and no circuit breaker, so a failing /props costs one request and
+// leaves the shared native-route budget untouched.
+func newPropsClient(baseURL string, hooks llmclient.Hooks, setHeader llmclient.HeaderSetter) *llmclient.Client {
+	return llmclient.New(llmclient.Config{
+		ProviderName: "llamacpp",
+		BaseURL:      passthroughBaseURL(baseURL),
+		Hooks:        hooks,
+	}, setHeader)
+}
+
 // NewWithHTTPClient creates a new llama.cpp provider with a custom HTTP client.
 // If httpClient is nil, http.DefaultClient is used.
 func NewWithHTTPClient(apiKey string, baseURL string, httpClient *http.Client, hooks llmclient.Hooks) *Provider {
@@ -81,13 +101,21 @@ func NewWithHTTPClient(apiKey string, baseURL string, httpClient *http.Client, h
 		rootClient: llmclient.NewWithHTTPClient(httpClient, rootClientCfg, func(req *http.Request) {
 			setHeaders(req, apiKey)
 		}),
+		propsClient: llmclient.NewWithHTTPClient(httpClient, llmclient.Config{
+			ProviderName: "llamacpp",
+			BaseURL:      passthroughBaseURL(resolvedBaseURL),
+			Hooks:        hooks,
+		}, func(req *http.Request) {
+			setHeaders(req, apiKey)
+		}),
 	}
 }
 
 // SetBaseURL allows configuring a custom base URL for the provider.
 func (p *Provider) SetBaseURL(url string) {
 	p.compatible.SetBaseURL(url)
 	p.rootClient.SetBaseURL(passthroughBaseURL(url))
+	p.propsClient.SetBaseURL(passthroughBaseURL(url))
 }
 
 func setHeaders(req *http.Request, apiKey string) {
@@ -108,11 +136,6 @@ func (p *Provider) StreamChatCompletion(ctx context.Context, req *core.ChatReque
 	return p.compatible.StreamChatCompletion(ctx, req)
 }
 
-// ListModels retrieves the list of available models from llama-server.
-func (p *Provider) ListModels(ctx context.Context) (*core.ModelsResponse, error) {
-	return p.compatible.ListModels(ctx)
-}
-
 // Responses sends a Responses API request to llama-server.
 func (p *Provider) Responses(ctx context.Context, req *core.ResponsesRequest) (*core.ResponsesResponse, error) {
 	return p.compatible.Responses(ctx, req)