fix(embeddings): validate width per model and stop guessing an Ollama target

Third round of review findings:

- Require an explicit OLLAMA_URL before resolving a model's width, matching what
  the embedding client already demands, so a loopback server cannot produce a
  knowledge base every later call refuses to serve
- Validate EMBEDDING_OUTPUT_DIMS against the selected model rather than its
  family, since the family pattern accepted widths the model silently overrides
- Raise a validation error for a missing model or unreportable width, so the
  route renders 400 instead of a generic 500
- Stop the Embeddings block auto-persisting whichever model Ollama lists first
- Do not offer back a configured value that the provider being set up rejects,
  so switching families cannot carry the previous family's model forward
- Cancel the response body of a rejected Ollama probe
- Make the drop-on-inspection-failure test actually exercise that path
- Correct the setup provider list, the older-Ollama filtering caveat, and the
  Sim Cloud OLLAMA_URL claim in the docs

Author: icecrasher321

apps/docs/content/docs/integrations/embeddings.mdx

@@ -19,7 +19,7 @@ Models differ in what they are good at and what they cost. `text-embedding-3-sma
 
 Two things worth knowing before you build on it. Vectors are only comparable when they come from the same model at the same size, so changing either means re-embedding everything you intend to compare. And input longer than the model's limit is shortened to fit rather than rejected, with a warning in the run, so chunk long documents yourself when the tail matters.
 
-Ollama is the exception to most of the above. It runs on your own deployment, so it needs no API key and adds no provider charge — Sim's own per-run charge still applies — and the model list is whatever you have pulled onto that server rather than a catalog Sim maintains. The block reads it live, drops the models that report a non-embedding capability, and shows each one's vector width next to its name where Ollama reports one. A server too old to report either will list its chat models too and label none of them, so check the model you pick. The block offers no task-type or dimension control for Ollama: task conditioning has no equivalent there, and while recent Ollama builds do accept a dimension override for Matryoshka models, older ones silently ignore it, so Sim uses each model's own width rather than one that may or may not take effect. Self-hosted deployments configure the server with `OLLAMA_URL`; on Sim Cloud there is no Ollama to reach, so the list comes back empty.
+Ollama is the exception to most of the above. It runs on your own deployment, so it needs no API key and adds no provider charge — Sim's own per-run charge still applies — and the model list is whatever you have pulled onto that server rather than a catalog Sim maintains. The block reads it live, drops the models that report a non-embedding capability, and shows each one's vector width next to its name where Ollama reports one. A server too old to report either will list its chat models too and label none of them, so check the model you pick. The block offers no task-type or dimension control for Ollama: task conditioning has no equivalent there, and while recent Ollama builds do accept a dimension override for Matryoshka models, older ones silently ignore it, so Sim uses each model's own width rather than one that may or may not take effect. Point Sim at the server with `OLLAMA_URL`. Sim Cloud runs no Ollama of its own, so without that variable the list comes back empty rather than dialling a loopback address that cannot answer — set it to a reachable server and Cloud will use it like any other deployment.
 
 Sim's knowledge bases embed separately: a base fixes one model and one vector width when it is created, from a smaller set of models. This block is for embedding text yourself inside a workflow.
 {/* MANUAL-CONTENT-END */}

apps/docs/content/docs/platform/self-hosting/environment-variables.mdx

@@ -214,15 +214,18 @@ surfaces as a failed document instead. Common pairings:
 | `3072` | `text-embedding-3-large`, `gemini-embedding-001` |
 
 On a Compose install or source checkout, `sim-setup add knowledge-embeddings` walks through all of
-this — pick OpenAI, Azure OpenAI, Gemini, or Ollama and it writes the variables that family needs.
+this — pick OpenAI, Azure OpenAI, OpenRouter, Gemini, or Ollama and it writes the variables that
+family needs. The three OpenAI-family transports share the same model and width prompts.
 `sim-setup config` then reports the one family your `KB_EMBEDDING_MODEL` actually selects, rather
 than every provider you happen to hold a key for. Helm releases set these values through your own
 chart values instead, and an Ollama server is yours to run either way — `sim-setup` configures Sim
 to reach one, never installs it.
 
 The Embeddings block reads the same `OLLAMA_URL`. It lists the models on that server that report an
 embedding capability, with the width each one emits where Ollama reports it, so a workflow can embed
-locally without an API key.
+locally without an API key. Ollama only began reporting capabilities in 0.5 — against an older
+server nothing can be filtered, so the list includes chat models and labels none of them with a
+width. Check what you pick there, or upgrade Ollama.
 
 ## Chat & PII
 

apps/sim/blocks/blocks/embeddings.ts

@@ -101,6 +101,14 @@ MODEL_SUB_BLOCKS.push({
   selectorKey: 'providers.ollamaEmbeddingModels',
   placeholder: 'Select a model on your Ollama server',
   required: true,
+  /**
+   * Suppresses the combobox's auto-select-first-option behaviour. Without it,
+   * opening the block persists whichever model the server happens to list
+   * first — and since each one emits a different width, that is a silent wrong
+   * answer rather than a harmless default. `required` still stands: the user
+   * must choose, they just are not chosen for.
+   */
+  emptyIsValid: true,
   condition: { field: 'provider', value: 'ollama' },
   dependsOn: ['provider'],
 })

apps/sim/lib/embeddings/knowledge-embedding-family.test.ts

@@ -2,11 +2,17 @@
  * @vitest-environment node
  */
 import { describe, expect, it } from 'vitest'
-import { knowledgeEmbeddingFamily } from '@/lib/core/config/env-capabilities'
+import {
+  inspectCapability,
+  KNOWLEDGE_EMBEDDINGS_CAPABILITY,
+  knowledgeEmbeddingFamily,
+} from '@/lib/core/config/env-capabilities'
 import {
   DEFAULT_EMBEDDING_MODEL,
   getEmbeddingModelInfo,
   getKbEligibleModels,
+  getKbEmbeddingDimensions,
+  KB_EMBEDDING_STORAGE_DIMENSIONS,
 } from '@/lib/embeddings/catalog'
 import { isKbEmbeddingModel } from '@/lib/knowledge/embedding-models'
 
@@ -28,6 +34,29 @@ describe('knowledgeEmbeddingFamily', () => {
     }
   })
 
+  /**
+   * The capability also duplicates each model's storable widths, to reject a
+   * width the selected model cannot emit. A width the catalog allows but the
+   * capability rejects blocks a working deployment; the reverse passes a
+   * configuration the runtime silently overrides.
+   */
+  it('accepts exactly the widths the catalog says each model can be indexed at', () => {
+    for (const model of getKbEligibleModels()) {
+      const widths = getKbEmbeddingDimensions(getEmbeddingModelInfo(model))
+      for (const width of KB_EMBEDDING_STORAGE_DIMENSIONS) {
+        const issues = inspectCapability(KNOWLEDGE_EMBEDDINGS_CAPABILITY, {
+          KB_EMBEDDING_MODEL: model,
+          EMBEDDING_OUTPUT_DIMS: String(width),
+          OPENAI_API_KEY: 'k',
+          GEMINI_API_KEY: 'k',
+        }).providers.flatMap((provider) => provider.invalidFields)
+        expect(issues.includes('EMBEDDING_OUTPUT_DIMS'), `${model} @ ${width}`).toBe(
+          !widths.includes(width)
+        )
+      }
+    }
+  })
+
   it('classifies any model on the deployment’s own Ollama by its routing prefix', () => {
     for (const model of ['ollama/nomic-embed-text', 'ollama/mxbai-embed-large:335m']) {
       expect(knowledgeEmbeddingFamily({ KB_EMBEDDING_MODEL: model }), model).toBe('ollama')

apps/sim/lib/embeddings/ollama-model-catalog.server.test.ts

@@ -92,20 +92,27 @@ describe('Ollama embedding model catalog', () => {
 
   it('keeps the rest of the catalog when one model cannot be inspected', async () => {
     serve({
-      broken: { capabilities: ['embedding'] },
       'all-minilm:latest': {
         capabilities: ['embedding'],
         modelInfo: { 'bert.embedding_length': 384 },
       },
     })
-    fetchMock.mockImplementationOnce(() =>
-      Promise.resolve(
-        Response.json({ models: [{ name: 'broken' }, { name: 'all-minilm:latest' }] })
-      )
-    )
+    const served = fetchMock.getMockImplementation()
+    /** `broken` has to fail its own `/api/show`, or the skip path never runs. */
+    fetchMock.mockImplementation((url: string, init?: RequestInit) => {
+      if (String(url).endsWith('/api/tags')) {
+        return Promise.resolve(
+          Response.json({ models: [{ name: 'broken' }, { name: 'all-minilm:latest' }] })
+        )
+      }
+      const { model } = JSON.parse(String(init?.body)) as { model: string }
+      if (model === 'broken') return Promise.reject(new Error('inspection failed'))
+      return served?.(url, init)
+    })
 
-    const catalog = await fetchOllamaEmbeddingModelCatalog()
-    expect(catalog).toContainEqual({ id: 'all-minilm:latest', dimensions: 384 })
+    await expect(fetchOllamaEmbeddingModelCatalog()).resolves.toEqual([
+      { id: 'all-minilm:latest', dimensions: 384 },
+    ])
   })
 
   it('offers nothing rather than throwing when no Ollama answers', async () => {

apps/sim/lib/embeddings/ollama-model-catalog.server.ts

@@ -60,6 +60,8 @@ async function fetchOllamaJson(path: string, init: RequestInit, signal?: AbortSi
     ...init,
   })
   if (!response.ok) {
+    /** Release the connection: nothing reads the body of a rejected probe. */
+    await response.body?.cancel().catch(() => {})
     throw new Error(`Ollama ${path} failed: ${response.status} ${response.statusText}`)
   }
   return readResponseJsonWithLimit(response, {
@@ -175,6 +177,15 @@ export async function getOllamaEmbeddingModelMetadata(
 ): Promise<Required<OllamaEmbeddingModel>> {
   const name = isOllamaEmbeddingModel(model) ? ollamaEmbeddingModelName(model) : model
   if (!name) throw new OllamaEmbeddingModelNotFoundError(model)
+  /**
+   * The same requirement `resolveProvider` enforces before embedding. Without it
+   * a developer machine with an unset `OLLAMA_URL` and a local server answering
+   * on the loopback default would resolve a width here and create a knowledge
+   * base that every later embedding call refuses to serve.
+   */
+  if (!isOllamaUrlConfigured()) {
+    throw new OllamaUnreachableError('OLLAMA_URL is not configured')
+  }
 
   let detail: { capabilities?: string[]; model_info?: Record<string, unknown> }
   try {

apps/sim/lib/knowledge/embeddings-configuration.test.ts

@@ -2,6 +2,8 @@
  * @vitest-environment node
  */
 import { beforeEach, describe, expect, it, vi } from 'vitest'
+import { OrchestrationError } from '@/lib/core/orchestration/types'
+import { OllamaEmbeddingModelNotFoundError } from '@/lib/embeddings/ollama-model-catalog.server'
 
 const { mockEnv, mockOllamaMetadata } = vi.hoisted(() => ({
   mockEnv: {} as { KB_EMBEDDING_MODEL?: string; EMBEDDING_OUTPUT_DIMS?: string },
@@ -18,7 +20,8 @@ vi.mock('@/lib/core/config/env', async (importOriginal) => ({
   env: mockEnv,
 }))
 
-vi.mock('@/lib/embeddings/ollama-model-catalog.server', () => ({
+vi.mock('@/lib/embeddings/ollama-model-catalog.server', async (importOriginal) => ({
+  ...(await importOriginal<typeof import('@/lib/embeddings/ollama-model-catalog.server')>()),
   getOllamaEmbeddingModelMetadata: mockOllamaMetadata,
 }))
 
@@ -98,6 +101,17 @@ describe('getConfiguredKbEmbedding', () => {
     await expect(getConfiguredKbEmbedding()).rejects.toThrow('Set EMBEDDING_OUTPUT_DIMS')
   })
 
+  /**
+   * A model the server does not have is the operator's to fix, so it is a
+   * validation failure the route renders as 400 rather than a generic 500.
+   */
+  it('classifies a missing model as caller-fixable', async () => {
+    mockEnv.KB_EMBEDDING_MODEL = 'ollama/not-pulled'
+    mockOllamaMetadata.mockRejectedValue(new OllamaEmbeddingModelNotFoundError('not-pulled'))
+
+    await expect(getConfiguredKbEmbedding()).rejects.toBeInstanceOf(OrchestrationError)
+  })
+
   it('refuses to create a base when the model emits a width no column can store', async () => {
     mockEnv.KB_EMBEDDING_MODEL = 'ollama/odd-width'
     mockOllamaMetadata.mockResolvedValue({ id: 'odd-width:latest', dimensions: 1152 })

apps/sim/lib/knowledge/embeddings.ts

@@ -7,9 +7,14 @@ import {
 import { recordUsage } from '@/lib/billing/core/usage-log'
 import { checkAndBillPayerOverageThreshold } from '@/lib/billing/threshold-billing'
 import { env, envNumber } from '@/lib/core/config/env'
+import { OrchestrationError } from '@/lib/core/orchestration/types'
 import { embedKnowledge } from '@/lib/embeddings'
 import { isOllamaEmbeddingModel } from '@/lib/embeddings/catalog'
-import { getOllamaEmbeddingModelMetadata } from '@/lib/embeddings/ollama-model-catalog.server'
+import {
+  getOllamaEmbeddingModelMetadata,
+  OllamaEmbeddingModelNotFoundError,
+  OllamaEmbeddingWidthUnknownError,
+} from '@/lib/embeddings/ollama-model-catalog.server'
 import {
   assertKbEmbeddingModel,
   DEFAULT_EMBEDDING_MODEL,
@@ -122,12 +127,25 @@ export async function getConfiguredKbEmbedding(): Promise<KbEmbeddingTarget> {
     try {
       dimensions = (await getOllamaEmbeddingModelMetadata(model)).dimensions
     } catch (error) {
-      throw new Error(
-        `Could not read the vector width of ${model} from the configured Ollama server (${getErrorMessage(error, 'Unknown error')}). Set EMBEDDING_OUTPUT_DIMS to the width it emits.`
-      )
+      /**
+       * A model the server does not have, or one whose width it will not report,
+       * is the operator's to fix and is surfaced as such. An unreachable server
+       * is a dependency failure and keeps its default classification — the
+       * orchestration vocabulary has no upstream-failure code, and the message
+       * carries the cause either way.
+       */
+      const message = `Could not read the vector width of ${model} from the configured Ollama server (${getErrorMessage(error, 'Unknown error')}). Set EMBEDDING_OUTPUT_DIMS to the width it emits.`
+      if (
+        error instanceof OllamaEmbeddingModelNotFoundError ||
+        error instanceof OllamaEmbeddingWidthUnknownError
+      ) {
+        throw new OrchestrationError('validation', message)
+      }
+      throw new Error(message, { cause: error })
     }
     if (!isKbEmbeddingDimensions(dimensions)) {
-      throw new Error(
+      throw new OrchestrationError(
+        'validation',
         `${model} emits ${dimensions}-dimensional vectors, which knowledge bases cannot store. Choose a model emitting one of ${KB_EMBEDDING_STORAGE_DIMENSIONS.join(', ')}.`
       )
     }

packages/deployment-config/src/env-capabilities.ts

@@ -1262,9 +1262,46 @@ export type KnowledgeEmbeddingFamily = 'openai' | 'gemini' | 'ollama'
 
 const OLLAMA_EMBEDDING_MODEL_PREFIX = 'ollama/'
 
+/**
+ * Widths each knowledge-base-eligible model can be indexed at, as the app
+ * catalog defines them. Duplicated here because packages cannot import the
+ * catalog; the parity test pins both halves, widths included.
+ */
+const KB_EMBEDDING_MODEL_WIDTHS: Readonly<Record<string, readonly number[]>> = {
+  'text-embedding-3-small': [1536, 1024, 768],
+  'text-embedding-3-large': [3072, 1536, 1024, 768],
+  'gemini-embedding-001': [3072, 1536, 768],
+}
+
 /** Knowledge-base-eligible Gemini model ids, as the app catalog defines them. */
 const GEMINI_KB_EMBEDDING_MODELS: readonly string[] = ['gemini-embedding-001']
 
+/**
+ * Rejects a width the *selected model* cannot emit, which the per-field pattern
+ * cannot: that pattern is the union across a family, so `text-embedding-3-small`
+ * at 3,072 satisfies it while the runtime falls back to 1,536 with a warning.
+ * A validation that passes for a configuration the runtime silently overrides is
+ * worse than none, so the two are checked together here.
+ */
+function validateEmbeddingModelWidth(
+  values: EnvCapabilityValues
+): readonly EnvProviderValidationIssue[] {
+  const model = String(readValue(values, 'KB_EMBEDDING_MODEL') ?? '')
+  const rawWidth = readValue(values, 'EMBEDDING_OUTPUT_DIMS')
+  if (!hasValue(values, 'EMBEDDING_OUTPUT_DIMS')) return []
+
+  const widths = KB_EMBEDDING_MODEL_WIDTHS[model]
+  if (!widths) return []
+  if (widths.includes(Number(String(rawWidth).trim()))) return []
+  return [
+    {
+      kind: 'invalid',
+      fields: ['EMBEDDING_OUTPUT_DIMS'],
+      message: `EMBEDDING_OUTPUT_DIMS must be one of ${widths.join(', ')} for ${model}`,
+    },
+  ]
+}
+
 export function knowledgeEmbeddingFamily(values: EnvCapabilityValues): KnowledgeEmbeddingFamily {
   /**
    * Read raw, not trimmed: the runtime looks the value up exactly as configured,
@@ -1346,6 +1383,7 @@ export const KNOWLEDGE_EMBEDDINGS_CAPABILITY = defineCapability({
         envField('KB_EMBEDDING_MODEL'),
         embeddingOutputDimsField(OPENAI_EMBEDDING_WIDTHS),
       ],
+      validate: validateEmbeddingModelWidth,
     },
     {
       id: 'openai',
@@ -1365,6 +1403,7 @@ export const KNOWLEDGE_EMBEDDINGS_CAPABILITY = defineCapability({
         envField('KB_EMBEDDING_MODEL'),
         embeddingOutputDimsField(OPENAI_EMBEDDING_WIDTHS),
       ],
+      validate: validateEmbeddingModelWidth,
     },
     {
       id: 'openrouter',
@@ -1381,6 +1420,7 @@ export const KNOWLEDGE_EMBEDDINGS_CAPABILITY = defineCapability({
         envField('KB_EMBEDDING_MODEL'),
         embeddingOutputDimsField(OPENAI_EMBEDDING_WIDTHS),
       ],
+      validate: validateEmbeddingModelWidth,
     },
     {
       id: 'gemini',
@@ -1400,6 +1440,7 @@ export const KNOWLEDGE_EMBEDDINGS_CAPABILITY = defineCapability({
         envField('KB_EMBEDDING_MODEL')
       ),
       optionalFields: [embeddingOutputDimsField(GEMINI_EMBEDDING_WIDTHS)],
+      validate: validateEmbeddingModelWidth,
     },
     {
       /**
@@ -1419,7 +1460,18 @@ export const KNOWLEDGE_EMBEDDINGS_CAPABILITY = defineCapability({
             message: 'must be a valid HTTP(S) URL',
           },
         }),
-        envField('KB_EMBEDDING_MODEL')
+        /**
+         * Constrained to the routing prefix, which is what makes it an Ollama
+         * model at all. It also stops the setup flow carrying a model from
+         * another family forward when an existing install switches to Ollama.
+         */
+        envField('KB_EMBEDDING_MODEL', {
+          validation: {
+            kind: 'pattern',
+            pattern: /^ollama\/.+/,
+            message: 'must name a model on your Ollama server, as ollama/<model>',
+          },
+        })
       ),
       optionalFields: [embeddingOutputDimsField(OLLAMA_EMBEDDING_WIDTHS)],
     },

packages/sim-setup/src/capability-setup.ts

@@ -404,7 +404,19 @@ async function promptField(
   const existing = state.currentValues.get(prompt.key)
   const current = hasEnvCapabilityValue(state.currentValues, prompt.key)
   const draft = getCapabilitySetupDraftValues(state.setup, state.optionId, state.values)
-  const initialValue = existing ?? draft[prompt.key] ?? prompt.defaultValue
+  /**
+   * A configured value is only offered back when it is valid for the provider
+   * being configured. Switching families otherwise pre-fills the previous
+   * family's model, and accepting the displayed value silently keeps it — the
+   * transition then rejects the provider the operator just chose.
+   */
+  const reusableExisting =
+    existing !== undefined &&
+    validateCapabilityFieldInput(state.setup.definition, prompt.key, existing, state.optionId) ===
+      undefined
+      ? existing
+      : undefined
+  const initialValue = reusableExisting ?? draft[prompt.key] ?? prompt.defaultValue
   const validate = (value: string): string | undefined => {
     if (!value) return prompt.required && !existing ? 'required' : undefined
     return prompt.validate