feat(knowledge): index Confluence image attachments

apps/docs/content/docs/knowledgebase/connectors.mdx

@@ -86,7 +86,7 @@ Each connector has source-specific fields that control what gets synced. Example
 - **Notion** — sync an entire workspace, a specific database, or a single page tree
 - **GitHub** — specify a repository, branch, and optional file extension filter
 - **GitLab** — specify a project path or ID and your instance host (leave blank for GitLab.com), then choose repository files, wiki pages, issues, or merge requests. Each connector syncs one project; submodules need their own connectors.
-- **Confluence** — enter your Atlassian domain and choose spaces, or **All** for all spaces accessible at each sync. Optionally filter by content type or label. PDF, Word (`.docx`, Word 97–2003 `.doc`), Excel (`.xlsx`), and PowerPoint (`.pptx`) attachments on matching pages and blog posts are included as separate documents.
+- **Confluence** — enter your Atlassian domain and choose spaces, or **All** for all spaces accessible at each sync. Optionally filter by content type or label. PDF, Word (`.docx`, Word 97–2003 `.doc`), Excel (`.xlsx`), and PowerPoint (`.pptx`) attachments on matching pages and blog posts are included as separate documents, and so are PNG, JPEG, WebP, and GIF images when **Image Attachments** is on.
 - **Azure DevOps** — choose what to sync (wiki pages, work items, repository files, or all), with optional work item type/state filters, a custom WIQL query, and repository/branch/path filters
 - **Amazon S3** — point at a bucket with an optional key prefix and a customizable file extension allowlist; S3-compatible stores (Cloudflare R2, MinIO) are supported via a custom endpoint
 - **YouTube** — sync a channel (by `@handle` or ID) or playlist, with an optional published-after date filter and the option to exclude Shorts
@@ -98,6 +98,8 @@ Configuration is validated on save — if a repository doesn't exist or a domain
 
 Confluence attachment indexing requires `read:attachment:confluence`. For a service account, include it when creating the scoped API token; see the [Confluence scope list](/search/confluence#using-a-service-account). Attachments are checked even when the parent page has not changed. Files over 100 MB appear as skipped; convert Word 6/95 files to `.docx`, and `.xls` and `.ppt` files to `.xlsx` and `.pptx`, before attaching them. Spaces that were already connected pick up newly supported formats on their next sync.
 
+Each image is read once by a vision model, which writes out what it holds: a table becomes one line per row with its column headers, a scan becomes its full text, and a screenshot or diagram becomes its visible text plus a description of what it shows. That text is what search finds. Logos, icons, and other decorative images are kept as documents but have nothing to search, so they show 0 chunks. Images under 2 KB, SVG, and HEIC files are not indexed. Image reading uses a Gemini key: the workspace's own key when one is added, otherwise the platform's, and each image is billed as a model call. Without any Gemini key, images are left out of the sync until one is added. One sync reads at most 500 new or changed images, and the rest are picked up by the next sync; a full resync does not read unchanged images again. New Confluence connectors read images by default. Connectors created before image support keep images off until you set **Image Attachments** to **Read with AI** in the connector's settings; the next sync then reads them. To stop indexing images, set it to **Don't index**: the next sync removes the images already indexed, and switching back reads them again, billed per image.
+
 </Step>
 <Step>
 

apps/docs/content/docs/platform/self-hosting/environment-variables.mdx

@@ -192,6 +192,9 @@ See [Observability](/platform/self-hosting/observability).
 
 | Variable | Description |
 |----------|-------------|
+| `KB_IMAGE_EXTRACTION_MODEL` | Gemini model that reads Confluence image attachments into searchable text. Defaults to `gemini-3.8-flash`; a value that is not a Gemini 3 model in the catalog falls back to the default. Needs `GEMINI_API_KEY` (or a workspace Gemini key); without one, image attachments are left out of syncs |
+| `KB_IMAGE_EXTRACTIONS_PER_SYNC` | New or changed images one connector sync sends for reading, at least `1`. Defaults to `500`; the rest wait for the next sync, which reports as partial |
+| `KB_CONFIG_VISION_REQUESTS_PER_MINUTE` | Image reads per minute per Gemini key, shared by every sync on the deployment. Defaults to `300`; lower it to match a key with a smaller Gemini quota |
 | `KB_EMBEDDING_MODEL` | Embedding model for new knowledge bases. Defaults to `text-embedding-3-small`; use `ollama/<model>` for a model on your own Ollama. An unrecognised hosted model id falls back to the default, but an `ollama/` id is taken at face value — if that model is not on the server, knowledge-base creation fails rather than falling back |
 | `EMBEDDING_OUTPUT_DIMS` | Vector width new knowledge bases are stored at: `384`, `768`, `1024`, `1536` (default), or `3072`. It must be a width the chosen model can emit; anything else falls back to `1536` with a warning |
 | `OPENROUTER_API_KEY` | Fallback route for the OpenAI embedding models — used when it is set and `OPENAI_API_KEY` is not the chosen path |

apps/sim/.env.example

@@ -259,6 +259,12 @@ CRON_SECRET=your_cron_secret # Use `openssl rand -hex 32` to generate. Authentic
 # keys in the same organization to one group using SHA-256 fingerprints, never raw keys.
 # MISTRAL_OCR_QUOTA_GROUPS={"<64-character lowercase key fingerprint>":"organization-id"}
 
+# Knowledge base images read with Gemini (Confluence "Image Attachments: Read with AI").
+# Needs GEMINI_API_KEY or a workspace Gemini key; without one, images are left out of syncs.
+# KB_IMAGE_EXTRACTION_MODEL=gemini-3.8-flash
+# KB_IMAGE_EXTRACTIONS_PER_SYNC=500
+# KB_CONFIG_VISION_REQUESTS_PER_MINUTE=300
+
 # Official Sim Search Slack app (optional; requires existing Search access)
 # Supply these through the deployment environment (for example, ECS Secrets Manager injection).
 # SLACK_SEARCH_APP_ID=

apps/sim/app/workspace/[workspaceId]/knowledge/[id]/[documentId]/document.tsx

@@ -402,6 +402,9 @@ export function Document({
   const DocumentIcon =
     ConnectorIcon || getDocumentIcon(documentData?.mimeType ?? '', effectiveDocumentName)
   const isCompleted = documentData?.processingStatus === 'completed'
+  /** A completed image with no chunks held nothing searchable, such as a logo or an icon. */
+  const isImageWithNothingToIndex =
+    isCompleted && documentData.mimeType.startsWith('image/') && documentData.chunkCount === 0
 
   /**
    * Kept separate from `documentError`: without the document there is no page to draw,
@@ -1284,6 +1287,12 @@ export function Document({
           emptyState={
             chunkError ? (
               <ChunkLoadError message={chunkError} kind={initialError ? 'load' : 'search'} />
+            ) : isImageWithNothingToIndex ? (
+              <EmptyState
+                graphic={<FileX className='size-[24px] text-[var(--text-icon)]' />}
+                title='Nothing to index'
+                description='This image holds no searchable information, such as a logo or an icon.'
+              />
             ) : undefined
           }
           selectable={chunkError ? undefined : selectableConfig}

apps/sim/app/workspace/[workspaceId]/knowledge/[id]/components/add-connector-modal/add-connector-modal.tsx

@@ -83,6 +83,17 @@ import { useConnectorSetupStore } from '@/stores/connector-setup/store'
 
 const CONNECTOR_ENTRIES = Object.entries(CONNECTOR_META_REGISTRY)
 
+/** The settings a new connector's form starts from; a Search source adds its own defaults. */
+function startingSourceConfig(
+  meta: Pick<ConnectorMeta, 'defaultSourceConfig' | 'searchDefaultSourceConfig'> | null | undefined,
+  isSearchIndex: boolean | undefined
+): Record<string, string> {
+  return {
+    ...meta?.defaultSourceConfig,
+    ...(isSearchIndex ? meta?.searchDefaultSourceConfig : undefined),
+  }
+}
+
 interface AddConnectorModalProps {
   scope?: ResourceScope
   open: boolean
@@ -327,9 +338,10 @@ export function AddConnectorModal({
   } = useConnectorConfigFields({
     connectorConfig,
     accessMode: access.accessMode,
-    initialSourceConfig: isSearchIndex
-      ? { ...connectorConfig?.searchDefaultSourceConfig, ...draft?.sourceConfig }
-      : draft?.sourceConfig,
+    initialSourceConfig: {
+      ...startingSourceConfig(connectorConfig, isSearchIndex),
+      ...draft?.sourceConfig,
+    },
     initialCanonicalModes: draft?.canonicalModes,
     initialSelectionLabels: draft?.selectionLabels,
   })
@@ -475,9 +487,7 @@ export function AddConnectorModal({
     if (setupDraftKey) useConnectorSetupStore.getState().clearDraft(setupDraftKey)
     gitlabPermissions.reset()
     setSelectedType(type)
-    setSourceConfig(
-      isSearchIndex ? { ...CONNECTOR_META_REGISTRY[type]?.searchDefaultSourceConfig } : {}
-    )
+    setSourceConfig(startingSourceConfig(CONNECTOR_META_REGISTRY[type], isSearchIndex))
     setSelectedCredentialId(null)
     setContentCredentialId(null)
     setAccess({

apps/sim/connectors/confluence/attachments.test.ts

@@ -16,8 +16,9 @@ vi.mock('@/lib/knowledge/documents/secure-fetch.server', async (importOriginal)
 }))
 
 const CONFIG = { domain: 'example.atlassian.net', spaceKey: 'ENG' }
+const IMAGES_ON = { ...CONFIG, imageAttachments: 'semantic' }
 const CONTEXT = { cloudId: 'cloud', spaceId: '1' }
-const INPUT = { accessToken: 'token', cloudId: 'cloud', domain: CONFIG.domain }
+const INPUT = { accessToken: 'token', cloudId: 'cloud', domain: CONFIG.domain, includeImages: true }
 const fetchMock = vi.fn<typeof fetch>()
 
 function file(overrides: Record<string, unknown> = {}) {
@@ -84,7 +85,10 @@ function fixture(attachment = file()) {
   })
 }
 
-async function get(externalId = 'attachment:page:p1:att123', config = CONFIG) {
+async function get(
+  externalId = 'attachment:page:p1:att123',
+  config: Record<string, unknown> = CONFIG
+) {
   return confluenceConnector.getDocument('token', config, externalId, { ...CONTEXT })
 }
 
@@ -404,6 +408,102 @@ describe('Confluence attachment listing', () => {
   })
 })
 
+describe('Confluence image attachments', () => {
+  async function listOne(attachment: Record<string, unknown>) {
+    fetchMock.mockResolvedValue(Response.json({ results: [attachment] }))
+    const result = await listConfluenceAttachments({
+      ...INPUT,
+      listParents: async () => ({ documents: [parent()], hasMore: false }),
+    })
+    return result.documents.slice(1)
+  }
+
+  it.each([
+    { title: 'Quarterly schedule.png', mediaType: 'image/png', mimeType: 'image/png' },
+    { title: 'image-20240101-000000', mediaType: 'image/jpeg', mimeType: 'image/jpeg' },
+    { title: 'Floor plan', mediaType: 'IMAGE/JPG', mimeType: 'image/jpeg' },
+    { title: 'Diagram.webp', mediaType: 'image/webp', mimeType: 'image/webp' },
+    { title: 'Spinner.gif', mediaType: 'image/gif', mimeType: 'image/gif' },
+  ])('lists $title by its media type', async ({ title, mediaType, mimeType }) => {
+    const [image] = await listOne(file({ title, mediaType, fileSize: 40_000 }))
+    expect(image).toMatchObject({ mimeType, contentDeferred: true })
+    expect(image.contentHash).toBe('confluence:attachment-v1:att123:page:p1:2:image-extract-v1')
+  })
+
+  it.each([
+    { title: 'Logo.svg', mediaType: 'image/svg+xml' },
+    { title: 'Photo.heic', mediaType: 'image/heic' },
+    { title: 'Photo.avif', mediaType: 'image/avif' },
+    { title: 'Archive', mediaType: 'application/zip' },
+    { title: 'Unknown', mediaType: undefined },
+  ])('drops $title', async ({ title, mediaType }) => {
+    expect(await listOne(file({ title, mediaType, fileSize: 40_000 }))).toEqual([])
+  })
+
+  it('drops images under the byte floor and keeps one whose size is unreported', async () => {
+    expect(
+      await listOne(file({ title: 'Icon.png', mediaType: 'image/png', fileSize: 900 }))
+    ).toEqual([])
+    const [unsized] = await listOne(
+      file({ title: 'Chart.png', mediaType: 'image/png', fileSize: undefined })
+    )
+    expect(unsized.mimeType).toBe('image/png')
+  })
+
+  it('keeps document hashes unchanged and still lists small documents', async () => {
+    const [pdf] = await listOne(file({ mediaType: 'application/pdf', fileSize: 12 }))
+    expect(pdf.contentHash).toBe('confluence:attachment-v1:att123:page:p1:2')
+  })
+
+  it('hydrates a listed image with its bytes and the same media type', async () => {
+    fixture(file({ title: 'Quarterly schedule', mediaType: 'image/png', fileSize: 40_000 }))
+    secureDownload.mockResolvedValue(new Response('png bytes'))
+    const hydrated = await get('attachment:page:p1:att123', IMAGES_ON)
+    expect(hydrated?.skippedReason).toBeUndefined()
+    expect(hydrated?.sourceFile).toMatchObject({
+      fileName: 'Quarterly schedule',
+      mimeType: 'image/png',
+    })
+  })
+
+  it('lists no images when the connector turns image attachments off, and keeps documents', async () => {
+    fetchMock.mockResolvedValue(
+      Response.json({
+        results: [
+          file({ title: 'Quarterly schedule.png', mediaType: 'image/png', fileSize: 40_000 }),
+          file({ id: 'att124', title: 'Guide.pdf', mediaType: 'application/pdf' }),
+        ],
+      })
+    )
+    const result = await listConfluenceAttachments({
+      ...INPUT,
+      includeImages: false,
+      listParents: async () => ({ documents: [parent()], hasMore: false }),
+    })
+    expect(result.documents.slice(1).map((doc) => doc.title)).toEqual(['Guide.pdf'])
+  })
+
+  it('replaces an indexed image once image attachments are turned off', async () => {
+    fixture(file({ title: 'Quarterly schedule', mediaType: 'image/png', fileSize: 40_000 }))
+    const hydrated = await get('attachment:page:p1:att123', { ...CONFIG, imageAttachments: 'off' })
+    expect(hydrated?.skippedReason).toBe('Image attachments are turned off for this connector')
+    expect(hydrated?.skippedExistingDisposition).toBe('replace')
+  })
+
+  it('keeps images off for a connector created before the setting existed', async () => {
+    fixture(file({ title: 'Quarterly schedule', mediaType: 'image/png', fileSize: 40_000 }))
+    const hydrated = await get('attachment:page:p1:att123', CONFIG)
+    expect(hydrated?.skippedReason).toBe('Image attachments are turned off for this connector')
+  })
+
+  it('replaces an attachment that stopped being a supported image', async () => {
+    fixture(file({ title: 'Logo', mediaType: 'image/svg+xml', fileSize: 40_000 }))
+    const hydrated = await get()
+    expect(hydrated?.skippedReason).toContain('supported image')
+    expect(hydrated?.skippedExistingDisposition).toBe('replace')
+  })
+})
+
 describe('Confluence attachment hydration', () => {
   it.each(['/attachments/att123', '/download'])(
     'reports missing attachment scope without invalidating the credential at %s',