fix(files): stop promising immutable caching for documents compiled against other files (#8129)

* fix(files): stop promising immutable caching for documents compiled against other files

A versioned serve URL (`?v=<updatedAt>`) was always answered with a one-year
`immutable` Cache-Control. That holds for a stored source — a content write
rotates the storage key, so a given key's bytes never change — but not for a
response the route resolves against OTHER files: a document compiled against the
files it references, or a sim page inlining its images, recompiles on every
request. Those bytes change when a referenced file changes, while this file's key
and `updatedAt` stay put, so the whole URL is unchanged and the browser served a
stale render from cache until the document itself was edited.

The resolver now reports when it read referenced content, and the route withholds
the immutable lifetime for exactly those responses, keeping it for stored sources
and self-contained artifacts.

Also corrects three comments that claimed generated docs are edited in place under
the same storage key. That stopped being true in #5545 (2026-07-13), which made
every content write allocate a new key; the caching rule above was reasoned from
the stale claim.

* improvement(files): make serve cacheability a declared, required property

An optional boolean let a branch added to the resolver inherit the cacheable
default by saying nothing — the exact failure this change exists to prevent.
Cacheability is now a required field every branch must declare, so forgetting it
fails the build rather than silently promising a year of immutability.

* improvement(files): answer a file revalidation with 304 instead of the whole body

A response the browser is told to revalidate carried no validator, so every check
re-sent the entire file. That is the cost a document compiled against other files
now pays on each window focus: it cannot be given a cache lifetime, because its
bytes really may have changed, so the only way to make the check cheap is to let
the client prove what it already holds.

Authorized serves now carry an ETag — the digest of the bytes about to be sent,
which is exact by construction however those bytes were produced — and answer 304
to a matching If-None-Match. Matching is weak, per RFC 9110, so a cache that
stored a weak validator still revalidates.

Kept out of createFileResponse deliberately: digesting costs a pass over the
buffer, up to the 100MB transfer ceiling, and a response served as immutable is
never revalidated, so it would pay that pass and never collect. Public assets and
the assistant-image path are unchanged.

* fix(files): report a reference dependency from the isolated-VM compile path

The isolated-VM fallback returns before the static reference scan, so a document
it compiled never reported one — yet that path reads workspace files live through
its broker, which is what `onWorkspaceFileAccess` records. A versioned request for
such a document therefore still took a one-year immutable lifetime, and changing a
referenced file left the browser serving a stale render.

Fixed at the root rather than in that one branch: the flag is now required on
CompiledDocResult, so every compile path must declare it and a new one cannot
inherit a cacheable-forever answer by staying silent. Each site reports the union
of what the source references statically and what the compile actually touched —
neither alone is sufficient, since a failed read records no access and the broker
reaches files the static scan cannot see.

Making it required immediately surfaced a second case: the process-local compile
cache is shared with compiles that carried a workspace, so a cached entry can hold
contributor identities even when the reading call passes none. That branch now
reads the cached identities instead of assuming independence.

* improvement(files): compute a validator only where a response can be revalidated

Both reviewers caught the same contradiction: the digest was documented as worth
paying only where a 304 can be collected, then applied to every authorized serve
including immutable ones, which are never revalidated. One place now decides, so
an immutable response takes the plain path and spends no pass over its buffer.

Also drops a redundant translation. The resolver was converting the compiler's
boolean into a string union and the cache rule was converting it straight back;
the producer's own required boolean now travels end to end, which is one fact in
one shape and keeps the same build-time guarantee that a new branch must declare it.

Author: waleedlatif1

apps/sim/app/api/files/serve/[...path]/route.test.ts

@@ -36,6 +36,7 @@ const {
   mockFindLocalFile,
   mockReadLocalFileWithinLimit,
   mockCreateFileResponse,
+  mockCreateConditionalFileResponse,
   mockCreateErrorResponse,
   FileNotFoundError,
   serveLogger,
@@ -64,6 +65,7 @@ const {
     mockFindLocalFile: vi.fn(),
     mockReadLocalFileWithinLimit: vi.fn(),
     mockCreateFileResponse: vi.fn(),
+    mockCreateConditionalFileResponse: vi.fn(),
     mockCreateErrorResponse: vi.fn(),
     FileNotFoundError: FileNotFoundErrorClass,
   }
@@ -129,6 +131,7 @@ vi.mock('@/lib/copilot/tools/server/files/doc-compile', () => ({
 vi.mock('@/app/api/files/utils', () => ({
   FileNotFoundError,
   createFileResponse: mockCreateFileResponse,
+  createConditionalFileResponse: mockCreateConditionalFileResponse,
   createErrorResponse: mockCreateErrorResponse,
   getContentType: mockGetContentType,
   extractStorageKey: vi.fn().mockImplementation((path: string) => path.split('/').pop()),
@@ -192,6 +195,11 @@ describe('File Serve API Route', () => {
         })
       }
     )
+    // Delegates so the existing assertions on the response payload — including its
+    // Cache-Control — read the same call list whichever helper the route reached for.
+    mockCreateConditionalFileResponse.mockImplementation((file: unknown) =>
+      mockCreateFileResponse(file)
+    )
     mockCreateErrorResponse.mockImplementation((error: Error) => {
       return new Response(JSON.stringify({ error: error.name, message: error.message }), {
         status: error.name === 'FileNotFoundError' ? 404 : 500,
@@ -439,6 +447,70 @@ describe('File Serve API Route', () => {
     expect(mockVerifyFileAccess).not.toHaveBeenCalled()
   })
 
+  describe('versioned cache lifetime', () => {
+    const principal = {
+      kind: 'delegated' as const,
+      serviceId: 'executor' as const,
+      subjectUserId: 'test-user-id',
+      workspaceId: 'test-workspace-id',
+      delegationId: 'delegation-1',
+      audience: 'sim:workspace-files',
+      issuedAt: new Date('2026-08-01T00:00:00Z'),
+      expiresAt: new Date('2026-08-01T01:00:00Z'),
+      delegationContext: {
+        kind: 'workflow_execution' as const,
+        workflowId: 'workflow-1',
+      },
+    }
+
+    async function serveVersionedDoc(dependsOnReferencedFiles: boolean) {
+      mockResolveStoredFileContext.mockResolvedValue('workspace')
+      mockParseWorkspaceFileKey.mockReturnValue('test-workspace-id')
+      mockAuthenticateWorkspaceFile.mockResolvedValue(principal)
+      mockResolveServableDocBytes.mockResolvedValue({
+        buffer: Buffer.from('compiled'),
+        contentType: 'application/pdf',
+        ...(dependsOnReferencedFiles ? { dependsOnReferencedFiles: true } : {}),
+      })
+
+      const req = new NextRequest(
+        'http://localhost:3000/api/files/serve/workspace/test-workspace-id/report.pdf?v=1756684800000'
+      )
+      await GET(req, {
+        params: Promise.resolve({ path: ['workspace', 'test-workspace-id', 'report.pdf'] }),
+      })
+      return mockCreateFileResponse.mock.calls.at(-1)?.[0]
+    }
+
+    it('caches a versioned document immutably when its bytes derive from the stored source alone', async () => {
+      expect(await serveVersionedDoc(false)).toEqual(
+        expect.objectContaining({ cacheControl: 'private, max-age=31536000, immutable' })
+      )
+    })
+
+    it('spends no validator on an immutable response, which is never revalidated', async () => {
+      await serveVersionedDoc(false)
+      expect(mockCreateConditionalFileResponse).not.toHaveBeenCalled()
+      expect(mockCreateFileResponse).toHaveBeenCalled()
+    })
+
+    it('attaches a validator to a revalidated response so the next check can be answered 304', async () => {
+      await serveVersionedDoc(true)
+      expect(mockCreateConditionalFileResponse).toHaveBeenCalled()
+    })
+
+    it('keeps a versioned document revalidated when it was compiled against referenced files', async () => {
+      /**
+       * The URL carries the file's own `updatedAt`, which does not move when a
+       * REFERENCED file changes — so an immutable lifetime would pin the stale
+       * render in the browser cache until the document itself is edited.
+       */
+      expect(await serveVersionedDoc(true)).toEqual(
+        expect.objectContaining({ cacheControl: 'private, no-cache, must-revalidate' })
+      )
+    })
+  })
+
   it('serves a mothership chat attachment stored under a workspace key', async () => {
     /**
      * The attachment shares the `workspace/…` prefix but is recorded as

apps/sim/app/api/files/serve/[...path]/route.ts

@@ -30,9 +30,11 @@ import { isSimPageSource, SIM_PAGE_CONTENT_TYPE } from '@/lib/workspace-files/pa
 import { renderSimPageDocumentWithAssets } from '@/lib/workspace-files/page-document.server'
 import { type KnowledgeFileAccess, verifyFileAccess } from '@/app/api/files/authorization'
 import {
+  createConditionalFileResponse,
   createErrorResponse,
   createFileResponse,
   FileNotFoundError,
+  type FileResponse,
   findLocalFile,
   getContentType,
   readLocalFileWithinLimit,
@@ -64,8 +66,25 @@ interface ServeOptions {
   raw: boolean
   /** `preview=1` — the caller renders these bytes rather than saving them. */
   preview: boolean
-  /** `v=<updatedAt>` — the URL addresses content-immutable bytes. */
+  /** `v=<updatedAt>` — the caller asserts the URL addresses one fixed content revision. */
   versioned: boolean
+  /** The request's `If-None-Match`, so a revalidation can be answered 304 instead of re-sending. */
+  ifNoneMatch: string | null
+}
+
+interface ServableBytes {
+  buffer: Buffer
+  contentType: string
+  /**
+   * These bytes were resolved against OTHER files' current content — a page inlining its images,
+   * or a document compiled against the files it references — so the same storage key can serve
+   * different bytes over time while this file's own key and `updatedAt` stay put.
+   *
+   * Required, so a branch added to the resolver cannot inherit the cacheable answer by saying
+   * nothing — the same reason the transfer ceiling is asserted where the branches converge rather
+   * than inside each one.
+   */
+  dependsOnReferencedFiles: boolean
 }
 
 /**
@@ -95,12 +114,16 @@ async function resolveServableBytes(params: {
   /** The stored record's content type, where the caller has the record. */
   fileType?: string
   signal: AbortSignal | undefined
-}): Promise<{ buffer: Buffer; contentType: string }> {
+}): Promise<ServableBytes> {
   // `raw` is the stored source, already bounded by the read that produced it, but it
   // goes through the same check so the ceiling holds for everything this returns
   // rather than for every branch someone remembered to cover.
-  const resolved = params.options.raw
-    ? { buffer: params.buffer, contentType: getContentType(params.filename) }
+  const resolved: ServableBytes = params.options.raw
+    ? {
+        buffer: params.buffer,
+        contentType: getContentType(params.filename),
+        dependsOnReferencedFiles: false,
+      }
     : await resolveTransformedBytes(params)
   assertKnownSizeWithinLimit(
     resolved.buffer.length,
@@ -120,7 +143,7 @@ async function resolveTransformedBytes(params: {
   filePrincipal?: Principal
   fileType?: string
   signal: AbortSignal | undefined
-}): Promise<{ buffer: Buffer; contentType: string }> {
+}): Promise<ServableBytes> {
   const {
     buffer,
     filename,
@@ -146,25 +169,36 @@ async function resolveTransformedBytes(params: {
         await renderSimPageDocumentWithAssets(text, { workspaceId }),
         'utf8'
       )
-      return { buffer: rendered, contentType: 'text/html' }
+      // Inlines the workspace images the page references, read at their CURRENT content.
+      return {
+        buffer: rendered,
+        contentType: 'text/html',
+        dependsOnReferencedFiles: true,
+      }
     }
   }
 
   if (options.preview) {
     // Images resolve independently of the document path: a HEIF has no compiled-source
     // concept, so it never reaches the doc branch.
     const image = await resolveServableImageBytes(buffer, storageKey)
-    if (image) return image
+    // Transcoded from THIS file's stored bytes, so it lives and dies with the storage key.
+    if (image) return { ...image, dependsOnReferencedFiles: false }
   }
 
-  return resolveServableDocBytes({
+  const doc = await resolveServableDocBytes({
     rawBuffer: buffer,
     fileName: filename,
     workspaceId,
     filePrincipal,
     ownerKey,
     signal,
   })
+  return {
+    buffer: doc.buffer,
+    contentType: doc.contentType,
+    dependsOnReferencedFiles: doc.dependsOnReferencedFiles,
+  }
 }
 
 const STORAGE_KEY_PREFIX_RE = /^\d{13}-[a-z0-9]{7}-/
@@ -184,18 +218,41 @@ const WORKSPACE_REVALIDATE_CACHE_CONTROL = 'private, no-cache, must-revalidate'
 const PUBLIC_ASSET_CACHE_CONTROL = 'public, max-age=31536000'
 
 /**
- * Cache-Control for a served file. A versioned request (`?v=<updatedAt>`) addresses
- * content-immutable bytes — generated docs are content-addressed and the version
- * bumps on every edit — so the browser may cache it indefinitely; re-opens and
- * focus refetches then resolve from cache with no round trip. Unversioned workspace
- * reads stay revalidated because the same storage key is edited in place.
+ * Cache-Control for a served file.
+ *
+ * A versioned request (`?v=<updatedAt>`) normally addresses content-immutable bytes: a workspace
+ * file's content write stores the new bytes under a NEW storage key, so a given key's stored
+ * source never changes and the browser may cache it indefinitely — re-opens and focus refetches
+ * then resolve from cache with no round trip.
+ *
+ * That promise does NOT hold when the response was resolved against other files' current content
+ * (`derived-from-referenced-files`): a document compiled against the files it references, or a page
+ * inlining its images, recompiles per request, so the same key serves different bytes once a
+ * referenced file changes — while this file's key and `updatedAt`, and therefore the whole URL,
+ * stay put. Promising immutability there pins a stale render in the browser cache for a year, so
+ * those responses stay revalidated whether or not the request carried a version.
  */
 function resolveServeCacheControl(
   versioned: boolean,
-  context: string | undefined
+  context: string | undefined,
+  dependsOnReferencedFiles: boolean
 ): string | undefined {
-  if (versioned) return IMMUTABLE_CACHE_CONTROL
-  return context === 'workspace' ? WORKSPACE_REVALIDATE_CACHE_CONTROL : undefined
+  if (versioned && !dependsOnReferencedFiles) return IMMUTABLE_CACHE_CONTROL
+  return context === 'workspace' || dependsOnReferencedFiles
+    ? WORKSPACE_REVALIDATE_CACHE_CONTROL
+    : undefined
+}
+
+/**
+ * Sends a resolved file, attaching a validator only where the client may actually revalidate.
+ *
+ * An immutable response is never revalidated, so digesting its buffer — a pass over up to the
+ * whole transfer ceiling — would cost the compute and never collect a single 304.
+ */
+function serveResolvedFile(file: FileResponse, ifNoneMatch: string | null): NextResponse {
+  return file.cacheControl === IMMUTABLE_CACHE_CONTROL
+    ? createFileResponse(file)
+    : createConditionalFileResponse(file, ifNoneMatch)
 }
 
 export const GET = withRouteHandler(
@@ -281,6 +338,7 @@ export const GET = withRouteHandler(
         raw: query.raw === '1',
         preview: query.preview === '1',
         versioned: query.v != null,
+        ifNoneMatch: request.headers.get('if-none-match'),
       }
 
       if (workspacePrincipal) {
@@ -381,12 +439,19 @@ async function handleWorkspaceFile(
     workspaceId,
     size: resolved.buffer.length,
   })
-  return createFileResponse({
-    buffer: resolved.buffer,
-    contentType: resolved.contentType,
-    filename: file.name,
-    cacheControl: resolveServeCacheControl(options.versioned, 'workspace'),
-  })
+  return serveResolvedFile(
+    {
+      buffer: resolved.buffer,
+      contentType: resolved.contentType,
+      filename: file.name,
+      cacheControl: resolveServeCacheControl(
+        options.versioned,
+        'workspace',
+        resolved.dependsOnReferencedFiles
+      ),
+    },
+    options.ifNoneMatch
+  )
 }
 
 async function handleLocalFile(
@@ -427,7 +492,11 @@ async function handleLocalFile(
     const segment = filename.split('/').pop() || filename
     const displayName = stripStorageKeyPrefix(segment)
     const workspaceId = getWorkspaceIdForCompile(filename)
-    const { buffer: fileBuffer, contentType } = await resolveServableBytes({
+    const {
+      buffer: fileBuffer,
+      contentType,
+      dependsOnReferencedFiles,
+    } = await resolveServableBytes({
       buffer: rawBuffer,
       filename: displayName,
       storageKey: filename,
@@ -439,12 +508,19 @@ async function handleLocalFile(
 
     logger.info('Local file served', { userId, filename, size: fileBuffer.length })
 
-    return createFileResponse({
-      buffer: fileBuffer,
-      contentType,
-      filename: displayName,
-      cacheControl: resolveServeCacheControl(options.versioned, context),
-    })
+    return serveResolvedFile(
+      {
+        buffer: fileBuffer,
+        contentType,
+        filename: displayName,
+        cacheControl: resolveServeCacheControl(
+          options.versioned,
+          context,
+          dependsOnReferencedFiles
+        ),
+      },
+      options.ifNoneMatch
+    )
   } catch (error) {
     logServeFailure('Error reading local file:', error)
     throw error
@@ -494,7 +570,11 @@ async function handleCloudProxy(
     const segment = cloudKey.split('/').pop() || 'download'
     const displayName = stripStorageKeyPrefix(segment)
     const workspaceId = getWorkspaceIdForCompile(cloudKey)
-    const { buffer: fileBuffer, contentType } = await resolveServableBytes({
+    const {
+      buffer: fileBuffer,
+      contentType,
+      dependsOnReferencedFiles,
+    } = await resolveServableBytes({
       buffer: rawBuffer,
       filename: displayName,
       storageKey: cloudKey,
@@ -511,12 +591,19 @@ async function handleCloudProxy(
       context,
     })
 
-    return createFileResponse({
-      buffer: fileBuffer,
-      contentType,
-      filename: displayName,
-      cacheControl: resolveServeCacheControl(options.versioned, context),
-    })
+    return serveResolvedFile(
+      {
+        buffer: fileBuffer,
+        contentType,
+        filename: displayName,
+        cacheControl: resolveServeCacheControl(
+          options.versioned,
+          context,
+          dependsOnReferencedFiles
+        ),
+      },
+      options.ifNoneMatch
+    )
   } catch (error) {
     logServeFailure('Error downloading from cloud storage:', error)
     throw error