Describe the bug
Think's workspace read tool classifies images correctly as kind: "image", but toModelOutput emits them as a file-data content part — the same part type it uses for PDFs. On the ai@7 peer this is harmless, because AI SDK core rewrites deprecated parts and the provider re-derives routing from mediaType. On the ai@6 peer there is no normalization step: file-data maps straight to a document part, so images are sent to the model as documents.
@cloudflare/think declares "ai": "^6.0.0 || ^7.0.0" as a peer, so ai@6 is a supported configuration. On it, reading an image from the workspace fails — loudly on OpenAI, silently on Anthropic. PDF and text reads are unaffected on both.
To Reproduce
- Install
@cloudflare/think@0.15.1 with ai@6.x and an OpenAI-backed model.
- Write a PNG into the workspace:
workspace.writeFileBytes("/screenshot.png", bytes, "image/png").
- Ask the agent to
read that path.
- The turn fails:
Invalid file data: 'input[3].output[1].file_data'. Expected a base64-encoded data URL
with a valid file MIME type (e.g. 'data:text/plain;base64,SGVsbG8sIFdvcmxkIQ=='),
but got unsupported MIME type 'image/png'.
With an Anthropic model instead, step 4 produces no error — its file-data case only handles application/pdf and pushes an unsupported tool content part type warning for anything else, dropping the part. The model receives the text note with no image and answers as though nothing was attached, which is the more dangerous of the two failures.
Changing only the emitted part type from file-data to image-data fixes both.
Expected behavior
read on an image returns it to the model as image content, so a vision-capable model can see it — the behaviour that shipped in #1435 and is still described by the tool's own description ("Images and PDFs are passed to capable models as file content") and by the docs table ("pass images and PDFs to multimodal models").
Screenshots
N/A — error output included above.
Version:
|
|
@cloudflare/think |
0.15.1 (latest) |
agents |
0.20.1 |
ai |
6.0.208 |
| provider |
@ai-sdk/gateway 3.x → OpenAI and Anthropic |
| runtime |
Cloudflare Workers + Durable Objects |
Additional context
Root cause
Pinned to main at e70f54ddcfaa55383fa1986d031be8a215282d85.
execute derives the distinction correctly — image at workspace.ts#L321-L330, PDF at #L331-L338:
if (mediaType.startsWith("image/")) {
return { kind: "image", ... };
}
if (mediaType === "application/pdf") {
return { kind: "file", ... };
}
toModelOutput then discards it. isModelFileReadOutput admits both kinds, and everything past that gate emits one shape at #L415-L429:
{
// `file-data` (base64 string `data`) is accepted by both AI SDK v6
// and v7. v7 also offers the newer `{ type: "file", data: { type:
// "data", data } }` shape, but that does not exist in v6, so we use
// the cross-major form here.
type: "file-data",
data,
mediaType: replayOutput.mediaType,
filename: replayOutput.name
}
The comment is accurate that both majors accept file-data. The gap is that they interpret it differently.
ai@7 rewrites deprecated tool-result parts in convert-to-language-model-prompt.ts before the provider sees them, preserving mediaType:
case 'file-data': {
return { type: 'file', data: { type: 'data', data: item.data },
filename: item.filename, mediaType: item.mediaType, ... };
}
The provider then routes on media type, so image/* still becomes input_image:
case "file": {
const topLevel = getTopLevelMediaType(item.mediaType);
if (topLevel === "image") return { type: "input_image", image_url: ... };
return { type: "input_file", filename: ..., file_data: ... };
}
ai@6 has no normalization step. image-data and file-data are distinct part types that map straight through, and the caller chooses:
image-data → input_image (OpenAI) / image block (Anthropic)
file-data → input_file (OpenAI) / document, PDF-only (Anthropic)
So on v6 the media type never gets a chance to influence routing, and an image emitted as file-data is a document by construction.
How it regressed
This worked when multimodal read landed. #1435 emitted image-data for images and file-data for PDFs, with tests asserting the split.
#1922 (feat: update AI SDK to v7) changed the image branch to file-data and updated both test expectations from image-data to file-data in the same commit, which is why CI stayed green. The PR notes describe the intent:
Prefer file over legacy image / image-data / media / file-data content parts. AI SDK v7 docs show multimodal tool results as { type: "file", mediaType, data: { type: "data", data } }.
That is correct for v7, where consolidating is safe precisely because the provider re-derives routing from mediaType. The regression is in the fallback: since { type: "file" } does not exist in v6, the code falls back to file-data on both majors — and on v6 that is document semantics, not a neutral container.
Suggested fix
Restoring the pre-#1922 branch is correct on both majors, so no peer detection is needed:
const isImage = replayOutput.kind === "image";
return {
type: "content",
value: [
{ type: "text", text: note },
isImage
? { type: "image-data", data, mediaType: replayOutput.mediaType }
: { type: "file-data", data, mediaType: replayOutput.mediaType,
filename: replayOutput.name }
]
};
On ai@7, image-data normalizes to { type: "file", mediaType } with the media type preserved (convert-to-language-model-prompt.ts, case 'image-data'), so the provider still produces input_image — same result as today, plus a deprecation warning. On ai@6 it produces input_image directly, which is the fix.
If you would rather not reintroduce a deprecated part type, the alternative is to emit v7's { type: "file", data: { type: "data", data }, mediaType } when the peer is v7 and image-data/file-data when it is v6 — but that needs version detection for no behavioural gain, since v7 accepts the deprecated forms until v8.
The two tests changed in #1922 are the ones that would have caught this, so reverting their expectations alongside the fix restores the guard.
Describe the bug
Think's workspace
readtool classifies images correctly askind: "image", buttoModelOutputemits them as afile-datacontent part — the same part type it uses for PDFs. On theai@7peer this is harmless, because AI SDK core rewrites deprecated parts and the provider re-derives routing frommediaType. On theai@6peer there is no normalization step:file-datamaps straight to a document part, so images are sent to the model as documents.@cloudflare/thinkdeclares"ai": "^6.0.0 || ^7.0.0"as a peer, soai@6is a supported configuration. On it, reading an image from the workspace fails — loudly on OpenAI, silently on Anthropic. PDF and text reads are unaffected on both.To Reproduce
@cloudflare/think@0.15.1withai@6.xand an OpenAI-backed model.workspace.writeFileBytes("/screenshot.png", bytes, "image/png").readthat path.With an Anthropic model instead, step 4 produces no error — its
file-datacase only handlesapplication/pdfand pushes anunsupported tool content part typewarning for anything else, dropping the part. The model receives the text note with no image and answers as though nothing was attached, which is the more dangerous of the two failures.Changing only the emitted part type from
file-datatoimage-datafixes both.Expected behavior
readon an image returns it to the model as image content, so a vision-capable model can see it — the behaviour that shipped in #1435 and is still described by the tool's own description ("Images and PDFs are passed to capable models as file content") and by the docs table ("pass images and PDFs to multimodal models").Screenshots
N/A — error output included above.
Version:
@cloudflare/thinkagentsai@ai-sdk/gateway3.x → OpenAI and AnthropicAdditional context
Root cause
Pinned to
mainate70f54ddcfaa55383fa1986d031be8a215282d85.executederives the distinction correctly — image atworkspace.ts#L321-L330, PDF at#L331-L338:toModelOutputthen discards it.isModelFileReadOutputadmits both kinds, and everything past that gate emits one shape at#L415-L429:The comment is accurate that both majors accept
file-data. The gap is that they interpret it differently.ai@7rewrites deprecated tool-result parts inconvert-to-language-model-prompt.tsbefore the provider sees them, preservingmediaType:The provider then routes on media type, so
image/*still becomesinput_image:ai@6has no normalization step.image-dataandfile-dataare distinct part types that map straight through, and the caller chooses:image-data→input_image(OpenAI) /imageblock (Anthropic)file-data→input_file(OpenAI) /document, PDF-only (Anthropic)So on v6 the media type never gets a chance to influence routing, and an image emitted as
file-datais a document by construction.How it regressed
This worked when multimodal read landed. #1435 emitted
image-datafor images andfile-datafor PDFs, with tests asserting the split.#1922 (
feat: update AI SDK to v7) changed the image branch tofile-dataand updated both test expectations fromimage-datatofile-datain the same commit, which is why CI stayed green. The PR notes describe the intent:That is correct for v7, where consolidating is safe precisely because the provider re-derives routing from
mediaType. The regression is in the fallback: since{ type: "file" }does not exist in v6, the code falls back tofile-dataon both majors — and on v6 that is document semantics, not a neutral container.Suggested fix
Restoring the pre-#1922 branch is correct on both majors, so no peer detection is needed:
On
ai@7,image-datanormalizes to{ type: "file", mediaType }with the media type preserved (convert-to-language-model-prompt.ts,case 'image-data'), so the provider still producesinput_image— same result as today, plus a deprecation warning. Onai@6it producesinput_imagedirectly, which is the fix.If you would rather not reintroduce a deprecated part type, the alternative is to emit v7's
{ type: "file", data: { type: "data", data }, mediaType }when the peer is v7 andimage-data/file-datawhen it is v6 — but that needs version detection for no behavioural gain, since v7 accepts the deprecated forms until v8.The two tests changed in #1922 are the ones that would have caught this, so reverting their expectations alongside the fix restores the guard.