Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/gemini-native-image-model-options.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
'@tanstack/ai-gemini': minor
---

Type Gemini-native image models with their own provider options. `GeminiImageModelProviderOptionsByName` mapped **every** image model to the Imagen-shaped `GeminiImageProviderOptions`, so `modelOptions: { safetySettings, thinkingConfig, imageConfig, systemInstruction }` was a compile error on `gemini-3.1-flash-image-preview`, `gemini-3.1-flash-lite-image`, `gemini-3-pro-image-preview`, and `gemini-2.5-flash-image` β€” even though those models are served by `generateContent`, whose `GenerateContentConfig` accepts all of them. The adapter compensated by forwarding only `seed`, silently dropping anything else.

The map now splits native vs Imagen, mirroring the split already used by `GeminiImageModelSizeByName` and `GeminiImageModelInputModalitiesByName`: native models get the new `GeminiNativeImageProviderOptions` (`seed`, `safetySettings`, `thinkingConfig`, `imageConfig`, `systemInstruction`), Imagen models keep `GeminiImageProviderOptions`. Both API paths now pick their config fields by name β€” never a wholesale spread β€” so neither shape's fields can reach the other's endpoint. Runtime routing moves the same way, off a `gemini-` prefix test onto membership in `GEMINI_NATIVE_IMAGE_MODELS`: a `gemini-*` image model not present in that list now routes to `generateImages` instead of `generateContent`, so it fails against that endpoint rather than silently taking the native path.

`responseModalities` stays a protected adapter default (`['TEXT', 'IMAGE']`) and is deliberately absent from the new type. `modelOptions.imageConfig` merges **over** the `imageConfig` derived from the portable `size` option, per field β€” passing only `imageConfig.imageSize` keeps the `aspectRatio` that `size` implied. `HarmCategory` and `HarmBlockThreshold` are now re-exported so `safetySettings` can be written without adding `@google/genai` to your own dependencies.

**BREAKING (types only):** Imagen fields no longer compile on the four Gemini-native image models β€” `aspectRatio`, `negativePrompt`, `personGeneration`, `safetyFilterLevel`, `addWatermark`, `language`, `outputMimeType`, `outputCompressionQuality`, `guidanceScale`, `enhancePrompt`, `includeSafetyAttributes`, `includeRaiReason`, `outputGcsUri`, `labels`. They previously type-checked but were already dropped at runtime (only `seed` was ever forwarded to `generateContent`), so no request behaviour changes β€” the compiler now reports what was already happening. Migrate `aspectRatio` to the portable `size` option (`'16:9_4K'`) or to `modelOptions.imageConfig`, and drop the rest. `GeminiImageAdapter.generateImages` (and its `~types.providerOptions`) also widens from `ImageGenerationOptions<GeminiImageProviderOptions>` to `ImageGenerationOptions<GeminiAnyImageProviderOptions>`, which affects code structurally annotated against the old signature.
26 changes: 25 additions & 1 deletion docs/adapters/gemini.md
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,7 @@ The Gemini adapter supports two types of image generation:
- **Gemini native image models** (NanoBanana) β€” Use the `generateContent` API with models like `gemini-3.1-flash-image-preview`. These support extended resolution tiers (1K, 2K, 4K) and aspect ratio control.
- **Imagen models** β€” Use the `generateImages` API with models like `imagen-4.0-generate-001`. These are dedicated image generation models with WIDTHxHEIGHT sizing.

The adapter automatically routes to the correct API based on the model name β€” models starting with `gemini-` use `generateContent`, while `imagen-` models use `generateImages`.
The adapter automatically routes to the correct API based on membership in the known list of Gemini native models (`GEMINI_NATIVE_IMAGE_MODELS`) β€” the Gemini native models listed above use `generateContent`, while Imagen models, and any model id this package doesn't know about, use `generateImages`.

### Example: Gemini Native Image Generation (NanoBanana)

Expand Down Expand Up @@ -494,6 +494,10 @@ const result = await generateImage({

### Image Model Options

`modelOptions` is typed per model family, because the two families hit different APIs.

Imagen models (`generateImages`) take `GenerateImagesConfig` fields:

Comment thread
coderabbitai[bot] marked this conversation as resolved.
```typescript ignore
import { generateImage } from "@tanstack/ai";
import { geminiImage } from "@tanstack/ai-gemini";
Expand All @@ -509,6 +513,26 @@ const result = await generateImage({
});
```

Gemini native models (`generateContent`) take `GenerateContentConfig` fields instead β€” `seed`, `safetySettings`, `thinkingConfig`, `imageConfig`, and `systemInstruction`:

```typescript
import { generateImage } from "@tanstack/ai";
import { geminiImage } from "@tanstack/ai-gemini";

const result = await generateImage({
adapter: geminiImage("gemini-3.1-flash-image-preview"),
prompt: "...",
size: "16:9_4K",
modelOptions: {
thinkingConfig: { thinkingBudget: 512 },
// Merged over the imageConfig derived from `size`, per field.
imageConfig: { imageSize: "2K" },
},
});
```

See [Image Generation](../media/image-generation) for the full native option list.

## Text-to-Speech (Experimental)

> **Note:** Gemini TTS is experimental and may require the Live API for full functionality.
Expand Down
4 changes: 2 additions & 2 deletions docs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -437,7 +437,7 @@
"label": "Image Generation",
"to": "media/image-generation",
"addedAt": "2026-04-15",
"updatedAt": "2026-08-13"
"updatedAt": "2026-08-14"
},
{
"label": "Video Generation",
Expand Down Expand Up @@ -796,7 +796,7 @@
"label": "Google Gemini",
"to": "adapters/gemini",
"addedAt": "2026-04-15",
"updatedAt": "2026-07-22"
"updatedAt": "2026-08-14"
},
{
"label": "Ollama",
Expand Down
36 changes: 35 additions & 1 deletion docs/media/image-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -634,7 +634,7 @@ const result = await generateImage({

#### Gemini Native Model Options (NanoBanana)

Gemini native image models accept `GenerateContentConfig` options directly in `modelOptions`:
Gemini native image models are served by `generateContent`, so their `modelOptions` are `GenerateContentConfig` fields β€” a different shape from the Imagen options above:

```typescript
import { generateImage } from "@tanstack/ai";
Expand All @@ -644,9 +644,43 @@ const result = await generateImage({
adapter: geminiImage("gemini-3.1-flash-image-preview"),
prompt: "A beautiful garden",
size: "16:9_4K",
modelOptions: {
seed: 42,
thinkingConfig: { thinkingBudget: 512 },
systemInstruction: "Always render in watercolor.",
// Merged over the imageConfig derived from `size`, per field: this keeps
// the 16:9 aspect ratio and overrides only the resolution tier.
imageConfig: { imageSize: "2K" },
},
});
```

`safetySettings` takes the SDK's `HarmCategory` / `HarmBlockThreshold` enums, so plain strings won't type-check. Both are re-exported from `@tanstack/ai-gemini` β€” you don't need `@google/genai` in your own dependencies:

```typescript
import { generateImage } from "@tanstack/ai";
import {
HarmBlockThreshold,
HarmCategory,
geminiImage,
} from "@tanstack/ai-gemini";

const result = await generateImage({
adapter: geminiImage("gemini-3.1-flash-image-preview"),
prompt: "A beautiful garden",
modelOptions: {
safetySettings: [
{
category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,
threshold: HarmBlockThreshold.BLOCK_ONLY_HIGH,
},
],
},
});
```

`responseModalities` is not accepted β€” the adapter always requests `['TEXT', 'IMAGE']`, so nothing can silently disable image output.

### Response Format

The image generation result includes:
Expand Down
124 changes: 90 additions & 34 deletions packages/ai-gemini/src/adapters/image.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import {
} from '../utils'
import { buildGeminiUsage } from '../usage'
import {
isGeminiNativeImageModel,
parseNativeImageSize,
sizeToAspectRatio,
validateImageSize,
Expand All @@ -15,10 +16,11 @@ import {
} from '../image/image-provider-options'
import type { GEMINI_IMAGE_MODELS } from '../model-meta'
import type {
GeminiAnyImageProviderOptions,
GeminiImageModelInputModalitiesByName,
GeminiImageModelProviderOptionsByName,
GeminiImageModelSizeByName,
GeminiImageProviderOptions,
GeminiNativeImageProviderOptions,
} from '../image/image-provider-options'
import type {
GeneratedImage,
Expand All @@ -35,6 +37,7 @@ import type {
GenerateImagesConfig,
GenerateImagesResponse,
GoogleGenAI,
ImageConfig,
Part,
} from '@google/genai'
import type { GeminiClientConfig } from '../utils/client'
Expand Down Expand Up @@ -65,7 +68,7 @@ export class GeminiImageAdapter<
TModel extends GeminiImageModel,
> extends BaseImageAdapter<
TModel,
GeminiImageProviderOptions,
GeminiAnyImageProviderOptions,
GeminiImageModelProviderOptionsByName,
GeminiImageModelSizeByName,
GeminiImageModelInputModalitiesByName
Expand All @@ -75,7 +78,7 @@ export class GeminiImageAdapter<

// Type-only property - never assigned at runtime
declare '~types': {
providerOptions: GeminiImageProviderOptions
providerOptions: GeminiAnyImageProviderOptions
modelProviderOptionsByName: GeminiImageModelProviderOptionsByName
modelSizeByName: GeminiImageModelSizeByName
modelInputModalitiesByName: GeminiImageModelInputModalitiesByName
Expand All @@ -89,7 +92,7 @@ export class GeminiImageAdapter<
}

async generateImages(
options: ImageGenerationOptions<GeminiImageProviderOptions>,
options: ImageGenerationOptions<GeminiAnyImageProviderOptions>,
): Promise<ImageGenerationResult> {
const { model, logger } = options

Expand Down Expand Up @@ -121,7 +124,7 @@ export class GeminiImageAdapter<
)
}

if (this.isGeminiImageModel(model)) {
if (isGeminiNativeImageModel(model)) {
return await this.generateWithGeminiApi(options, resolved)
}

Expand Down Expand Up @@ -155,29 +158,40 @@ export class GeminiImageAdapter<
}
}

private isGeminiImageModel(model: string): boolean {
return model.startsWith('gemini-')
}

private async generateWithGeminiApi(
options: ImageGenerationOptions<GeminiImageProviderOptions>,
options: ImageGenerationOptions<GeminiNativeImageProviderOptions>,
resolved: ResolvedMediaPrompt,
): Promise<ImageGenerationResult> {
const { model, size, numberOfImages, modelOptions } = options

const parsedSize = size ? parseNativeImageSize(size) : undefined

// GeminiImageProviderOptions is Imagen-shaped β€” most fields
// (personGeneration, safetyFilterLevel, addWatermark, outputMimeType,
// outputCompressionQuality, guidanceScale, enhancePrompt,
// includeSafetyAttributes, includeRaiReason, outputGcsUri, labels,
// negativePrompt, language) are only valid on GenerateImagesConfig and
// would be rejected by the Gemini-native generateContent path. Pick only
// the fields that are valid on GenerateContentConfig instead of spreading
// the whole options object.
const nativeConfig: GenerateContentConfig = {}
if (modelOptions?.seed !== undefined) {
nativeConfig.seed = modelOptions.seed
// The portable `size` option is the baseline; modelOptions.imageConfig is
// the provider escape hatch and wins per field, so a caller passing only
// `imageConfig.imageSize` keeps the aspectRatio derived from `size`.
const imageConfig: ImageConfig = {
...(parsedSize?.aspectRatio && { aspectRatio: parsedSize.aspectRatio }),
...(parsedSize?.resolution && { imageSize: parsedSize.resolution }),
...modelOptions?.imageConfig,
}

// Named picks, never a wholesale spread: the Imagen-shaped fields of
// GeminiImageProviderOptions (personGeneration, safetyFilterLevel,
// addWatermark, outputMimeType, …) are only valid on GenerateImagesConfig
// and would be rejected by generateContent. Picking by name means no
// Imagen field can reach this path even if one slips past the per-model
// provider-options map.
const nativeConfig: GenerateContentConfig = {
...(modelOptions?.seed !== undefined && { seed: modelOptions.seed }),
...(modelOptions?.safetySettings !== undefined && {
safetySettings: modelOptions.safetySettings,
}),
...(modelOptions?.thinkingConfig !== undefined && {
thinkingConfig: modelOptions.thinkingConfig,
}),
...(modelOptions?.systemInstruction !== undefined && {
systemInstruction: modelOptions.systemInstruction,
}),
}

const config: GenerateContentConfig = {
Expand All @@ -186,16 +200,7 @@ export class GeminiImageAdapter<
// IMPORTANT: responseModalities is a protected default β€” set it AFTER
// nativeConfig so nothing can silently disable image output.
responseModalities: ['TEXT', 'IMAGE'],
...(parsedSize && {
imageConfig: {
...(parsedSize.aspectRatio && {
aspectRatio: parsedSize.aspectRatio,
}),
...(parsedSize.resolution && {
imageSize: parsedSize.resolution,
}),
},
}),
...(Object.keys(imageConfig).length > 0 && { imageConfig }),
}

const contents = this.buildContents(resolved, numberOfImages)
Expand Down Expand Up @@ -321,19 +326,70 @@ export class GeminiImageAdapter<
}

private buildImagenConfig(
options: ImageGenerationOptions<GeminiImageProviderOptions>,
options: ImageGenerationOptions<GeminiAnyImageProviderOptions>,
): GenerateImagesConfig {
const { size, numberOfImages, modelOptions } = options

// Build with conditional spreads β€” under exactOptionalPropertyTypes the
// vendor `GenerateImagesConfig` fields are `field?: T` (no `| undefined`),
// so we can only assign the property when we actually have a value.
const sizeAspectRatio = size ? sizeToAspectRatio(size) : undefined

// Named picks, never a wholesale spread β€” the mirror image of the native
// path below. A native-only field (safetySettings, thinkingConfig,
// imageConfig, systemInstruction) belongs to GenerateContentConfig and is
// rejected by generateImages with 400 INVALID_ARGUMENT, so it must not be
// able to reach here even when the caller's `modelOptions` was typed
// against both shapes at once (e.g. an adapter inferred from a union of
// model names).
return {
numberOfImages: numberOfImages ?? 1,
// Map size to aspect ratio if provided (modelOptions.aspectRatio will override)
// Map size to aspect ratio if provided; modelOptions.aspectRatio,
// picked after it, overrides.
...(sizeAspectRatio !== undefined && { aspectRatio: sizeAspectRatio }),
...modelOptions,
...(modelOptions?.aspectRatio !== undefined && {
aspectRatio: modelOptions.aspectRatio,
}),
...(modelOptions?.personGeneration !== undefined && {
personGeneration: modelOptions.personGeneration,
}),
...(modelOptions?.safetyFilterLevel !== undefined && {
safetyFilterLevel: modelOptions.safetyFilterLevel,
}),
...(modelOptions?.seed !== undefined && { seed: modelOptions.seed }),
...(modelOptions?.addWatermark !== undefined && {
addWatermark: modelOptions.addWatermark,
}),
...(modelOptions?.language !== undefined && {
language: modelOptions.language,
}),
...(modelOptions?.negativePrompt !== undefined && {
negativePrompt: modelOptions.negativePrompt,
}),
...(modelOptions?.outputMimeType !== undefined && {
outputMimeType: modelOptions.outputMimeType,
}),
...(modelOptions?.outputCompressionQuality !== undefined && {
outputCompressionQuality: modelOptions.outputCompressionQuality,
}),
...(modelOptions?.guidanceScale !== undefined && {
guidanceScale: modelOptions.guidanceScale,
}),
...(modelOptions?.enhancePrompt !== undefined && {
enhancePrompt: modelOptions.enhancePrompt,
}),
...(modelOptions?.includeSafetyAttributes !== undefined && {
includeSafetyAttributes: modelOptions.includeSafetyAttributes,
}),
...(modelOptions?.includeRaiReason !== undefined && {
includeRaiReason: modelOptions.includeRaiReason,
}),
...(modelOptions?.outputGcsUri !== undefined && {
outputGcsUri: modelOptions.outputGcsUri,
}),
...(modelOptions?.labels !== undefined && {
labels: modelOptions.labels,
}),
}
}

Expand Down
Loading