Skip to content

feat(google-gemini): update model YAMLs [bot]#902

Open
harshiv-26 wants to merge 1 commit intomainfrom
bot/update-google-gemini-20260501-020709
Open

feat(google-gemini): update model YAMLs [bot]#902
harshiv-26 wants to merge 1 commit intomainfrom
bot/update-google-gemini-20260501-020709

Conversation

@harshiv-26
Copy link
Copy Markdown
Collaborator

@harshiv-26 harshiv-26 commented May 1, 2026

Auto-generated by poc-agent for provider google-gemini.


Note

Medium Risk
Medium risk because it changes model configuration used for pricing calculations and enforces a significantly smaller context_window/max_input_tokens for gemini-robotics-er-1.6-preview, which could impact workloads relying on prior limits.

Overview
Updates Google Gemini model metadata YAMLs.

Adds batched input pricing (input_cost_per_token_batches) to gemini-3.1-flash-tts-preview.

Reduces gemini-robotics-er-1.6-preview limits by lowering context_window and max_input_tokens from 1048576 to 131072.

Reviewed by Cursor Bugbot for commit 619d9b9. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

/test-models

@harshiv-26
Copy link
Copy Markdown
Collaborator Author

Gateway test results

  • Total: 17
  • Passed: 15
  • Failed: 0
  • Validation failed: 2
  • Errored: 0
  • Skipped: 0
  • Success rate: 88.24%
Provider Model Scenarios
google-gemini gemini-3.1-flash-tts-preview success: params
google-gemini gemini-robotics-er-1.6-preview success: tool-call, params, tool-call:stream, structured-output:stream, tool-call:stream:google-genai, params:stream, tool-call:google-genai, params:google-genai, params:stream:google-genai, structured-output:stream:google-genai, structured-output, structured-output:google-genai, reasoning:stream:google-genai, reasoning:google-genai

validation_failure: reasoning, reasoning:stream
Failures (2)

google-gemini/gemini-robotics-er-1.6-preview — reasoning (validation_failure)

Error:

Traceback (most recent call last):
  File "/tmp/tmpa5o9yuyh/snippet.py", line 42, in <module>
    raise Exception("VALIDATION FAILED: reasoning - no reasoning information in response")
Exception: VALIDATION FAILED: reasoning - no reasoning information in response
Code snippet
from openai import OpenAI

client = OpenAI(api_key="***", base_url="https://internal.devtest.truefoundry.tech/api/llm")

response = client.chat.completions.create(
    model="test-v2-gemini/gemini-robotics-er-1.6-preview",
    messages=[
        {"role": "user", "content": "Hi"},
        {"role": "assistant", "content": "Hi, how can I help you"},
        {"role": "user", "content": "How to calculate 3^3^3^3? Think step by step and show all reasoning."},
    ],
    reasoning_effort="medium",
    stream=False,
)

_usage = getattr(response, "usage", None)
_reasoning_detected = False

_choices = getattr(response, "choices", None)
if _choices and len(_choices) > 0:
    _message = getattr(_choices[0], "message", None)
else:
    _message = None

if _message and getattr(_message, "content", None) is not None:
    print(_message.content)

if _usage is not None:
    _output_token_details = getattr(_usage, "completion_tokens_details", None)
    if _output_token_details and getattr(_output_token_details, "reasoning_tokens", 0) > 0:
        _reasoning_detected = True
    elif getattr(_usage, "reasoning", None) is not None:
        _reasoning_detected = True

if getattr(_message, "reasoning_content", None) is not None:
    _reasoning_detected = True
elif getattr(_message, "reasoning", None) is not None:
    _reasoning_detected = True

if not _reasoning_detected:
    print("Response: ", response)
    raise Exception("VALIDATION FAILED: reasoning - no reasoning information in response")
print("VALIDATION: reasoning SUCCESS")

google-gemini/gemini-robotics-er-1.6-preview — reasoning:stream (validation_failure)

Error:

Traceback (most recent call last):
  File "/tmp/tmpxz5clhg2/snippet.py", line 34, in <module>
    raise Exception("VALIDATION FAILED: reasoning stream - no reasoning information in stream")
Exception: VALIDATION FAILED: reasoning stream - no reasoning information in stream
Code snippet
from openai import OpenAI

client = OpenAI(api_key="***", base_url="https://internal.devtest.truefoundry.tech/api/llm")

response = client.chat.completions.create(
    model="test-v2-gemini/gemini-robotics-er-1.6-preview",
    messages=[
        {"role": "user", "content": "Hi"},
        {"role": "assistant", "content": "Hi, how can I help you"},
        {"role": "user", "content": "How to calculate 3^3^3^3? Think step by step and show all reasoning."},
    ],
    reasoning_effort="medium",
    stream=True,
)

_reasoning_detected = False
for chunk in response:
    if chunk.choices and len(chunk.choices) > 0:
        delta = chunk.choices[0].delta
        if delta.content is not None:
            print(delta.content, end="", flush=True)
        if getattr(delta, "reasoning_content", None) is not None:
            _reasoning_detected = True
        if getattr(delta, "reasoning", None) is not None:
            _reasoning_detected = True

    _usage = getattr(chunk, "usage", None)
    if _usage is not None:
        _details = getattr(_usage, "completion_tokens_details", None)
        if _details and getattr(_details, "reasoning_tokens", 0) > 0:
            _reasoning_detected = True

if not _reasoning_detected:
    raise Exception("VALIDATION FAILED: reasoning stream - no reasoning information in stream")
print("\nVALIDATION: reasoning stream SUCCESS")

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 619d9b9. Configure here.

@@ -1,5 +1,6 @@
costs:
- input_cost_per_token: 0.000001
input_cost_per_token_batches: 5e-7
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing output batch cost in TTS model pricing

Medium Severity

Adding input_cost_per_token_batches without a corresponding output_cost_per_token_batches is inconsistent with the other TTS models. Both gemini-2.5-pro-preview-tts.yaml (which has identical per-token pricing) and gemini-2.5-flash-preview-tts.yaml include output_cost_per_token_batches at a 50% discount of their output_cost_per_audio_token. This model's output_cost_per_audio_token is 0.00002, so the expected batch output cost would be 0.00001.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 619d9b9. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant