[Bug] LiteLlm sends invalid PDF content for Azure-backed models through litellm_proxy

[ross-p]

🔴 Required Information

Describe the Bug:

google.adk.models.lite_llm.LiteLlm does not correctly handle PDF
attachments when an Azure-backed model is accessed through a LiteLLM Proxy
using a nested model identifier such as:

litellm_proxy/azure/<deployment>

ADK derives provider-specific file behavior from the first segment of the
model identifier. _get_provider_from_model() therefore classifies the
provider as litellm_proxy, rather than recognizing that the proxied model
ultimately uses Azure.

This bypasses ADK's OpenAI/Azure file-upload path. Instead of uploading the PDF
and sending a file_id, ADK produces a generic inline file block containing
only file_data. When the proxy translates that block for the Azure Responses
API, Azure rejects the resulting input_file content item as incomplete.

Steps to Reproduce:

1. Install the relevant packages:

   pip install google-adk litellm google-genai

2. Configure a LiteLLM Proxy model backed by an Azure OpenAI Responses API
   deployment.

3. Construct an ADK LiteLlm using a nested proxy model identifier:

   LiteLlm(model="litellm_proxy/azure/<deployment>")

4. Send a user message containing an application/pdf inline_data part.

5. Observe that Azure rejects the request before inference with an error such
   as:

   BadRequestError: AzureException BadRequestError
   Missing required parameter: 'input[0].content[3]'
   

   The indexed content item is the PDF attachment.

A network-free reproduction of the conversion decision is:

import asyncio

from google.adk.models import lite_llm
from google.genai import types

model = "litellm_proxy/azure/example-deployment"

parts = [
    types.Part(text="Context"),
    types.Part(text="Describe this PDF"),
    types.Part(text="Attached file: example.pdf"),
    types.Part(
        inline_data=types.Blob(
            mime_type="application/pdf",
            data=b"%PDF-1.4\n% minimal reproduction\n",
        )
    ),
]

provider = lite_llm._get_provider_from_model(model)
content = asyncio.run(
    lite_llm._get_content(parts, provider=provider, model=model)
)

print("provider:", provider)
print("file keys:", content[-1]["file"].keys())

Observed output:

provider: litellm_proxy
file keys: dict_keys(['file_data'])

The resulting file block is equivalent to:

{
  "type": "file",
  "file": {
    "file_data": "data:application/pdf;base64,..."
  }
}

It has no Azure/OpenAI file_id and no filename.

Expected Behavior:

PDF attachments should remain usable when an Azure-backed model is accessed
through LiteLLM Proxy.

ADK should either:

• preserve enough underlying-provider information to select the appropriate
  Azure/OpenAI file representation, or
• provide another supported way to communicate the proxied provider's file
  requirements.

At minimum, it should not produce a request that is reported as supported but
is deterministically rejected by the downstream provider.

Observed Behavior:

ADK treats litellm_proxy as the final provider and takes its generic inline
file-data path. The proxy subsequently forwards an incomplete input_file
representation to Azure, which rejects the request with HTTP 400 before any
model inference occurs.

Environment Details:

• ADK Library Version: google-adk 2.6.0
• Desktop OS: macOS
• Python Version: 3.11
• LiteLLM Version: 1.94.1
• google-genai Version: 2.16.0

Model Information:

• Are you using LiteLLM: Yes
• Which model is being used: an Azure OpenAI Responses API deployment accessed
  through litellm_proxy/azure/<deployment>

---

🟡 Optional Information

Regression:

Not known to have worked in an earlier ADK release. Current provider detection
takes only the first segment of a LiteLLM model identifier, so this appears to
be a missing proxy-routing case rather than a regression.

Logs:

BadRequestError: litellm.BadRequestError: Error code: 400
AzureException BadRequestError:
Missing required parameter: 'input[0].content[3]'.
type: invalid_request_error
code: missing_required_parameter

No model call is completed and no tokens are consumed.

Additional Context:

The direct-provider cases have related prior reports:

• Cannot send attach file to OpenAI throw LiteLlm #2862: attachment requests missing file.file_id
• [BUG] LiteLLM (OpenAI/gpt-4o-mini): load_artifacts PDF attachment fails with “Invalid file data: 'file_id' … expected application/pdf, got None” #4174: PDF upload lost MIME metadata for direct OpenAI/Azure routes
• Title LiteLlm: Azure OpenAI file_id with assistant- prefix bypasses Responses-API content block #5664: Azure assistant- file IDs bypassed the Responses API content block
• LiteLLM wrapper doesn't support GCS file URIs (gs://) when using proxy #3131: file URIs were dropped when using a LiteLLM proxy

Those reports cover adjacent file-conversion defects, but they do not appear to
cover provider-specific file handling being lost because the model identifier
starts with litellm_proxy/.

The relevant provider detection currently behaves approximately as follows:

provider, _ = model.split("/", 1)

For litellm_proxy/azure/<deployment>, this necessarily returns
litellm_proxy. The PDF converter's upload path, however, is selected only
when the provider is exactly openai or azure.

How often has this issue occurred?:

• Always (100%) for this route and PDF input

Relationship to #6539

This issue is related to #6539 because both affect PDF input through LiteLlm, but they are not duplicates:

• This issue ([Bug] LiteLlm sends invalid PDF content for Azure-backed models through litellm_proxy #6538) occurs for litellm_proxy/azure/<deployment>. ADK identifies the provider as litellm_proxy, never enters the Azure/OpenAI upload branch, and emits inline file_data that becomes an invalid Azure input_file.
• [Bug] LiteLlm uploads PDFs to Azure without filename or MIME metadata #6539 occurs for the direct azure/<deployment> route. ADK correctly identifies Azure and does attempt a Files API upload, but supplies anonymous raw bytes without a filename or MIME metadata.

Fixing #6539 alone would not make the proxy route enter Azure-specific file handling. Conversely, fixing this issue so the nested route selects the Azure upload path may then expose #6539 unless the upload-metadata defect is also resolved.

[SHAI-nikhil-chaudhary]

Hi @ross-p ,
I just submitted PR #6548 to fix the related issue #6539 regarding the missing Azure metadata tuple, I'd love to take a swing at fixing this proxy routing issue as well!
Looking at the root cause, it seems we need to update the provider in _FILE_ID_REQUIRED_PROVIDERS check to also inspect the second segment of the model string if the first segment is litellm_proxy.
I was thinking of an approach like this:

proxied_provider = model.split("/")[1] if is_proxy and len(model.split("/")) > 1 else None

if provider in _FILE_ID_REQUIRED_PROVIDERS or proxied_provider in _FILE_ID_REQUIRED_PROVIDERS:
    # Trigger the Azure/OpenAI multipart upload