Why is there an audioTimeStamp flag that can not be used for Gemini File API

[Zonkodonko]

Hi.
Im using google-genai:1.24 and I want to create transcriptions with timestamps and speaker diarization.
For that I use gemini-2.5-flash.
My content config looks like this:

GenerateContentConfig contentConfig = GenerateContentConfig.builder()  
						.audioTimestamp(true)  
						.responseSchema(responseSchema)   
						.build();
GenerateContentResponse contentResponse = client.models.generateContent(MODEL, requestContent,contentConfig) 

But this code throws:

java.lang.IllegalArgumentException: audioTimestamp parameter is not supported in Gemini API.

Why is there and audioTimestamp flag when I can not use it?
I read that this is used for the VertexAI Speech-To-Text but I couldn't find any documentation where this function or even this library is used.
Isn't the library for Speech-To-Text Vertex AI "google-cloud-speech" ?
Im really confused about all those google genai librarys.

[hemasekhar-p]

Hi @Zonkodonko, Thank you for reaching out. The audioTimestamp feature is currently exclusive to the Vertex AI backend. It is not supported when using the apiKey. To enable this functionality, you will need to switch your client initialization to Vertex AI. This requires a Google Cloud project and a specified location. A possible solution is shown below And you may also find the this documentation helpful.

Solution :

public static void solution() {
        Client client = Client.builder()
            .vertexAI(true)
            .project(GenAIConstants.GOOGLE_CLOUD_PROJECT)
            .location("us-central1")
            .build();

        GenerateContentConfig config = GenerateContentConfig.builder()
            .audioTimestamp(true)
            .build();

        Part audioPart = Part.builder()
            .fileData(com.google.genai.types.FileData.builder()
                .mimeType("audio/mp3")
                .fileUri("pixel.mp3")  //PROVIDE AUDIO URL 
                .build())
            .build();

        Content content = Content.builder()
            .role("user")
            .parts(Collections.singletonList(audioPart))
            .build();

        try {
            GenerateContentResponse response = client.models.generateContent(
                "gemini-2.5-flash", 
                content, 
                config
            );

            System.out.println("Response: " + response.text());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    

[Zonkodonko]

Hello @hemasekhar-p

Thank you for the response. I have some follow up questions:

1. When I experimented with Vertex AI, I used the com.google.cloud.speech.v1.SpeechClient. Did I use the wrong library? It seems like it since your response is based on based on the com.google.genai.Client
2. Why is there a audioTimestamp flag anyway? I can just add to the prompt or to the response schema that I want a timestamp.
3. I have a big problem undestanding the difference between Vertex AI and Gemini File API. Is there documentation that explains it better? The documentation you send is not really good in explaining when the code examples work for Vertex AI and when it works for Gemini File API. What are the key differences?

[hemasekhar-p]

Hello @Zonkodonko, please find the following clarifications regarding your questions.

Library Diff: It depends on your usecase. SpeechClient is for the dedicated Speech-to-Text service (transcription only). genai.Client is for the Gemini multimodal model (understanding + transcription). If you want Gemini's reasoning power, stick with genai.Client.

AudioTimestamp Flag: Prompting for timestamps often results in 'hallucinations' (incorrect times). The audioTimestamp flag activates a specialized mode in Gemini on Vertex AI that links the output directly to the audio clock for high-precision accuracy that prompting alone cannot match.

Documentation:
Gemini File API
Vertex AI
Compare Gemini API and Vertex AI