hallucinations_v1: sentence validator ignores grounding_metadata

[ruben-cid]

🔴 Required Information

Describe the Bug:
HallucinationsV1Evaluator (google/adk/evaluation/hallucinations_v1.py) never includes model-supplied grounding metadata (InvocationEvent.grounding_metadata) in the context it builds for the sentence-validator judge. _create_context_for_step() only assembles developer instructions, user prompt, tool declarations, and per-event function_call/function_response/text parts — it never calls the existing get_grounding_metadata_as_json_str() helper (defined in llm_as_judge_utils.py) the way the sibling rubric_based_final_response_quality_v1.py metric does. For agents that use built-in/model-internal grounding tools (e.g. VertexAiSearchTool, google_search), search results only ever arrive as groundingMetadata on the event, never as a function_response part. As a result, sentences that are genuinely grounded by search have no supporting evidence in the judge's <context>, and the judge labels them unsupported, inflating the hallucination rate for any search-grounded agent.

Steps to Reproduce:

1. Build an ADK agent whose only retrieval mechanism is a model-internal grounding tool, e.g. VertexAiSearchTool (no explicit function_call/function_response in the event stream, only event.grounding_metadata).
2. Run an eval with metrics_to_run: [hallucination] (HallucinationsCriterion, hallucinations_v1.py) against a case where the final response correctly cites/uses retrieved document content.
3. Inspect the per-sentence validator output (or just the resulting score) for the invocation.
4. Compare hallucinations_v1.py::_create_context_for_step against rubric_based_final_response_quality_v1.py::_build_auto_rater_prompt, which imports and injects get_grounding_metadata_as_json_str(...) — hallucinations_v1.py has no equivalent import/usage.

Expected Behavior:
Sentences that are supported by the retrieved grounding chunks (search results attached via event.grounding_metadata) should be scored supported, since the information genuinely came from a trusted retrieval source.

Observed Behavior:
Sentences grounded only via grounding_metadata (no matching function_call/function_response part) are scored unsupported/hallucinated by the sentence validator, because the context string passed to the judge never contains the grounding chunks/search results — only explicit tool call/response JSON. This drags down the metric's Accuracy Score for any agent that relies on model-internal search grounding rather than explicit function-tool calls.

Environment Details:

• ADK Library Version (pip show google-adk): 2.9.1
• Desktop OS: macOS (Darwin 24.6.0, arm64)
• Python Version (python -V): 3.12.14

Model Information:

• Are you using LiteLLM: No
• Which model is being used: gemini-3.1-flash-lite (agent model, via Gemini(model=GEMINI_MODEL)); judge model for the hallucination criterion uses the ADK default (not overridden in this project's eval config)

---

🟡 Optional Information

Regression:
N/A — this appears to be a design gap since the metric's introduction, not a regression. rubric_based_final_response_quality_v1.py already handles grounding metadata correctly, so the pattern to fix it exists in the same codebase.

Logs:

N/A — behavioral/code-path issue, not a crash. Confirmed via static review of
google/adk/evaluation/hallucinations_v1.py (context builder, lines ~309-494)
vs. google/adk/evaluation/rubric_based_final_response_quality_v1.py (lines ~28,
50, 308-347) and google/adk/evaluation/llm_as_judge_utils.py
(get_grounding_metadata_as_json_str, lines ~209-240).

Screenshots / Video:
N/A

Additional Context:
We independently worked around an analogous problem for a different metric in our own eval config (tests/eval/eval_config.yaml, search_evidence custom metric): VertexAiSearchTool only ever adds grounding_metadata, never a function_call event, so adaptive-rubric judges without grounding-metadata support routinely mark search/grounding criteria as unverifiable even when a search genuinely happened. The same root cause applies here to hallucinations_v1.py.

This is the same class of bug already fixed for a sibling metric: PR #5834 ("fix: include grounding metadata in rubric judge prompt", merged 2026-08-14, resolving #5831) added exactly this — grounding-metadata plumbing into the judge prompt — but only for rubric_based_final_response_quality_v1.py (touching eval_case.py, evaluation_generator.py, llm_as_judge_utils.py, and that file). hallucinations_v1.py was not updated in that PR and still has the gap described above. Suggested fix: apply the same pattern PR #5834 used — call get_grounding_metadata_as_json_str(...) in _create_context_for_step and add validator-prompt instructions analogous to the rubric-based metric's: "model-supplied grounding metadata is trusted evidence for model-internal tools such as google_search/VertexAiSearchTool whose raw search results may not appear as function tool responses." Note PR #5834's own stated limitation ("final answer text is still not treated as evidence") likely applies equally here and is worth calling out if filing upstream.

Minimal Reproduction Code:

# No standalone repro needed — the gap is visible by diffing the two files:
#   google/adk/evaluation/hallucinations_v1.py::_create_context_for_step
#   google/adk/evaluation/rubric_based_final_response_quality_v1.py (imports
#   get_grounding_metadata_as_json_str and injects a <grounding_metadata> block;
#   hallucinations_v1.py does neither).

How often has this issue occurred?:

• Always (100%) — for any invocation where a sentence's only support is grounding metadata rather than an explicit tool call/response.

[chelsealong]

Picking this one up now — opening a PR shortly. Flagging it here so nobody duplicates the work; if someone is already on it, say so and I will drop mine.

[surajksharma07]

@ruben-cid Confirmed the gap by diffing against rubric_based_final_response_quality_v1.py, the context string really does drop grounding_metadata on search-only steps. chelsealong's #7159 patches it using the same get_grounding_metadata_as_json_str pattern the rubric metric already relies on. Could you pull that branch and re-run your eval against the VertexAiSearchTool case to confirm it fixes the scoring before it merges? If it holds on your end flag it on the PR so we can move it toward merge.

[ruben-cid]

@surajksharma07

Hello.

I tried with the new changes and it works. The metric returns the expected range of scores.