Description
When running the automated evaluation using gemini-3.1-flash-lite, the Rust LiveKit runner successfully completes the session (interview_complete). However, the agent fails during the final report generation. The LLM outputs an unsupported string (unsupported delivery or personality judgment) for the $.improvementPlan[3].selfReview and successCriterion fields, which violates the defined JSON schema. The system attempts to auto-repair the JSON twice but fails, ultimately crashing without generating the final assessment scores.
Error Logs
Rust LiveKit runner ended (interview_complete) but Gemini report generation failed for model gemini-3.1-flash-lite on remove-element.
Final editor state: 251 bytes of c.
Error: Gemini report failed schema validation after 2 repairs: $.improvementPlan[3].selfReview[0]: unsupported delivery or personality judgment; $.improvementPlan[3].successCriterion: unsupported delivery or personality judgment.
Your session ran end-to-end, but no scores were produced, so nothing here is an assessment of your work. Check the agent logs and GOOGLE_API_KEY, then try again.
Expected vs Actual Behavior
- Expected Behavior: The agent should successfully validate the LLM's structured output against the schema, generate the final evaluation report, and provide the assessment scores.
- Actual Behavior: The agent framework fails to parse the string, exhausts its repair attempts, and aborts the grading process entirely.
Environment
- OS: Windows
- Access Method: Web browser via local server (
http://127.0.0.1:3000)
Description
When running the automated evaluation using
gemini-3.1-flash-lite, the Rust LiveKit runner successfully completes the session (interview_complete). However, the agent fails during the final report generation. The LLM outputs an unsupported string (unsupported delivery or personality judgment) for the$.improvementPlan[3].selfReviewandsuccessCriterionfields, which violates the defined JSON schema. The system attempts to auto-repair the JSON twice but fails, ultimately crashing without generating the final assessment scores.Error Logs
Rust LiveKit runner ended (interview_complete) but Gemini report generation failed for model gemini-3.1-flash-lite on remove-element.$.improvementPlan[3].selfReview[0]: unsupported delivery or personality judgment; $ .improvementPlan[3].successCriterion: unsupported delivery or personality judgment.
Final editor state: 251 bytes of c.
Error: Gemini report failed schema validation after 2 repairs:
Your session ran end-to-end, but no scores were produced, so nothing here is an assessment of your work. Check the agent logs and GOOGLE_API_KEY, then try again.
Expected vs Actual Behavior
Environment
http://127.0.0.1:3000)