Skip to content

Fixing MiniCPM5 and Mistral bfcl errors - #4479

Open
przepeck wants to merge 7 commits into
mainfrom
przepeck/parser_minor_fixes
Open

Fixing MiniCPM5 and Mistral bfcl errors#4479
przepeck wants to merge 7 commits into
mainfrom
przepeck/parser_minor_fixes

Conversation

@przepeck

Copy link
Copy Markdown
Collaborator

🛠 Summary

CVS-192979
Hanlding exceptions thrown in tool parsers, fixing Mistrals.
Temporary fix for MiniCPM5 util servable refactor is introduced

🧪 Checklist

  • Unit tests added.
  • The documentation updated.
  • Change follows security best practices.
    ``

@przepeck
przepeck requested review from mzegla and pgladkows and a lite review from Copilot August 26, 2026 06:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves robustness of OVMS LLM streaming by preventing tool/output-parser exceptions from unwinding GenAI’s internal generate() loop (which can leave the underlying InferRequest “busy”), and adds regression tests for Mistral and MiniCPM5 parser edge cases.

Changes:

  • Catch output-parser exceptions inside OVMSTextStreamer flush and cancel generation gracefully while recording a parser-error flag.
  • Propagate parser failure to legacy executors by checking OVMSTextStreamer::hadParserError() after generate().
  • Add tests for Mistral streaming tool-call ordering and MiniCPM5 unterminated reasoning blocks; harden MiniCPM5 reasoning token stripping for missing end token.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/test/llm/output_parsers/mistral_output_parser_test.cpp Adds a regression test for a streaming tool-call JSON shape where arguments appears before name.
src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Adds a regression test ensuring unterminated <think>/reasoning does not crash and still yields tool calls.
src/test/http_openai_handler_test.cpp Adds an integration-style test verifying parser exception cancels streaming and leads to request failure.
src/llm/visual_language_model/legacy/legacy_executor.cpp Marks generation as failed when the text streamer reports a parser error.
src/llm/ovms_text_streamer.hpp Exposes hadParserError() to let callers detect parser failures after generation returns.
src/llm/ovms_text_streamer.cpp Wraps parseChunk() in try/catch; cancels generation and emits a final callback on parser errors.
src/llm/omni_model/legacy/legacy_executor.cpp Marks generation as failed when the text streamer reports a parser error.
src/llm/language_model/legacy/legacy_executor.cpp Marks generation as failed when the text streamer reports a parser error.
src/llm/io_processing/minicpm5/minicpm5_tool_parser.cpp Avoids UB/crash when the reasoning end token is missing by keeping tokens unchanged.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/test/llm/output_parsers/mistral_output_parser_test.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants