[BUG] Local vllm devstral2 -> OpenAI completion error

[maksir]

Problem (one or two sentences)

I am using Roo-Code v3.39.3 on VS Code 1.108.0
Provider: local vllm serve mistralai/Devstral-Small-2-24B-Instruct-2512 --served-model-name devstral

`Date/time: 2026-01-13T11:54:57.088Z
Extension version: 3.39.3
Provider: openai
Model: devstral

400
OpenAI completion error: 400 Unexpected role 'user' after role 'tool' Unexpected role 'user' after role 'tool'`

Context (who is affected and when)

..

Reproduction steps

..

Expected result

no error

Actual result

error 400

Variations tried (optional)

No response

App Version

3.39.3

API Provider (optional)

None

Model Used (optional)

No response

Roo Code Task Links (optional)

No response

Relevant logs or errors (optional)

[roomote]

I have analyzed this issue and here is my proposed implementation plan:

Root Cause Analysis

The error "Unexpected role 'user' after role 'tool'" occurs because:

1. When Roo Code sends tool calls to your vLLM/Devstral endpoint, the conversation history includes tool results followed by user feedback text
2. The current OpenAI provider converts this to separate messages: role: "tool" (for results) followed by role: "user" (for feedback text)
3. Many OpenAI-compatible servers (including vLLM with Mistral models) do not accept a user message immediately after a tool message

Proposed Fix

Modify src/api/providers/openai.ts to enable the existing mergeToolResultText option when calling convertToOpenAiMessages(). This option:

• Merges any text content following tool results into the last tool message
• Prevents creating a separate user message after tool messages
• Is already successfully used by other providers (DeepSeek, Z.ai) for similar compatibility issues

Files to modify:

1. src/api/providers/openai.ts - Update all calls to convertToOpenAiMessages() to pass { mergeToolResultText: true }

   • Line 125: streaming mode
   • Line 232: non-streaming mode
   • Lines 354, 392: O3 family handlers

2. Add/update tests to verify the fix works correctly

This approach:

• Uses an existing, well-tested solution pattern
• Provides broader compatibility with OpenAI-compatible APIs (vLLM, llama.cpp, Ollama, etc.)
• Has minimal risk since the same pattern is already used by DeepSeek and Z.ai providers

Please mention @roomote in a comment saying that you approve this plan, and I'll proceed with the implementation.

[maksir]

@roomote Isn't this related to #10540 with the same error message for the same model but a different provider?

[roomote]

@maksir Yes, you are correct! Issue #10684 and #10540 share the same root cause - the "Unexpected role 'user' after role 'tool'" error occurs because strict OpenAI-compatible servers (including vLLM and NVIDIA NIM with Devstral models) do not accept a user message immediately after a tool message.

PR #10598 addresses this for OpenAI Compatible providers by adding the openAiStrictToolMessageOrdering setting. Since this issue (#10684) also involves the same underlying problem (user using local vLLM which presents as an OpenAI-compatible endpoint), that PR should resolve both issues once merged.

Thank you for pointing out the connection - I'll mark this issue as related to #10540 so they can be tracked together.

[Hullabalo]

Hi @maksir . this is not the same model as in #10540, (Devstral2 123b vs. 24b). But I can reproduce the same error as yours with Devstral2 24b using LMStudio/llama.cpp. Locally it's possible to resolve this by using a custom Jinja template (I suppose this solution could also works with vllm ?), for example something like this :

{%- set today_date = strftime_now("%Y-%m-%d") %}

{%- set default_system_message =
"You are Devstral-Small-2-24B-Instruct-2512, a Large Language Model created by Mistral AI.
Your knowledge base was last updated on 2023-10-01.
The current date is " + today_date + ".

You follow the instructions in all languages and always reply in the user's language." %}

{{- bos_token }}

{# ---------- SYSTEM PROMPT ---------- #}
{%- if messages and messages[0]['role'] == 'system' %}
    {{- '[SYSTEM_PROMPT]' }}
    {{- messages[0]['content'] }}
    {{- '[/SYSTEM_PROMPT]' }}
    {%- set loop_messages = messages[1:] %}
{%- else %}
    {{- '[SYSTEM_PROMPT]' + default_system_message + '[/SYSTEM_PROMPT]' }}
    {%- set loop_messages = messages %}
{%- endif %}

{# ---------- TOOLS ---------- #}
{%- if tools is defined and tools and tools|length > 0 %}
    {{- '[AVAILABLE_TOOLS]' + (tools | tojson) + '[/AVAILABLE_TOOLS]' }}
{%- endif %}

{# ---------- CONVERSATION ---------- #}
{%- for message in loop_messages %}

    {%- if message['role'] == 'user' %}
        {{- '[INST]' }}
        {%- if message['content'] is string %}
            {{- message['content'] }}
        {%- elif message['content'] is iterable %}
            {%- for block in message['content'] %}
                {%- if block['type'] == 'text' %}
                    {{- block['text'] }}
                {%- elif block['type'] in ['image', 'image_url'] %}
                    {{- '[IMG]' }}
                {%- endif %}
            {%- endfor %}
        {%- endif %}
        {{- '[/INST]' }}

    {%- elif message['role'] == 'assistant' %}
        {%- if message['content'] %}
            {%- if message['content'] is string %}
                {{- message['content'] }}
            {%- elif message['content'] is iterable %}
                {%- for block in message['content'] %}
                    {%- if block['type'] == 'text' %}
                        {{- block['text'] }}
                    {%- endif %}
                {%- endfor %}
            {%- endif %}
        {%- endif %}

        {%- if message['tool_calls'] is defined and message['tool_calls'] %}
            {%- for tool in message['tool_calls'] %}
                {%- set args = tool['function']['arguments'] %}
                {{- '[TOOL_CALLS]' + tool['function']['name'] + '[ARGS]' +
                    (args if args is string else args | tojson) }}
            {%- endfor %}
        {%- endif %}

        {{- eos_token }}

    {%- elif message['role'] == 'tool' %}
        {{- '[TOOL_RESULTS]' + message['content']|string + '[/TOOL_RESULTS]' }}

    {%- endif %}

{%- endfor %}

[PeterSmarthome]

@roomote Isn't this related to #10618 with the same error message for the same model-family (mistralai/*-2512) but a different provider?

[roomote]

@PeterSmarthome Yes, you're right! Issue #10618 (OpenRouter + Mistral/Devstral) and this issue (#10684 - local vLLM + Devstral) share the same root cause: Mistral-family models enforce strict message ordering and reject a "user" role message immediately after a "tool" role message.

The fix is the same approach across all providers - using the mergeToolResultText option to merge text content into the last tool message instead of creating a separate user message. Here's the landscape of fixes:

• [BUG] Code mode does not work with devstral2 123b 2512 (NVIDIA NIM provider) #10540 (OpenAI Compatible + Mistral): Fixed by PR feat: add strict tool message ordering setting for OpenAI Compatible providers #10598 with a user setting
• [BUG] openrouter:mistralai/devstral-2512 -> "completion error" #10618 (OpenRouter + Mistral/Devstral): PR fix: add mergeToolResultText for Mistral/Devstral models in OpenRouter #10630 adds automatic mergeToolResultText for Mistral models
• [BUG] Local vllm devstral2 -> OpenAI completion error #10684 (OpenAI provider + local vLLM/Devstral): Needs the same fix in src/api/providers/openai.ts

I'll proceed with implementing the fix for the OpenAI provider now, which will address this issue for local vLLM users.

[roomote]

Opened PR #10695 with my attempt at addressing this for the OpenAI provider.

The fix adds mergeToolResultText: true to all convertToOpenAiMessages() calls in src/api/providers/openai.ts, which merges text content (like environment_details) into the last tool message instead of creating a separate user message. This prevents the "Unexpected role 'user' after role 'tool'" error.

This is the same fix approach used by:

• PR fix: add mergeToolResultText for Mistral/Devstral models in OpenRouter #10630 for OpenRouter's Mistral/Devstral models
• DeepSeek, Z.ai, and MiniMax providers for similar issues

Feedback welcome!