ReActV2 finish: :submit: a thought-only step continues before submit is forced

CHANGELOG.md

@@ -34,6 +34,29 @@ User-visible changes to Imp are recorded here.
 
 ### Changed
 
+- Under ReActV2's `finish: :submit`, a step that writes a thought and calls
+  no tool is now followed by an ordinary step (every tool,
+  `tool_choice: "auto"`, the thought shown as the request's last message when
+  the tools are held natively)
+  instead of the forced `submit`. A second thought-only step in a row, or a
+  step that says nothing or only whitespace, still forces `submit`
+  (`:empty_tool_calls`), as does `max_iters`. Once the deadline has passed,
+  no further request is sent, continued or forced. A model that announced an action can take
+  it, and a reply in the thought is usually submitted unforced, also on
+  providers that cannot force a tool. Other signatures with `submit` are
+  unchanged.
+- Once the process's `Imp.Deadline` has passed, ReActV2 sends no forced
+  `submit`, no `"required"` retry and no extraction request (and, under
+  `finish: :submit`, no continued step); the turn is `:incomplete` with
+  `termination_cause: :deadline_exceeded`, as the last text request already
+  was. This applies to every signature with `submit`, `finish: :auto`
+  included.
+- An `:extracted` ReActV2 turn's history now ends with the extracted
+  outputs, recorded as a step that called no tool (with any inputs no step
+  spent), so the history holds the turn's answer. This applies under
+  `finish: :auto` too. Such a step shows the model nothing when the history
+  is replayed: the request still ends on the thought before it, shown as a
+  user message.
 - A ReActV2 forced `submit` request no longer ends on the model's own message.
   When the step before it wrote a thought and called nothing, the thought was
   the request's last message, as an assistant turn, and providers refuse that:

decisions.md

@@ -8,6 +8,7 @@ not necessarily when it was made.
 
 | Date | Decision | Source and reason | Status | Retires when |
 | --- | --- | --- | --- | --- |
+| 2026-10-16 | Under `finish: :submit`, a ReActV2 step that writes a thought and calls no tool is followed by an ordinary step (every tool, `tool_choice: "auto"`, the thought as the request's last user message when the tools are held natively), not at once by the forced `submit`; a second thought-only step in a row, or a step that says nothing or only whitespace, is the `:empty_tool_calls` interruption. `max_iters` ends the turn with the forced `submit` as before; once the deadline has passed no further request is sent and the turn is `:incomplete` (`:deadline_exceeded`). With written tool calls the request ends on the tool listing, after the thought. Signatures with `submit` under `finish: :auto` keep DSPy's ending. | `lib/imp/predict/react_v2.ex` `continue_after_thought?/3`, `test/react_v2_thought_continues_test.exs`. Forcing at once took every other tool away from a model that had just announced its next action, and added a request whose failure lost the reply the thought held. Measured 2026-10-16 on 42 real final steps from 11 models through OpenRouter, sending that ordinary step: most samples called `submit` with the thought's reply, copied or paraphrased (Meta's models 6 of 6, unforced; Kimi, GPT and Gemini models likewise); a thought that was a closing note was submitted as an empty answer; a DeepSeek model went on to fetch what it had said it would read; 5 of 42 wrote text again. DSPy's ReActV2 forces `submit` after any step that calls no tool. | In force. | A measurement shows a different bound, or forcing at once, loses fewer answers. |
 | 2026-10-16 | ReActV2's forced `submit` request, when the step before it wrote a thought and called nothing, shows that thought as the request's last message, a user message under its field's marker (`[[ ## next_thought ## ]]`), not as the assistant turn the history records it as. DSPy's ReActV2 leaves the thought out of the forced request. | `lib/imp/adapter/chat.ex` `thought_as_request/4`, `test/react_v2_finish_submit_test.exs`. As a trailing assistant turn the request is refused by providers that do not take an assistant prefix: Mistral ("Cannot set an assistant prefix with function calling") and Gemini ("Requests ending with a model turn are not supported"), probed through OpenRouter 2026-10-16. Dropping it, as DSPy does, made models regenerate finished answers with reasoning off, 2 of 16 to the output limit (deepfates/imp#336). Measured 2026-10-16 on real final steps re-sent through OpenRouter: with the thought as this user message, the submitted answer kept the plain-text reply (text similarity 0.8 to 1.0 across Kimi, GPT, DeepSeek and Muse models), a model whose plain text was a closing note submitted an empty answer as it intended, Meta's models (which accept only `tool_choice: auto`) copied the reply into `submit` 3 of 3, and Gemini accepted every request; with the thought dropped, the models wrote a different answer (similarity about 0.05), one shrank to 30 characters, and the closing-note model submitted a 170-character answer. | In force. | A measurement shows the thought is better dropped, or DSPy keeps it in a form providers accept. |
 | 2026-10-12 | In that plain-text step (row of 2026-10-07; a program with `finish: :auto`, the default), the text output is not presented as a field: the system message lists no output fields, the default agent line names no output ("You are an Agent. Use the supplied tools to gather information and take actions."), there is no "The outputs to produce are:" list, and the finish line is "When the final answer is ready, write it as plain text without calling a tool." followed by the field's description, once. Amended 2026-10-15: the finish line was "... reply without calling a tool."; "reply" is a common tool name, so for a program holding a `reply` tool it read as an instruction about that tool, and it now uses the sentence the other tool-less text answer already used. A host's `agent_line` still replaces the agent line. | `lib/imp/adapter/chat.ex` `default_system/2`, `with_guidance/5`; `test/react_v2_plain_text_answer_test.exs`. The answer is the whole message, so a named, typed output field described a place in the message that does not exist; models on several providers wrote the name into the message as a label (`answer:`, `**answer**`, `[[ ## answer ## ]]`), after notes about the turn, and the whole message, label and notes included, was the answer. The description was listed twice. DSPy has no plain-text finish to follow (row of 2026-10-07). | In force. | ReActV2 gives that signature a `submit` tool, or DSPy defines a plain-text finish whose prompt differs. |
 | 2026-10-08 | A stored history turn shows the outputs it recorded and nothing for the rest: no filler, and not DSPy 3.2.1's `None`. A turn with no recorded output is its user message alone. A predictor that names a tool-calls output and is not a tool loop keeps its recorded outputs as the assistant text on native replay. The system message of every format leaves out a history input, since history arrives as turns: a field typed `history`, or one the call supplies as an `Imp.History`. | deepfates/imp#261, deepfates/imp#258. `lib/imp/adapter/chat.ex` `recorded_reply/3`, `system_signature/2`, `test/history_test.exs`. A filler or `None` puts words in the model's mouth that it never wrote, and a `[[ ## history ## ]]` section in the structure is one no message in the request contains. DSPy 3.2.1 does both (`ChatAdapter.format_assistant_message_content`, `Adapter.format`). | In force. | DSPy stops writing missing history outputs or listing the history input, and Imp follows it. |

docs/diving-deeper/react.md

@@ -52,15 +52,28 @@ A constrained output never takes that path: its allowed values are in
 `submit`'s schema, and text is not an answer to it.
 
 **`finish: :submit`** gives the one-text-output signature `submit` as well,
-so its turn ends as DSPy's ReActV2 ends every turn: a step that writes text
-and calls no tool is a thought (`:empty_tool_calls`), and the forced
-`submit` follows (section 3). Use it when the model writes text it does not
-mean as the answer: a sentence about what it will do next, or a report of
-what it just did. The text then stays beside the calls as the step's
-thought, and only `submit`'s argument is the answer. It costs one more
-request whenever the model ends in prose, and it needs a provider that can
-force a tool, or the extractor answers. The default, `finish: :auto`, lets
-the signature decide as above. The option is saved with the program.
+so its turn ends only through `submit`: a step that writes text and calls no
+tool is a thought, not the answer. Use it when the model writes text it does
+not mean as the answer: a sentence about what it will do next, or a report
+of what it just did. The text then stays in the history as the step's
+thought, and only `submit`'s argument is the answer.
+
+After such a step the loop goes on with an ordinary step: every tool,
+`tool_choice: "auto"`, and the thought shown as the request's last message
+when the tools are held natively (as in section 3; with written tool calls
+the request ends on the tool listing, after the thought). The model can
+then do what it said it would, or submit what it wrote; a thought that was
+only a closing note can be submitted as an empty answer. A second
+thought-only step in a row, or a step that says nothing, interrupts the turn
+(`:empty_tool_calls`) and the forced `submit` follows. Once the process's
+`Imp.Deadline` has passed, no further request is sent. Measured on real final steps from 11 models, most submitted the
+thought's reply on that ordinary step, unforced, Meta's models included,
+whose provider cannot force a tool; some went on with the work they had
+announced; about one in eight wrote text again. Forcing at once, as DSPy
+does, took the announced action away and added a request whose failure
+lost the reply. It costs one more request whenever the model ends in prose.
+The default, `finish: :auto`, lets the signature decide as above. The
+option is saved with the program.
 
 The prompt says the same. When the provider holds the tools natively, the
 Chat adapter's system message for `Imp.react("intent -> answer", [look])`
@@ -104,12 +117,15 @@ parse fails, or when a step gives neither a tool call nor an answer. Rather
 than return nothing, Imp makes one more request:
 
 - With `submit`, the request names `submit` as the required tool, as DSPy
-  does (`:forced_submit`). A provider that refuses a named `tool_choice`
+  does (`:forced_submit`). If the process's `Imp.Deadline` has already
+  passed, no request is made and the turn is `:incomplete`
+  (`:deadline_exceeded`). A provider that refuses a named `tool_choice`
   (its error names `tool_choice` and says what it supports, as Meta's
   "only `auto` is supported" does through OpenRouter) is asked again with
   `"required"` and `submit` as the only tool; if it refuses
   that too, or answers without `submit`, a typed extractor with no tools
-  reads the history and fills the outputs (`:extracted`).
+  reads the history and fills the outputs (`:extracted`). The history then
+  ends with those outputs, recorded as a step that called no tool.
 
   When the step before it wrote a thought and called nothing, that thought
   is the newest thing in the history, and the request would end on the
@@ -284,20 +300,23 @@ answer has to follow a tool call, give the signature a second output, so the
 turn ends only through `submit`, or end it from the tool with `finish_on:`.
 
 Or keep the one output and pass `finish: :submit`. Text without a tool call
-is then a thought, and the turn ends through `submit`:
+is then a thought, not the answer. Here the model announces a lookup and
+forgets to make it; the next step sees what it wrote, makes the lookup, and
+ends the turn through `submit`:
 
 ```elixir
 lm =
   script.([
     %{next_thought: "Let me check who is on call for atlas.", tool_calls: []},
+    %{tool_calls: [%{name: "on_call", arguments: %{"team" => "atlas"}}]},
     %{tool_calls: [%{name: "submit", arguments: %{"reply" => "Maya from atlas is on it."}}]}
   ])
 
 reply = Imp.react(Imp.signature("ticket -> reply"), [on_call], lm: lm, finish: :submit)
 {:ok, prediction} = Imp.call(reply, %{ticket: "We were charged twice this month."})
 
 {Imp.get(prediction, :reply), prediction.metadata.termination_reason}
-#=> {"Maya from atlas is on it.", :forced_submit}
+#=> {"Maya from atlas is on it.", :submit}
 ```
 
 ### One more request on a history you hold
@@ -468,7 +487,7 @@ step wins.
 | `:submit` | the model called `submit` with valid outputs |
 | `:finished_by_tool` | a `finish_on` function finished the turn; `finished_by_tool` names the tool |
 | `:forced_submit` | interrupted; the last request, with `submit` required, answered |
-| `:extracted` | interrupted; the provider could not require `submit`, and the extractor answered |
+| `:extracted` | interrupted; the provider could not require `submit`, and the extractor answered; the history ends with its outputs |
 | `:last_text` | interrupted; one text output, and the last request's text is the answer |
 | `:incomplete` | interrupted, and there is no answer |
 
@@ -479,7 +498,7 @@ When the turn was interrupted, `termination_cause` says why:
 | `:max_iters` | the step budget ran out |
 | `:parse_error` | a step's reply could not be read |
 | `:prediction_error` | a step's request failed |
-| `:empty_tool_calls` | with `submit` (every signature but one unconstrained text output under `finish: :auto`): a step called no tool |
+| `:empty_tool_calls` | with `submit` (every signature but one unconstrained text output under `finish: :auto`): a step called no tool; under `finish: :submit`, a second such step in a row, or one that said nothing |
 | `:context_window_exceeded` | the prompt no longer fits, even with old episodes left out |
 | `:deadline_exceeded` | the process's `Imp.Deadline` passed |
 
@@ -572,7 +591,9 @@ What differs:
 
 - A signature with one text output has no `submit`; the model's text is the
   answer (`:answered`, and `:last_text` when interrupted). `finish: :submit`
-  gives it `submit`, as DSPy does. Its last request
+  gives it `submit`, as DSPy does, but a thought-only step is followed by an
+  ordinary step rather than at once by the forced `submit`, which DSPy sends
+  after every step that calls no tool. A plain-text turn's last request
   runs no tool by default; `last_request_tools:` names the ones it may run,
   the counterpart of DSPy executing `submit` in its forced last request.
 - `finish_on:` lets a tool end the turn (`:finished_by_tool`).

lib/imp/adapter/chat.ex

@@ -1308,15 +1308,35 @@ defmodule Imp.Adapter.Chat do
     end
   end
 
+  # The last turn that is a step with something to show. A step that called
+  # nothing and wrote nothing renders no assistant message (ReActV2 records
+  # its extracted outputs that way), so the message before it is the thought
+  # it follows. One that carries inputs renders them as a user message,
+  # which already ends the request.
   defp last_history_turn(signature, inputs) do
     Enum.find_value(Enum.reverse(signature.inputs), fn field ->
       case fetch_field(inputs, field.name) do
-        %Imp.History{} = history -> history |> Imp.History.messages() |> List.last()
-        _other -> nil
+        %Imp.History{} = history ->
+          history
+          |> Imp.History.messages()
+          |> Enum.reverse()
+          |> Enum.drop_while(&silent_step?/1)
+          |> List.first()
+
+        _other ->
+          nil
       end
     end)
   end
 
+  defp silent_step?(turn) do
+    turn = Imp.Example.new(turn) |> Imp.Example.to_map()
+
+    native_tool_history_turn?(turn) and
+      normalize_history_tool_calls(fetch_field(turn, :tool_calls)) == [] and
+      String.trim(blank_to_empty(fetch_field(turn, :next_thought))) == ""
+  end
+
   defp thought_only_step?(nil), do: false
 
   defp thought_only_step?(turn) do