ReActV2 finish: :submit: keep the thought as the model's message and say it was not sent

CHANGELOG.md

@@ -36,14 +36,20 @@ User-visible changes to Imp are recorded here.
 
 - Under ReActV2's `finish: :submit`, a step that writes a thought and calls
   no tool is now followed by an ordinary step (every tool,
-  `tool_choice: "auto"`, the thought shown as the request's last message when
-  the tools are held natively)
-  instead of the forced `submit`. A second thought-only step in a row, or a
-  step that says nothing or only whitespace, still forces `submit`
-  (`:empty_tool_calls`), as does `max_iters`. Once the deadline has passed,
-  no further request is sent, continued or forced. A model that announced an action can take
-  it, and a reply in the thought is usually submitted unforced, also on
-  providers that cannot force a tool. Other signatures with `submit` are
+  `tool_choice: "auto"`), in which the thought stays the model's own message
+  and a user message after it says "Your message above was not sent. To
+  reply, call `submit` with `answer`; to keep working, call a tool." (naming
+  the signature's outputs), instead of the forced `submit`. The line follows
+  every thought-only step, in later requests and on replay too, except a
+  thought the turn's extracted answer follows, and the forced `submit`'s own
+  last thought, which is shown as the request's last user message instead.
+  With written tool calls the request ends on the tool listing after it. A
+  second thought-only step in a row, or a step that says nothing or only
+  whitespace, still forces `submit` (`:empty_tool_calls`), as does
+  `max_iters`. Once the deadline has passed, no further request is sent,
+  continued or forced. A model that announced an action can take it, and a
+  reply in the thought is usually submitted unforced, also on providers that
+  cannot force a tool. Other signatures with `submit` are
   unchanged.
 - Once the process's `Imp.Deadline` has passed, ReActV2 sends no forced
   `submit`, no `"required"` retry and no extraction request (and, under

decisions.md

@@ -8,7 +8,8 @@ not necessarily when it was made.
 
 | Date | Decision | Source and reason | Status | Retires when |
 | --- | --- | --- | --- | --- |
-| 2026-10-16 | Under `finish: :submit`, a ReActV2 step that writes a thought and calls no tool is followed by an ordinary step (every tool, `tool_choice: "auto"`, the thought as the request's last user message when the tools are held natively), not at once by the forced `submit`; a second thought-only step in a row, or a step that says nothing or only whitespace, is the `:empty_tool_calls` interruption. `max_iters` ends the turn with the forced `submit` as before; once the deadline has passed no further request is sent and the turn is `:incomplete` (`:deadline_exceeded`). With written tool calls the request ends on the tool listing, after the thought. Signatures with `submit` under `finish: :auto` keep DSPy's ending. | `lib/imp/predict/react_v2.ex` `continue_after_thought?/3`, `test/react_v2_thought_continues_test.exs`. Forcing at once took every other tool away from a model that had just announced its next action, and added a request whose failure lost the reply the thought held. Measured 2026-10-16 on 42 real final steps from 11 models through OpenRouter, sending that ordinary step: most samples called `submit` with the thought's reply, copied or paraphrased (Meta's models 6 of 6, unforced; Kimi, GPT and Gemini models likewise); a thought that was a closing note was submitted as an empty answer; a DeepSeek model went on to fetch what it had said it would read; 5 of 42 wrote text again. DSPy's ReActV2 forces `submit` after any step that calls no tool. | In force. | A measurement shows a different bound, or forcing at once, loses fewer answers. |
+| 2026-10-16 | Under `finish: :submit`, every thought-only ReActV2 step is followed, in every request and on replay, by one user message: "Your message above was not sent. To reply, call `submit` with <the outputs>; to keep working, call a tool." The thought stays the model's own (assistant) message. Two exceptions: the forced `submit`'s own last thought, when that request has nothing new to ask, is shown as the request's last user message (the row on the forced request), with no note after it; and a thought that the turn's extracted outputs directly follow was answered and gets no note. This is a sentence Imp writes in a request, where otherwise it adds nothing but a host's `last_request_note`. | `lib/imp/predict/react_v2.ex` `unsent_note/2`, `lib/imp/adapter/chat.ex` `unsent_note_messages/3`, `test/react_v2_unsent_note_test.exs`. With the thought shown back as a user message under `[[ ## next_thought ## ]]`, some models read it as an echo or a pipeline artifact, or as their own message already sent, and ended with an empty `submit`: 4 of 74 owed replies lost in a rehearsal. Measured 2026-10-16 on 21 real final steps, 2 samples each, across 11 model families, after a plain-text step: thought as a user message, 5 samples wrote text again; thought kept with this note, none did, replies were submitted intact (similarity about 1.0), Meta's models accepted it with `tool_choice: auto`, and one family once submitted a closing note as its answer; with only the note's first sentence, models probed their status and took extra steps. The note is a sentence Imp writes, which the last request does not; it states a fact about the step (only `submit` is sent) and the two ways on. | In force. | A measurement finds a wording or shape that loses fewer replies. |
+| 2026-10-16 | Under `finish: :submit`, a ReActV2 step that writes a thought and calls no tool is followed by an ordinary step (every tool, `tool_choice: "auto"`; amended the same day: the thought stays the model's own message, followed by the unsent note, the row on that note), not at once by the forced `submit`; a second thought-only step in a row, or a step that says nothing or only whitespace, is the `:empty_tool_calls` interruption. `max_iters` ends the turn with the forced `submit` as before; once the deadline has passed no further request is sent and the turn is `:incomplete` (`:deadline_exceeded`). With written tool calls the request ends on the tool listing, after the thought. Signatures with `submit` under `finish: :auto` keep DSPy's ending. | `lib/imp/predict/react_v2.ex` `continue_after_thought?/3`, `test/react_v2_thought_continues_test.exs`. Forcing at once took every other tool away from a model that had just announced its next action, and added a request whose failure lost the reply the thought held. Measured 2026-10-16 on 42 real final steps from 11 models through OpenRouter, sending that ordinary step: most samples called `submit` with the thought's reply, copied or paraphrased (Meta's models 6 of 6, unforced; Kimi, GPT and Gemini models likewise); a thought that was a closing note was submitted as an empty answer; a DeepSeek model went on to fetch what it had said it would read; 5 of 42 wrote text again. DSPy's ReActV2 forces `submit` after any step that calls no tool. | In force. | A measurement shows a different bound, or forcing at once, loses fewer answers. |
 | 2026-10-16 | ReActV2's forced `submit` request, when the step before it wrote a thought and called nothing, shows that thought as the request's last message, a user message under its field's marker (`[[ ## next_thought ## ]]`), not as the assistant turn the history records it as. DSPy's ReActV2 leaves the thought out of the forced request. | `lib/imp/adapter/chat.ex` `thought_as_request/4`, `test/react_v2_finish_submit_test.exs`. As a trailing assistant turn the request is refused by providers that do not take an assistant prefix: Mistral ("Cannot set an assistant prefix with function calling") and Gemini ("Requests ending with a model turn are not supported"), probed through OpenRouter 2026-10-16. Dropping it, as DSPy does, made models regenerate finished answers with reasoning off, 2 of 16 to the output limit (deepfates/imp#336). Measured 2026-10-16 on real final steps re-sent through OpenRouter: with the thought as this user message, the submitted answer kept the plain-text reply (text similarity 0.8 to 1.0 across Kimi, GPT, DeepSeek and Muse models), a model whose plain text was a closing note submitted an empty answer as it intended, Meta's models (which accept only `tool_choice: auto`) copied the reply into `submit` 3 of 3, and Gemini accepted every request; with the thought dropped, the models wrote a different answer (similarity about 0.05), one shrank to 30 characters, and the closing-note model submitted a 170-character answer. | In force. | A measurement shows the thought is better dropped, or DSPy keeps it in a form providers accept. |
 | 2026-10-12 | In that plain-text step (row of 2026-10-07; a program with `finish: :auto`, the default), the text output is not presented as a field: the system message lists no output fields, the default agent line names no output ("You are an Agent. Use the supplied tools to gather information and take actions."), there is no "The outputs to produce are:" list, and the finish line is "When the final answer is ready, write it as plain text without calling a tool." followed by the field's description, once. Amended 2026-10-15: the finish line was "... reply without calling a tool."; "reply" is a common tool name, so for a program holding a `reply` tool it read as an instruction about that tool, and it now uses the sentence the other tool-less text answer already used. A host's `agent_line` still replaces the agent line. | `lib/imp/adapter/chat.ex` `default_system/2`, `with_guidance/5`; `test/react_v2_plain_text_answer_test.exs`. The answer is the whole message, so a named, typed output field described a place in the message that does not exist; models on several providers wrote the name into the message as a label (`answer:`, `**answer**`, `[[ ## answer ## ]]`), after notes about the turn, and the whole message, label and notes included, was the answer. The description was listed twice. DSPy has no plain-text finish to follow (row of 2026-10-07). | In force. | ReActV2 gives that signature a `submit` tool, or DSPy defines a plain-text finish whose prompt differs. |
 | 2026-10-08 | A stored history turn shows the outputs it recorded and nothing for the rest: no filler, and not DSPy 3.2.1's `None`. A turn with no recorded output is its user message alone. A predictor that names a tool-calls output and is not a tool loop keeps its recorded outputs as the assistant text on native replay. The system message of every format leaves out a history input, since history arrives as turns: a field typed `history`, or one the call supplies as an `Imp.History`. | deepfates/imp#261, deepfates/imp#258. `lib/imp/adapter/chat.ex` `recorded_reply/3`, `system_signature/2`, `test/history_test.exs`. A filler or `None` puts words in the model's mouth that it never wrote, and a `[[ ## history ## ]]` section in the structure is one no message in the request contains. DSPy 3.2.1 does both (`ChatAdapter.format_assistant_message_content`, `Adapter.format`). | In force. | DSPy stops writing missing history outputs or listing the history input, and Imp follows it. |

docs/diving-deeper/react.md

@@ -58,18 +58,32 @@ not mean as the answer: a sentence about what it will do next, or a report
 of what it just did. The text then stays in the history as the step's
 thought, and only `submit`'s argument is the answer.
 
-After such a step the loop goes on with an ordinary step: every tool,
-`tool_choice: "auto"`, and the thought shown as the request's last message
-when the tools are held natively (as in section 3; with written tool calls
-the request ends on the tool listing, after the thought). The model can
-then do what it said it would, or submit what it wrote; a thought that was
+After such a step the loop goes on with an ordinary step: every tool and
+`tool_choice: "auto"`. The thought stays the model's own message, and one
+user message follows it, naming the signature's outputs:
+
+```text
+Your message above was not sent. To reply, call `submit` with `answer`; to keep working, call a tool.
+```
+
+That line follows every thought-only step, in later requests and on replay
+too, so each request is the previous one plus the newest exchange. It is
+left out after a thought the turn's extracted answer follows (that thought
+was answered), and the forced `submit`'s own last thought is shown on the
+user side instead (section 3). With
+written tool calls the request then ends, as every such request does, on
+the tool listing. The model can then do what it said it would, or submit
+what it wrote; a thought that was
 only a closing note can be submitted as an empty answer. A second
 thought-only step in a row, or a step that says nothing, interrupts the turn
 (`:empty_tool_calls`) and the forced `submit` follows. Once the process's
-`Imp.Deadline` has passed, no further request is sent. Measured on real final steps from 11 models, most submitted the
-thought's reply on that ordinary step, unforced, Meta's models included,
-whose provider cannot force a tool; some went on with the work they had
-announced; about one in eight wrote text again. Forcing at once, as DSPy
+`Imp.Deadline` has passed, no further request is sent. Measured on real
+final steps from 11 model families, with that line after the thought no
+sample wrote text again, and replies were submitted intact, Meta's models
+included, whose provider cannot force a tool. Shown the thought back as a
+user message instead, some models took it for an echo of their own output
+and submitted nothing; given only "Your message above was not sent.", they
+asked about their status and took extra steps. Forcing at once, as DSPy
 does, took the announced action away and added a request whose failure
 lost the reply. It costs one more request whenever the model ends in prose.
 The default, `finish: :auto`, lets the signature decide as above. The
@@ -145,7 +159,9 @@ If the last request does not produce an answer either, the turn ends
 `termination_cause`, and `Imp.Prediction.complete?/1` is false for it; a turn
 without an answer is never reported complete.
 
-Imp adds nothing to that request unless you pass `last_request_note:`, one
+Imp adds nothing to that request (beyond the unsent line that follows each
+earlier thought-only step under `finish: :submit`, section 2) unless you
+pass `last_request_note:`, one
 line of your own text, sent as a user message and kept in the history.
 
 Some programs act through a tool: they publish, send or record something by