From 171e42f320a8565f832e4e9be6648201827b1fbc Mon Sep 17 00:00:00 2001 From: deepfates Date: Fri, 16 Oct 2026 12:56:27 -0700 Subject: [PATCH 1/4] ReActV2: finish: :tools and history_replay: :actions finish: :tools ends a turn only through a finish_on tool. Text without a tool call is a thought followed by one ordinary step and a line naming the ending tools (unsent_note: replaces it); a second thought, an empty step, the step limit or a failed step ends the turn :incomplete with nothing forced. A call may give its own finish:. history_replay: :actions shows earlier turns as inputs, calls with results, and outputs, without the text and reasoning beside the calls. Opt-in; the default :trajectory replays whole, as DSPy's ReActV2 does. --- CHANGELOG.md | 13 ++ decisions.md | 2 + docs/diving-deeper/react.md | 31 ++++ lib/imp/adapter/chat.ex | 25 +++- lib/imp/predict/react_v2.ex | 196 +++++++++++++++++++++++--- lib/imp/saving.ex | 20 +++ priv/public_api.json | 4 +- test/react_v2_finish_tools_test.exs | 183 ++++++++++++++++++++++++ test/react_v2_history_replay_test.exs | 106 ++++++++++++++ 9 files changed, 557 insertions(+), 23 deletions(-) create mode 100644 test/react_v2_finish_tools_test.exs create mode 100644 test/react_v2_history_replay_test.exs diff --git a/CHANGELOG.md b/CHANGELOG.md index e6bfcc50..7ed6b7f1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,19 @@ User-visible changes to Imp are recorded here. ## Unreleased +### Added + +- ReActV2 `finish: :tools`: only a `finish_on` tool ends a turn. Text without + a tool call is a thought, followed by one ordinary step and a line naming + the ending tools; a second thought in a row, an empty step, the step limit + or a failed step ends the turn `:incomplete`, with no further request and + nothing forced. `unsent_note:` replaces that line. A call may give its own + `finish:`. +- ReActV2 `history_replay: :actions`: earlier turns are shown as their + inputs, calls with results, and outputs, without the text and reasoning + written beside the calls. Opt-in; the default `:trajectory` is unchanged, + and the returned history keeps every step. + ## 0.14.1 — 2026-10-16 ### Fixed diff --git a/decisions.md b/decisions.md index feb8ab71..14598901 100644 --- a/decisions.md +++ b/decisions.md @@ -8,6 +8,8 @@ not necessarily when it was made. | Date | Decision | Source and reason | Status | Retires when | | --- | --- | --- | --- | --- | +| 2026-10-17 | ReActV2 `history_replay: :actions` (opt-in) shows earlier turns without the text and reasoning their steps wrote beside their calls, and without steps that only wrote text. Default `:trajectory` replays whole, as DSPy's ReActV2 does (its history events carry `next_thought` and the calls with results, and the adapter renders every event). | A host whose loop does not speak for itself (its text is read by nobody) found the model rebuilding a persona from its own earlier narration on every turn. The divergence is opt-in; the record keeps every step, only the prompt changes. | settled | DSPy offers a comparable history view. | +| 2026-10-17 | ReActV2 `finish: :tools` ends a turn only through `finish_on` and never forces: past the one continued step after a thought, the step limit or a failed step, the turn ends `:incomplete`. | Forcing a call from a loop whose ending tools act in the world would make it act without having decided to; the host knows what an unfinished turn means. | settled | A host needs a forced ending tool. | | 2026-10-16 | Under `finish: :submit`, every thought-only ReActV2 step is followed, in every request and on replay, by one user message: "Your message above was not sent. To reply, call `submit` with ; to keep working, call a tool." The thought stays the model's own (assistant) message. Two exceptions: the forced `submit`'s own last thought, when that request has nothing new to ask, is shown as the request's last user message (the row on the forced request), with no note after it; and a thought that the turn's extracted outputs directly follow was answered and gets no note. This is a sentence Imp writes in a request, where otherwise it adds nothing but a host's `last_request_note`. | `lib/imp/predict/react_v2.ex` `unsent_note/2`, `lib/imp/adapter/chat.ex` `unsent_note_messages/3`, `test/react_v2_unsent_note_test.exs`. With the thought shown back as a user message under `[[ ## next_thought ## ]]`, some models read it as an echo or a pipeline artifact, or as their own message already sent, and ended with an empty `submit`: 4 of 74 owed replies lost in a rehearsal. Measured 2026-10-16 on 21 real final steps, 2 samples each, across 11 model families, after a plain-text step: thought as a user message, 5 samples wrote text again; thought kept with this note, none did, replies were submitted intact (similarity about 1.0), Meta's models accepted it with `tool_choice: auto`, and one family once submitted a closing note as its answer; with only the note's first sentence, models probed their status and took extra steps. The note is a sentence Imp writes, which the last request does not; it states a fact about the step (only `submit` is sent) and the two ways on. | In force. | A measurement finds a wording or shape that loses fewer replies. | | 2026-10-16 | Under `finish: :submit`, a ReActV2 step that writes a thought and calls no tool is followed by an ordinary step (every tool, `tool_choice: "auto"`; amended the same day: the thought stays the model's own message, followed by the unsent note, the row on that note), not at once by the forced `submit`; a second thought-only step in a row, or a step that says nothing or only whitespace, is the `:empty_tool_calls` interruption. `max_iters` ends the turn with the forced `submit` as before; once the deadline has passed no further request is sent and the turn is `:incomplete` (`:deadline_exceeded`). With written tool calls the request ends on the tool listing, after the thought. Signatures with `submit` under `finish: :auto` keep DSPy's ending. | `lib/imp/predict/react_v2.ex` `continue_after_thought?/3`, `test/react_v2_thought_continues_test.exs`. Forcing at once took every other tool away from a model that had just announced its next action, and added a request whose failure lost the reply the thought held. Measured 2026-10-16 on 42 real final steps from 11 models through OpenRouter, sending that ordinary step: most samples called `submit` with the thought's reply, copied or paraphrased (Meta's models 6 of 6, unforced; Kimi, GPT and Gemini models likewise); a thought that was a closing note was submitted as an empty answer; a DeepSeek model went on to fetch what it had said it would read; 5 of 42 wrote text again. DSPy's ReActV2 forces `submit` after any step that calls no tool. | In force. | A measurement shows a different bound, or forcing at once, loses fewer answers. | | 2026-10-16 | ReActV2's forced `submit` request, when the step before it wrote a thought and called nothing, shows that thought as the request's last message, a user message under its field's marker (`[[ ## next_thought ## ]]`), not as the assistant turn the history records it as. DSPy's ReActV2 leaves the thought out of the forced request. | `lib/imp/adapter/chat.ex` `thought_as_request/4`, `test/react_v2_finish_submit_test.exs`. As a trailing assistant turn the request is refused by providers that do not take an assistant prefix: Mistral ("Cannot set an assistant prefix with function calling") and Gemini ("Requests ending with a model turn are not supported"), probed through OpenRouter 2026-10-16. Dropping it, as DSPy does, made models regenerate finished answers with reasoning off, 2 of 16 to the output limit (deepfates/imp#336). Measured 2026-10-16 on real final steps re-sent through OpenRouter: with the thought as this user message, the submitted answer kept the plain-text reply (text similarity 0.8 to 1.0 across Kimi, GPT, DeepSeek and Muse models), a model whose plain text was a closing note submitted an empty answer as it intended, Meta's models (which accept only `tool_choice: auto`) copied the reply into `submit` 3 of 3, and Gemini accepted every request; with the thought dropped, the models wrote a different answer (similarity about 0.05), one shrank to 30 characters, and the closing-note model submitted a 170-character answer. | In force. | A measurement shows the thought is better dropped, or DSPy keeps it in a form providers accept. | diff --git a/docs/diving-deeper/react.md b/docs/diving-deeper/react.md index f3841e28..b62cdd73 100644 --- a/docs/diving-deeper/react.md +++ b/docs/diving-deeper/react.md @@ -504,6 +504,37 @@ When one step calls several finishing tools, the first in call order ends the turn; the others still run and are recorded, and a `submit` in the same step wins. +**Only a tool ends the turn: `finish: :tools`.** Some programs act only +through tools: what they say goes out through a tool, and their text is +for nobody. With `finish: :tools`, nothing but a `finish_on` tool ends the +turn. There is no `submit`, and the signature's outputs come from the +`finish_on` functions. A step that writes text and calls no tool is a +thought, as under `finish: :submit`: the next request is an ordinary step +in which the thought stays the model's own message, followed by one line, +by default "Your message above was not sent. To end the turn, call +`decline` or `send`; to keep working, call a tool." Pass `unsent_note:` to +say what ending the turn means in your program. Nothing is ever forced: a +second thought in a row, a step that says nothing, the step limit or a +failed step ends the turn `:incomplete`, with its `termination_cause` and +no further request, and the host decides what that means. A program built +with `finish: :tools` needs at least one `finish_on` tool. + +A call may give its own `finish:`, as it gives its own `max_iters`. A +program with `finish: :tools` that needs one text answer for one call, a +summary of the conversation say, calls with `finish: :auto, max_iters: 0` +and a note. + +**Earlier turns as actions: `history_replay: :actions`.** By default a +history replays every earlier step whole, as DSPy's ReActV2 does: the text +each step wrote beside its calls, its reasoning, the calls and their +results. With `history_replay: :actions`, the steps of earlier turns lose +the text and reasoning written beside their calls, and a step that only +wrote text is left out; earlier turns read as their inputs, the calls with +their results, and the outputs. The current turn is always shown whole, and +the history the prediction returns keeps every step. Use it when the +model's own narration should not become something it reads back on every +later turn. + ### Why a turn ended `termination_reason` says how the turn ended: diff --git a/lib/imp/adapter/chat.ex b/lib/imp/adapter/chat.ex index 30dbad33..4feca117 100644 --- a/lib/imp/adapter/chat.ex +++ b/lib/imp/adapter/chat.ex @@ -777,10 +777,17 @@ defmodule Imp.Adapter.Chat do defp with_guidance(instructions, %{} = guidance, signature, plain?, agent_line) do names = fn key -> guidance |> Map.get(key, []) |> Enum.map_join(", ", &"`#{&1}`") end plain_finish? = plain? and Map.get(guidance, :submit_tool, :submit) == nil + finish_tools = Map.get(guidance, :finish_tools) finish = - case Map.get(guidance, :submit_tool, :submit) do - nil when plain? -> + case {Map.get(guidance, :finish_tools), Map.get(guidance, :submit_tool, :submit)} do + # `finish: :tools`: only these tools end the turn, and what the model + # writes without one is not sent (`Imp.Predict.ReActV2`). + {[_ | _] = tools, _submit} -> + "The turn ends only when you call " <> + or_names(tools) <> ". Text you write is not sent." + + {_tools, nil} when plain? -> [ "When the final answer is ready, write it as plain text without calling a tool.", field_description(output_field(signature, meta_field(signature, :text_field))) @@ -788,10 +795,10 @@ defmodule Imp.Adapter.Chat do |> Enum.reject(&(&1 == "")) |> Enum.join(" ") - nil -> + {_tools, nil} -> text_answer(signature) - tool -> + {_tools, tool} -> "When the final answer is ready, call `#{tool}` with #{names.(:output_names)}." end @@ -804,6 +811,9 @@ defmodule Imp.Adapter.Chat do outputs = case Map.get(guidance, :outputs, []) do _fields when plain_finish? -> nil + # Under `finish: :tools` the tools that end the turn give its outputs; + # the model writes none. + _fields when is_list(finish_tools) -> nil [] -> nil fields -> "The outputs to produce are:\n" <> render_field_list(fields) end @@ -825,6 +835,13 @@ defmodule Imp.Adapter.Chat do # without a tool call, so the opening line does not say the tools # produce it: they gather information and act. Nor does it name the # answer as an output to produce (see `with_guidance/5`). + defp or_names([one]), do: "`#{one}`" + + defp or_names(names) do + quoted = Enum.map(names, &"`#{&1}`") + Enum.join(Enum.drop(quoted, -1), ", ") <> " or " <> List.last(quoted) + end + defp default_agent_line(guidance, plain?, names) do if plain? and Map.get(guidance, :submit_tool, :submit) == nil do "You are an Agent. Use the supplied tools to gather information and take actions." diff --git a/lib/imp/predict/react_v2.ex b/lib/imp/predict/react_v2.ex index c5939ee4..b69c48f6 100644 --- a/lib/imp/predict/react_v2.ex +++ b/lib/imp/predict/react_v2.ex @@ -286,6 +286,8 @@ defmodule Imp.Predict.ReActV2 do tool_policy: :allow, finish_on: %{}, finish: :auto, + unsent_note: nil, + history_replay: :trajectory, last_request_tools: [] ] @@ -360,7 +362,7 @@ defmodule Imp.Predict.ReActV2 do "See \"How a turn ends\" above." ], finish: [ - type: {:in, [:auto, :submit]}, + type: {:in, [:auto, :submit, :tools]}, default: :auto, doc: "How the model ends its turn. `:auto` lets the signature decide: one " <> @@ -368,9 +370,30 @@ defmodule Imp.Predict.ReActV2 do "`submit`. `:submit` ends every turn through `submit`, as DSPy's ReActV2 does, " <> "so a step that writes text and calls no tool is a thought, not the answer; " <> "the next step continues with every tool, and a second such step in a row " <> - "forces `submit`. " <> + "forces `submit`. `:tools` ends a turn only through a `finish_on` tool: text " <> + "without a tool call is a thought as under `:submit`, there is no `submit`, and " <> + "a second such step in a row, the step limit or a failed step ends the turn " <> + "`:incomplete`, with no further request. " <> "See \"Which signatures get `submit`\" above." ], + unsent_note: [ + type: {:or, [:string, nil]}, + default: nil, + doc: + "The line shown after a thought-only step under `finish: :submit` or `:tools`, " <> + "in place of the default, so it can name what ending the turn means for this host." + ], + history_replay: [ + type: {:in, [:trajectory, :actions]}, + default: :trajectory, + doc: + "How the steps of earlier turns in a `history` are shown to the model. " <> + "`:trajectory` shows them whole, as DSPy's ReActV2 does. `:actions` leaves out " <> + "the text and reasoning each earlier step wrote beside its calls, and drops a " <> + "step that only wrote text: earlier turns read as their inputs, the calls made " <> + "with their results, and the outputs. The current turn is always whole, and the " <> + "history the prediction returns keeps everything." + ], last_request_tools: [ type: {:list, {:or, [:atom, :string]}}, default: [], @@ -411,6 +434,7 @@ defmodule Imp.Predict.ReActV2 do end reject_reserved_tools!(signature, "Imp.Predict.ReActV2.new/3") + validate_tools_finish!(opts[:finish], opts[:finish_on], "Imp.Predict.ReActV2.new/3") tools = put_submit(tools, signature, opts[:finish]) @@ -440,7 +464,7 @@ defmodule Imp.Predict.ReActV2 do adapter_opts = Keyword.merge( Keyword.get(opts, :adapter_opts, []), - loop_adapter_opts(signature, tools, opts[:finish]) + loop_adapter_opts(signature, tools, opts[:finish], opts[:unsent_note], opts[:finish_on]) ) %__MODULE__{ @@ -459,6 +483,8 @@ defmodule Imp.Predict.ReActV2 do last_request_note: opts[:last_request_note], finish_on: resolve_finish_on!(opts[:finish_on], tools), finish: opts[:finish], + unsent_note: opts[:unsent_note], + history_replay: opts[:history_replay], last_request_tools: resolve_last_request_tools!( opts[:last_request_tools], @@ -480,7 +506,7 @@ defmodule Imp.Predict.ReActV2 do # can have several typed outputs, and those signatures still get it. # `finish: :submit` gives that signature `submit` as well, as DSPy does. def put_submit(tools, signature, finish \\ :auto) do - if plain_text?(signature, finish), + if plain_text?(signature, finish) or finish == :tools, do: tools, else: Map.put( @@ -490,6 +516,32 @@ defmodule Imp.Predict.ReActV2 do ) end + # `finish: :tools` ends a turn only through a `finish_on` tool, so a loop + # with none could never end but by interruption. + defp validate_tools_finish!(:tools, finish_on, context) when finish_on == %{}, + do: + raise( + ArgumentError, + "#{context}: finish: :tools needs at least one finish_on tool to end a turn" + ) + + defp validate_tools_finish!(_finish, _finish_on, _context), do: :ok + + # A call's own `finish`, for one call: the loop as `new/3` would build it with + # that ending, from the same tools and options. + defp with_finish(%__MODULE__{finish: finish} = react, finish), do: react + + defp with_finish(%__MODULE__{} = react, finish) do + validate_tools_finish!(finish, react.finish_on, "Imp.Predict.ReActV2 call") + tools = react.tools |> Map.delete(:submit) |> put_submit(react.signature, finish) + restore_loop(%{react | finish: finish, tools: tools}) + end + + defp validate_call_finish(finish) when finish in [:auto, :submit, :tools], do: :ok + + defp validate_call_finish(finish), + do: {:error, {:invalid_react_v2_finish, finish}} + @doc false def validate_finish_on(finish_on) when is_map(finish_on) do invalid = @@ -568,9 +620,9 @@ defmodule Imp.Predict.ReActV2 do # What the loop itself tells the adapter: its guidance, and a request shape # that ends on the newest exchange with no response instruction. - defp loop_adapter_opts(signature, tools, finish), + defp loop_adapter_opts(signature, tools, finish, unsent_note, finish_on), do: [ - guidance: guidance(signature, tools, finish), + guidance: guidance(signature, tools, finish, unsent_note, finish_on), response_instruction: false, omit_empty_request: true ] @@ -607,7 +659,13 @@ defmodule Imp.Predict.ReActV2 do adapter_opts: Keyword.merge( react.adapter_opts, - loop_adapter_opts(agent.signature, agent.tools, agent.finish) + loop_adapter_opts( + agent.signature, + agent.tools, + agent.finish, + agent.unsent_note, + agent.finish_on + ) ), signature: %{ react.signature @@ -635,7 +693,7 @@ defmodule Imp.Predict.ReActV2 do Keyword.put( agent.react.adapter_opts, :guidance, - guidance(agent.signature, tools, agent.finish) + guidance(agent.signature, tools, agent.finish, agent.unsent_note, agent.finish_on) ) } @@ -669,6 +727,9 @@ defmodule Imp.Predict.ReActV2 do defp do_call(%__MODULE__{} = react, inputs, execution) do with {:ok, inputs} <- normalize_inputs(inputs), + {finish, inputs} <- pop_call_option(inputs, :finish, react.finish), + :ok <- validate_call_finish(finish), + react = with_finish(react, finish), {max_iters, inputs} <- pop_max_iters(inputs, react.max_iters), :ok <- validate_call_max_iters(max_iters), {note, inputs} <- pop_call_option(inputs, :last_request_note, react.last_request_note), @@ -700,7 +761,7 @@ defmodule Imp.Predict.ReActV2 do run( react, - history_context(history, react.signature), + history_context(history, react.signature, react.history_replay), pending, pending, 0, @@ -841,7 +902,8 @@ defmodule Imp.Predict.ReActV2 do # once. @thought_steps_continued 1 - defp continue_after_thought?(%{finish: :submit}, prediction, thoughts) do + defp continue_after_thought?(%{finish: finish}, prediction, thoughts) + when finish in [:submit, :tools] do thoughts < @thought_steps_continued and String.trim(to_string(Imp.get(prediction, :next_thought) || "")) != "" and not deadline_passed?() @@ -851,6 +913,21 @@ defmodule Imp.Predict.ReActV2 do # The one place an interrupted turn goes: the last request for a signature # with one text output, the forced submit for every other. + # Under `finish: :tools` nothing ends the turn but a `finish_on` tool, and + # nothing is forced: the turn ends `:incomplete`, with no further request, + # and the host decides what that means. + defp interrupted( + %{finish: :tools}, + history, + _inputs, + _pending, + cause, + _turn, + error, + _execution + ), + do: incomplete_prediction(history, cause, error) + defp interrupted(react, history, inputs, pending, cause, turn, error, execution) do if plain_text?(react.signature, react.finish), do: last_text(react, history, inputs, pending, cause, turn, error, execution), @@ -1710,6 +1787,7 @@ defmodule Imp.Predict.ReActV2 do # text output, unless the program asked for `finish: :submit`. defp plain_text?(signature, :auto), do: single_text_output?(signature) defp plain_text?(_signature, :submit), do: false + defp plain_text?(_signature, :tools), do: false # Text is the answer only when any text is a valid answer: a constrained # string (an enum, a pattern, an answer shape) keeps `submit`, whose schema @@ -1901,7 +1979,7 @@ defmodule Imp.Predict.ReActV2 do # Full history remains the durable result. Only whole prior episode groups are # eligible for prompt projection; tool observations appended in this call are # protected even if they alone exceed the model window. - defp history_context(history, signature) do + defp history_context(history, signature, replay) do outputs = Imp.Signature.output_names(signature) size = length(history.messages) @@ -1919,7 +1997,64 @@ defmodule Imp.Predict.ReActV2 do boundaries = if size > 0 and List.last(boundaries) != size, do: boundaries ++ [size], else: boundaries - %{full: history, boundaries: boundaries, omitted: 0, omitted_groups: 0, retries: 0} + %{ + full: history, + boundaries: boundaries, + omitted: 0, + omitted_groups: 0, + retries: 0, + # `history_replay: :actions`: the steps given before this call are + # earlier turns, shown as their actions (`actions_view/2`). + actions_before: if(replay == :actions, do: size), + inputs: Imp.Signature.input_names(signature), + outputs: outputs + } + end + + # An earlier turn's step as `history_replay: :actions` shows it. A step that + # carries the task's outputs was the answer and is shown whole. Any other + # step loses the text and reasoning the model wrote beside its calls; one + # that called nothing keeps only the inputs it carried, and goes when it + # carried none. Only the request changes: the history keeps every step. + # Divergence from DSPy's ReActV2, whose history replays every step whole + # (decisions.md). + @step_words [:next_thought, :reasoning_content, :reasoning_details] + + defp actions_view(step, context) do + if Enum.any?(context.outputs, &(not is_nil(Imp.FieldMap.get(step, &1)))) do + [step] + else + step = Enum.reduce(@step_words, step, &Imp.FieldMap.delete(&2, &1)) + + cond do + history_calls(step) != [] -> [step] + Enum.any?(context.inputs, &Imp.FieldMap.has_key?(step, &1)) -> [step] + true -> [] + end + end + end + + defp history_calls(step) do + case Imp.FieldMap.get(step, :tool_calls) do + %ToolCalls{tool_calls: calls} -> calls + %{tool_calls: calls} when is_list(calls) -> calls + %{"tool_calls" => calls} when is_list(calls) -> calls + calls when is_list(calls) -> calls + _ -> [] + end + end + + defp projected_messages(context) do + messages = Enum.drop(context.full.messages, context.omitted) + + case context do + %{actions_before: before} when is_integer(before) -> + {earlier, current} = Enum.split(messages, max(before - context.omitted, 0)) + Enum.flat_map(earlier, &actions_view(&1, context)) ++ current + + _context -> + messages + end end # A stored turn that carries the answer of a one-text-output task but no @@ -1951,7 +2086,7 @@ defmodule Imp.Predict.ReActV2 do do: %{context | full: Imp.History.append(context.full, event)} defp context_call(context, call) do - projected = %{context.full | messages: Enum.drop(context.full.messages, context.omitted)} + projected = %{context.full | messages: projected_messages(context)} case call.(projected) do {:ok, prediction} -> @@ -2140,17 +2275,25 @@ defmodule Imp.Predict.ReActV2 do # when the signature has no `submit` and the answer is text. `outputs` # are the task's output fields, so each step says what every output means; # otherwise their descriptions reach the model only inside `submit`'s schema. - defp guidance(signature, tools, finish) do + defp guidance(signature, tools, finish, unsent_note, finish_on) do %{ - submit_tool: if(plain_text?(signature, finish), do: nil, else: :submit), + submit_tool: if(plain_text?(signature, finish) or finish == :tools, do: nil, else: :submit), input_names: Imp.Signature.input_names(signature), output_names: Imp.Signature.output_names(signature), outputs: signature.outputs, tool_names: tools |> Map.keys() |> Enum.sort(), - unsent_note: unsent_note(signature, finish) + unsent_note: unsent_note || unsent_note(signature, finish, finish_on) } + |> maybe_put(:finish_tools, finish_tools(finish, finish_on)) end + # The tools that end a `finish: :tools` turn, in name order, for the + # guidance and the default unsent note. + defp finish_tools(:tools, finish_on), + do: finish_on |> Map.keys() |> Enum.map(&to_string/1) |> Enum.sort() + + defp finish_tools(_finish, _finish_on), do: nil + # Under `finish: :submit` a thought-only step is followed by an ordinary # step, and the adapter tells the model, after that step's message, what # became of it and what its two ways on are. This is a fact about the step @@ -2162,7 +2305,9 @@ defmodule Imp.Predict.ReActV2 do # user message, some models took it for an echo of their own output and # submitted nothing; with the first sentence alone, models probed their # status and took extra steps. - defp unsent_note(signature, :submit) do + defp unsent_note(signature, finish, finish_on) + + defp unsent_note(signature, :submit, _finish_on) do fields = signature |> Imp.Signature.output_names() |> Enum.map(&"`#{&1}`") with_fields = @@ -2180,10 +2325,25 @@ defmodule Imp.Predict.ReActV2 do # half of the note, so the thought does not read as a message that was # sent. A history written in a plain-text loop has no such step (its text # steps carry the answer), so its requests are unchanged. - defp unsent_note(signature, :auto) do + defp unsent_note(signature, :auto, _finish_on) do if single_text_output?(signature), do: "Your message above was not sent." end + # Under `finish: :tools` only a `finish_on` tool ends the turn, so the line + # names them. A host that knows what those tools mean passes its own + # (`:unsent_note`). + defp unsent_note(_signature, :tools, finish_on) do + "Your message above was not sent. To end the turn, call " <> + or_list(finish_tools(:tools, finish_on)) <> "; to keep working, call a tool." + end + + defp or_list([one]), do: "`#{one}`" + + defp or_list(names) do + quoted = Enum.map(names, &"`#{&1}`") + Enum.join(Enum.drop(quoted, -1), ", ") <> " or " <> List.last(quoted) + end + defp provider_tool_config(tools, order, signature) do [ tools: tools |> ordered_tools(order) |> Enum.map(&tool_description(&1, signature)), diff --git a/lib/imp/saving.ex b/lib/imp/saving.ex index 3ab70fcd..cc4c8b5a 100644 --- a/lib/imp/saving.ex +++ b/lib/imp/saving.ex @@ -307,6 +307,9 @@ defmodule Imp.Saving do "last_request_tools" => react.last_request_tools, "tool_policy" => dump_tool_policy(react.tool_policy, "ReActV2 tool policy") } + # Saved only when set, so a program that uses neither saves as before. + |> maybe_put_saved("unsent_note", react.unsent_note, nil) + |> maybe_put_saved("history_replay", react.history_replay, :trajectory) end defp dump_state(%Imp.Predict.CodeAct{} = code_act) do @@ -647,6 +650,8 @@ defmodule Imp.Saving do last_request_note: load_react_v2_last_request_note!(state["last_request_note"]), finish_on: load_finish_on!(state["finish_on"]), finish: finish, + unsent_note: load_react_v2_last_request_note!(state["unsent_note"]), + history_replay: load_react_v2_history_replay!(state["history_replay"]), last_request_tools: load_last_request_tools!(state["last_request_tools"], tools, finish), tool_policy: load_tool_policy!(state["tool_policy"], "ReActV2 tool policy") } @@ -1310,10 +1315,25 @@ defmodule Imp.Saving do defp load_react_v2_finish!(nil), do: :auto defp load_react_v2_finish!("auto"), do: :auto defp load_react_v2_finish!("submit"), do: :submit + defp load_react_v2_finish!("tools"), do: :tools defp load_react_v2_finish!(other), do: raise(ArgumentError, "invalid saved ReActV2 finish: #{inspect(other)}") + defp maybe_put_saved(state, _key, value, default) when value in [nil, default], do: state + + defp maybe_put_saved(state, key, value, _default) when is_atom(value), + do: Map.put(state, key, Atom.to_string(value)) + + defp maybe_put_saved(state, key, value, _default), do: Map.put(state, key, value) + + defp load_react_v2_history_replay!(nil), do: :trajectory + defp load_react_v2_history_replay!("trajectory"), do: :trajectory + defp load_react_v2_history_replay!("actions"), do: :actions + + defp load_react_v2_history_replay!(other), + do: raise(ArgumentError, "invalid saved ReActV2 history_replay: #{inspect(other)}") + defp load_react_v2_last_request_note!(nil), do: nil defp load_react_v2_last_request_note!(note) when is_binary(note), do: note diff --git a/priv/public_api.json b/priv/public_api.json index 0d102a86..2f821714 100644 --- a/priv/public_api.json +++ b/priv/public_api.json @@ -8039,6 +8039,7 @@ "struct_fields": [ "finish", "finish_on", + "history_replay", "last_request_note", "last_request_tools", "max_iters", @@ -8046,7 +8047,8 @@ "signature", "tool_order", "tool_policy", - "tools" + "tools", + "unsent_note" ], "types": [ { diff --git a/test/react_v2_finish_tools_test.exs b/test/react_v2_finish_tools_test.exs new file mode 100644 index 00000000..6766c728 --- /dev/null +++ b/test/react_v2_finish_tools_test.exs @@ -0,0 +1,183 @@ +defmodule ReActV2FinishToolsTest do + use ExUnit.Case, async: true + + # `finish: :tools`: only a `finish_on` tool ends the turn. Text without a + # tool call is a thought, as under `finish: :submit`, but there is no + # `submit` and nothing is forced: a second thought in a row, an empty step + # or the step limit ends the turn `:incomplete`, with no further request. + + defp look, do: Imp.tool(:look, "Look at a thing", fn _arguments -> "a cat" end) + defp send_tool, do: Imp.tool(:send, "Send the reply", fn _arguments -> "sent" end) + defp decline, do: Imp.tool(:decline, "Say nothing", fn _arguments -> "ok" end) + + @thought %{next_thought: "A cat, I think.", tool_calls: []} + @look %{next_thought: "Looking first.", tool_calls: [%{id: "c1", name: "look", arguments: %{}}]} + @send %{tool_calls: [%{id: "s1", name: "send", arguments: %{}}]} + + defp finish_on, + do: %{ + send: fn _arguments, _result, _inputs -> {:finish, %{reply: "sent"}} end, + decline: fn _arguments, _result, _inputs -> {:finish, %{reply: ""}} end + } + + defp scripted(owner, replies) do + {:ok, agent} = Agent.start_link(fn -> replies end) + + Imp.LM.Static.new( + handler: fn messages, opts -> + send(owner, {:request, messages, opts}) + + Agent.get_and_update(agent, fn + [reply | rest] -> {reply, rest} + [] -> {%{tool_calls: []}, []} + end) + end + ) + end + + defp requests(acc \\ []) do + receive do + {:request, messages, opts} -> requests([{messages, opts} | acc]) + after + 0 -> Enum.reverse(acc) + end + end + + defp program(replies, opts \\ []) do + Imp.react( + "ticket -> reply", + [look(), send_tool(), decline()], + [lm: scripted(self(), replies), finish: :tools, finish_on: finish_on()] ++ opts + ) + end + + defp call(replies, opts \\ []), + do: replies |> program(opts) |> Imp.call(%{ticket: "What is in the box?"}) + + defp tool_names(opts), do: Enum.map(opts[:tools], & &1.function.name) + + test "a thought continues, the turn ends only through a finish_on tool, and nothing is submit" do + assert {:ok, prediction} = call([@thought, @send]) + assert prediction.metadata.termination_reason == :finished_by_tool + assert prediction.metadata.finished_by_tool == "send" + assert Imp.get(prediction, :reply) == "sent" + + [{[system | _], first}, {second_messages, second}] = requests() + assert tool_names(first) == ["look", "send", "decline"] + assert first[:tool_choice] == "auto" and second[:tool_choice] == "auto" + + # The guidance names the ending tools; nothing asks for an answer in text. + assert system.content =~ "The turn ends only when you call `decline` or `send`." + refute system.content =~ "write it as plain text" + refute system.content =~ "submit" + + # The thought stays the model's own message, then the default line. + [thought, line] = Enum.take(second_messages, -2) + assert thought.role == :assistant and thought.content =~ "A cat, I think." + + assert line == %{ + role: :user, + content: + "Your message above was not sent. To end the turn, call `decline` or `send`; " <> + "to keep working, call a tool." + } + end + + test "a second thought in a row ends the turn :incomplete, with no further request" do + assert {:ok, prediction} = call([@thought, @thought, @send]) + assert prediction.metadata.termination_reason == :incomplete + assert prediction.metadata.termination_cause == :empty_tool_calls + refute Imp.Prediction.complete?(prediction) + assert length(requests()) == 2 + end + + test "the step limit ends the turn :incomplete, and nothing is forced" do + assert {:ok, prediction} = call([@look, @look], max_iters: 2) + assert prediction.metadata.termination_reason == :incomplete + assert prediction.metadata.termination_cause == :max_iters + + assert Enum.map(requests(), fn {_messages, opts} -> opts[:tool_choice] end) == [ + "auto", + "auto" + ] + end + + test "a host's own unsent note replaces the default" do + note = "Your message above was not sent. To speak, call `send`; to keep looking, call a tool." + assert {:ok, _prediction} = call([@thought, @send], unsent_note: note) + [_first, {messages, _opts}] = requests() + assert List.last(messages) == %{role: :user, content: note} + end + + test "finish: :tools without a finish_on tool is refused" do + assert_raise ArgumentError, ~r/finish: :tools needs at least one finish_on tool/, fn -> + Imp.react("ticket -> reply", [look()], finish: :tools) + end + end + + test "a call's own finish overrides the program's, for that call" do + program = program(["In a box: a cat."]) + + assert {:ok, prediction} = + Imp.call(program, %{ticket: "What is in the box?", finish: :auto, max_iters: 0}) + + assert Imp.get(prediction, :reply) == "In a box: a cat." + assert prediction.metadata.termination_reason == :last_text + + # And the other way: a plain-text program called with `finish: :tools`. + plain = + Imp.react("ticket -> reply", [look(), send_tool(), decline()], + lm: scripted(self(), [@thought, @thought]), + finish_on: finish_on() + ) + + _ = requests() + + assert {:ok, prediction} = Imp.call(plain, %{ticket: "What is in the box?", finish: :tools}) + assert prediction.metadata.termination_reason == :incomplete + end + + test "an answer recorded under :auto replays as an answer, with no unsent note" do + history = + Imp.History.new([ + %{ticket: "Hello?", next_thought: "Hi there.", reply: "Hi there.", tool_calls: []} + ]) + + assert {:ok, _prediction} = + [@send] |> program() |> Imp.call(%{ticket: "And now?", history: history}) + + [{messages, _opts}] = requests() + assert Enum.any?(messages, &(&1.role == :assistant and &1.content =~ "Hi there.")) + refute Enum.any?(messages, &(to_string(&1.content) =~ "was not sent")) + end + + test "the program saves and loads with its ending, note and replay" do + runner = fn _arguments -> "x" end + finish = fn _arguments, _result, _inputs -> {:finish, %{reply: ""}} end + registry = Imp.Saving.Registry.new(look_runner: runner, finish_cb: finish) + + dumped = + Imp.react("ticket -> reply", [Imp.tool(:look, "Look", runner)], + finish: :tools, + finish_on: %{look: finish}, + unsent_note: "Not sent.", + history_replay: :actions + ) + |> Imp.dump(registry: registry) + + assert {dumped["finish"], dumped["unsent_note"], dumped["history_replay"]} == + {"tools", "Not sent.", "actions"} + + loaded = Imp.load!(dumped, registry: registry) + + assert {loaded.finish, loaded.unsent_note, loaded.history_replay} == + {:tools, "Not sent.", :actions} + + # A program that sets neither saves as before. + plain = + Imp.react("ticket -> reply", [Imp.tool(:look, "Look", runner)]) + |> Imp.dump(registry: registry) + + refute Map.has_key?(plain, "unsent_note") or Map.has_key?(plain, "history_replay") + end +end diff --git a/test/react_v2_history_replay_test.exs b/test/react_v2_history_replay_test.exs new file mode 100644 index 00000000..6b5a3262 --- /dev/null +++ b/test/react_v2_history_replay_test.exs @@ -0,0 +1,106 @@ +defmodule ReActV2HistoryReplayTest do + use ExUnit.Case, async: true + + # `history_replay: :actions` shows earlier turns as their inputs, the calls + # they made with the results, and their outputs, without the text and + # reasoning written beside the calls. The current turn is shown whole, and + # the history the prediction returns keeps every step. + + defp look, do: Imp.tool(:look, "Look at a thing", fn _arguments -> "a cat" end) + + defp scripted(owner, replies) do + {:ok, agent} = Agent.start_link(fn -> replies end) + + Imp.LM.Static.new( + handler: fn messages, opts -> + send(owner, {:request, messages, opts}) + + Agent.get_and_update(agent, fn + [reply | rest] -> {reply, rest} + [] -> {%{tool_calls: []}, []} + end) + end + ) + end + + # An earlier turn: a call with narration beside it, a thought-only step, + # then the answer. + defp earlier do + {:ok, first} = + Imp.react("ticket -> reply", [look()], + lm: + scripted(self(), [ + %{ + reply: "Earlier narration beside a call.", + tool_calls: [%{id: "c0", name: "look", arguments: %{}}] + }, + %{reply: "A cat.", tool_calls: []} + ]) + ) + |> Imp.call(%{ticket: "What is in the box?"}) + + flush() + [call, answer] = first.metadata.history.messages + thought = %{next_thought: "An earlier thought nobody answered.", tool_calls: []} + Imp.History.new([call, thought, answer]) + end + + defp flush do + receive do + {:request, _, _} -> flush() + after + 0 -> :ok + end + end + + defp texts(messages), do: Enum.map_join(messages, "\n", &to_string(&1.content)) + + defp call(replay) do + Imp.react("ticket -> reply", [look()], + lm: + scripted(self(), [ + %{ + reply: "This turn's narration.", + tool_calls: [%{id: "c1", name: "look", arguments: %{}}] + }, + "Still a cat." + ]), + history_replay: replay + ) + |> Imp.call(%{ticket: "And now?", history: earlier()}) + end + + defp second_request do + receive do + {:request, _first, _opts} -> + receive do + {:request, messages, _opts} -> messages + end + end + end + + test ":actions leaves earlier narration and thoughts out of the request, and keeps the current turn whole" do + assert {:ok, prediction} = call(:actions) + text = texts(second_request()) + + refute text =~ "Earlier narration beside a call." + refute text =~ "An earlier thought nobody answered." + # The earlier call, its result, the answer and the current turn remain. + assert text =~ "What is in the box?" + assert text =~ "a cat" + assert text =~ "A cat." + assert text =~ "This turn's narration." + + # The history returned keeps everything. + kept = prediction.metadata.history.messages |> Enum.map(&inspect/1) |> Enum.join() + assert kept =~ "Earlier narration beside a call." + assert kept =~ "An earlier thought nobody answered." + end + + test ":trajectory, the default, shows earlier steps whole" do + assert {:ok, _prediction} = call(:trajectory) + text = texts(second_request()) + assert text =~ "Earlier narration beside a call." + assert text =~ "An earlier thought nobody answered." + end +end From 8e82f9ce16b7b289d94cbf4f8846adcbd93c4c52 Mon Sep 17 00:00:00 2001 From: deepfates Date: Fri, 16 Oct 2026 13:33:35 -0700 Subject: [PATCH 2/4] ReActV2 :tools/:actions review fixes - A task input named finish is the input, not the per-call option. - :actions keeps the current turn whole on a resume without new inputs: it starts at the last step that carries inputs, so in-turn reasoning stays. - Under :tools, a failed request gets one more ordinary step, once per turn; a passed deadline ends the turn :deadline_exceeded. - :tools refuses last_request_note/last_request_tools and a blank unsent_note; a call's :tools without finish_on returns an error; loading validates the same; the unsent_note load error names unsent_note. - The written-tool agent line under :tools no longer says the tools produce the outputs. - Docs: per-call finish, :tools in how a turn ends, the retry, :actions' current-turn rule and output steps, reasoning on earlier turns (Kimi K3 through OpenRouter accepted it in a probe), :tools context recovery. - Tests discriminate: distinct results, reasoning fields, a mid-turn resume. --- docs/diving-deeper/react.md | 13 +- lib/imp/adapter/chat.ex | 5 +- lib/imp/predict/react_v2.ex | 172 +++++++++++++++++++++++--- lib/imp/saving.ex | 9 +- test/react_v2_finish_tools_test.exs | 103 +++++++++++++++ test/react_v2_history_replay_test.exs | 168 +++++++++++++++++-------- 6 files changed, 394 insertions(+), 76 deletions(-) diff --git a/docs/diving-deeper/react.md b/docs/diving-deeper/react.md index b62cdd73..4079de4f 100644 --- a/docs/diving-deeper/react.md +++ b/docs/diving-deeper/react.md @@ -513,10 +513,12 @@ thought, as under `finish: :submit`: the next request is an ordinary step in which the thought stays the model's own message, followed by one line, by default "Your message above was not sent. To end the turn, call `decline` or `send`; to keep working, call a tool." Pass `unsent_note:` to -say what ending the turn means in your program. Nothing is ever forced: a +say what ending the turn means in your program. Nothing is ever forced. A +step whose request fails gets one more ordinary step, once per turn. A second thought in a row, a step that says nothing, the step limit or a -failed step ends the turn `:incomplete`, with its `termination_cause` and -no further request, and the host decides what that means. A program built +second failure ends the turn `:incomplete`, with its `termination_cause` +(`:deadline_exceeded` when the deadline has passed) and no further request, +and the host decides what that means. A program built with `finish: :tools` needs at least one `finish_on` tool. A call may give its own `finish:`, as it gives its own `max_iters`. A @@ -533,7 +535,10 @@ wrote text is left out; earlier turns read as their inputs, the calls with their results, and the outputs. The current turn is always shown whole, and the history the prediction returns keeps every step. Use it when the model's own narration should not become something it reads back on every -later turn. +later turn. A step that carries the task's outputs (an answer) is shown +whole. On a call without new inputs (a resume mid-turn), the current turn +starts at the last step that carries inputs. Some thinking APIs reject an +earlier tool-call message without its reasoning content; check yours. ### Why a turn ended diff --git a/lib/imp/adapter/chat.ex b/lib/imp/adapter/chat.ex index 4feca117..ee71437b 100644 --- a/lib/imp/adapter/chat.ex +++ b/lib/imp/adapter/chat.ex @@ -843,7 +843,10 @@ defmodule Imp.Adapter.Chat do end defp default_agent_line(guidance, plain?, names) do - if plain? and Map.get(guidance, :submit_tool, :submit) == nil do + # Under `finish: :tools` the ending tools give the outputs, so the tools do + # not "produce" them from the inputs, with native or written calls alike. + if is_list(Map.get(guidance, :finish_tools)) or + (plain? and Map.get(guidance, :submit_tool, :submit) == nil) do "You are an Agent. Use the supplied tools to gather information and take actions." else "You are an Agent. Use the supplied tools to produce #{names.(:output_names)} " <> diff --git a/lib/imp/predict/react_v2.ex b/lib/imp/predict/react_v2.ex index b69c48f6..c03785ee 100644 --- a/lib/imp/predict/react_v2.ex +++ b/lib/imp/predict/react_v2.ex @@ -35,6 +35,32 @@ defmodule Imp.Predict.ReActV2 do DSPy's ending: its first thought-only step is that interruption. The default, `finish: :auto`, leaves the choice to the signature. + `finish: :tools` is for a program that acts only through tools: its turn + ends only through a `finish_on` tool. There is no `submit`, and the outputs + come from the `finish_on` functions. A thought-only step is followed by one + ordinary step, as under `:submit`, with a line naming the ending tools + (`unsent_note:` replaces it). Nothing is forced and no last request is + made. A step whose request fails gets one more ordinary step, once per turn + (there is no last request to absorb a transient failure). A second + thought-only step in a row, a step that says nothing, the step limit or a + second failure ends the turn `:incomplete` with that cause, or + `:deadline_exceeded` when the deadline has passed. A program with + `finish: :tools` needs a `finish_on` tool, and refuses `last_request_note` + and `last_request_tools`, which it would never use. Its context recovery + (a full window) has no answer steps to mark episodes, so it can only drop + the whole history. + + `history_replay: :actions` shows the steps of earlier turns without the + text and reasoning written beside their calls, and leaves out steps that + only wrote text; a step that carries the task's outputs (an answer) is shown + whole. The current turn is shown whole: it starts with the call's new + inputs, or, on a call without them, at the last step that carries inputs. + DeepSeek accepts earlier turns without `reasoning_content`; Anthropic and + Gemini do not validate reasoning on earlier turns. Some thinking APIs reject + an earlier tool-call message without its reasoning content (Kimi-style + thinking modes); Kimi K3 through OpenRouter accepted one in a probe + (2026-10-17), but check the provider before relying on it. + ## The prediction The prediction's fields are the signature's outputs and nothing else, so an @@ -92,7 +118,8 @@ defmodule Imp.Predict.ReActV2 do and its last request answered (below). * `:incomplete`. The turn was interrupted and has no answer: its last request was answered with no valid answer in it, or the turn ran out of - time or context window. + time or context window, or, under `finish: :tools`, it ended without a + `finish_on` tool. A turn that could not get a model response returns `{:error, %Imp.Predict.ReActV2.StepError{}}` rather than a prediction: its @@ -190,7 +217,10 @@ defmodule Imp.Predict.ReActV2 do `Imp.Tool`s that the program holds, each the program's own in name, description and schema, whose runners the call uses in place of the program's: the request is unchanged, and only what a call to those tools - does is the call's. Any other value is refused before a model is called. + does is the call's. `finish` (`:auto`, `:submit` or `:tools`) replaces the + program's ending for the call, as `new/3` would build it; it is an option + only when the task has no input named `finish`, which is then that input. + Any other value is refused before a model is called. Together they continue a conversation the caller holds. A call with a `:history`, `max_iters: 0` and a note makes one request, the last request: @@ -380,8 +410,9 @@ defmodule Imp.Predict.ReActV2 do type: {:or, [:string, nil]}, default: nil, doc: - "The line shown after a thought-only step under `finish: :submit` or `:tools`, " <> - "in place of the default, so it can name what ending the turn means for this host." + "The line shown after each thought-only step, in place of Imp's: under " <> + "`finish: :submit` and `:tools`, and on a plain-text loop's replay of a history " <> + "written under `:submit`. A blank line is refused." ], history_replay: [ type: {:in, [:trajectory, :actions]}, @@ -435,6 +466,8 @@ defmodule Imp.Predict.ReActV2 do reject_reserved_tools!(signature, "Imp.Predict.ReActV2.new/3") validate_tools_finish!(opts[:finish], opts[:finish_on], "Imp.Predict.ReActV2.new/3") + validate_tools_last_request!(opts, "Imp.Predict.ReActV2.new/3") + validate_unsent_note!(opts[:unsent_note], "Imp.Predict.ReActV2.new/3") tools = put_submit(tools, signature, opts[:finish]) @@ -527,16 +560,76 @@ defmodule Imp.Predict.ReActV2 do defp validate_tools_finish!(_finish, _finish_on, _context), do: :ok + @doc false + # Shared with `Imp.Saving`, so a loaded program is held to the same rules. + def validate_tools_program!(%__MODULE__{} = react, context) do + validate_tools_finish!(react.finish, react.finish_on, context) + + validate_tools_last_request!( + [ + finish: react.finish, + last_request_note: react.last_request_note, + last_request_tools: react.last_request_tools + ], + context + ) + + validate_unsent_note!(react.unsent_note, context) + react + end + + defp validate_tools_last_request!(opts, context) do + if opts[:finish] == :tools and + (is_binary(opts[:last_request_note]) or (opts[:last_request_tools] || []) != []), + do: raise(ArgumentError, "#{context}: " <> tools_last_request_reason()) + + :ok + end + + # An empty line would be sent as an empty user message. + defp validate_unsent_note!(note, context) when is_binary(note) do + if String.trim(note) == "", + do: raise(ArgumentError, "#{context}: unsent_note must say something, or be left out") + + :ok + end + + defp validate_unsent_note!(_note, _context), do: :ok + # A call's own `finish`, for one call: the loop as `new/3` would build it with # that ending, from the same tools and options. - defp with_finish(%__MODULE__{finish: finish} = react, finish), do: react + defp with_finish(%__MODULE__{finish: finish} = react, finish), do: {:ok, react} + + defp with_finish(%__MODULE__{finish_on: finish_on}, :tools) when finish_on == %{}, + do: {:error, {:invalid_react_v2_finish, "finish: :tools needs at least one finish_on tool"}} + + defp with_finish(%__MODULE__{} = react, :tools) + when react.last_request_tools != [] or is_binary(react.last_request_note), + do: {:error, {:invalid_react_v2_finish, tools_last_request_reason()}} defp with_finish(%__MODULE__{} = react, finish) do - validate_tools_finish!(finish, react.finish_on, "Imp.Predict.ReActV2 call") tools = react.tools |> Map.delete(:submit) |> put_submit(react.signature, finish) - restore_loop(%{react | finish: finish, tools: tools}) + {:ok, restore_loop(%{react | finish: finish, tools: tools})} + end + + # `finish` is a call option unless the task has an input of that name, which + # is then the input, as it was before the option existed. + defp pop_call_finish(inputs, react) do + if Enum.any?(react.signature.inputs, &Imp.FieldMap.same_name?(&1.name, :finish)), + do: {react.finish, inputs}, + else: pop_call_option(inputs, :finish, react.finish) end + defp tools_last_request_reason, + do: + "finish: :tools makes no last request (an interrupted turn ends :incomplete), so " <> + "last_request_note and last_request_tools would never be used" + + defp validate_tools_call_note(:tools, note) when is_binary(note), + do: {:error, {:invalid_react_v2_last_request_note, tools_last_request_reason()}} + + defp validate_tools_call_note(_finish, _note), do: :ok + defp validate_call_finish(finish) when finish in [:auto, :submit, :tools], do: :ok defp validate_call_finish(finish), @@ -727,13 +820,14 @@ defmodule Imp.Predict.ReActV2 do defp do_call(%__MODULE__{} = react, inputs, execution) do with {:ok, inputs} <- normalize_inputs(inputs), - {finish, inputs} <- pop_call_option(inputs, :finish, react.finish), + {finish, inputs} <- pop_call_finish(inputs, react), :ok <- validate_call_finish(finish), - react = with_finish(react, finish), + {:ok, react} <- with_finish(react, finish), {max_iters, inputs} <- pop_max_iters(inputs, react.max_iters), :ok <- validate_call_max_iters(max_iters), {note, inputs} <- pop_call_option(inputs, :last_request_note, react.last_request_note), :ok <- validate_call_note(note), + :ok <- validate_tools_call_note(react.finish, note), {config, inputs} <- pop_call_option(inputs, :config, []), :ok <- validate_call_config(config), {tools, inputs} <- pop_call_option(inputs, :tools, []), @@ -761,19 +855,20 @@ defmodule Imp.Predict.ReActV2 do run( react, - history_context(history, react.signature, react.history_replay), + history_context(history, react.signature, react.history_replay, pending), pending, pending, 0, max_iters, execution, - 0 + %{thoughts: 0, retried: false} ) end end - # `thoughts` counts the thought-only steps since the last step that called - # a tool (see `continue_after_thought?/3`). + # `steps.thoughts` counts the thought-only steps since the last step that + # called a tool (see `continue_after_thought?/3`); `steps.retried` whether a + # `finish: :tools` turn has spent its one retry of a failed request. defp run(react, history, inputs, pending, turn, max_iters, execution, _thoughts) when turn >= max_iters, do: interrupted(react, history, inputs, pending, :max_iters, turn, nil, execution) @@ -803,7 +898,7 @@ defmodule Imp.Predict.ReActV2 do emit_reasoning(prediction, turn) {history, pending} = append_thought_only_step(history, pending, prediction, calls) - if continue_after_thought?(react, prediction, thoughts), + if continue_after_thought?(react, prediction, thoughts.thoughts), do: run( react, @@ -813,7 +908,7 @@ defmodule Imp.Predict.ReActV2 do turn + 1, max_iters, execution, - thoughts + 1 + %{thoughts | thoughts: thoughts.thoughts + 1} ), else: interrupted(react, history, inputs, pending, cause, turn, nil, execution) end @@ -840,7 +935,10 @@ defmodule Imp.Predict.ReActV2 do }) true -> - run(react, history, inputs, %{}, turn + 1, max_iters, execution, 0) + run(react, history, inputs, %{}, turn + 1, max_iters, execution, %{ + thoughts + | thoughts: 0 + }) end end end @@ -867,6 +965,14 @@ defmodule Imp.Predict.ReActV2 do match?(%Imp.LMError{content_filtered: true}, reason) -> step_error(history, reason) + # Under `finish: :tools` there is no last request to absorb a + # transient failure, so the turn gets one more ordinary step, once. + react.finish == :tools and not thoughts.retried and not deadline_passed?() -> + run(react, history, inputs, pending, turn + 1, max_iters, execution, %{ + thoughts + | retried: true + }) + true -> interrupted( react, @@ -925,8 +1031,10 @@ defmodule Imp.Predict.ReActV2 do _turn, error, _execution - ), - do: incomplete_prediction(history, cause, error) + ) do + cause = if deadline_passed?(), do: :deadline_exceeded, else: cause + incomplete_prediction(history, cause, error) + end defp interrupted(react, history, inputs, pending, cause, turn, error, execution) do if plain_text?(react.signature, react.finish), @@ -1979,7 +2087,7 @@ defmodule Imp.Predict.ReActV2 do # Full history remains the durable result. Only whole prior episode groups are # eligible for prompt projection; tool observations appended in this call are # protected even if they alone exceed the model window. - defp history_context(history, signature, replay) do + defp history_context(history, signature, replay, pending) do outputs = Imp.Signature.output_names(signature) size = length(history.messages) @@ -2005,7 +2113,7 @@ defmodule Imp.Predict.ReActV2 do retries: 0, # `history_replay: :actions`: the steps given before this call are # earlier turns, shown as their actions (`actions_view/2`). - actions_before: if(replay == :actions, do: size), + actions_before: if(replay == :actions, do: current_turn_start(history, signature, pending)), inputs: Imp.Signature.input_names(signature), outputs: outputs } @@ -2020,6 +2128,30 @@ defmodule Imp.Predict.ReActV2 do # (decisions.md). @step_words [:next_thought, :reasoning_content, :reasoning_details] + # Where the current turn starts in a history passed to a call. A call with + # new inputs starts a turn, so every passed step is earlier. A call without + # them (a resume mid-turn) continues the turn the last step carrying the + # task's inputs opened; that step and those after it are the current turn, + # shown whole, since a provider that keeps reasoning within a turn + # (DeepSeek's `reasoning_content`, Anthropic's thinking blocks, Gemini's + # thought signatures) needs it on the turn's own calls. + defp current_turn_start(history, signature, pending) do + if pending != %{} do + length(history.messages) + else + inputs = Imp.Signature.input_names(signature) + + history.messages + |> Enum.with_index() + |> Enum.filter(fn {step, _} -> Enum.any?(inputs, &Imp.FieldMap.has_key?(step, &1)) end) + |> List.last() + |> case do + {_step, index} -> index + nil -> 0 + end + end + end + defp actions_view(step, context) do if Enum.any?(context.outputs, &(not is_nil(Imp.FieldMap.get(step, &1)))) do [step] diff --git a/lib/imp/saving.ex b/lib/imp/saving.ex index cc4c8b5a..756bd01e 100644 --- a/lib/imp/saving.ex +++ b/lib/imp/saving.ex @@ -650,11 +650,12 @@ defmodule Imp.Saving do last_request_note: load_react_v2_last_request_note!(state["last_request_note"]), finish_on: load_finish_on!(state["finish_on"]), finish: finish, - unsent_note: load_react_v2_last_request_note!(state["unsent_note"]), + unsent_note: load_react_v2_unsent_note!(state["unsent_note"]), history_replay: load_react_v2_history_replay!(state["history_replay"]), last_request_tools: load_last_request_tools!(state["last_request_tools"], tools, finish), tool_policy: load_tool_policy!(state["tool_policy"], "ReActV2 tool policy") } + |> Imp.Predict.ReActV2.validate_tools_program!("loading a ReActV2 program") |> Imp.Predict.ReActV2.restore_loop() end @@ -1327,6 +1328,12 @@ defmodule Imp.Saving do defp maybe_put_saved(state, key, value, _default), do: Map.put(state, key, value) + defp load_react_v2_unsent_note!(nil), do: nil + defp load_react_v2_unsent_note!(note) when is_binary(note), do: note + + defp load_react_v2_unsent_note!(other), + do: raise(ArgumentError, "invalid saved ReActV2 unsent_note: #{inspect(other)}") + defp load_react_v2_history_replay!(nil), do: :trajectory defp load_react_v2_history_replay!("trajectory"), do: :trajectory defp load_react_v2_history_replay!("actions"), do: :actions diff --git a/test/react_v2_finish_tools_test.exs b/test/react_v2_finish_tools_test.exs index 6766c728..01d4651b 100644 --- a/test/react_v2_finish_tools_test.exs +++ b/test/react_v2_finish_tools_test.exs @@ -180,4 +180,107 @@ defmodule ReActV2FinishToolsTest do refute Map.has_key?(plain, "unsent_note") or Map.has_key?(plain, "history_replay") end + + test "a task input named finish is the input, not the call option" do + program = + Imp.react("start, finish -> reply", [look()], + lm: scripted(self(), [%{reply: "Eight hours.", tool_calls: []}]) + ) + + assert {:ok, prediction} = Imp.call(program, %{start: "9am", finish: "5pm"}) + assert Imp.get(prediction, :reply) == "Eight hours." + [{messages, _opts}] = requests() + assert List.last(messages).content =~ "5pm" + end + + test "a passed deadline ends a :tools turn :deadline_exceeded, with no further request" do + slow = + Imp.LM.Static.new( + handler: fn messages, opts -> + send(self(), {:request, messages, opts}) + Process.sleep(30) + @thought + end + ) + + program = + Imp.react("ticket -> reply", [look(), send_tool(), decline()], + lm: slow, + finish: :tools, + finish_on: finish_on() + ) + + assert {:ok, prediction} = + Imp.Deadline.with_deadline(10, fn -> Imp.call(program, %{ticket: "?"}) end) + + assert prediction.metadata.termination_reason == :incomplete + assert prediction.metadata.termination_cause == :deadline_exceeded + end + + test "a failed request gets one more ordinary step, once per turn" do + {:ok, count} = Agent.start_link(fn -> 0 end) + + flaky = fn replies -> + Imp.LM.Static.new( + handler: fn _messages, _opts -> + n = Agent.get_and_update(count, &{&1, &1 + 1}) + + case Enum.at(replies, n) do + :fail -> raise "provider down" + reply -> reply + end + end + ) + end + + program = fn replies -> + Agent.update(count, fn _ -> 0 end) + + Imp.react("ticket -> reply", [look(), send_tool(), decline()], + lm: flaky.(replies), + finish: :tools, + finish_on: finish_on() + ) + end + + assert {:ok, prediction} = Imp.call(program.([:fail, @send]), %{ticket: "?"}) + assert prediction.metadata.termination_reason == :finished_by_tool + + assert {:ok, prediction} = Imp.call(program.([:fail, :fail, @send]), %{ticket: "?"}) + assert prediction.metadata.termination_reason == :incomplete + assert Agent.get(count, & &1) == 2 + end + + test "settings :tools would never use, a blank note and a call's :tools without finish_on are refused" do + assert_raise ArgumentError, ~r/finish: :tools makes no last request/, fn -> + program([], last_request_note: "Last one.") + end + + assert_raise ArgumentError, ~r/unsent_note must say something/, fn -> + program([], unsent_note: " ") + end + + assert {:error, {:invalid_react_v2_last_request_note, _}} = + Imp.call(program([]), %{ticket: "?", last_request_note: "Last one."}) + + plain = Imp.react("ticket -> reply", [look()], lm: scripted(self(), [])) + + assert {:error, {:invalid_react_v2_finish, _}} = + Imp.call(plain, %{ticket: "?", finish: :tools}) + + runner = fn _arguments -> "x" end + finish = fn _arguments, _result, _inputs -> {:finish, %{reply: ""}} end + registry = Imp.Saving.Registry.new(look_runner: runner, finish_cb: finish) + + dumped = + Imp.react("ticket -> reply", [Imp.tool(:look, "Look", runner)], + finish: :tools, + finish_on: %{look: finish} + ) + |> Imp.dump(registry: registry) + + assert_raise ArgumentError, ~r/finish: :tools needs at least one finish_on tool/, fn -> + dumped |> Map.put("finish_on", %{}) |> Imp.load!(registry: registry) + end + end end diff --git a/test/react_v2_history_replay_test.exs b/test/react_v2_history_replay_test.exs index 6b5a3262..91ea0a3a 100644 --- a/test/react_v2_history_replay_test.exs +++ b/test/react_v2_history_replay_test.exs @@ -6,7 +6,11 @@ defmodule ReActV2HistoryReplayTest do # reasoning written beside the calls. The current turn is shown whole, and # the history the prediction returns keeps every step. - defp look, do: Imp.tool(:look, "Look at a thing", fn _arguments -> "a cat" end) + defp look, do: Imp.tool(:look, "Look at a thing", fn %{"at" => at} -> "seen: #{at}" end) + + @schema %{"type" => "object", "properties" => %{"at" => %{"type" => "string"}}} + defp look_tool, + do: Imp.tool(:look, "Look at a thing", fn %{"at" => at} -> "seen: #{at}" end, schema: @schema) defp scripted(owner, replies) do {:ok, agent} = Agent.start_link(fn -> replies end) @@ -23,17 +27,25 @@ defmodule ReActV2HistoryReplayTest do ) end - # An earlier turn: a call with narration beside it, a thought-only step, - # then the answer. + defp flush do + receive do + {:request, _, _} -> flush() + after + 0 -> :ok + end + end + + defp call_step(id, at, text), + do: %{reply: text, tool_calls: [%{id: id, name: "look", arguments: %{"at" => at}}]} + + # An earlier turn: a call with narration and reasoning beside it, a + # thought-only step, then the answer. defp earlier do {:ok, first} = - Imp.react("ticket -> reply", [look()], + Imp.react("ticket -> reply", [look_tool()], lm: scripted(self(), [ - %{ - reply: "Earlier narration beside a call.", - tool_calls: [%{id: "c0", name: "look", arguments: %{}}] - }, + call_step("c0", "the box", "Earlier narration."), %{reply: "A cat.", tool_calls: []} ]) ) @@ -41,66 +53,122 @@ defmodule ReActV2HistoryReplayTest do flush() [call, answer] = first.metadata.history.messages + + call = + call + |> Map.put(:reasoning_content, "earlier reasoning") + |> Map.put(:reasoning_details, [%{"text" => "earlier detail"}]) + thought = %{next_thought: "An earlier thought nobody answered.", tool_calls: []} Imp.History.new([call, thought, answer]) end - defp flush do - receive do - {:request, _, _} -> flush() - after - 0 -> :ok - end - end - - defp texts(messages), do: Enum.map_join(messages, "\n", &to_string(&1.content)) - - defp call(replay) do - Imp.react("ticket -> reply", [look()], - lm: - scripted(self(), [ - %{ - reply: "This turn's narration.", - tool_calls: [%{id: "c1", name: "look", arguments: %{}}] - }, - "Still a cat." - ]), - history_replay: replay - ) - |> Imp.call(%{ticket: "And now?", history: earlier()}) - end + defp program(replay, replies), + do: + Imp.react("ticket -> reply", [look_tool()], + lm: scripted(self(), replies), + history_replay: replay + ) - defp second_request do + defp last_request do receive do - {:request, _first, _opts} -> + {:request, messages, _opts} -> receive do - {:request, messages, _opts} -> messages + {:request, _, _} = next -> + send(self(), next) + last_request() + after + 0 -> messages end end end - test ":actions leaves earlier narration and thoughts out of the request, and keeps the current turn whole" do - assert {:ok, prediction} = call(:actions) - text = texts(second_request()) + defp assistant_with_call(messages, id), + do: + Enum.find( + messages, + &(&1.role == :assistant and + Enum.any?(&1[:tool_calls] || [], fn c -> c[:id] == id or c["id"] == id end)) + ) + + defp tool_result(messages, id), + do: + Enum.find( + messages, + &(&1.role == :tool and Enum.any?(&1[:tool_calls] || [], fn c -> c[:id] == id end)) + ) + + test ":actions strips earlier narration, reasoning and thoughts, keeps calls paired with their results, and keeps the current turn whole" do + replies = [ + call_step("c1", "the lid", "This turn's narration."), + %{reply: "Still a cat.", tool_calls: []} + ] + + assert {:ok, prediction} = + Imp.call(program(:actions, replies), %{ticket: "And now?", history: earlier()}) - refute text =~ "Earlier narration beside a call." + messages = last_request() + text = Enum.map_join(messages, "\n", &inspect/1) + + refute text =~ "Earlier narration." + refute text =~ "earlier reasoning" + refute text =~ "earlier detail" refute text =~ "An earlier thought nobody answered." - # The earlier call, its result, the answer and the current turn remain. - assert text =~ "What is in the box?" - assert text =~ "a cat" - assert text =~ "A cat." + + # The earlier call is still there, paired with its own result. + earlier_call = assistant_with_call(messages, "c0") + assert earlier_call + assert tool_result(messages, "c0").content =~ "seen: the box" + # The current turn is whole: its narration and its call with its result. assert text =~ "This turn's narration." + assert tool_result(messages, "c1").content =~ "seen: the lid" + assert text =~ "A cat." - # The history returned keeps everything. - kept = prediction.metadata.history.messages |> Enum.map(&inspect/1) |> Enum.join() - assert kept =~ "Earlier narration beside a call." - assert kept =~ "An earlier thought nobody answered." + kept = inspect(prediction.metadata.history.messages) + assert kept =~ "Earlier narration." and kept =~ "earlier reasoning" end test ":trajectory, the default, shows earlier steps whole" do - assert {:ok, _prediction} = call(:trajectory) - text = texts(second_request()) - assert text =~ "Earlier narration beside a call." + replies = [%{reply: "Still a cat.", tool_calls: []}] + + assert {:ok, _} = + Imp.call(program(:trajectory, replies), %{ticket: "And now?", history: earlier()}) + + text = Enum.map_join(last_request(), "\n", &inspect/1) + assert text =~ "Earlier narration." assert text =~ "An earlier thought nobody answered." end + + test "a resume with no new inputs keeps the in-progress turn whole, reasoning included" do + # The history ends mid-turn: the turn's inputs, then a call with reasoning + # beside it and its result, and no answer yet. + {:ok, stopped} = + Imp.react("ticket -> reply", [look_tool()], + lm: + scripted(self(), [ + %{ + next_thought: "Checking the lid.", + tool_calls: [%{id: "c2", name: "look", arguments: %{"at" => "the lid"}}] + } + ]), + finish: :tools, + finish_on: %{look: fn _arguments, _result, _inputs -> :continue end}, + max_iters: 1 + ) + |> Imp.call(%{ticket: "And the lid?", history: earlier()}) + + flush() + {done, [current]} = Enum.split(stopped.metadata.history.messages, -1) + in_progress = done ++ [Map.put(current, :reasoning_content, "current reasoning")] + + assert {:ok, _} = + Imp.call(program(:actions, [%{reply: "Shut.", tool_calls: []}]), %{ + history: Imp.History.new(in_progress) + }) + + text = Enum.map_join(last_request(), "\n", &inspect/1) + assert text =~ "Checking the lid." + assert text =~ "current reasoning" + refute text =~ "earlier reasoning" + end end From 9639713742da06cb6f4a0ba3b19db81529c32215 Mon Sep 17 00:00:00 2001 From: deepfates Date: Fri, 16 Oct 2026 13:43:02 -0700 Subject: [PATCH 3/4] ReActV2 :actions: a call's own note starts a turn, so a summary request reads every passed turn as earlier --- docs/diving-deeper/react.md | 5 +++-- lib/imp/predict/react_v2.ex | 18 ++++++++++++++---- test/react_v2_history_replay_test.exs | 13 +++++++++++++ 3 files changed, 30 insertions(+), 6 deletions(-) diff --git a/docs/diving-deeper/react.md b/docs/diving-deeper/react.md index 4079de4f..d6d4c772 100644 --- a/docs/diving-deeper/react.md +++ b/docs/diving-deeper/react.md @@ -536,8 +536,9 @@ their results, and the outputs. The current turn is always shown whole, and the history the prediction returns keeps every step. Use it when the model's own narration should not become something it reads back on every later turn. A step that carries the task's outputs (an answer) is shown -whole. On a call without new inputs (a resume mid-turn), the current turn -starts at the last step that carries inputs. Some thinking APIs reject an +whole. A call with new inputs or its own `last_request_note` starts a turn; +on a call with neither (a resume mid-turn), the current turn starts at the +last step that carries inputs. Some thinking APIs reject an earlier tool-call message without its reasoning content; check yours. ### Why a turn ended diff --git a/lib/imp/predict/react_v2.ex b/lib/imp/predict/react_v2.ex index c03785ee..1f17d375 100644 --- a/lib/imp/predict/react_v2.ex +++ b/lib/imp/predict/react_v2.ex @@ -54,7 +54,8 @@ defmodule Imp.Predict.ReActV2 do text and reasoning written beside their calls, and leaves out steps that only wrote text; a step that carries the task's outputs (an answer) is shown whole. The current turn is shown whole: it starts with the call's new - inputs, or, on a call without them, at the last step that carries inputs. + inputs or its own `last_request_note`, or, on a call with neither (a resume + mid-turn), at the last step that carries inputs. DeepSeek accepts earlier turns without `reasoning_content`; Anthropic and Gemini do not validate reasoning on earlier turns. Some thinking APIs reject an earlier tool-call message without its reasoning content (Kimi-style @@ -825,6 +826,7 @@ defmodule Imp.Predict.ReActV2 do {:ok, react} <- with_finish(react, finish), {max_iters, inputs} <- pop_max_iters(inputs, react.max_iters), :ok <- validate_call_max_iters(max_iters), + note_given? = is_binary(inputs[:last_request_note] || inputs["last_request_note"]), {note, inputs} <- pop_call_option(inputs, :last_request_note, react.last_request_note), :ok <- validate_call_note(note), :ok <- validate_tools_call_note(react.finish, note), @@ -855,7 +857,12 @@ defmodule Imp.Predict.ReActV2 do run( react, - history_context(history, react.signature, react.history_replay, pending), + history_context( + history, + react.signature, + react.history_replay, + if(note_given?, do: :new_request, else: pending) + ), pending, pending, 0, @@ -2129,13 +2136,16 @@ defmodule Imp.Predict.ReActV2 do @step_words [:next_thought, :reasoning_content, :reasoning_details] # Where the current turn starts in a history passed to a call. A call with - # new inputs starts a turn, so every passed step is earlier. A call without - # them (a resume mid-turn) continues the turn the last step carrying the + # new inputs or its own `last_request_note` starts a turn, so every passed + # step is earlier. A call with neither (a resume mid-turn) continues the turn + # the last step carrying the # task's inputs opened; that step and those after it are the current turn, # shown whole, since a provider that keeps reasoning within a turn # (DeepSeek's `reasoning_content`, Anthropic's thinking blocks, Gemini's # thought signatures) needs it on the turn's own calls. defp current_turn_start(history, signature, pending) do + # A call that brings new inputs, or its own note (a new user message, such + # as a request for a summary), starts a turn: every passed step is earlier. if pending != %{} do length(history.messages) else diff --git a/test/react_v2_history_replay_test.exs b/test/react_v2_history_replay_test.exs index 91ea0a3a..9e7e3df0 100644 --- a/test/react_v2_history_replay_test.exs +++ b/test/react_v2_history_replay_test.exs @@ -171,4 +171,17 @@ defmodule ReActV2HistoryReplayTest do assert text =~ "current reasoning" refute text =~ "earlier reasoning" end + + test "a call with its own note and no inputs (a summary request) reads every passed turn as earlier" do + assert {:ok, _} = + Imp.call(program(:actions, ["Summary."]), %{ + history: earlier(), + max_iters: 0, + last_request_note: "Summarise this conversation." + }) + + text = Enum.map_join(last_request(), "\n", &inspect/1) + refute text =~ "Earlier narration." + assert text =~ "Summarise this conversation." + end end From 1a9df00fd9fb9650ef87245249c1105809d81018 Mon Sep 17 00:00:00 2001 From: deepfates Date: Fri, 16 Oct 2026 14:02:07 -0700 Subject: [PATCH 4/4] ReActV2 :tools: no retry on the last allowed step, so the error is kept; docs and dates match the retry --- CHANGELOG.md | 8 +++++--- decisions.md | 4 ++-- lib/imp/predict/react_v2.ex | 16 +++++++++++----- test/react_v2_finish_tools_test.exs | 28 ++++++++++++++++++++++++++++ 4 files changed, 46 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7ed6b7f1..ed801323 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,9 +8,11 @@ User-visible changes to Imp are recorded here. - ReActV2 `finish: :tools`: only a `finish_on` tool ends a turn. Text without a tool call is a thought, followed by one ordinary step and a line naming - the ending tools; a second thought in a row, an empty step, the step limit - or a failed step ends the turn `:incomplete`, with no further request and - nothing forced. `unsent_note:` replaces that line. A call may give its own + the ending tools. A failed request gets one more ordinary step, once per + turn (not on the last allowed step). A second thought in a row, an empty + step, the step limit or a second failure ends the turn `:incomplete`, with + no further request and nothing forced; a passed deadline ends it + `:deadline_exceeded`. `unsent_note:` replaces that line. A call may give its own `finish:`. - ReActV2 `history_replay: :actions`: earlier turns are shown as their inputs, calls with results, and outputs, without the text and reasoning diff --git a/decisions.md b/decisions.md index 14598901..5b95c969 100644 --- a/decisions.md +++ b/decisions.md @@ -8,8 +8,8 @@ not necessarily when it was made. | Date | Decision | Source and reason | Status | Retires when | | --- | --- | --- | --- | --- | -| 2026-10-17 | ReActV2 `history_replay: :actions` (opt-in) shows earlier turns without the text and reasoning their steps wrote beside their calls, and without steps that only wrote text. Default `:trajectory` replays whole, as DSPy's ReActV2 does (its history events carry `next_thought` and the calls with results, and the adapter renders every event). | A host whose loop does not speak for itself (its text is read by nobody) found the model rebuilding a persona from its own earlier narration on every turn. The divergence is opt-in; the record keeps every step, only the prompt changes. | settled | DSPy offers a comparable history view. | -| 2026-10-17 | ReActV2 `finish: :tools` ends a turn only through `finish_on` and never forces: past the one continued step after a thought, the step limit or a failed step, the turn ends `:incomplete`. | Forcing a call from a loop whose ending tools act in the world would make it act without having decided to; the host knows what an unfinished turn means. | settled | A host needs a forced ending tool. | +| 2026-10-16 | ReActV2 `history_replay: :actions` (opt-in) shows earlier turns without the text and reasoning their steps wrote beside their calls, and without steps that only wrote text. Default `:trajectory` replays whole, as DSPy's ReActV2 does (its history events carry `next_thought` and the calls with results, and the adapter renders every event). | A host whose loop does not speak for itself (its text is read by nobody) found the model rebuilding a persona from its own earlier narration on every turn. The divergence is opt-in; the record keeps every step, only the prompt changes. | settled | DSPy offers a comparable history view. | +| 2026-10-16 | ReActV2 `finish: :tools` ends a turn only through `finish_on` and never forces: past the one continued step after a thought, the step limit, or a second failed request (a failed request gets one more step, once per turn, not on the last allowed step), the turn ends `:incomplete`. | Forcing a call from a loop whose ending tools act in the world would make it act without having decided to; the host knows what an unfinished turn means. | settled | A host needs a forced ending tool. | | 2026-10-16 | Under `finish: :submit`, every thought-only ReActV2 step is followed, in every request and on replay, by one user message: "Your message above was not sent. To reply, call `submit` with ; to keep working, call a tool." The thought stays the model's own (assistant) message. Two exceptions: the forced `submit`'s own last thought, when that request has nothing new to ask, is shown as the request's last user message (the row on the forced request), with no note after it; and a thought that the turn's extracted outputs directly follow was answered and gets no note. This is a sentence Imp writes in a request, where otherwise it adds nothing but a host's `last_request_note`. | `lib/imp/predict/react_v2.ex` `unsent_note/2`, `lib/imp/adapter/chat.ex` `unsent_note_messages/3`, `test/react_v2_unsent_note_test.exs`. With the thought shown back as a user message under `[[ ## next_thought ## ]]`, some models read it as an echo or a pipeline artifact, or as their own message already sent, and ended with an empty `submit`: 4 of 74 owed replies lost in a rehearsal. Measured 2026-10-16 on 21 real final steps, 2 samples each, across 11 model families, after a plain-text step: thought as a user message, 5 samples wrote text again; thought kept with this note, none did, replies were submitted intact (similarity about 1.0), Meta's models accepted it with `tool_choice: auto`, and one family once submitted a closing note as its answer; with only the note's first sentence, models probed their status and took extra steps. The note is a sentence Imp writes, which the last request does not; it states a fact about the step (only `submit` is sent) and the two ways on. | In force. | A measurement finds a wording or shape that loses fewer replies. | | 2026-10-16 | Under `finish: :submit`, a ReActV2 step that writes a thought and calls no tool is followed by an ordinary step (every tool, `tool_choice: "auto"`; amended the same day: the thought stays the model's own message, followed by the unsent note, the row on that note), not at once by the forced `submit`; a second thought-only step in a row, or a step that says nothing or only whitespace, is the `:empty_tool_calls` interruption. `max_iters` ends the turn with the forced `submit` as before; once the deadline has passed no further request is sent and the turn is `:incomplete` (`:deadline_exceeded`). With written tool calls the request ends on the tool listing, after the thought. Signatures with `submit` under `finish: :auto` keep DSPy's ending. | `lib/imp/predict/react_v2.ex` `continue_after_thought?/3`, `test/react_v2_thought_continues_test.exs`. Forcing at once took every other tool away from a model that had just announced its next action, and added a request whose failure lost the reply the thought held. Measured 2026-10-16 on 42 real final steps from 11 models through OpenRouter, sending that ordinary step: most samples called `submit` with the thought's reply, copied or paraphrased (Meta's models 6 of 6, unforced; Kimi, GPT and Gemini models likewise); a thought that was a closing note was submitted as an empty answer; a DeepSeek model went on to fetch what it had said it would read; 5 of 42 wrote text again. DSPy's ReActV2 forces `submit` after any step that calls no tool. | In force. | A measurement shows a different bound, or forcing at once, loses fewer answers. | | 2026-10-16 | ReActV2's forced `submit` request, when the step before it wrote a thought and called nothing, shows that thought as the request's last message, a user message under its field's marker (`[[ ## next_thought ## ]]`), not as the assistant turn the history records it as. DSPy's ReActV2 leaves the thought out of the forced request. | `lib/imp/adapter/chat.ex` `thought_as_request/4`, `test/react_v2_finish_submit_test.exs`. As a trailing assistant turn the request is refused by providers that do not take an assistant prefix: Mistral ("Cannot set an assistant prefix with function calling") and Gemini ("Requests ending with a model turn are not supported"), probed through OpenRouter 2026-10-16. Dropping it, as DSPy does, made models regenerate finished answers with reasoning off, 2 of 16 to the output limit (deepfates/imp#336). Measured 2026-10-16 on real final steps re-sent through OpenRouter: with the thought as this user message, the submitted answer kept the plain-text reply (text similarity 0.8 to 1.0 across Kimi, GPT, DeepSeek and Muse models), a model whose plain text was a closing note submitted an empty answer as it intended, Meta's models (which accept only `tool_choice: auto`) copied the reply into `submit` 3 of 3, and Gemini accepted every request; with the thought dropped, the models wrote a different answer (similarity about 0.05), one shrank to 30 characters, and the closing-note model submitted a 170-character answer. | In force. | A measurement shows the thought is better dropped, or DSPy keeps it in a form providers accept. | diff --git a/lib/imp/predict/react_v2.ex b/lib/imp/predict/react_v2.ex index 1f17d375..fd36a38d 100644 --- a/lib/imp/predict/react_v2.ex +++ b/lib/imp/predict/react_v2.ex @@ -43,7 +43,9 @@ defmodule Imp.Predict.ReActV2 do made. A step whose request fails gets one more ordinary step, once per turn (there is no last request to absorb a transient failure). A second thought-only step in a row, a step that says nothing, the step limit or a - second failure ends the turn `:incomplete` with that cause, or + second failure (or a failure on the last allowed step) ends the turn + `:incomplete` with that cause and the request's error in + `termination_error`, or `:deadline_exceeded` when the deadline has passed. A program with `finish: :tools` needs a `finish_on` tool, and refuses `last_request_note` and `last_request_tools`, which it would never use. Its context recovery @@ -60,7 +62,7 @@ defmodule Imp.Predict.ReActV2 do Gemini do not validate reasoning on earlier turns. Some thinking APIs reject an earlier tool-call message without its reasoning content (Kimi-style thinking modes); Kimi K3 through OpenRouter accepted one in a probe - (2026-10-17), but check the provider before relying on it. + (2026-10-16), but check the provider before relying on it. ## The prediction @@ -403,8 +405,9 @@ defmodule Imp.Predict.ReActV2 do "the next step continues with every tool, and a second such step in a row " <> "forces `submit`. `:tools` ends a turn only through a `finish_on` tool: text " <> "without a tool call is a thought as under `:submit`, there is no `submit`, and " <> - "a second such step in a row, the step limit or a failed step ends the turn " <> - "`:incomplete`, with no further request. " <> + "a failed request gets one more step, once per turn; a second such step in a " <> + "row, the step limit or a second failure ends the turn `:incomplete`, with no " <> + "further request. " <> "See \"Which signatures get `submit`\" above." ], unsent_note: [ @@ -974,7 +977,10 @@ defmodule Imp.Predict.ReActV2 do # Under `finish: :tools` there is no last request to absorb a # transient failure, so the turn gets one more ordinary step, once. - react.finish == :tools and not thoughts.retried and not deadline_passed?() -> + # A retry on the last allowed step would end at the step limit and + # lose the error, so the turn ends on the error there instead. + react.finish == :tools and not thoughts.retried and turn + 1 < max_iters and + not deadline_passed?() -> run(react, history, inputs, pending, turn + 1, max_iters, execution, %{ thoughts | retried: true diff --git a/test/react_v2_finish_tools_test.exs b/test/react_v2_finish_tools_test.exs index 01d4651b..aaffc13e 100644 --- a/test/react_v2_finish_tools_test.exs +++ b/test/react_v2_finish_tools_test.exs @@ -283,4 +283,32 @@ defmodule ReActV2FinishToolsTest do dumped |> Map.put("finish_on", %{}) |> Imp.load!(registry: registry) end end + + test "a failure on the last allowed step ends the turn on the error, not the step limit" do + {:ok, count} = Agent.start_link(fn -> 0 end) + + lm = + Imp.LM.Static.new( + handler: fn _messages, _opts -> + case Agent.get_and_update(count, &{&1, &1 + 1}) do + 0 -> @look + _ -> raise "provider down" + end + end + ) + + program = + Imp.react("ticket -> reply", [look(), send_tool(), decline()], + lm: lm, + finish: :tools, + finish_on: finish_on(), + max_iters: 2 + ) + + assert {:ok, prediction} = Imp.call(program, %{ticket: "?"}) + assert prediction.metadata.termination_reason == :incomplete + assert prediction.metadata.termination_cause == :prediction_error + assert prediction.metadata.termination_error + assert Agent.get(count, & &1) == 2 + end end