diff --git a/CHANGELOG.md b/CHANGELOG.md index 8d2f3c67..be419d5b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ User-visible changes to Imp are recorded here. ## Unreleased +## 0.16.0 — 2026-10-17 + ### Added - `Imp.Run.start/3` `request_retry: [waits: [...], reserve: ms]`: a model diff --git a/README.md b/README.md index a90f25d7..70571e93 100644 --- a/README.md +++ b/README.md @@ -145,7 +145,7 @@ text or JSON you can score, such as an agent's tool descriptions. ## Install ```elixir -{:imp, "~> 0.15.0"} +{:imp, "~> 0.16.0"} ``` Imp needs Elixir 1.19 or later and a C and C++ compiler, for the native code @@ -162,7 +162,7 @@ large-scale benchmarking. Bug reports and pull requests are welcome. - [Getting started](docs/getting-started/index.md) builds one program step by step, from the first call to a supervised server, with real scores. - [Coming from DSPy](docs/coming-from-dspy.md) maps DSPy's names to Imp's. -- [Tutorials](https://github.com/deepfates/imp/tree/v0.15.0/livebooks) are Livebook notebooks +- [Tutorials](https://github.com/deepfates/imp/tree/v0.16.0/livebooks) are Livebook notebooks you can run offline or with a key. - The [cheatsheet](docs/cheatsheet.cheatmd) has the common calls on one page. diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 45a509e5..e0a2d4f1 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,21 +1,22 @@ -# Imp v0.15.0 +# Imp v0.16.0 Imp is a framework for typed, optimizable language-model programs on the BEAM. -This release adds two ReActV2 options a caller opts into: `finish: :tools`, -which ends a turn only through a `finish_on` tool, for a program that acts -only through its tools, and `history_replay: :actions`, which shows earlier -turns without the text and reasoning written beside their calls. Programs that -use neither send byte-for-byte the requests 0.14.1 sent and save as before. -New options make this a minor release under Imp's versioning rule. +This release adds `request_retry:` to `Imp.Run.start/3`, which sends a model +request the provider failed in a way that may pass again in the same run, after +a wait. It also changes two things a caller may rely on: an LM client other +than ReqLLM reports its charge as metadata `:cost`, because `Imp.Core` no longer +reads it from the usage map; and a ReActV2 forced `submit` with invalid +arguments goes to the extractor. A new option and changed behaviour make this +a minor release under Imp's versioning rule. ## Install ```elixir -{:imp, "~> 0.15.0"} +{:imp, "~> 0.16.0"} ``` -`~> 0.15.0` takes later 0.15 patch releases and nothing newer. The two-part -form, `~> 0.15`, means `>= 0.15.0 and < 1.0.0` in Mix, so `mix deps.update +`~> 0.16.0` takes later 0.16 patch releases and nothing newer. The two-part +form, `~> 0.16`, means `>= 0.16.0 and < 1.0.0` in Mix, so `mix deps.update imp` under it takes every later 0.x release, including minor releases that change what callers receive. @@ -28,38 +29,43 @@ in its release definition; see [releases that use MCP or ACP](docs/production.md#releases-that-use-mcp-or-acp). Ordinary Imp startup starts no protocol endpoint. -## Changes and upgrading from 0.14 +## Changes and upgrading from 0.15 -Change the Imp dependency to `{:imp, "~> 0.15.0"}`, run `mix deps.update imp`, -and commit `mix.lock`. No code or data migration is needed: saved programs -without the new options load with their defaults. +Change the Imp dependency to `{:imp, "~> 0.16.0"}`, run `mix deps.update imp`, +and commit `mix.lock`. No data migration is needed. Check the second item if +you use an LM client of your own, or build response envelopes by hand. -- **Added: `finish: :tools`.** Only a `finish_on` tool ends a ReActV2 turn. - There is no `submit`; the outputs come from the `finish_on` functions. A - step that writes text and calls no tool is a thought, followed by one - ordinary step and a line naming the ending tools (`unsent_note:` replaces - it). Nothing is forced and no last request is made: a failed request gets - one more ordinary step, once per turn and not on the last allowed step; a - second thought in a row, a step that says nothing, the step limit or a - second failure ends the turn `:incomplete`, with the request's error in - `termination_error`, or `:deadline_exceeded` when the deadline has passed. - A program with `finish: :tools` needs a `finish_on` tool and refuses - `last_request_note` and `last_request_tools`. A call may give its own - `finish:` (when the task has no input of that name), so a `:tools` program - can ask for one text answer, such as a summary, with `finish: :auto`. -- **Added: `history_replay: :actions`.** Earlier turns are shown as their - inputs, the calls with their results, and their outputs; the text and - reasoning written beside the calls, and steps that only wrote text, are left - out. The current turn is shown whole: it starts with the call's new inputs - or its own `last_request_note`, or, on a resume with neither, at the last - step that carries inputs. The returned history keeps every step. This - diverges from DSPy's ReActV2, which replays every step whole; the default, - `:trajectory`, still does. Some thinking APIs reject an earlier tool-call - message without its reasoning content; check your provider. -- **Added: `unsent_note:`.** The line shown after each thought-only step, in - place of Imp's. A blank line is refused. +- **Added: `request_retry:` on `Imp.Run.start/3`.** `request_retry: [waits: + [ms, ...], reserve: ms]` sends a model request that failed with an error + `Imp.Errors.retryable?/1` accepts (408, 425, 429, 5xx, a failed connection) + again inside `Imp.LM.request/2`, after each wait in turn. Under a deadline + the last wait repeats while it leaves `reserve` (60 s by default) of the + deadline; without one, the list is the bound. A provider's `retry-after` + longer than the wait is waited instead. Each attempt is its own + `:model_request` and `:model_response`, and a failed response followed by + another attempt carries `retry_in_ms`. Only the outermost request in a + process retries, so an LM that wraps an LM does not multiply attempts. + Streamed requests are not sent again. `waits` must not be empty. Off unless + given. A timed-out attempt may have been finished and billed by the + provider without a response to record, so a retry can be billed twice; + under a deadline each attempt is capped to the time left. +- **Changed: where a call's charge is read.** OpenRouter bring-your-own-key + billing moved into `Imp.Clients.ReqLLM`, which records the charge it + resolves as `:charge` in its `req_llm` response metadata. `Imp.Core` reads + `LMResponse.cost` from that `:charge` (final when present, `nil` included), + and otherwise from metadata `:cost`; it no longer reads usage `"cost"` or + `"is_byok"`. An LM client other than ReqLLM, or a hand-built response + envelope, must report its charge as `:cost` in its metadata (or + `req_llm.charge`), or its cost is `nil`. +- **Fixed: a forced `submit` with invalid arguments is extracted.** A ReActV2 + forced `submit` whose arguments fail validation now goes to the typed + extractor and ends `:extracted`, where it ended `:incomplete` with no + outputs. If that extraction request fails with an LM error, the call + returns `{:error, %Imp.Predict.ReActV2.StepError{}}`, where it returned + `{:ok, prediction}` ending `:incomplete`. A forced `submit` the tool policy + refuses still ends `:incomplete`. -Rollback means restoring `0.14.1` in the lock and removing the new options. +Rollback means restoring `0.15.0` in the lock and removing `request_retry:`. See the [changelog](CHANGELOG.md) for every change. ## Known limits diff --git a/docs/coming-from-dspy.md b/docs/coming-from-dspy.md index 15d37baf..ad4307e9 100644 --- a/docs/coming-from-dspy.md +++ b/docs/coming-from-dspy.md @@ -12,7 +12,7 @@ brought in. ## The five-minute version ```elixir -# pip install dspy -> {:imp, "~> 0.15.0"} +# pip install dspy -> {:imp, "~> 0.16.0"} # lm = dspy.LM("openai/...") -> lm = Imp.req_llm("openai:gpt-5.4-mini", api_key: ...) # dspy.Predict("q -> a") -> program = Imp.predict("q -> a", lm: lm) # program(q="...") -> {:ok, pred} = Imp.call(program, %{q: "..."}) @@ -87,7 +87,7 @@ error row too. Each model request has a receive timeout, and has no time limit of its own (an MCP tool call does, 30 seconds by default), so to bound an agent, start it with `Imp.start_run/3` and cancel it when you choose. The -[deployment example](https://github.com/deepfates/imp/blob/v0.15.0/examples/deployment/README.md) +[deployment example](https://github.com/deepfates/imp/blob/v0.16.0/examples/deployment/README.md) is a complete OTP application. **Optimizers see what you name.** DSPy finds predictors by walking a module's diff --git a/docs/diving-deeper/modules-and-composition.md b/docs/diving-deeper/modules-and-composition.md index b78eb154..4521f171 100644 --- a/docs/diving-deeper/modules-and-composition.md +++ b/docs/diving-deeper/modules-and-composition.md @@ -149,7 +149,7 @@ A custom module is your code, so it is not saved as a whole. What an optimizer chose for it is data: `Imp.ProgramParameters.values/1` reads it, `Imp.ProgramParameters.apply_values/2` puts it on a freshly built program, and `Imp.Optimizer.Artifact` writes it to a checksummed file. The -[deployment example](https://github.com/deepfates/imp/blob/v0.15.0/examples/deployment/README.md) +[deployment example](https://github.com/deepfates/imp/blob/v0.16.0/examples/deployment/README.md) does this in a supervised application. ### Built-in variants diff --git a/docs/diving-deeper/saving-and-artifacts.md b/docs/diving-deeper/saving-and-artifacts.md index a94e665e..ebd689d3 100644 --- a/docs/diving-deeper/saving-and-artifacts.md +++ b/docs/diving-deeper/saving-and-artifacts.md @@ -21,7 +21,7 @@ Save the whole program when it is made of Imp's modules: `Imp.predict/2`, when the program is your own struct implementing `Imp.Module`, or when you want the program's code to live in your release and only its tuned text to cross the persistence boundary. The -[deployment example](https://github.com/deepfates/imp/tree/v0.15.0/examples/deployment) +[deployment example](https://github.com/deepfates/imp/tree/v0.16.0/examples/deployment) does the second. ### 2. JSON with a checksum diff --git a/docs/getting-started/running-in-your-application.md b/docs/getting-started/running-in-your-application.md index 9af41763..e2807cdc 100644 --- a/docs/getting-started/running-in-your-application.md +++ b/docs/getting-started/running-in-your-application.md @@ -121,7 +121,7 @@ Changing models is then a configuration change, and rotating a key is a restart. The -[deployment example](https://github.com/deepfates/imp/blob/v0.15.0/examples/deployment/README.md) +[deployment example](https://github.com/deepfates/imp/blob/v0.16.0/examples/deployment/README.md) is the complete version of this page: a two-stage program like `TicketTriage`, learned parameters loaded and reloaded as an `Imp.Optimizer.Artifact`, crashed and timed-out calls contained, and the whole thing restarted in a fresh OS diff --git a/docs/getting-started/setting-up.md b/docs/getting-started/setting-up.md index 5d9caad9..7f1edc70 100644 --- a/docs/getting-started/setting-up.md +++ b/docs/getting-started/setting-up.md @@ -5,7 +5,7 @@ Add Imp to a Mix project: ~~~elixir def deps do [ - {:imp, "~> 0.15.0"} + {:imp, "~> 0.16.0"} ] end ~~~ @@ -13,7 +13,7 @@ end or, for a script or a Livebook notebook: ~~~elixir -Mix.install([{:imp, "~> 0.15.0"}]) +Mix.install([{:imp, "~> 0.16.0"}]) ~~~ Imp needs Elixir 1.19 and a C and C++ compiler, for the native code in two diff --git a/docs/getting-started/where-to-go-next.md b/docs/getting-started/where-to-go-next.md index 17db5848..62fa74aa 100644 --- a/docs/getting-started/where-to-go-next.md +++ b/docs/getting-started/where-to-go-next.md @@ -51,7 +51,7 @@ The module documentation is the reference for every function. [Running Imp in production](../production.md) covers what the last page began: supervision, provider failures, runs you can observe and cancel, and telemetry. The -[deployment example](https://github.com/deepfates/imp/blob/v0.15.0/examples/deployment/README.md) +[deployment example](https://github.com/deepfates/imp/blob/v0.16.0/examples/deployment/README.md) is a complete application to copy from. ## Try it in a notebook diff --git a/docs/production.md b/docs/production.md index 08598b84..441d9f41 100644 --- a/docs/production.md +++ b/docs/production.md @@ -5,16 +5,16 @@ where its credentials come from, how long a call may take and what it may cost. This page covers those decisions in the order you meet them: starting Imp, credentials, concurrency, timeouts, cost and caching, persistence and telemetry. The -[deployment example](https://github.com/deepfates/imp/tree/v0.15.0/examples/deployment) +[deployment example](https://github.com/deepfates/imp/tree/v0.16.0/examples/deployment) is a complete OTP application that puts them together. ## Imp in your supervision tree -Imp is an OTP application, and adding `{:imp, "~> 0.15.0"}` to your +Imp is an OTP application, and adding `{:imp, "~> 0.16.0"}` to your dependencies starts it with yours. It supervises its settings, its response cache and the task pools that bound its work. It opens no listener and starts no subprocess, and the MCP and ACP runtimes start only when you use -them. In a script, `Mix.install([{:imp, "~> 0.15.0"}])` starts it too. +them. In a script, `Mix.install([{:imp, "~> 0.16.0"}])` starts it too. ## Credentials at runtime @@ -263,7 +263,7 @@ standard error. ## The reference application -[`examples/deployment`](https://github.com/deepfates/imp/tree/v0.15.0/examples/deployment) +[`examples/deployment`](https://github.com/deepfates/imp/tree/v0.16.0/examples/deployment) is a small OTP application that loads a checksummed artifact at startup, reads its key from the environment, serves calls from bounded supervised tasks, answers overload and timeouts with errors instead of blocking, and reloads diff --git a/examples/deployment/README.md b/examples/deployment/README.md index f6527c52..7787ce15 100644 --- a/examples/deployment/README.md +++ b/examples/deployment/README.md @@ -150,4 +150,4 @@ Use a whole-program artifact for a supported portable Imp shape. Use its selected predictor parameters should cross the persistence boundary. During source development set `IMP_PATH` to the Imp checkout. Other -applications use the Hex dependency in `mix.exs`, `{:imp, "~> 0.15.0"}`. +applications use the Hex dependency in `mix.exs`, `{:imp, "~> 0.16.0"}`. diff --git a/examples/deployment/mix.exs b/examples/deployment/mix.exs index aca65a98..7149c877 100644 --- a/examples/deployment/mix.exs +++ b/examples/deployment/mix.exs @@ -11,7 +11,7 @@ defmodule ImpDeployment.MixProject do defp deps do case System.get_env("IMP_PATH") do - nil -> [{:imp, "~> 0.15.0"}] + nil -> [{:imp, "~> 0.16.0"}] path -> [{:imp, path: path}] end end diff --git a/livebooks/01_real_lm_front_door.livemd b/livebooks/01_real_lm_front_door.livemd index 23f05644..ba9629a6 100644 --- a/livebooks/01_real_lm_front_door.livemd +++ b/livebooks/01_real_lm_front_door.livemd @@ -7,7 +7,7 @@ imp_path = System.get_env("IMP_PATH") lockfile = imp_path && Path.join(imp_path, "mix.lock") cond do - is_nil(imp_path) -> Mix.install([{:imp, "~> 0.15.0"}]) + is_nil(imp_path) -> Mix.install([{:imp, "~> 0.16.0"}]) File.regular?(lockfile) -> Mix.install([{:imp, path: imp_path}], lockfile: lockfile) true -> Mix.install([{:imp, path: imp_path}]) end diff --git a/livebooks/02_without_a_provider.livemd b/livebooks/02_without_a_provider.livemd index 98fe2ecb..47176b20 100644 --- a/livebooks/02_without_a_provider.livemd +++ b/livebooks/02_without_a_provider.livemd @@ -7,7 +7,7 @@ imp_path = System.get_env("IMP_PATH") lockfile = imp_path && Path.join(imp_path, "mix.lock") cond do - is_nil(imp_path) -> Mix.install([{:imp, "~> 0.15.0"}]) + is_nil(imp_path) -> Mix.install([{:imp, "~> 0.16.0"}]) File.regular?(lockfile) -> Mix.install([{:imp, path: imp_path}], lockfile: lockfile) true -> Mix.install([{:imp, path: imp_path}]) end diff --git a/livebooks/03_evaluate_and_optimize.livemd b/livebooks/03_evaluate_and_optimize.livemd index 003fc39a..81997f94 100644 --- a/livebooks/03_evaluate_and_optimize.livemd +++ b/livebooks/03_evaluate_and_optimize.livemd @@ -7,7 +7,7 @@ imp_path = System.get_env("IMP_PATH") lockfile = imp_path && Path.join(imp_path, "mix.lock") cond do - is_nil(imp_path) -> Mix.install([{:imp, "~> 0.15.0"}]) + is_nil(imp_path) -> Mix.install([{:imp, "~> 0.16.0"}]) File.regular?(lockfile) -> Mix.install([{:imp, path: imp_path}], lockfile: lockfile) true -> Mix.install([{:imp, path: imp_path}]) end diff --git a/livebooks/04_tools_agents_mcp_rlm.livemd b/livebooks/04_tools_agents_mcp_rlm.livemd index e507ee61..1844ec03 100644 --- a/livebooks/04_tools_agents_mcp_rlm.livemd +++ b/livebooks/04_tools_agents_mcp_rlm.livemd @@ -7,7 +7,7 @@ imp_path = System.get_env("IMP_PATH") lockfile = imp_path && Path.join(imp_path, "mix.lock") cond do - is_nil(imp_path) -> Mix.install([{:imp, "~> 0.15.0"}]) + is_nil(imp_path) -> Mix.install([{:imp, "~> 0.16.0"}]) File.regular?(lockfile) -> Mix.install([{:imp, path: imp_path}], lockfile: lockfile) true -> Mix.install([{:imp, path: imp_path}]) end diff --git a/livebooks/05_operating_imp.livemd b/livebooks/05_operating_imp.livemd index 10567eb4..58f189a5 100644 --- a/livebooks/05_operating_imp.livemd +++ b/livebooks/05_operating_imp.livemd @@ -7,7 +7,7 @@ imp_path = System.get_env("IMP_PATH") lockfile = imp_path && Path.join(imp_path, "mix.lock") cond do - is_nil(imp_path) -> Mix.install([{:imp, "~> 0.15.0"}]) + is_nil(imp_path) -> Mix.install([{:imp, "~> 0.16.0"}]) File.regular?(lockfile) -> Mix.install([{:imp, path: imp_path}], lockfile: lockfile) true -> Mix.install([{:imp, path: imp_path}]) end @@ -103,7 +103,7 @@ Enum.each(busy, &Task.await/1) refused ``` -The [deployment example](https://github.com/deepfates/imp/tree/v0.15.0/examples/deployment) +The [deployment example](https://github.com/deepfates/imp/tree/v0.16.0/examples/deployment) wraps the same pattern in a `ProgramServer` that also reloads parameters while serving. diff --git a/mix.exs b/mix.exs index 6a467746..a0f291cc 100644 --- a/mix.exs +++ b/mix.exs @@ -1,7 +1,7 @@ defmodule Imp.MixProject do use Mix.Project - @version "0.15.0" + @version "0.16.0" def project do [ diff --git a/priv/public_api.json b/priv/public_api.json index f3344007..65ccb546 100644 --- a/priv/public_api.json +++ b/priv/public_api.json @@ -10359,7 +10359,7 @@ "types": [] } ], - "package_version": "0.15.0", + "package_version": "0.16.0", "schema_version": 3, "scope": "Curated Imp API compiled from package-shipped sources. Exports come from non-hidden documentation entries; callbacks, types, and struct fields are category- and policy-gated." } diff --git a/test/package_contract_test.exs b/test/package_contract_test.exs index ada33961..d5833ffe 100644 --- a/test/package_contract_test.exs +++ b/test/package_contract_test.exs @@ -114,7 +114,7 @@ defmodule PackageContractTest do version = Mix.Project.config()[:version] dependency = hex_dependency(version) - assert version == "0.15.0" + assert version == "0.16.0" assert File.read!("RELEASE_NOTES.md") =~ "# Imp v#{version}" assert File.read!("CHANGELOG.md") =~ "## #{version}" assert File.read!("examples/deployment/mix.exs") =~ dependency