Move OpenRouter BYOK billing out of Imp.Core

[mvanhorn]

What changed

Imp.Clients.ReqLLM.resolve_openrouter_charge/2 now resolves a charge only for an OpenRouter model whose usage has "is_byok": the fee plus upstream_inference_cost when both are numbers, otherwise nil. Response metadata and stream normalization store that figure as :charge, and a cache hit drops it so a repeat does not bill the stored charge. Core treats a present :charge as final, nil included, and otherwise still reads usage "cost" or metadata :cost; usage sanitization leaves Decimal values intact so a non-numeric figure stays unknown instead of raising. The BYOK cases now live with the ReqLLM client and stream tests, and a raw envelope that carries no resolved charge reports the fee alone.

Callers of the provider-neutral response path were billed with OpenRouter's bring-your-own-key rule: a usage map marked is_byok turned the reported cost into OpenRouter's fee plus the upstream inference charge, or left the charge unknown when either figure was missing. That one provider's accounting showed up on every core response, including raw envelopes that had never been normalized by an OpenRouter client. Imp.Core.reported_cost/2 matched usage with "is_byok" and added "cost" to cost_details.upstream_inference_cost. The OpenRouter field names and the fee-plus-upstream rule sat in the provider-neutral cost reader instead of with the client that sees OpenRouter usage.

Fixes #265

How it was verified

Run mix check — it is the ordinary merge signal and it needs no credentials.
Paste the summary line. Run any further gate your change touches
(mix protocol.check, mix package.check, mix quality.check,
mix dialyzer.check); CONTRIBUTING.md lists them all.

Say what you could not run and why. Provider-backed, research-scale and
benchmark checks need credentials, datasets or spend: they are the
maintainer's to run, and an outside contributor is not expected to.

Documentation

Update ExDoc, guides or Livebooks alongside a public API change.

[deepfates]

Thanks for taking this on. The rule now sits with the ReqLLM client and core has no provider names, which is what #265 asked for. A few things before this can merge:

1. Drop the model check in resolve_openrouter_charge/2. openrouter_model?/1 doesn't match model specs the client otherwise accepts, such as {:openrouter, "id"}, {:openrouter, "id", opts} and %{"provider" => "openrouter"} (provider_label/1 handles all three). For those specs, a BYOK call now reports only OpenRouter's fee, where before it reported fee plus upstream cost. "is_byok" in the usage map is enough to identify the case. Please also remove the "not OpenRouter reports the fee" test.
2. Keep existing behaviour for envelopes that carry no resolved charge. Please don't rewrite the core test to expect the fee alone. Reporting only the fee understates spend, and the issue asks for BYOK behaviour to stay the same. One neutral approach: the client always sets :charge (the reported "cost", or the BYOK sum), and core reads :charge and then metadata :cost. A missing charge then reads as unknown (nil), never as a lower figure.
3. Add one test that fails on main and passes with your change. For example: a BYOK response on a {:openrouter, "id"} model spec, costed as fee plus upstream through the client.
4. The Decimal pass-through in sanitize_usage_value/1 is outside this issue's scope, and ReqLLM doesn't put Decimals in usage. Please drop it and its test cases, or move it to its own PR with a reason.
5. Paste the mix check summary line into the description, as the PR template asks.

[mvanhorn]

@deepfates Thanks, all four code points are in 5df1cfc. The model check is gone, so "is_byok" alone picks the BYOK path, and the "not OpenRouter" test is removed. The client now always records :charge (the BYOK sum or the reported cost), and a raw BYOK envelope with no resolved charge reads as nil rather than the fee. The new test costs a BYOK response on {:openrouter, "id"}, {:openrouter, "id", []} and %{"provider" => "openrouter"} as fee plus upstream, and it fails on main. The Decimal pass-through and its cases are dropped.

On point 5: I don't have a clean mix check line yet. On my machine (Elixir 1.20.2) it stops on compiler warnings in files this PR doesn't touch, so I'll add the summary to the description once I've run it on the project's toolchain. The two affected test files pass (116 tests).

[deepfates]

Thanks for the quick turnaround, and sorry for the wait. I checked 5df1cfc against each point and merged it into current main locally. It merges cleanly, and mix check passes there (Elixir 1.19.5 / OTP 28: 59 doctests, 9 properties, 3652 tests, 0 failures, 13 skipped (221 excluded)).

Points 1, 3 and 4 are done. The :charge handling for streams and cache hits looks right, and the stream-failure test is a nice addition.

Remaining before merge:

1. "is_byok" is still in core (lib/imp/core.ex:303, unresolved_cost(_metadata, %{"is_byok" => true})). OpenRouter's BYOK billing rule lives in Imp.Core #265 is done when core names no provider fields. The client now always sets :charge, so core can read :charge and then metadata :cost, and stop reading usage "cost" itself. The raw-envelope tests in test/model_response_cost_test.exs that rely on usage "cost" would then either set :charge in their envelope or move beside the client tests.

2. Please update the PR description. It still says three things that aren't true at 5df1cfc:

   • the charge is resolved only for an OpenRouter model;
   • usage sanitization keeps Decimals;
   • a raw envelope with no charge reports the fee alone.

   Please also fill in "How it was verified". If the 1.20 warnings still block you locally, saying so is fine; the run above can stand in.

3. Optional: resolve_openrouter_charge/2 now applies to every provider, ignores its first argument and always returns {:resolved, _}. A charge(usage) returning the value would read more plainly, both in the client and in Imp.Streaming.Execution.

Thanks again. This is close.

[mvanhorn]

Point 1 is in 4b5d920. Core no longer reads usage "cost" or "is_byok". It reads the client's :charge and then metadata :cost, and an envelope with no resolved charge reads as nil. The raw-envelope tests in test/model_response_cost_test.exs now set :charge in their envelope instead of usage "cost".

I took the optional point 3 as well: resolve_openrouter_charge/2 is now Imp.Clients.ReqLLM.charge/1, which returns the figure directly, and Imp.Streaming.Execution calls it too.

mix check exits 0 at 4b5d920 on Elixir 1.19.5 / OTP 28, after mix format --check-formatted and mix compile --warnings-as-errors.