Repository navigation
Preserve billed usage when OpenAI response parsing fails - #3
Open
isaacbmiller wants to merge 2 commits into
Open
isaacbmiller wants to merge 2 commits into
isaacbmiller wants to merge 2 commits into
Conversation
Keep the original ProviderError and attach an accounting-only partial Response using each dialect’s existing usage reader. Decode the body once; leave successful output parsing and HTTP diagnostics unchanged. Signed-off-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03 Co-authored-by: Isaac Miller <isaac@cmpnd.ai>
Signed-off-by: Amp <amp@ampcode.com> Co-authored-by: Isaac Miller <isaac@cmpnd.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03
Author
|
@greptileai review |
isaacbmiller
pushed a commit
to stanfordnlp/dspy
that referenced
this pull request
Oct 23, 2026
Use bundled lm15 for built-in LM execution without compatibility fallback. Move hosted embeddings to the OpenAI SDK, replace the proxy test fixture, and update installation, provider, cache, deployment and API documentation. Import the source-owned billed-usage fix from cmpnd-ai/lm15-python#3 so malformed native replies retain their accounting without retries, cache or history writes. Co-authored-by: Isaac Miller <isaac@cmpnd.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03
isaacbmiller
pushed a commit
to stanfordnlp/dspy
that referenced
this pull request
Oct 23, 2026
Use bundled lm15 for built-in LM execution without compatibility fallback. Move hosted embeddings to the OpenAI SDK, replace the proxy test fixture, and update installation, provider, cache, deployment and API documentation. Import the source-owned billed-usage fix from cmpnd-ai/lm15-python#3 so malformed native replies retain their accounting without retries, cache or history writes. Co-authored-by: Isaac Miller <isaac@cmpnd.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03
isaacbmiller
pushed a commit
to stanfordnlp/dspy
that referenced
this pull request
Oct 26, 2026
Use bundled lm15 for built-in LM execution without compatibility fallback. Move hosted embeddings to the OpenAI SDK, replace the proxy test fixture, and update installation, provider, cache, deployment and API documentation. Import the source-owned billed-usage fix from cmpnd-ai/lm15-python#3 so malformed native replies retain their accounting without retries, cache or history writes. Co-authored-by: Isaac Miller <isaac@cmpnd.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03
isaacbmiller
pushed a commit
to stanfordnlp/dspy
that referenced
this pull request
Oct 26, 2026
Use bundled lm15 for built-in LM execution without compatibility fallback. Move hosted embeddings to the OpenAI SDK, replace the proxy test fixture, and update installation, provider, cache, deployment and API documentation. Import the source-owned billed-usage fix from cmpnd-ai/lm15-python#3 so malformed native replies retain their accounting without retries, cache or history writes. Co-authored-by: Isaac Miller <isaac@cmpnd.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03
isaacbmiller
pushed a commit
to stanfordnlp/dspy
that referenced
this pull request
Oct 26, 2026
Use bundled lm15 for built-in LM execution without compatibility fallback. Move hosted embeddings to the OpenAI SDK, replace the proxy test fixture, and update installation, provider, cache, deployment and API documentation. Import the source-owned billed-usage fix from cmpnd-ai/lm15-python#3 so malformed native replies retain their accounting without retries, cache or history writes. Co-authored-by: Isaac Miller <isaac@cmpnd.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03
isaacbmiller
pushed a commit
to stanfordnlp/dspy
that referenced
this pull request
Oct 26, 2026
Use bundled lm15 for built-in LM execution without compatibility fallback. Move hosted embeddings to the OpenAI SDK, replace the proxy test fixture, and update installation, provider, cache, deployment and API documentation. Import the source-owned billed-usage fix from cmpnd-ai/lm15-python#3 so malformed native replies retain their accounting without retries, cache or history writes. Co-authored-by: Isaac Miller <isaac@cmpnd.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03
isaacbmiller
pushed a commit
to stanfordnlp/dspy
that referenced
this pull request
Oct 27, 2026
Use bundled lm15 for built-in LM execution without compatibility fallback. Move hosted embeddings to the OpenAI SDK, replace the proxy test fixture, and update installation, provider, cache, deployment and API documentation. Import the source-owned billed-usage fix from cmpnd-ai/lm15-python#3 so malformed native replies retain their accounting without retries, cache or history writes. Co-authored-by: Isaac Miller <isaac@cmpnd.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03
isaacbmiller
pushed a commit
to stanfordnlp/dspy
that referenced
this pull request
Oct 28, 2026
Use bundled lm15 for built-in LM execution without compatibility fallback. Move hosted embeddings to the OpenAI SDK, replace the proxy test fixture, and update installation, provider, cache, deployment and API documentation. Import the source-owned billed-usage fix from cmpnd-ai/lm15-python#3 so malformed native replies retain their accounting without retries, cache or history writes. Co-authored-by: Isaac Miller <isaac@cmpnd.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03
isaacbmiller
pushed a commit
to stanfordnlp/dspy
that referenced
this pull request
Oct 28, 2026
Use bundled lm15 for built-in LM execution without compatibility fallback. Move hosted embeddings to the OpenAI SDK, replace the proxy test fixture, and update installation, provider, cache, deployment and API documentation. Import the source-owned billed-usage fix from cmpnd-ai/lm15-python#3 so malformed native replies retain their accounting without retries, cache or history writes. Co-authored-by: Isaac Miller <isaac@cmpnd.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03
isaacbmiller
pushed a commit
to stanfordnlp/dspy
that referenced
this pull request
Oct 28, 2026
Use bundled lm15 for built-in LM execution without compatibility fallback. Move hosted embeddings to the OpenAI SDK, replace the proxy test fixture, and update installation, provider, cache, deployment and API documentation. Import the source-owned billed-usage fix from cmpnd-ai/lm15-python#3 so malformed native replies retain their accounting without retries, cache or history writes. Co-authored-by: Isaac Miller <isaac@cmpnd.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03
isaacbmiller
added a commit
to stanfordnlp/dspy
that referenced
this pull request
Oct 28, 2026
* Remove LiteLLM and document the native-only migration Use bundled lm15 for built-in LM execution without compatibility fallback. Move hosted embeddings to the OpenAI SDK, replace the proxy test fixture, and update installation, provider, cache, deployment and API documentation. Import the source-owned billed-usage fix from cmpnd-ai/lm15-python#3 so malformed native replies retain their accounting without retries, cache or history writes. Co-authored-by: Isaac Miller <isaac@cmpnd.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03 * Preserve OpenRouter cross-model fallback in native requests Accept the provider-owned models/route policy only on OpenRouter and document the flat generation-option migration. Cover public sync/async Predict wire behavior and refusal on unrelated routes. Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03 Co-authored-by: Isaac Miller <isaac@cmpnd.ai> * Keep LiteLLM available as an optional installation extra Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03 Co-authored-by: Isaac Miller <isaac@cmpnd.ai> * Retain optional LiteLLM compatibility and restore docs Pipenv files Keep native-first auto selection, strict lm15, and explicit litellm with the optional extra. Preserve pre-I/O fallback, native endpoint improvements, and lazy SDK imports. Exercise installed-wheel compatibility separately from dependency-free core CI. Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03 Co-authored-by: Isaac Miller <isaac@cmpnd.ai> * Stop testing optional LiteLLM compatibility in CI Amp-Thread-ID: https://ampcode.com/threads/T-01a135b1-7d59-73cc-a17c-57b7b3454c03 Co-authored-by: Isaac Miller <isaac@cmpnd.ai> --------- Co-authored-by: Amp <amp@ampcode.com> Co-authored-by: Isaac Miller <isaac@cmpnd.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
A completed OpenAI Chat/Responses reply can contain valid billed usage and a malformed tool call (for example, no function name). lm15 raises the correct
ProviderError, but discards the usage. DSPy's native-only migration then loses those billed tokens.Fix
Preserve the original error and attach an accounting-only
error.partialresponse with empty content andfinish_reason="error". Use each dialect's existing usage reader on the already-decoded body; do not reparse malformed output or treat it as a successful response. Invalid telemetry cannot mask the original error. Sync and async callers share these readers.This covers explicit OpenAI Chat/Responses parser errors and compatible providers sharing those parsers. HTTP-status failures, streaming and other dialects are unchanged.
CONTRACT_PINis unchanged.This PR targets
integration/lm15-1.5.0-patches, DSPy's current source pin, to keep the diff limited to this fix. It does not propose merging the existing downstream patch stack into forkmain.Validation