Skip to content

ENG-747: read status-error codes from the shape the SDK actually stores — revives the dead model-403 card#247

Merged
alecantu7 merged 2 commits into
mainfrom
alejandrocantu/eng-747-sdk-unwraps-error-envelope
Jul 13, 2026
Merged

ENG-747: read status-error codes from the shape the SDK actually stores — revives the dead model-403 card#247
alecantu7 merged 2 commits into
mainfrom
alejandrocantu/eng-747-sdk-unwraps-error-envelope

Conversation

@alecantu7

Copy link
Copy Markdown
Contributor

Summary

Fixes ENG-747: the ENG-598 model-unavailable card has been dead in production since it shipped — observed live on a fully-updated free-tier machine (2026-07-13) and reproduced deterministically.

Root cause: the OpenAI SDK unwraps the error envelope before storing it (openai/_client.py:416, data = body.get("error", body)), so the gateway's 403 {"error": {"code": "model_access_denied", …}} reaches the mapper as the inner dict with code at top level. The mapper read body["error"]["code"] — a key the SDK had already peeled — so every model-403 fell to the generic "temporarily unavailable" path: no typed error, no turn error, no card.

Why 429 worked all along: the gateway's 429 is FastAPI-style ({"detail": …}, no envelope) — the SDK's unwrap is a no-op for it. The working token-limit card was load-bearing on the gateway's dialect inconsistency.

Fix

  • _raise_for_status_error: read code and detail from the top level first, envelope fallback second — correct for both gateway dialects today, and the 429 card now survives a future gateway standardization onto the OpenAI envelope (previously a silent time bomb).
  • Tests rebuilt on the real SDK: every APIStatusError is now constructed by the pinned SDK (openai 2.21.0) from a raw HTTP response via httpx.MockTransport — the duck-typed _FakeStatusError (which hand-built the wire shape the SDK never stores, letting the suite validate the spec against itself) is gone from the status-error path. A dedicated test_sdk_unwraps_error_envelope documents the SDK behavior so any future SDK parsing change fails loudly. Coverage: 403 envelope × both codes, 403 pre-unwrapped, 429 both dialects, 429-beats-403 precedence on the unwrapped shape, 401 JSON + 401 nginx-HTML, WAF HTML 403, code-less 403, 500.

Verification

  • Mapper suite: 15 passed. Full suite: 1170 passed, 17 skipped.
  • Repro chain (SDK source line, MockTransport assert-proof with the live-captured prod gateway body, field observation) documented on the ticket.
  • End-to-end confirmation available: the live free-tier repro machine — fresh message on a locked model should show the ModelUnavailableCard after this ships.

Security pass

Reviewed the changed surface: no credentials or user content in error messages beyond the model alias; the HTML-body cases confirm non-JSON error pages can't inject into curated copy (fall through to the generic string); no logging added.

Notes

🤖 Generated with Claude Code

alecantu7 and others added 2 commits July 13, 2026 13:42
…res (ENG-747)

The OpenAI SDK unwraps the error envelope before storing it
(openai/_client.py: data = body.get('error', body)), so the gateway's
403 body {'error': {'code': 'model_access_denied', ...}} reaches the
mapper as the INNER dict with 'code' at top level. The shipped ENG-598
mapper read only body['error']['code'] — a key the SDK had already
peeled off — so every model-403 fell through to the generic
'temporarily unavailable' path and the model-unavailable card never
fired in production. The 429/token-limit card only worked because the
gateway's 429 is FastAPI-style ({'detail': ...}, no envelope to unwrap).

- Read code and detail from the top level first with an envelope
  fallback: correct for both gateway dialects today, and the 429 card
  now survives a future gateway standardization onto the OpenAI
  envelope (which would previously have killed it the same way).
- Replace the duck-typed _FakeStatusError fixtures with exceptions
  constructed by the REAL pinned SDK from raw HTTP responses
  (httpx.MockTransport). The old fixtures hand-built the wire envelope
  shape the SDK never stores, so the suite validated the spec against
  itself and shipped a dead card. A dedicated test documents the SDK
  unwrap so any future SDK parsing change fails loudly.
- Root-cause chain, live prod evidence, and repro on the ticket.

Suite: 1170 passed, 17 skipped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…cstring (ENG-747)

Adversarial self-review findings, addressed:

- The envelope fallback was untestable by the MockTransport harness (the
  pinned SDK ALWAYS unwraps, so _sdk_error physically cannot produce an
  envelope-shaped exc.body) — it was pinned by nothing, and the
  'unwrapped shape' test's comment described the opposite of what it fed.
  New _wire_shaped_error helper constructs a real openai.APIStatusError
  DIRECTLY with an explicit body, pinning envelope.get('code') and
  envelope.get('detail'); mutation-checked (removing either fallback
  clause fails exactly those tests).
- Docstring overclaimed '429 survives a future standardization': a real
  move to OpenAI's quota dialect carries message, not detail, and stays
  generic on purpose (classifying arbitrary provider messages would put
  an upsell CTA on BYOK errors). Replaced with an explicit known-limits
  paragraph, including the SDK discarding top-level siblings of 'error'
  (the old precedence test's wire shape — unrecoverable from exc.body).
- 429 detail is now str-gated: a FastAPI validation-error LIST no longer
  renders its Python repr into user-facing copy with an upgrade CTA.

Mapper suite: 17 passed. Full suite: 1172 passed, 17 skipped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@alecantu7

Copy link
Copy Markdown
Contributor Author

Adversarial self-review → hardening commit bc5b419

Ran the 8-angle adversarial review before requesting human review. The fix itself held (no caller breaks, no real-provider behavior changes — BYOK OpenAI/Anthropic/Azure error shapes verified unchanged through the real SDK; cowork-server's detection is isinstance-based, no strings touched). Three findings on my own work, all addressed:

  1. The envelope fallback was pinned by nothing — and my own harness couldn't pin it. The pinned SDK always unwraps, so _sdk_error physically cannot produce an envelope-shaped exc.body; deleting the fallback clauses left the suite green. And test_unwrapped_403_shape_also_maps's comment described the opposite of what it fed. Fixed with _wire_shaped_error — a real openai.APIStatusError constructed directly with an explicit body — pinning both envelope.get("code") and envelope.get("detail"); mutation-checked (removing either clause fails exactly those tests).
  2. Docstring overclaim: "survives a future 429 standardization" was false comfort — OpenAI's real quota dialect carries message, not detail, and deliberately stays generic (an upsell CTA on BYOK errors would be wrong). Replaced with an explicit known-limits paragraph, including that the SDK discards top-level siblings of error (the old precedence test's wire shape is unrecoverable from exc.body — that test had pinned a shape production never delivers).
  3. 429 detail is now str-gated — a FastAPI validation-error LIST no longer renders its Python repr into user-facing copy with an upgrade CTA attached (+ regression test).

Mapper suite: 17 passed. Full suite: 1172 passed, 17 skipped. uv.lock still untouched.

🤖 Generated with Claude Code

@alecantu7
alecantu7 changed the base branch from staging to main July 13, 2026 21:03
@alecantu7

Copy link
Copy Markdown
Contributor Author

Retargeted stagingmain as a hotfix (Ale's call): the dead model-403 card affects every free-tier user in the current prod release (v2.26.7.13.1), and the fix is two files with no schema/config coupling — the definition of hotfix-shaped. staging is currently an ancestor of main, so the diff is unchanged by the retarget.

Post-merge note for whoever back-fills: staging will need these two commits too (same branch merges cleanly into staging, or a mainstaging back-merge) so the next weekly promotion doesn't ship without them.

🤖 Generated with Claude Code

@pnewsam pnewsam left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving -- will merge once we have Devops unlock main for the hotfixes

@alecantu7
alecantu7 added this pull request to the merge queue Jul 13, 2026
Merged via the queue into main with commit 385d3b9 Jul 13, 2026
5 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 13, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants