ENG-747: read status-error codes from the shape the SDK actually stores — revives the dead model-403 card#247
Conversation
…res (ENG-747)
The OpenAI SDK unwraps the error envelope before storing it
(openai/_client.py: data = body.get('error', body)), so the gateway's
403 body {'error': {'code': 'model_access_denied', ...}} reaches the
mapper as the INNER dict with 'code' at top level. The shipped ENG-598
mapper read only body['error']['code'] — a key the SDK had already
peeled off — so every model-403 fell through to the generic
'temporarily unavailable' path and the model-unavailable card never
fired in production. The 429/token-limit card only worked because the
gateway's 429 is FastAPI-style ({'detail': ...}, no envelope to unwrap).
- Read code and detail from the top level first with an envelope
fallback: correct for both gateway dialects today, and the 429 card
now survives a future gateway standardization onto the OpenAI
envelope (which would previously have killed it the same way).
- Replace the duck-typed _FakeStatusError fixtures with exceptions
constructed by the REAL pinned SDK from raw HTTP responses
(httpx.MockTransport). The old fixtures hand-built the wire envelope
shape the SDK never stores, so the suite validated the spec against
itself and shipped a dead card. A dedicated test documents the SDK
unwrap so any future SDK parsing change fails loudly.
- Root-cause chain, live prod evidence, and repro on the ticket.
Suite: 1170 passed, 17 skipped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…cstring (ENG-747)
Adversarial self-review findings, addressed:
- The envelope fallback was untestable by the MockTransport harness (the
pinned SDK ALWAYS unwraps, so _sdk_error physically cannot produce an
envelope-shaped exc.body) — it was pinned by nothing, and the
'unwrapped shape' test's comment described the opposite of what it fed.
New _wire_shaped_error helper constructs a real openai.APIStatusError
DIRECTLY with an explicit body, pinning envelope.get('code') and
envelope.get('detail'); mutation-checked (removing either fallback
clause fails exactly those tests).
- Docstring overclaimed '429 survives a future standardization': a real
move to OpenAI's quota dialect carries message, not detail, and stays
generic on purpose (classifying arbitrary provider messages would put
an upsell CTA on BYOK errors). Replaced with an explicit known-limits
paragraph, including the SDK discarding top-level siblings of 'error'
(the old precedence test's wire shape — unrecoverable from exc.body).
- 429 detail is now str-gated: a FastAPI validation-error LIST no longer
renders its Python repr into user-facing copy with an upgrade CTA.
Mapper suite: 17 passed. Full suite: 1172 passed, 17 skipped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adversarial self-review → hardening commit
|
|
Retargeted Post-merge note for whoever back-fills: 🤖 Generated with Claude Code |
pnewsam
left a comment
There was a problem hiding this comment.
Approving -- will merge once we have Devops unlock main for the hotfixes
Summary
Fixes ENG-747: the ENG-598 model-unavailable card has been dead in production since it shipped — observed live on a fully-updated free-tier machine (2026-07-13) and reproduced deterministically.
Root cause: the OpenAI SDK unwraps the error envelope before storing it (
openai/_client.py:416,data = body.get("error", body)), so the gateway's 403{"error": {"code": "model_access_denied", …}}reaches the mapper as the inner dict withcodeat top level. The mapper readbody["error"]["code"]— a key the SDK had already peeled — so every model-403 fell to the generic "temporarily unavailable" path: no typed error, no turn error, no card.Why 429 worked all along: the gateway's 429 is FastAPI-style (
{"detail": …}, no envelope) — the SDK's unwrap is a no-op for it. The working token-limit card was load-bearing on the gateway's dialect inconsistency.Fix
_raise_for_status_error: readcodeanddetailfrom the top level first, envelope fallback second — correct for both gateway dialects today, and the 429 card now survives a future gateway standardization onto the OpenAI envelope (previously a silent time bomb).APIStatusErroris now constructed by the pinned SDK (openai 2.21.0) from a raw HTTP response viahttpx.MockTransport— the duck-typed_FakeStatusError(which hand-built the wire shape the SDK never stores, letting the suite validate the spec against itself) is gone from the status-error path. A dedicatedtest_sdk_unwraps_error_envelopedocuments the SDK behavior so any future SDK parsing change fails loudly. Coverage: 403 envelope × both codes, 403 pre-unwrapped, 429 both dialects, 429-beats-403 precedence on the unwrapped shape, 401 JSON + 401 nginx-HTML, WAF HTML 403, code-less 403, 500.Verification
Security pass
Reviewed the changed surface: no credentials or user content in error messages beyond the model alias; the HTML-body cases confirm non-JSON error pages can't inject into curated copy (fall through to the generic string); no logging added.
Notes
v2.26.7.13.1; they were starved of input, not broken.uv.lockintentionally untouched (therich <15resync drift is pre-existing and already flagged on feat(prompts): move backend + HTML-dashboard contracts into built-in skills #244).🤖 Generated with Claude Code