feat(app-api): gzip JSON responses, with SSE excluded and un-buffered

[philmerrell]

Nothing compressed app-api responses before this. CloudFront can't do it for us: the /api/* behaviour is deliberately compress: false so the edge never buffers a text/event-stream, which leaves compression to the origin — the only layer that knows a response's content type rather than guessing it from a path pattern.

The whole risk is the SSE surface, so the new StreamSafeGZipMiddleware is defined by what it refuses to touch.

The trap is not where you'd expect

The pinned starlette==1.3.1 already excludes SSE:

DEFAULT_EXCLUDED_CONTENT_TYPES = ("text/event-stream",)

Stock GZipMiddleware skips text/event-stream by content type and skips any body that already carries a Content-Encoding. A bare add_middleware(GZipMiddleware) would not have gzipped the chat stream.

It would still have broken something. Both Starlette responders withhold http.response.start until the first http.response.body arrives — they can't know whether they'll need to set Content-Encoding until they see a body. On a chat turn that chunk is the model's first token, so response headers end up behind the agent's entire thinking time. Measured against the real middleware stack on a turn that stalls 2s before its first event:

	headers	first chunk
stock GZipMiddleware	2011 ms	2012 ms
StreamSafeGZipMiddleware	5 ms	2008 ms

Identical without Accept-Encoding: gzip — the identity responder buffers too, so a non-gzip client got the delay as well.

What's here

A ~30-line subclass, not route scoping. Route scoping can't work: proxy_routes.py:241 picks SSE-vs-JSON from the upstream's content type at runtime, so the same route is both. Content type is the only honest signal.

It reuses Starlette's exclusion machinery (a test pins the constant, so a version bump can't silently un-exclude SSE) and adds the one thing Starlette doesn't — when the response-start message shows the response is excluded, it forwards those headers immediately and passes the rest through untouched. Also excludes already-compressed payload types (zip, raster images, media, woff): CPU for ~0% gain.

compresslevel=6 rather than Starlette's 9 — level 9 buys 3% more for 4× the CPU on a 1 MB body (89 ms → 21 ms). Measured ratios at 6: ~3.0× on a conversation-history response, ~3.9× on a 5,000-row grid, 4.9× on a 748 B catalog.

Ordering is now ProxiedRedirect → GZip → SessionRefresh → CSRF → AgentCoreContext → CORS → router. ProxiedRedirect stays outermost as its docstring requires; the two can't collide since it only rewrites Location on 3xx. Comment extended, order pinned by a test.

inference-api was already broken

It already had GZipMiddleware, commented "for SSE streams… reducing bandwidth by 50-70%". Both halves are wrong: it compresses nothing on /invocations, and it was adding the full header delay above. That delay wasn't contained there — app-api's proxy reads the upstream content-type before it can open its own stream to the SPA, so it propagated to the browser. Swapped to the shared middleware with its settings unchanged; re-measured at 5 ms.

Verification

• Accept-Encoding reaches the origin. Read the live managed policies: Managed-CachingDisabled has EnableAcceptEncodingGzip/Brotli both false, and Managed-AllViewerExceptHostHeader is allExcept: [host]. AWS docs for exactly this case: "When caching compressed objects is disabled for both compression formats, CloudFront treats the Accept-Encoding header the same as any other HTTP header… You can include it in the headers list in a cache policy or an origin request policy." No normalization path applies.
• CloudFront passes through the origin's Content-Encoding. "When a response from an origin includes the Content-Encoding header, CloudFront doesn't compress the object… CloudFront sends the response to the viewer." With CACHING_DISABLED it's a pure proxy anyway.
• SSE streams incrementally. Against the real apis.app_api.main:app on uvicorn: headers at 5 ms, 8 frames 250 ms apart, 148 B each, un-coalesced, no Content-Encoding.
• JSON compresses through the whole stack: 120,900 B → 5,302 B on the wire, byte-identical after decode.
• Tests: 8,605 passed, 3 skipped (16 new).

Not verified

No real Bedrock chat turn through POST /invocations. The dev-ai SSO token is expired and the session couldn't re-auth, so a dev-configured app-api wouldn't boot. The probe reproduces the proxy's response shape, not its upstream. Worth one manual turn on dev after merge.

Note on the brief

GET /files/{id}/sheet-preview doesn't exist on this branch — no sheet route anywhere in app_api, nothing in the SPA referencing it. Benchmarks above are payload shapes, not that endpoint. Separately, the test baseline here is 8,592 collected, not the 8,663 quoted; the 71-test gap is pre-existing and unrelated to this change.

🤖 Generated with Claude Code