Retain provider timing and reasoning usage for timeout diagnosis

[jmlago]

Provider timeouts currently collapse into a generic 504: operators cannot distinguish waiting for response headers from an incomplete response body, and successful OpenAI-compatible calls discard upstream generation IDs, provider names and reasoning-token usage. Preserve bounded HTTP transport timings and available upstream metadata per attempt in the response trace and the existing call ledger. Expose reported reasoning counts in usage.completion_tokens_details.

This supports investigation of intermittent DeepSeek timeouts. The same saved request has both timed out at 40.3 seconds and completed through the public router in 18.0 seconds buffered / 18.5 seconds streamed; these observations do not establish the cause. This PR adds evidence, not a latency fix. Provider request bodies, routing, retries, output limits and deadlines stay unchanged. No production deployment has been performed.

Validation: 60 distinct focused tests pass locally (one existing PostgreSQL-dependent skip). New tests use real TCP responses to distinguish stalled headers from a trickling body; verify HTTP success/error and ledger propagation, concurrent-request isolation, bounded metadata, absent/zero reasoning usage, and exclusion of headers/prompts/reasoning text. Rebuilding the saved failed request before and after the change produces the identical outbound URL/body hash. Full CI passes on head b863b34: Python suite, policy core assertions, router/AntSeed image build and boot.

See docs/provider-diagnostics.md for field semantics and limitations, including missing metadata on incomplete buffered JSON and outer cancellation before an adapter result is returned.

Summary by CodeRabbit

• New Features
  • Added bounded provider timing diagnostics for asynchronous OpenAI-compatible requests, including successful, failed, buffered, and streaming calls.
  • Added visibility into request phases, status, timeouts, provider identifiers, model families, and token usage.
  • Preserved reasoning-token usage metadata while excluding private reasoning content and sensitive request details.
  • Documented diagnostic fields, limits, and streaming behavior.

[coderabbitai]

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: ababb50c-b0f6-484c-bec4-34cef52d7dc0

📥 Commits

Reviewing files that changed from the base of the PR and between 8bae535 and b863b34.

📒 Files selected for processing (7)

• docs/provider-diagnostics.md
• host_store.py
• llm_router_host.py
• provider_adapters/diagnostics.py
• provider_adapters/openai_compatible.py
• shim.py
• tests/test_provider_diagnostics.py

 __________________________________________________
< Veni, Vidi, Validavi. I came, I saw, I reviewed. >
 --------------------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}