feat(switchyard-llm-client): add HTTP LLM client crate

[messiaen]

Summary

Adds switchyard-llm-client — an HTTP LLM client that speaks Switchyard's neutral IR
end-to-end, built on the protocol traits and translation helpers from the base branches. Main class is TranslatingLlmClient, which implements RoutedLlmClient (libsy).

1. Looks up the model by API name → Backend (OpenAiChat / OpenAiResponses / Anthropic).
2. Encodes the IR request into that backend's wire format via switchyard-translation.
3. Applies auth (Bearer for OpenAI, x-api-key + anthropic-version for Anthropic), forwards caller headers (minus reserved ones), rewrites model upstream.
4. Calls with a shared reqwest::Client.
5. Decodes the reply back into a switchyard_protocol::Response — buffered (LlmResponse::Agg) or streamed (LlmResponse::Stream) based on request.llm_request.stream.

Changes

• New crate crates/libsy-llm-client/ (package switchyard-llm-client):
  • TranslatingLlmClient — maps a model API name (e.g. "openai/gpt-oss-20b", not "strong") + wire format to a per-model Backend (built from a list of ModelConfigs), encodes the request, makes the HTTP call (auth + caller-header forwarding), and decodes the response — buffered JSON or SSE — back to the IR.
  • Implements RoutedLlmClient, so it can back a libsy LlmTarget directly.
  • call_rewrite_model_raw(ctx, raw_http_request, http_headers, model, wire_format) — the
    whole decode → call → encode path in one call, returning a buffered JSON body or an
    unframed stream of wire events (RawResponse); SSE framing stays a transport concern, so
    the crate has no HTTP-server dependency.

Example usage

(thanks Claude!)

  use std::collections::BTreeMap;
  use switchyard_llm_client::{Backend, HttpBackendConfig, ModelConfig, TranslatingLlmClient};                                                                                                                                                                 
  use switchyard_protocol::{completion_text, text_request, Context, LlmResponse, Request};
                                                                                                                                                                                                                                                              
  let openai = HttpBackendConfig {                                
      base_url: "https://api.openai.com/v1".to_string(),                                                                                                                                                                                                      
      api_key: std::env::var("OPENAI_API_KEY").ok(),                                                                                                                                                                                                          
      extra_headers: BTreeMap::new(),
  };                                                                                                                                                                                                                                                          
                                                                  
  let client = TranslatingLlmClient::new(&[ModelConfig::new(                                                                                                                                                                                                  
      "gpt-4o-mini",                                              
      Backend::OpenAiChat(openai),
      None,                                                                                                                                                                                                                                                   
  )])?;
                                                                                                                                                                                                                                                              
  let request = Request {                                         
      llm_request: text_request(None, "Say hello in five words."),
      raw_request: None,
      metadata: None,
  };                                                                                                                                                                                                                                                          
  
  let response = client                                                                                                                                                                                                                                       
      .call_rewrite_model(Context::default(), request, Some("gpt-4o-mini"))
      .await?;

  if let LlmResponse::Agg(agg) = response.llm_response {                                                                                                                                                                                                      
      println!("{}", completion_text(&agg));
  }     

Summary by CodeRabbit

• New Features
  • Added an HTTP client for connecting to LLM backends using OpenAI Chat, OpenAI Responses, and Anthropic formats.
  • Supports buffered and streaming responses, request translation, model routing, authentication, header forwarding, and model-name rewriting.
  • Added structured error handling, including context-window overflow detection.
  • Added configuration and documentation for integrating supported backends.

[nv-msvoboda]

@sir-merge-a-lot groom

[nv-sir-merge-a-lot]

Previous updates (collapsed)

🤖 sir-merge-a-lot

Resolving conflicts

🔄 sir-merge-a-lot is resolving conflicts on this MR.

Detected 3 conflicting file(s) (REGENERATE). I'll update this comment with the result. This can take a couple of minutes.

---

🤖 sir-merge-a-lot

Coding agent working

🤖 sir-merge-a-lot handed a large conflict to its sandboxed coding agent.

The LLM tier could not resolve crates/switchyard-translation/src/helpers.rs (too large or timed out), so I escalated this single file to the coding agent. I'll update this comment with the result. This can take several minutes.

🤖 sir-merge-a-lot

Coding agent could not resolve the conflict

⚠️ The coding agent could not resolve crates/switchyard-translation/src/helpers.rs (reason: multi_file_conflict). This file needs manual conflict resolution. Re-invoke @sir-merge-a-lot-bot groom to try again, or resolve it by hand.

Resolution ladder trace

Rung	Result	Details
Server-side rebase	skipped	Skipped because conflicts were detected during local analysis; GitLab rebase would not produce a clean branch. Files: Cargo.lock, crates/switchyard-translation/src/helpers.rs, crates/switchyard-translation/src/sse.rs
Client-side rebase	skipped	Skipped because conflicts were detected during local analysis; using the conflict-resolution ladder instead of force-pushing a conflicted rebase. Files: Cargo.lock, crates/switchyard-translation/src/helpers.rs, crates/switchyard-translation/src/sse.rs
Rerere	miss	No cached rerere resolution matched the remaining conflicts. Files: Cargo.lock, crates/switchyard-translation/src/helpers.rs, crates/switchyard-translation/src/sse.rs
Mergiraf	declined	Mergiraf ran but left conflicts in the remaining files. Files: Cargo.lock, crates/switchyard-translation/src/helpers.rs, crates/switchyard-translation/src/sse.rs
Auto-regenerable fallback	resolved	Took whole-file source/MR side for known-regenerable paths (lock files / vendored deps / generated schemas) so the LLM does not burn tokens on files that will be regenerated by the next install or schema-gen step. Files: Cargo.lock
LLM	invoked	Escalating remaining files to LLM for bounded LLM resolution. Files: crates/switchyard-translation/src/helpers.rs, crates/switchyard-translation/src/sse.rs
LLM	resolved	LLM produced accepted bounded-edit resolutions. Files: crates/switchyard-translation/src/sse.rs
LLM	failed	LLM could not produce an accepted resolution for the remaining files. Failure reason: llm_timeout. Files: crates/switchyard-translation/src/helpers.rs
Coding agent	failed	multi_file_conflict Files: crates/switchyard-translation/src/helpers.rs

[coderabbitai]

Walkthrough

Adds the switchyard-llm-client workspace crate, defining provider backends, model and wire-format routing, HTTP translation, buffered and streaming responses, error handling, routed-client integration, tests, and usage documentation.

Changes

LLM Client Crate

Layer / File(s)	Summary
Crate API and configuration Cargo.toml, crates/libsy-llm-client/Cargo.toml, crates/libsy-llm-client/src/lib.rs, crates/libsy-llm-client/src/raw.rs, crates/libsy-llm-client/src/client.rs	Registers the crate, defines dependencies and public exports, adds RawResponse, and introduces model configuration.
Backend and error handling crates/libsy-llm-client/src/backend.rs, crates/libsy-llm-client/src/error.rs	Adds OpenAI and Anthropic backend configuration, URL and authentication handling, overflow detection, shared errors, and unit tests.
Translation client and usage crates/libsy-llm-client/src/client.rs, crates/libsy-llm-client/README.md	Implements model and format resolution, HTTP translation, streaming and raw response paths, routed-client integration, comprehensive tests, and usage documentation.

Estimated code review effort: 4 (Complex) | ~60 minutes

Poem

A rabbit hops through JSON snow,
With backend maps arranged in rows.
Streams sparkle, headers fly,
Errors bloom beneath the sky.
“Translate on!” the bunny cries—
New client magic starts to rise.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: adding the HTTP LLM client crate.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/libsy-llm-client/README.md`:
- Around line 8-15: Rewrite all README examples to use the crate’s actual API:
import ModelConfig and TranslatingLlmClient from switchyard_llm_client,
construct the client with TranslatingLlmClient::new, and invoke
call_rewrite_model instead of the unsupported LlmModelClient, libsy_llm_client,
nested backend maps, call, formats_for, or aggregate interfaces. Remove the
unused LlmModelClient link definition while preserving the examples’ intended
behavior.

In `@crates/libsy-llm-client/src/backend.rs`:
- Around line 33-41: Replace the derived Debug implementation on
HttpBackendConfig with a manual implementation that preserves base_url and
extra_headers while rendering any present api_key as “[REDACTED]” rather than
its value. Keep Debug available for containing types such as Backend and
ModelConfig without exposing credentials.

In `@crates/libsy-llm-client/src/client.rs`:
- Around line 491-495: Replace the `.expect("buffered response")` calls in the
aggregate-response handling at crates/libsy-llm-client/src/client.rs lines
491-495 and 683-687 with explicit matching, asserting or handling the success
and failure cases directly in each test while preserving the existing successful
aggregate behavior.
- Around line 25-38: Update the header-filtering logic using RESERVED_HEADERS so
credential-bearing headers such as cookie and proxy-authorization are never
forwarded from caller metadata. Prefer an explicit allowlist for headers
eligible for forwarding; otherwise expand the denylist to cover all browser,
proxy, and authentication credential headers, and add tests covering the
forwarding policy.

In `@crates/libsy-llm-client/src/lib.rs`:
- Around line 14-19: Update the module-level documentation near the
dependency-boundary explanation to remove the speculative
future-refactor/project-management statement, while retaining the explanation of
the vendored context-overflow detection and crate dependencies.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5051dedb-431a-4206-b555-88a8b1b832ea

📥 Commits

Reviewing files that changed from the base of the PR and between 8b8dad8 and a3c4498.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock, !Cargo.lock

📒 Files selected for processing (8)

• Cargo.toml
• crates/libsy-llm-client/Cargo.toml
• crates/libsy-llm-client/README.md
• crates/libsy-llm-client/src/backend.rs
• crates/libsy-llm-client/src/client.rs
• crates/libsy-llm-client/src/error.rs
• crates/libsy-llm-client/src/lib.rs
• crates/libsy-llm-client/src/raw.rs

[grahamking]

Codex Findings

Checked all 6 findings, none were relevant here, although I can see why Codex thought they were.