Skip to content

feat(agent): support OpenAI-compatible gateways via vendor/model names #100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kannan983 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat(agent): support OpenAI-compatible gateways via vendor/model names #100

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ ANTHROPIC_API_KEY=
OPENAI_API_KEY=

# chat-latest follows ChatGPT Instant. gpt-* uses OpenAI; claude-* uses Anthropic.
# vendor/model names (e.g. deepseek/deepseek-chat) use an OpenAI-compatible
# gateway: set OPENAI_BASE_URL (e.g. https://openrouter.ai/api/v1) and put the
# gateway key in OPENAI_API_KEY. Server keys only; browser keys use OpenAI.
# OPENAI_BASE_URL=
# Model access/availability must be verified with your provider.
# Blank or unsupported model prefixes fail at startup.
# Keys are optional for BYOK-only hosting; visitors enter both keys in the app.
Expand Down
13 changes: 10 additions & 3 deletions apps/agent/src/model.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,18 @@

def build_model(*, allow_unconfigured: bool = False) -> BaseChatModel:
model_name = os.environ.get("LLM_MODEL", DEFAULT_MODEL).strip()
is_openai = model_name == "chat-latest" or model_name.startswith("gpt-")
# vendor/model names (e.g. deepseek/deepseek-chat) target an OpenAI-compatible
# gateway such as OpenRouter; set OPENAI_BASE_URL and OPENAI_API_KEY for it.
is_openai = (
model_name == "chat-latest"
or model_name.startswith("gpt-")
or "/" in model_name
)
if not is_openai and not model_name.startswith("claude-"):
raise ValueError(
"LLM_MODEL must name a supported provider: claude-* (Anthropic) "
"or gpt-*/chat-latest (OpenAI). Unset LLM_MODEL to use the default."
"LLM_MODEL must name a supported provider: claude-* (Anthropic), "
"gpt-*/chat-latest (OpenAI), or vendor/model with OPENAI_BASE_URL. "
"Unset LLM_MODEL to use the default."
)
key_name = "OPENAI_API_KEY" if is_openai else "ANTHROPIC_API_KEY"
if not os.environ.get(key_name, "").strip():
Expand Down
9 changes: 9 additions & 0 deletions apps/agent/tests/test_model.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,3 +84,12 @@ def test_missing_provider_key_has_actionable_error(monkeypatch, model, key, valu
monkeypatch.setenv(key, value)
with pytest.raises(ValueError, match=key):
build_model()


def test_vendor_model_names_route_to_openai_compatible_gateway(monkeypatch):
monkeypatch.setenv("LLM_MODEL", "deepseek/deepseek-chat")
monkeypatch.setenv("OPENAI_BASE_URL", "https://openrouter.ai/api/v1")
model = build_model()
assert isinstance(model, ChatOpenAI)
assert model.model_name == "deepseek/deepseek-chat"
assert str(model.root_client.base_url).startswith("https://openrouter.ai/api/v1")
2 changes: 1 addition & 1 deletion docs/deployment.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ For shared credentials, the blueprint and Python factory default to `LLM_MODEL=c

| Variable | Service | Purpose |
| ----------------------------------------- | ------------- | ----------------------------------------------------------------------------------- |
| `LLM_MODEL` | Agent | `claude-*` routes to Anthropic; `gpt-*` routes to OpenAI |
| `LLM_MODEL` | Agent | `claude-*` routes to Anthropic; `gpt-*` to OpenAI; `vendor/model` to `OPENAI_BASE_URL` |
| `ANTHROPIC_API_KEY` | Agent | Required when selecting Anthropic |
| `OPENAI_API_KEY` | Agent | Required when selecting OpenAI |
| `LANGSMITH_API_KEY` | As configured | Optional tracing configuration |
Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ TYPESAFE_API_KEY=your-typesafe-key
JEV_MODEL=jev-latest
```

The model factory accepts `claude-*` names through Anthropic and `chat-latest` or `gpt-*` names through OpenAI. To override the default, set a model available to your account in `LLM_MODEL` and supply its provider key. Shared mode requires the selected answer provider’s key and the Jev key; BYOK requests supply their own pair. Unset `LLM_MODEL` uses the local default; an empty value or unsupported prefix fails with a configuration error.
The model factory accepts `claude-*` names through Anthropic and `chat-latest` or `gpt-*` names through OpenAI. `vendor/model` names, such as `deepseek/deepseek-chat`, go through an OpenAI-compatible gateway such as OpenRouter: set `OPENAI_BASE_URL=https://openrouter.ai/api/v1` and put the gateway key in `OPENAI_API_KEY`. Browser-supplied keys always use OpenAI. To override the default, set a model available to your account in `LLM_MODEL` and supply its provider key. Shared mode requires the selected answer provider’s key and the Jev key; BYOK requests supply their own pair. Unset `LLM_MODEL` uses the local default; an empty value or unsupported prefix fails with a configuration error.

Check model availability and access with a real request. This repository does not guarantee that every model name works or implement a separate native GPT-6 API. Never put shared provider keys in public frontend environment variables. See [BYOK behavior](../README.md#use-your-own-api-keys).

Expand Down