Skip to content
4 changes: 2 additions & 2 deletions .env.template
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,9 @@
# Allow optional /p/{provider}/v1/... passthrough aliases while keeping /p/{provider}/... canonical (default: true)
# ALLOW_PASSTHROUGH_V1_ALIAS=true

# Comma-separated list of provider types enabled for /p/{provider}/... passthrough (default: openai,anthropic,openrouter,kilo,zai,sglang,vllm,llamacpp,llmd,deepseek)
# Comma-separated list of provider types enabled for /p/{provider}/... passthrough (default: openai,anthropic,openrouter,kilo,zai,sglang,vllm,llamacpp,llmd,deepseek,hetzner)
# Cohere native passthrough is opt-in; add cohere when those routes are needed.
# ENABLED_PASSTHROUGH_PROVIDERS=openai,anthropic,cohere,openrouter,kilo,zai,sglang,vllm,llamacpp,llmd,deepseek
# ENABLED_PASSTHROUGH_PROVIDERS=openai,anthropic,cohere,openrouter,kilo,zai,sglang,vllm,llamacpp,llmd,deepseek,hetzner

# Enable the realtime (speech-to-speech) endpoints (default: true): the /v1/realtime
# websocket (and /p/{provider}/v1/realtime passthrough upgrade), the WebRTC SDP
Expand Down
1 change: 1 addition & 0 deletions docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@
"providers/llmd",
"providers/multiple-ollama",
"providers/kimicode",
"providers/hetzner",
{
"group": "Cloud Platforms",
"icon": "cloud",
Expand Down
100 changes: 100 additions & 0 deletions docs/providers/hetzner.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
---
title: "Hetzner"
description: "Configure Hetzner's experimental OpenAI-compatible inference API in GoModel."
icon: "server"
keywords: ["Hetzner", "experimental", "inference", "provider setup"]
---

<Warning>
**Experimental**: Hetzner declares this inference API as experimental. Expect breaking
changes, no SLA, and no availability guarantees. Do not use it for production workloads.
Hetzner may change models, limits, or the endpoint itself without notice while the
experiment runs.
</Warning>

Hetzner Inference is an OpenAI-compatible REST API served at
`https://inference.hetzner.com/api/v1`. GoModel routes chat, model listing, and
passthrough requests through the shared OpenAI adapter. The `/v1/responses` endpoint is
translated through chat completions. Files, batches, and embeddings are not supported —
Hetzner exposes no `/v1/embeddings` endpoint. Embedding requests fail fast with a typed
"not supported" error; no upstream call is made.

<Note>
Passthrough is a generic forwarder: it sends any path you give it to Hetzner
unchanged. Hetzner's tolerance for arbitrary upstream paths is unverified while the
API is experimental — expect HTTP 404 or 405 for paths outside `/v1/models`,
`/v1/completions`, and `/v1/chat/completions`. `hetzner` is in the default
`ENABLED_PASSTHROUGH_PROVIDERS` allowlist, so `/p/hetzner/...` routes work
without operator opt-in.
</Note>

## Configure

Create an API token in the [Hetzner Experiments console](https://experiments.hetzner.com/inference)
and set:

```bash
HETZNER_API_KEY=...
```

Or in `config.yaml`:

```yaml
providers:
hetzner:
type: hetzner
base_url: "https://inference.hetzner.com/api/v1"
api_key: "${HETZNER_API_KEY}"
```

You can also override the base URL and model list with:

```bash
HETZNER_BASE_URL=https://inference.hetzner.com/api/v1
HETZNER_MODELS=Qwen/Qwen3.6-35B-A3B-FP8
```

<Note>
The model ID above is the example from the
[official Hetzner inference docs](https://docs.hetzner.com/general/company-and-policy/experiments/inference/)
(checked 2026-08-17). The catalogue is experimental and changes; confirm the current
IDs with `GET /v1/models` before you copy the example.
</Note>

## Models

The model catalogue changes while the experiment runs. Query the live list instead of
relying on documentation snapshots:

```bash
curl -s https://inference.hetzner.com/api/v1/models \
-H "Authorization: Bearer $HETZNER_API_KEY"
```

GoModel also exposes this list through its own `/v1/models` endpoint once the provider
is configured. Vision-capable models accept OpenAI-standard `image_url` content parts
unchanged.

## Rate limits

Hetzner enforces per-key rate limits on input tokens and output tokens.
Exceeding either limit returns HTTP 429. The documented windows are 3M input tokens /
60k output tokens per 60s and 500M input / 5M output per 24h. The exact values change
while the experiment runs, so check the
[official inference docs](https://docs.hetzner.com/general/company-and-policy/experiments/inference/)
for the current numbers. Prefer conservative retry settings:

```yaml
providers:
hetzner:
type: hetzner
api_key: "${HETZNER_API_KEY}"
retries: 2
```

## Pricing

The API is free of charge while it remains in experimental status. Hetzner states it will
notify users by email before billing begins. GoModel's usage-cost tracking reports `cost`
as zero for Hetzner requests until upstream pricing exists, so `cost` load-balancing cannot
rank this provider by price.
14 changes: 14 additions & 0 deletions docs/providers/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ support, not every individual model capability exposed by an upstream provider.
| ElevenLabs (voice only) | `ELEVENLABS_API_KEY` (`ELEVENLABS_BASE_URL` optional) | `eleven_multilingual_v2` | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | [ElevenLabs](/providers/elevenlabs) |
| OpenCode Go | `OPENCODE_GO_API_KEY` (`OPENCODE_GO_BASE_URL` optional) | `glm-5.1` | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | [OpenCode Go](/providers/opencode-go) |
| Kimi Code | `KIMICODE_API_KEY` | `kimi-for-coding` | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | [Kimi Code](/providers/kimicode) |
| Hetzner (experimental) | `HETZNER_API_KEY` (`HETZNER_BASE_URL` optional) | `Qwen/Qwen3.6-35B-A3B-FP8` | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | [Hetzner](/providers/hetzner) |
Comment thread
weselben marked this conversation as resolved.
| Azure OpenAI | `AZURE_API_KEY` + `AZURE_BASE_URL` (`AZURE_API_VERSION` optional) | `gpt-5` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | [Azure OpenAI](/providers/azure) |
| Oracle GenAI | `ORACLE_API_KEY` + `ORACLE_BASE_URL` | `openai.gpt-oss-120b` | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | [Oracle GenAI](/providers/oracle) |
| Ollama | `OLLAMA_BASE_URL` | `llama3.2` | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | [Ollama](/providers/multiple-ollama) |
Expand Down Expand Up @@ -130,6 +131,19 @@ support, not every individual model capability exposed by an upstream provider.
constrained by a rolling 5-hour window. Usage-cost tracking reports zero for
Kimi Code, so `cost` load-balancing cannot price it; prefer conservative
retry strategies.
- **Hetzner (experimental)** — Hetzner declares the inference API experimental:
expect breaking changes and no SLA. The model catalogue and rate-limit windows
change while the experiment runs; query the live `/v1/models` endpoint and the
official docs instead of relying on snapshots. The example model ID in the table
above (`Qwen/Qwen3.6-35B-A3B-FP8`) is the entry from the official Hetzner docs as
of 2026-08-17 and may differ at read time. Free while experimental, so
usage-cost tracking reports zero and `cost` load-balancing cannot price it.
No embeddings endpoint; chat, `/v1/responses` (via chat), model listing, and
passthrough only. Passthrough is a generic forwarder — the ✅ marks adapter
capability; Hetzner's tolerance for arbitrary upstream paths is unverified
while the API is experimental. `hetzner` is included in the default
`ENABLED_PASSTHROUGH_PROVIDERS` allowlist so `/p/hetzner/...` routes work
without operator opt-in; remove it from the list to gate them.
- **Configured model lists** — available for every provider with
`<PROVIDER>_MODELS`, for example
`OPENROUTER_MODELS=openai/gpt-oss-120b,anthropic/claude-sonnet-4` or
Expand Down
26 changes: 26 additions & 0 deletions internal/providers/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,9 @@ var testDiscoveryConfigs = map[string]DiscoveryConfig{
"kimicode": {
DefaultBaseURL: "https://api.kimi.com/coding/v1",
},
"hetzner": {
DefaultBaseURL: "https://inference.hetzner.com/api/v1",
},
}

// --- buildProviderConfig ---
Expand Down Expand Up @@ -1897,3 +1900,26 @@ func TestResolveProviders_NoProvidersNoEnvVars(t *testing.T) {
t.Errorf("expected empty filtered raw, got %d entries", len(filteredRaw))
}
}

func TestBuildProviderConfig_Hetzner_ResolvesBaseURL(t *testing.T) {
t.Setenv("HETZNER_API_KEY", "hetzner-test-key")

raw := map[string]config.RawProviderConfig{
"hetzner": {Type: "hetzner", APIKey: "hetzner-test-key"},
}
got := applyProviderEnvVars(raw, testDiscoveryConfigs)

p, exists := got["hetzner"]
if !exists {
t.Fatal("hetzner not discovered by config parser")
}
if p.Type != "hetzner" {
t.Errorf("Type = %q, want hetzner", p.Type)
}
if p.APIKey != "hetzner-test-key" {
t.Errorf("APIKey = %q, want hetzner-test-key", p.APIKey)
}
if p.BaseURL != testDiscoveryConfigs["hetzner"].DefaultBaseURL {
t.Errorf("BaseURL = %q, want %q", p.BaseURL, testDiscoveryConfigs["hetzner"].DefaultBaseURL)
}
}
68 changes: 68 additions & 0 deletions internal/providers/hetzner/hetzner.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
// Package hetzner provides Hetzner Inference API integration for the LLM gateway.
//
// The "hetzner" provider routes to Hetzner's experimental OpenAI-compatible
// inference endpoint, so all transport goes through the shared chat-centric
// adapter and model IDs are forwarded unchanged.
//
// Note: Hetzner declares this inference API as experimental. Breaking changes
// may ship without notice and there is no SLA. Hetzner does not document an
// embeddings endpoint; chat completions, model listing, and passthrough are
// the supported surfaces.
package hetzner

import (
"context"
"net/http"

"github.com/enterpilot/gomodel/internal/core"
"github.com/enterpilot/gomodel/internal/llmclient"
"github.com/enterpilot/gomodel/internal/providers"
"github.com/enterpilot/gomodel/internal/providers/openai"
)

const defaultBaseURL = "https://inference.hetzner.com/api/v1"

// Registration provides factory registration for the Hetzner provider.
var Registration = providers.Registration{
Type: "hetzner",
New: New,
Discovery: providers.DiscoveryConfig{
DefaultBaseURL: defaultBaseURL,
},
}

// Provider implements the core.Provider interface for Hetzner. Hetzner is
// OpenAI-compatible, so all transport goes through the shared chat-centric
// adapter: chat completions, model listing, and passthrough are exposed via
// the embedded *openai.ChatCompatible. Hetzner documents no embeddings
// endpoint, so Embeddings is overridden to fail fast with a typed error.
type Provider struct {
*openai.ChatCompatible
}

var _ core.Provider = (*Provider)(nil)

// New creates a new Hetzner provider.
func New(cfg providers.ProviderConfig, opts providers.ProviderOptions) core.Provider {
return &Provider{openai.NewChatCompatible(cfg.APIKey, opts, openai.CompatibleProviderConfig{
ProviderName: "hetzner",
BaseURL: providers.ResolveBaseURL(cfg.BaseURL, defaultBaseURL),
})}
}

// NewWithHTTPClient creates a new Hetzner provider with a custom HTTP client.
// If httpClient is nil, http.DefaultClient is used.
//
// The signature is intentionally stable and matches every other chat-compatible
// provider on main: (apiKey, baseURL, httpClient, hooks).
func NewWithHTTPClient(apiKey string, baseURL string, httpClient *http.Client, hooks llmclient.Hooks) *Provider {
return &Provider{openai.NewChatCompatibleWithHTTPClient(apiKey, httpClient, hooks, openai.CompatibleProviderConfig{
ProviderName: "hetzner",
BaseURL: providers.ResolveBaseURL(baseURL, defaultBaseURL),
})}
}

// Embeddings returns an error because Hetzner does not expose an embeddings endpoint.
func (p *Provider) Embeddings(_ context.Context, _ *core.EmbeddingRequest) (*core.EmbeddingResponse, error) {
return nil, core.NewInvalidRequestError("hetzner does not support embeddings", nil)
}
Loading