-
-
Notifications
You must be signed in to change notification settings - Fork 88
feat(providers): add Hetzner experimental inference provider #701
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
SantiagoDePolonia
merged 9 commits into
ENTERPILOT:main
from
weselben:feat/hetzner-provider
Aug 17, 2026
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
e5f681c
feat(providers): add hetzner experimental provider
weselben 132297f
feat(run): register hetzner provider in factory
weselben bca0b60
test(providers): add hetzner to config parser test fixtures
weselben a5b86c9
docs(providers): add hetzner provider guide
weselben e54d52f
test(providers): hetzner unit tests, 100% statement coverage
weselben 30e5052
fix(providers): hetzner embeddings typed error, test newline
weselben 695f1db
fix(providers): review findings round 2
weselben 7bde193
docs(providers): document hetzner model-ID provenance and passthrough…
weselben e0fd293
fix(providers): review-bot findings on PR #701
weselben File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| --- | ||
| title: "Hetzner" | ||
| description: "Configure Hetzner's experimental OpenAI-compatible inference API in GoModel." | ||
| icon: "server" | ||
| keywords: ["Hetzner", "experimental", "inference", "provider setup"] | ||
| --- | ||
|
|
||
| <Warning> | ||
| **Experimental**: Hetzner declares this inference API as experimental. Expect breaking | ||
| changes, no SLA, and no availability guarantees. Do not use it for production workloads. | ||
| Hetzner may change models, limits, or the endpoint itself without notice while the | ||
| experiment runs. | ||
| </Warning> | ||
|
|
||
| Hetzner Inference is an OpenAI-compatible REST API served at | ||
| `https://inference.hetzner.com/api/v1`. GoModel routes chat, model listing, and | ||
| passthrough requests through the shared OpenAI adapter. The `/v1/responses` endpoint is | ||
| translated through chat completions. Files, batches, and embeddings are not supported — | ||
| Hetzner exposes no `/v1/embeddings` endpoint. Embedding requests fail fast with a typed | ||
| "not supported" error; no upstream call is made. | ||
|
|
||
| <Note> | ||
| Passthrough is a generic forwarder: it sends any path you give it to Hetzner | ||
| unchanged. Hetzner's tolerance for arbitrary upstream paths is unverified while the | ||
| API is experimental — expect HTTP 404 or 405 for paths outside `/v1/models`, | ||
| `/v1/completions`, and `/v1/chat/completions`. `hetzner` is in the default | ||
| `ENABLED_PASSTHROUGH_PROVIDERS` allowlist, so `/p/hetzner/...` routes work | ||
| without operator opt-in. | ||
| </Note> | ||
|
|
||
| ## Configure | ||
|
|
||
| Create an API token in the [Hetzner Experiments console](https://experiments.hetzner.com/inference) | ||
| and set: | ||
|
|
||
| ```bash | ||
| HETZNER_API_KEY=... | ||
| ``` | ||
|
|
||
| Or in `config.yaml`: | ||
|
|
||
| ```yaml | ||
| providers: | ||
| hetzner: | ||
| type: hetzner | ||
| base_url: "https://inference.hetzner.com/api/v1" | ||
| api_key: "${HETZNER_API_KEY}" | ||
| ``` | ||
|
|
||
| You can also override the base URL and model list with: | ||
|
|
||
| ```bash | ||
| HETZNER_BASE_URL=https://inference.hetzner.com/api/v1 | ||
| HETZNER_MODELS=Qwen/Qwen3.6-35B-A3B-FP8 | ||
| ``` | ||
|
|
||
| <Note> | ||
| The model ID above is the example from the | ||
| [official Hetzner inference docs](https://docs.hetzner.com/general/company-and-policy/experiments/inference/) | ||
| (checked 2026-08-17). The catalogue is experimental and changes; confirm the current | ||
| IDs with `GET /v1/models` before you copy the example. | ||
| </Note> | ||
|
|
||
| ## Models | ||
|
|
||
| The model catalogue changes while the experiment runs. Query the live list instead of | ||
| relying on documentation snapshots: | ||
|
|
||
| ```bash | ||
| curl -s https://inference.hetzner.com/api/v1/models \ | ||
| -H "Authorization: Bearer $HETZNER_API_KEY" | ||
| ``` | ||
|
|
||
| GoModel also exposes this list through its own `/v1/models` endpoint once the provider | ||
| is configured. Vision-capable models accept OpenAI-standard `image_url` content parts | ||
| unchanged. | ||
|
|
||
| ## Rate limits | ||
|
|
||
| Hetzner enforces per-key rate limits on input tokens and output tokens. | ||
| Exceeding either limit returns HTTP 429. The documented windows are 3M input tokens / | ||
| 60k output tokens per 60s and 500M input / 5M output per 24h. The exact values change | ||
| while the experiment runs, so check the | ||
| [official inference docs](https://docs.hetzner.com/general/company-and-policy/experiments/inference/) | ||
| for the current numbers. Prefer conservative retry settings: | ||
|
|
||
| ```yaml | ||
| providers: | ||
| hetzner: | ||
| type: hetzner | ||
| api_key: "${HETZNER_API_KEY}" | ||
| retries: 2 | ||
| ``` | ||
|
|
||
| ## Pricing | ||
|
|
||
| The API is free of charge while it remains in experimental status. Hetzner states it will | ||
| notify users by email before billing begins. GoModel's usage-cost tracking reports `cost` | ||
| as zero for Hetzner requests until upstream pricing exists, so `cost` load-balancing cannot | ||
| rank this provider by price. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| // Package hetzner provides Hetzner Inference API integration for the LLM gateway. | ||
| // | ||
| // The "hetzner" provider routes to Hetzner's experimental OpenAI-compatible | ||
| // inference endpoint, so all transport goes through the shared chat-centric | ||
| // adapter and model IDs are forwarded unchanged. | ||
| // | ||
| // Note: Hetzner declares this inference API as experimental. Breaking changes | ||
| // may ship without notice and there is no SLA. Hetzner does not document an | ||
| // embeddings endpoint; chat completions, model listing, and passthrough are | ||
| // the supported surfaces. | ||
| package hetzner | ||
|
|
||
| import ( | ||
| "context" | ||
| "net/http" | ||
|
|
||
| "github.com/enterpilot/gomodel/internal/core" | ||
| "github.com/enterpilot/gomodel/internal/llmclient" | ||
| "github.com/enterpilot/gomodel/internal/providers" | ||
| "github.com/enterpilot/gomodel/internal/providers/openai" | ||
| ) | ||
|
|
||
| const defaultBaseURL = "https://inference.hetzner.com/api/v1" | ||
|
|
||
| // Registration provides factory registration for the Hetzner provider. | ||
| var Registration = providers.Registration{ | ||
| Type: "hetzner", | ||
| New: New, | ||
| Discovery: providers.DiscoveryConfig{ | ||
| DefaultBaseURL: defaultBaseURL, | ||
| }, | ||
| } | ||
|
|
||
| // Provider implements the core.Provider interface for Hetzner. Hetzner is | ||
| // OpenAI-compatible, so all transport goes through the shared chat-centric | ||
| // adapter: chat completions, model listing, and passthrough are exposed via | ||
| // the embedded *openai.ChatCompatible. Hetzner documents no embeddings | ||
| // endpoint, so Embeddings is overridden to fail fast with a typed error. | ||
| type Provider struct { | ||
| *openai.ChatCompatible | ||
| } | ||
|
|
||
| var _ core.Provider = (*Provider)(nil) | ||
|
|
||
| // New creates a new Hetzner provider. | ||
| func New(cfg providers.ProviderConfig, opts providers.ProviderOptions) core.Provider { | ||
| return &Provider{openai.NewChatCompatible(cfg.APIKey, opts, openai.CompatibleProviderConfig{ | ||
| ProviderName: "hetzner", | ||
| BaseURL: providers.ResolveBaseURL(cfg.BaseURL, defaultBaseURL), | ||
| })} | ||
| } | ||
|
|
||
| // NewWithHTTPClient creates a new Hetzner provider with a custom HTTP client. | ||
| // If httpClient is nil, http.DefaultClient is used. | ||
| // | ||
| // The signature is intentionally stable and matches every other chat-compatible | ||
| // provider on main: (apiKey, baseURL, httpClient, hooks). | ||
| func NewWithHTTPClient(apiKey string, baseURL string, httpClient *http.Client, hooks llmclient.Hooks) *Provider { | ||
| return &Provider{openai.NewChatCompatibleWithHTTPClient(apiKey, httpClient, hooks, openai.CompatibleProviderConfig{ | ||
| ProviderName: "hetzner", | ||
| BaseURL: providers.ResolveBaseURL(baseURL, defaultBaseURL), | ||
| })} | ||
| } | ||
|
|
||
| // Embeddings returns an error because Hetzner does not expose an embeddings endpoint. | ||
| func (p *Provider) Embeddings(_ context.Context, _ *core.EmbeddingRequest) (*core.EmbeddingResponse, error) { | ||
| return nil, core.NewInvalidRequestError("hetzner does not support embeddings", nil) | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.