You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/docs/concepts/gateways.md
+59-32Lines changed: 59 additions & 32 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -32,8 +32,6 @@ domain: example.com
32
32
33
33
</div>
34
34
35
-
A domain name is required to create a gateway.
36
-
37
35
To create or update the gateway, simply call the [`dstack apply`](../reference/cli/dstack/apply.md) command:
38
36
39
37
<div class="termy">
@@ -53,6 +51,12 @@ Provisioning...
53
51
54
52
## Configuration options
55
53
54
+
### Domain
55
+
56
+
A gateway requires a `domain` to be specified in the configuration before creation. The domain is used to generate service endpoints (e.g. `<run name>.<gateway domain>`).
57
+
58
+
Once the gateway is created and assigned a hostname, configure your DNS by adding a wildcard record for `*.<gateway domain>` (e.g. `*.example.com`). The record should point to the gateway's hostname and should be of type `A` if the hostname is an IP address (most cases), or of type `CNAME` if the hostname is another domain (some private gateways and Kubernetes).
59
+
56
60
### Backend
57
61
58
62
You can create gateways with the `aws`, `azure`, `gcp`, or `kubernetes` backends, but that does not limit where services run. A gateway can use one backend while services run on any other backend supported by dstack, including backends where gateways themselves cannot be created.
@@ -61,27 +65,6 @@ You can create gateways with the `aws`, `azure`, `gcp`, or `kubernetes` backends
61
65
Gateways in `kubernetes` backend require an external load balancer. Managed Kubernetes solutions usually include a load balancer.
62
66
For self-hosted Kubernetes, you must provide a load balancer by yourself.
63
67
64
-
### Instance type
65
-
66
-
By default, `dstack` provisions a small, low-cost instance for the gateway. If you expect to run high-traffic services, you can configure a larger instance type using the `instance_type` property.
67
-
68
-
<div editor-title="gateway.dstack.yml">
69
-
70
-
```yaml
71
-
type: gateway
72
-
name: example-gateway
73
-
74
-
backend: aws
75
-
region: eu-west-1
76
-
77
-
# (Optional) Override the gateway instance type
78
-
instance_type: t3.large
79
-
80
-
domain: example.com
81
-
```
82
-
83
-
</div>
84
-
85
68
### Router
86
69
87
70
By default, the gateway uses its own load balancer to route traffic between replicas. However, you can delegate this responsibility to a specific router by setting the `router` property. Currently, the only supported external router is `sglang`.
@@ -123,21 +106,65 @@ router:
123
106
>
124
107
> Support for prefill/decode disaggregation and auto-scaling based on inter-token latency is coming soon.
125
108
109
+
### Certificate
110
+
111
+
By default, when you run a service with a gateway, `dstack` provisions an SSL certificate via Let's Encrypt for the configured domain. This automatically enables HTTPS for the service endpoint.
112
+
113
+
If you disable [public IP](#public-ip) (e.g. to make the gateway private) or if you simply don't need HTTPS, you can set `certificate` to `null`.
114
+
115
+
> Note, by default services set [`https`](../reference/dstack.yml/service.md#https) to `true` which requires a certificate. You can set `https` to `auto` to detect if the gateway supports HTTPS or not automatically.
116
+
117
+
??? info "Certificate types"
118
+
`dstack` supports the following certificate types:
119
+
120
+
* `lets-encrypt` (default) — Automatic certificates via [Let's Encrypt](https://letsencrypt.org/). Requires a [public IP](#public-ip).
121
+
* `acm` — Certificates managed by [AWS Certificate Manager](https://aws.amazon.com/certificate-manager/). AWS-only. TLS is terminated at the load balancer, not at the gateway.
122
+
* `null` — No certificate. Services will use HTTP.
123
+
126
124
### Public IP
127
125
128
-
If you don't need/want a public IP for the gateway, you can set the `public_ip` to `false` (the default value is `true`), making the gateway private.
126
+
If you don't need a public IP for the gateway, you can set `public_ip` to `false` (the default is `true`), making the gateway private.
127
+
129
128
Private gateways are currently supported in `aws` and `gcp` backends.
130
129
131
-
!!! info "Reference"
132
-
For all gateway configuration options, refer to the [reference](../reference/dstack.yml/gateway.md).
130
+
<div editor-title="gateway.dstack.yml">
133
131
134
-
## Update DNS records
132
+
```yaml
133
+
type: gateway
134
+
name: private-gateway
135
+
136
+
backend: aws
137
+
region: eu-west-1
138
+
domain: example.com
139
+
140
+
public_ip: false
141
+
certificate: null
142
+
```
143
+
144
+
</div>
145
+
146
+
### Instance type
147
+
148
+
By default, `dstack` provisions a small, low-cost instance for the gateway. If you expect to run high-traffic services, you can configure a larger instance type using the `instance_type` property.
149
+
150
+
<div editor-title="gateway.dstack.yml">
135
151
136
-
Once the gateway is assigned a hostname, go to your domain's DNS settings
137
-
and add a DNS record for `*.<gateway domain>`, e.g. `*.example.com`.
138
-
The record should point to the gateway's hostname shown in `dstack`
139
-
and should be of type `A` if the hostname is an IP address (most cases),
140
-
or of type `CNAME` if the hostname is another domain (some private gateways and Kubernetes).
152
+
```yaml
153
+
type: gateway
154
+
name: example-gateway
155
+
156
+
backend: aws
157
+
region: eu-west-1
158
+
159
+
instance_type: t3.large
160
+
161
+
domain: example.com
162
+
```
163
+
164
+
</div>
165
+
166
+
!!! info "Reference"
167
+
For all gateway configuration options, refer to the [reference](../reference/dstack.yml/gateway.md).
0 commit comments