Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ jobs:
run: |
set -euo pipefail
helm lint deploy/rustfs-operator
helm lint deploy/rustfs-tenant

# Default: cosiDriver disabled, nothing COSI-related should render.
helm template test deploy/rustfs-operator > /tmp/helm-default.yaml
Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/helm-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,15 +84,18 @@ jobs:
run: |
set -eux
helm package ./deploy/rustfs-operator \
--destination deploy/rustfs-operator/ \
--destination helm-packages/ \
--version "${{ steps.version.outputs.chart_version }}" \
--app-version "${{ steps.version.outputs.chart_version }}"
helm package ./deploy/rustfs-tenant \
--destination helm-packages/ \
--version "${{ steps.version.outputs.chart_version }}"

- name: Upload helm package as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: helm-package
path: deploy/rustfs-operator/*.tgz
path: helm-packages/*.tgz
retention-days: 1

publish-helm-package:
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ CONSOLE-INTEGRATION-SUMMARY.md
SCRIPTS-UPDATE.md
AGENTS.md
docs/*
!docs/tenant-networking.md
!docs/operator-user-guide.md
!docs/operator-user-guide.zh-CN.md
.codex
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ All notable changes to RustFS Operator are documented in this file. The format i

### Added

- Optional `rustfs-tenant` Helm chart for a Tenant and separate API/Console Ingress
or HTTPRoute resources, with backend TLS policies, native YAML/Kustomize examples,
and networking lifecycle documentation. Existing CRDs and controller behavior are unchanged.

- Tenant `spec.oidc.extraCaCertSecretRef` for validated, reloadable custom CA trust used by
RustFS OIDC outbound connections.
- Tenant `spec.additionalVolumes` and `spec.additionalVolumeMounts` for Kubernetes-native file
Expand Down
4 changes: 4 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@

## 📋 Code Quality Requirements

Helm 3 must be on `PATH` for `cargo test --all` and `make pre-commit`: the Tenant
chart contract tests render the chart and verify its backends against Operator
Service generation. CI installs Helm before running these tests.

### 🔧 Code Formatting Rules

**MANDATORY**: All code must be properly formatted before committing. This project enforces strict formatting standards to maintain code consistency and readability.
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ Then use `http://127.0.0.1:19000` for the Tenant S3 API and `http://127.0.0.1:19
|-----|---------|
| [CONTRIBUTING.md](CONTRIBUTING.md) | Quality gates, `make pre-commit`, PR rules. |
| [examples/README.md](examples/README.md) | Tenant manifests and usage notes. |
| [docs/tenant-networking.md](docs/tenant-networking.md) | Tenant networking with native YAML, Kustomize, or the optional Tenant chart. |
| [deploy/README.md](deploy/README.md) | Helm and Kubernetes deployment entry point. |
| [deploy/rustfs-operator/README.md](deploy/rustfs-operator/README.md) | Helm chart values and examples. |
| [console-web/README.md](console-web/README.md) | Operator console frontend development and deployment. |
Expand Down
11 changes: 10 additions & 1 deletion deploy/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# RustFS Operator Deployment

This directory contains the Helm chart for deploying the RustFS Kubernetes operator.
This directory contains separate charts for installing the RustFS Operator and deploying Tenants.

## Quick Start

Expand All @@ -20,6 +20,15 @@ helm install rustfs-operator deploy/rustfs-operator/ \
- CRD installation
- Example Tenant resources

- **rustfs-tenant/** - Optional Tenant deployment chart with API/Console Ingress or HTTPRoute.
It requires an existing Operator; it does not change existing Tenant YAML or install a Gateway.

## Tenant networking

Use [plain Kubernetes YAML/Kustomize](../docs/tenant-networking.md) or the
[separate Tenant chart](rustfs-tenant/README.md). All external endpoints are disabled
by default in the chart. Existing deployments do not need to migrate.

## Documentation

See the [Helm chart README](rustfs-operator/README.md) for detailed configuration options and usage examples.
Expand Down
26 changes: 26 additions & 0 deletions deploy/rustfs-tenant/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Copyright 2026 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v2
name: rustfs-tenant
description: A RustFS Tenant with optional S3 and Console ingress resources
type: application
version: 0.0.6
home: https://rustfs.com
sources:
- https://github.com/rustfs/operator
keywords:
- storage
- s3
- rustfs
145 changes: 145 additions & 0 deletions deploy/rustfs-tenant/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
<!--
Copyright 2026 RustFS Team

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

# RustFS Tenant chart

Deploy a Tenant and optional S3 API / Tenant Console networking as one Helm release,
or render ordinary Kubernetes YAML for GitOps and `kubectl apply -f`.
Install the RustFS Operator and its CRD separately; this chart has no Operator dependency.
See the [networking guide](../../docs/tenant-networking.md) for plain YAML, TLS,
virtual-hosted S3, verification, and lifecycle details.

## Resource ownership

| Owner | Resources |
| --- | --- |
| This chart / your GitOps application | Tenant, optional Ingress / HTTPRoute / BackendTLSPolicy |
| RustFS Operator | Tenant Services, StatefulSets and other existing managed resources |
| Platform administrator | GatewayClass, Gateway, controllers, DNS and external certificates |
| Secret management system / administrator | Credentials, referenced TLS Secrets and CA ConfigMaps |

The chart does not change the Tenant CRD, controller RBAC, or existing Operator chart.
`ingress` and `httpRoute` are **chart values**, not new Tenant.spec fields.
The Operator Console's `console.ingress` setting belongs to the Operator chart and
is independent of the Tenant Console configured here.

## Install from this repository

1. Install the Operator and CRD using the [Operator guide](../../docs/operator-user-guide.md).
2. Create `rustfs-storage` and a `rustfs-credentials` Secret in that namespace,
containing UTF-8 `accesskey` and `secretkey` values (minimum 8 characters each).
Use your secret manager; do not commit credentials in values files.
3. Copy [values.yaml](values.yaml) to `tenant-values.yaml`. Set a tested image,
storage class, pool layout and capacity for your environment. The default is
a **single-node, single-disk development** Tenant, not an HA layout.
4. Install without external access:

```bash
helm upgrade --install rustfs ./deploy/rustfs-tenant \
--namespace rustfs-storage --values tenant-values.yaml
```

To enable networking, edit the hostnames and platform references in one example:

```bash
# Choose Ingress OR Gateway API per endpoint.
helm upgrade --install rustfs ./deploy/rustfs-tenant \
--namespace rustfs-storage --values tenant-values.yaml \
--values deploy/rustfs-tenant/examples/gateway.yaml
```

The Ingress alternative is [examples/ingress.yaml](examples/ingress.yaml).
The example requires an installed IngressClass and a pre-created edge TLS Secret.
The Gateway example requires Gateway API CRDs, a controller, and an existing HTTPS
Gateway listener allowing routes from `rustfs-storage`. It does not create them.

These commands use the source chart. Use a published repository chart only after
a release containing `rustfs-tenant` is available; do not assume the existing
published `0.0.6` release already contains this addition. The release workflow
packages both charts under the release version. The Tenant chart deliberately
has no `appVersion`: it does not select a RustFS image through chart metadata.

## Values contract

| Value | Meaning |
| --- | --- |
| `tenant.metadata.name` | Tenant name; defaults to release name; DNS-1035, max 55 characters |
| `tenant.metadata.labels`, `annotations` | Native metadata maps for the Tenant |
| `tenant.spec` | Native Tenant spec, rendered without `tpl` evaluation or environment injection |
| `ingress.api`, `ingress.console` | Independent Ingress settings; disabled by default |
| `ingress.*.hosts` | Explicit DNS hostnames; no catch-all rule |
| `ingress.*.ingressClassName`, `tls` | Native Ingress fields; TLS uses existing Secrets |
| `ingress.*.labels`, `annotations` | Resource metadata, including controller-specific settings |
| `httpRoute.api`, `httpRoute.console` | Independent HTTPRoute settings; disabled by default |
| `httpRoute.*.parentRefs`, `hostnames` | Native Gateway references and explicit hostnames |
| `httpRoute.*.timeouts` | Native rule timeouts; depends on controller support |
| `httpRoute.*.labels`, `annotations` | HTTPRoute metadata |
| `httpRoute.*.backendTLS` | Optional native BackendTLSPolicy `spec.validation`; requires v1.4+ CRDs and controller support |

Namespace comes from `--namespace`. Services are always `<tenant>-io:9000` and
`<tenant>-console:9001`, matching the existing Operator contract. Each route uses
`/` with prefix matching. The chart does not add rewrites, shared Gateways, DNS,
certificates, arbitrary extra resources, or controller-specific policies.
Use separate standard manifests for advanced platform policies.

Helm merges maps but **replaces lists**, including `tenant.spec.pools` and `env`.
Keep the complete desired lists in your values file. Clearing a default map uses
`null`, e.g. `tenant.spec.credsSecret: null` when using another supported credential
source. The installed Tenant CRD remains the authority for validating native spec
fields; chart schema validation covers the chart-owned settings.

## Upgrade and remove

Keep the release name, namespace and Tenant name stable. Renaming a Tenant creates
a different storage cluster; this chart does not implement data migration.
Do not adopt an existing Tenant or user-managed networking by adding Helm ownership
annotations or using `--take-ownership`. Existing installations can keep using
plain YAML unchanged. Plan any ownership migration separately.

Set an endpoint's `enabled: false` and apply a Helm upgrade to remove its chart-owned
Ingress/HTTPRoute and its optional backend TLS policy. Disabled endpoints may
retain their configuration for later reuse. Reverting to an internal-only
deployment must disable all previously
enabled endpoints. Use complete desired values; avoid `--reuse-values` when removing
old exposure settings. Switching Ingress to HTTPRoute can interrupt traffic and
needs a planned cutover.

`helm uninstall rustfs -n rustfs-storage` deletes the chart-owned Tenant and routes;
Tenant deletion triggers the Operator/Kubernetes workload cleanup. Treat it as a
storage teardown, not a networking-only operation. Check PVC/PV retention and
backups before teardown. Referenced credential/certificate resources and shared
Gateways are not deleted by this chart.

Deleting only the Tenant CR does **not** garbage-collect the chart-owned routes:
there are intentionally no synthetic ownerReferences or lookup hooks. Use the
Helm release lifecycle, or explicitly manage every resource with GitOps.
Helm is not a continuous reconciler; use a GitOps controller if drift correction
of networking resources is required.

## Validation

From the repository root (Helm 3 required):

```bash
helm lint deploy/rustfs-tenant
cargo test --test tenant_chart
kubectl kustomize examples/networking/ingress
kubectl kustomize examples/networking/gateway
```

Rendering does not verify the installed CRDs, Gateway attachment, DNS, certificates,
or S3 traffic. Follow the [runtime verification steps](../../docs/tenant-networking.md#verify-the-data-path)
on your target platform.
50 changes: 50 additions & 0 deletions deploy/rustfs-tenant/examples/backend-tls.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Copyright 2026 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Layer over examples/gateway.yaml. Requires cert-manager, an existing Issuer,
# a trusted CA ConfigMap (ca.crt) and Gateway API v1.4+ with BackendTLSPolicy support.
tenant:
spec:
tls:
mode: certManager
enableInternodeHttps: true
certManager:
manageCertificate: true
secretName: rustfs-server-tls
issuerRef:
group: cert-manager.io
kind: Issuer
name: rustfs-issuer
includeGeneratedDnsNames: true
dnsNames:
- rustfs-io.rustfs-storage.svc.cluster.local
- rustfs-console.rustfs-storage.svc.cluster.local
caTrust:
source: CertificateSecretCa
trustSystemCa: false
httpRoute:
api:
backendTLS:
hostname: rustfs-io.rustfs-storage.svc.cluster.local
caCertificateRefs:
- group: ""
kind: ConfigMap
name: rustfs-backend-ca
console:
backendTLS:
hostname: rustfs-console.rustfs-storage.svc.cluster.local
caCertificateRefs:
- group: ""
kind: ConfigMap
name: rustfs-backend-ca
34 changes: 34 additions & 0 deletions deploy/rustfs-tenant/examples/gateway.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Copyright 2026 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# The existing Gateway owns the public certificate and HTTPS listener.
httpRoute:
api:
enabled: true
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: shared-gateway
namespace: gateway-system
sectionName: https
hostnames: [s3.example.com]
console:
enabled: true
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: shared-gateway
namespace: gateway-system
sectionName: https
hostnames: [console.example.com]
30 changes: 30 additions & 0 deletions deploy/rustfs-tenant/examples/ingress.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Copyright 2026 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Values file for: helm upgrade --install rustfs deploy/rustfs-tenant -n rustfs-storage -f ...
ingress:
api:
enabled: true
ingressClassName: traefik
hosts: [s3.example.com]
tls:
- hosts: [s3.example.com]
secretName: rustfs-edge-tls
console:
enabled: true
ingressClassName: traefik
hosts: [console.example.com]
tls:
- hosts: [console.example.com]
secretName: rustfs-edge-tls
Loading
Loading