diff --git a/docs/deployment-preparation/software-requirements.md b/docs/deployment-preparation/software-requirements.md index db0a8b5c..223fdbe2 100644 --- a/docs/deployment-preparation/software-requirements.md +++ b/docs/deployment-preparation/software-requirements.md @@ -40,9 +40,15 @@ For Kubernetes-based deployments, the following Kubernetes environments and dist | Google GKE | 1.30 and higher - non production only! | | K3s | 1.33 and higher | | Kubernetes (vanilla) | 1.33 and higher | +| RKE2 | 1.33 and higher | | Talos | 1.6.7 and higher | | OpenShift | 4.19 and higher | +!!! info + SUSE Rancher is a management plane, not a Kubernetes distribution. Simplyblock is installed into the downstream + cluster it manages, which must run a supported RKE2 or K3s version. For details, see + [SUSE Rancher and RKE2](../kubernetes/installation/rancher.md). + Additionally, there are verified and supported operating systems for the Kubernetes worker nodes. A full reference is available at the [Supported Linux Distributions](../reference/supported-linux-distributions.md#kubernetes-hyper-converged-control-plane-and-storage-plane) reference. diff --git a/docs/kubernetes/installation/index.md b/docs/kubernetes/installation/index.md index e1a30261..133f2582 100644 --- a/docs/kubernetes/installation/index.md +++ b/docs/kubernetes/installation/index.md @@ -47,5 +47,6 @@ For detailed CRD documentation, see [Simplyblock Operator](../../reference/opera ## Platform-Specific Notes - [OpenShift](openshift.md): Additional configuration for OpenShift clusters. +- [SUSE Rancher and RKE2](rancher.md): Permissions and kubelet configuration for RKE2 and K3s clusters. - [Talos](talos.md): Specifics for Talos-based OS images. - [Volume Encryption](../usage/volume-encryption.md): End-to-end encryption with customer-managed keys. diff --git a/docs/kubernetes/installation/rancher.md b/docs/kubernetes/installation/rancher.md new file mode 100644 index 00000000..cdc209e5 --- /dev/null +++ b/docs/kubernetes/installation/rancher.md @@ -0,0 +1,94 @@ +--- +title: "SUSE Rancher and RKE2" +description: "Installing simplyblock into RKE2 or K3s clusters managed by SUSE Rancher, including the permissions required on CIS-hardened clusters." +weight: 40150 +--- + +[SUSE Rancher](https://www.rancher.com/){:target="_blank" rel="noopener"} (Rancher Manager, Rancher Prime) is a +management plane for Kubernetes clusters and not a Kubernetes distribution of its own. Simplyblock is installed into a +downstream cluster managed by Rancher, never into the Rancher management cluster. + +A Rancher-based deployment is made up of three layers, and the simplyblock requirements differ per layer: + +| Layer | Examples | Simplyblock requirements | +|-------------------------|-------------------------------------------------------------------------------------------------------------------------|---------------------------------------------| +| Management plane | Rancher Manager, Rancher Prime | None | +| Kubernetes distribution | [RKE2](https://docs.rke2.io/){:target="_blank" rel="noopener"}, [K3s](https://k3s.io/){:target="_blank" rel="noopener"} | Privileged permissions on hardened clusters | +| Worker node OS | SUSE Linux Enterprise, SUSE Linux Micro, RHEL, Ubuntu | Supported Linux distribution | + +!!! info + Not all worker nodes of a downstream cluster have to host storage components. + Simplyblock uses node labels to identify nodes that participate in the storage cluster. + Storage workloads can be isolated on dedicated worker nodes or node pools. + +## Prerequisites + +The downstream cluster has to be operational, and cluster administrator privileges are required. `kubectl` is +configured against the downstream cluster with the kubeconfig provided by the Rancher UI, not with the kubeconfig of +the management cluster. + +The cluster and its worker nodes have to meet the general simplyblock requirements: + +- [Software Requirements](../../deployment-preparation/software-requirements.md) +- [Hardware Requirements](../../deployment-preparation/hardware-requirements.md) +- [Supported Linux Distributions](../../reference/supported-linux-distributions.md#kubernetes-hyper-converged-control-plane-and-storage-plane) + +## Required Permissions + +The Simplyblock CSI driver connects NVMe over Fabrics devices, formats them, and mounts them. Therefore, it runs as a +privileged container. + +By default, RKE2 ships a Pod Security Admission configuration that enforces the `privileged` standard cluster-wide, and +nothing else has to be granted. On a cluster provisioned with the CIS hardening profile, the `restricted` standard is +enforced instead, and only the RKE2 system namespaces are exempted from it. The simplyblock namespace therefore has to +be labeled with the exemptions itself. + +```yaml title="Example of the simplyblock namespace with privileged exemptions (simplyblock-namespace.yaml)" +apiVersion: v1 +kind: Namespace +metadata: + name: simplyblock + labels: + pod-security.kubernetes.io/enforce: privileged + pod-security.kubernetes.io/enforce-version: latest + pod-security.kubernetes.io/audit: privileged + pod-security.kubernetes.io/audit-version: latest + pod-security.kubernetes.io/warn: privileged + pod-security.kubernetes.io/warn-version: latest +``` + +```bash title="Creating the simplyblock namespace with the privileged exemptions" +kubectl apply -f simplyblock-namespace.yaml +``` + +The namespace is created before the operator is installed. Otherwise, the Helm chart creates an unlabeled namespace of +its own, and the exemptions never take effect. + +## CPU Topology and Core Isolation + +The kubelet CPU topology and the core isolation of the storage nodes are configured by simplyblock through the Helm +values of the operator. The cluster definition of RKE2 or K3s stays untouched: + +| Helm value | Purpose | +|----------------------------------------|----------------------------------------------------------------| +| `storagenode.enableCpuTopology` | Enables the CPU topology configuration on storage nodes | +| `storagenode.isolateCores` | Enables the automatic core isolation | +| `storagenode.skipKubeletConfiguration` | Skips the kubelet CPU topology configuration if already set up | +| `storagenode.reservedSystemCpu` | Reserves CPU cores for the host and system workloads | + +The full list of values is documented in the [Kubernetes Reference](../../reference/kubernetes/index.md). + +## SUSE Linux Micro Nodes + +[SUSE Linux Micro](https://www.suse.com/products/micro/){:target="_blank" rel="noopener"} (SL Micro, formerly SLE Micro) +is supported as a worker node operating system and needs no image customization. Huge pages are allocated by +simplyblock automatically, and `/etc` is writable, so configuration drop-ins are applied directly. + +Additional packages cannot be installed in place, because the root filesystem is read-only. Such a change is applied +with `transactional-update` and takes effect after a reboot. + +## Installation of Simplyblock + +To install the simplyblock components on a Rancher-managed cluster, follow the instructions to +[install the Simplyblock Operator](k8s-control-plane.md) and follow the instructions to [deploy the storage nodes and +CSI driver](k8s-storage-plane.md). diff --git a/docs/reference/supported-linux-distributions.md b/docs/reference/supported-linux-distributions.md index 4f91ba52..2b2b3500 100644 --- a/docs/reference/supported-linux-distributions.md +++ b/docs/reference/supported-linux-distributions.md @@ -6,8 +6,11 @@ weight: 20200 Simplyblock requires a Linux Kernel 5.19 or later with NVMe over Fabrics and NVMe over TCP enabled. However, `{{ cliname }}`, the simplyblock commandline interface, requires some additional tools and expects certain -conventions for configuration files and locations. Therefore, simplyblock officially only supports Red Hat-based Linux -distributions as of now. +conventions for configuration files and locations. For **plain Linux deployments**, which are managed through +`{{ cliname }}`, simplyblock therefore officially only supports Red Hat-based Linux distributions as of now. + +**Kubernetes deployments** do not use `{{ cliname }}` on the nodes. The distributions supported for the Kubernetes +worker nodes are listed below. While others may work, manual intervention may be required, and simplyblock cannot support those. @@ -36,16 +39,22 @@ The following Linux distributions are considered tested and supported to run a d The following Linux distributions are considered tested and supported to run a hyper-converged control and storage plane: -| Distribution | Version | Architecture | Support Level | -|--------------------------|-----------------|-------------------------|-----------------| -| Red Hat Enterprise Linux | 9 and later | x86-64, ARM64 (AArch64) | Fully supported | -| Rocky Linux | 9 and later | x86-64, ARM64 (AArch64) | Fully supported | -| Alma Linux | 9 and later | x86-64, ARM64 (AArch64) | Fully supported | -| Ubuntu | 22.04 and later | x86-64, ARM64 (AArch64) | Fully supported | -| Debian | 12 or later | x86-64, ARM64 (AArch64) | Fully supported | -| Amazon Linux 2 (AL2) | - | x86-64, ARM64 (AArch64) | Fully supported | -| Amazon Linux 2023 | - | x86-64, ARM64 (AArch64) | Fully supported | -| Talos | 1.6.7 or later | x86-64, ARM64 (AArch64) | Fully supported | +| Distribution | Version | Architecture | Support Level | +|--------------------------|-----------------|-------------------------|-----------------------------| +| Red Hat Enterprise Linux | 9 and later | x86-64, ARM64 (AArch64) | Fully supported | +| Rocky Linux | 9 and later | x86-64, ARM64 (AArch64) | Fully supported | +| Alma Linux | 9 and later | x86-64, ARM64 (AArch64) | Fully supported | +| Ubuntu | 22.04 and later | x86-64, ARM64 (AArch64) | Fully supported | +| Debian | 12 or later | x86-64, ARM64 (AArch64) | Fully supported | +| Amazon Linux 2 (AL2) | - | x86-64, ARM64 (AArch64) | Fully supported | +| Amazon Linux 2023 | - | x86-64, ARM64 (AArch64) | Fully supported | +| SUSE Linux Enterprise | 15 SP7 or later | x86-64, ARM64 (AArch64) | Fully supported | +| SUSE Linux Micro | 6.1 or later | x86-64, ARM64 (AArch64) | Fully supported | +| Talos | 1.6.7 or later | x86-64, ARM64 (AArch64) | Fully supported1 | + +1 Talos is an immutable distribution shipping a minimal kernel +module set. It requires a custom node image and additional preparation to load the required kernel modules and reserve +huge pages. See [Talos](../kubernetes/installation/talos.md) for details. ## Hosts (Initiators accessing Storage Cluster over NVMf) @@ -61,10 +70,10 @@ The following Linux distributions are considered tested and supported as NVMe-oF | Ubuntu | 20.04 | x86-64, ARM64 (AArch64) | Fully supported | | Ubuntu | 22.04 | x86-64, ARM64 (AArch64) | Fully supported | | Debian | 12 or later | x86-64, ARM64 (AArch64) | Fully supported | -| Amazon Linux 2 (AL2) | - | x86-64, ARM64 (AArch64) | Partially supported1 | -| Amazon Linux 2023 | - | x86-64, ARM64 (AArch64) | Partially supported1 | +| Amazon Linux 2 (AL2) | - | x86-64, ARM64 (AArch64) | Partially supported2 | +| Amazon Linux 2023 | - | x86-64, ARM64 (AArch64) | Partially supported2 | -1 Amazon Linux 2 and Amazon Linux 2023 have a bug with +2 Amazon Linux 2 and Amazon Linux 2023 have a bug with [NVMe over Fabrics Multipathing](../important-notes/terminology.md#multipathing). That means that NVMe over Fabrics on any Amazon Linux operates in a degraded state with the risk of connection outages. Alternatively, multipathing must be configured using the Linux Device Manager (dm) via DM-MPIO.