diff --git a/docs/en/networking/how_to/kube_ovn/configure_egress_gateway.mdx b/docs/en/networking/how_to/kube_ovn/configure_egress_gateway.mdx index a32a34b7b..3588fd6da 100644 --- a/docs/en/networking/how_to/kube_ovn/configure_egress_gateway.mdx +++ b/docs/en/networking/how_to/kube_ovn/configure_egress_gateway.mdx @@ -106,6 +106,11 @@ Before creating the gateway, prepare the following resources: - An external subnet - A Network Attachment Definition (NAD) for that subnet +:::note + The NetworkAttachmentDefinition name MUST NOT contain a dot (`.`). + Each NetworkAttachmentDefinition can be used by only one subnet. Do not reuse the same NetworkAttachmentDefinition across multiple subnets. +::: + The following example uses a Kube-OVN underlay subnet as the external network. :::note @@ -428,6 +433,44 @@ Routing Policies 29000 ip4.src == $subnet1.kube.ovn.worker_ip4 reroute 100.64.0.4 ``` +### Optional: Configure Bandwidth Limits + +To limit the bandwidth of selected namespaces or workloads, set _.spec.bandwidth.ingress_ and _.spec.bandwidth.egress_. +The values are in Mbps and apply to each gateway instance. + +:::note + If _.spec.replicas_ is greater than 1, each replica gets the configured limit. +::: + +The following example limits the selected namespaces to 100 Mbps ingress bandwidth and 200 Mbps egress bandwidth: + +```yaml +apiVersion: kubeovn.io/v1 +kind: VpcEgressGateway +metadata: + name: gateway-bandwidth + namespace: default +spec: + replicas: 1 + internalSubnet: ovn-default + externalSubnet: underlay-ext + externalIPs: + - 172.17.0.21 + bandwidth: + ingress: 100 + egress: 200 + selectors: + - namespaceSelector: + matchExpressions: + - key: kubernetes.io/metadata.name + operator: In + values: + - ecommerce + - ecommerce-test +``` + +In this example, the direction is relative to the selected namespaces or workloads: `ingress` limits traffic from the external network to them, and `egress` limits traffic from them to the external network. + ### Optional: Enable BFD-based High Availability BFD-based failover depends on the VPC BFD LRP. @@ -615,6 +658,25 @@ Session 1 If all gateway instances are down, egress traffic handled by the VPC Egress Gateway is dropped. ::: +## Validated Capacity Guidance + +The following results were validated on ACP 4.3.2 with Kube-OVN v1.15.13, two replicas for each VPC Egress Gateway, and bfdd CPU request/limit `50m/200m`. +Use them as sizing guidance, not as a hard product limit. + +:::note + The validated capacity is for reference only. Actual capacity can vary with hardware performance and cluster load. +::: + +| BFD setting | Validated scale | Recommendation | +| --- | --- | --- | +| `100ms / 100ms x2` | 100 gateways / 200 gateway Pods | Use only for smaller deployments that require aggressive detection. | +| `100ms / 100ms x5` | 150 gateways / 300 gateway Pods | Use when 100 ms probing is required and a larger multiplier is acceptable. | +| `200ms / 200ms x5` | 300 gateways / 600 gateway Pods | Suitable for medium scale. | +| `300ms / 300ms x5` | 400 gateways / 800 gateway Pods | Keep capacity headroom when using this setting. | +| `500ms / 500ms x3` | 500 gateways / 1000 gateway Pods | Recommended for 500-gateway deployments. | + +For dedicated Egress Gateway nodes with about 32 vCPUs and 96 GiB memory, the same `500ms / 500ms x3` BFD setting and bfdd `50m/200m` CPU request/limit validated up to about 200 gateway Pods per node. + ## Operations That May Interrupt Traffic The following operations may briefly interrupt egress traffic because they delete or recreate gateway instances: diff --git a/docs/en/networking/how_to/kube_ovn/multiple_networks.mdx b/docs/en/networking/how_to/kube_ovn/multiple_networks.mdx index 7fbf15e92..ad023ffa3 100644 --- a/docs/en/networking/how_to/kube_ovn/multiple_networks.mdx +++ b/docs/en/networking/how_to/kube_ovn/multiple_networks.mdx @@ -46,6 +46,9 @@ Example: `network-attachment-definition.yml` :::note The provider format in config is `..ovn`, where `` and `` are the name and namespace of this NetworkAttachmentDefinition CR respectively. +The NetworkAttachmentDefinition name MUST NOT contain a dot (`.`). +Each NetworkAttachmentDefinition can be used by only one subnet. Do not reuse the same NetworkAttachmentDefinition across multiple subnets. + If you use an underlay secondary network, set `spec.vlan` in the Subnet to the target VLAN CR name. ::: diff --git a/docs/en/networking/observability/operator-deployment.mdx b/docs/en/networking/observability/operator-deployment.mdx index 7ff4c07e5..61f6d19e2 100644 --- a/docs/en/networking/observability/operator-deployment.mdx +++ b/docs/en/networking/observability/operator-deployment.mdx @@ -78,15 +78,23 @@ metadata: spec: configuration: files: - disable-trace-log: | + 99-disable-system-logs.xml: | # [!code callout] - - - - - - - + + + + + + + + + + + + + + + clusters: - layout: @@ -98,7 +106,7 @@ spec: - 127.0.0.1 - 0.0.0.0/0 - ::/0 - clickhouse_root/password_sha256_hex: + clickhouse_root/password_sha256_hex: # [!code callout] defaults: templates: dataVolumeClaimTemplate: default @@ -152,6 +160,26 @@ spec: storage: 15Gi ``` + + +1. Additional ClickHouse configuration file for disabling system logs that are not required by NetObserv. The `99-` prefix makes the file load after the default configuration, and `remove="1"` removes the corresponding log configuration entries. +2. SHA-256 hash of the password for the ClickHouse user `clickhouse_root`. Generate the hash from the original plaintext password and replace `` with the command output. + + + +For example, to use `NetObserv@123` as the ClickHouse password, run: + +```bash +printf '%s' 'NetObserv@123' | sha256sum | awk '{print $1}' +``` + +Use the generated hash in `clickhouse_root/password_sha256_hex`. +When you create the `ck-auth` secret later, use `clickhouse_root` as the username and the original plaintext password as the password. + +If you want to use a different ClickHouse username, replace `clickhouse_root` in both `clickhouse_root/networks/ip` and `clickhouse_root/password_sha256_hex`. +For example, for a user named `netobserv_user`, use `netobserv_user/networks/ip` and `netobserv_user/password_sha256_hex`. +Then use the same username in the `ck-auth` secret. + After the ClickHouse instance is ready, get the service endpoint by inspecting field `.status.endpoint` of the ClickHouseInstallation instance. You can also run the following command to get the endpoint: @@ -190,10 +218,13 @@ This secret is referenced by `.spec.clickhouse.authSecret.name` in the FlowColle 1. Create a local file named _ck-auth.env_ with the following content: ```ini - username= - password= + username=clickhouse_root + password= ``` + If you use the ClickHouseInstallation example in this section, set `password` to the original plaintext password that was used to generate ``. + If you use a different ClickHouse username, set `username` to that value. + 2. Create the Kubernetes secret: ```bash @@ -281,4 +312,4 @@ spec: If you are using an existing ClickHouse server, make sure that the FlowCollector can reach the server address and that the referenced secret contains valid credentials. -After the FlowCollector instance is created, the eBPF agents and the flow logs pipeline are deployed to the namespace specified by `.spec.namespace`. \ No newline at end of file +After the FlowCollector instance is created, the eBPF agents and the flow logs pipeline are deployed to the namespace specified by `.spec.namespace`.