Skip to content

docs: update NetObserv and Kube-OVN networking guidance #877

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zhangzujian wants to merge 1 commit into
base: master
Choose a base branch
from
Open

docs: update NetObserv and Kube-OVN networking guidance #877

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions docs/en/networking/how_to/kube_ovn/configure_egress_gateway.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,11 @@ Before creating the gateway, prepare the following resources:
- An external subnet
- A Network Attachment Definition (NAD) for that subnet

:::note
The NetworkAttachmentDefinition name MUST NOT contain a dot (`.`).
Each NetworkAttachmentDefinition can be used by only one subnet. Do not reuse the same NetworkAttachmentDefinition across multiple subnets.
:::

The following example uses a Kube-OVN underlay subnet as the external network.

:::note
Expand Down Expand Up @@ -428,6 +433,44 @@ Routing Policies
29000 ip4.src == $subnet1.kube.ovn.worker_ip4 reroute 100.64.0.4
```

### Optional: Configure Bandwidth Limits

To limit the bandwidth of selected namespaces or workloads, set _.spec.bandwidth.ingress_ and _.spec.bandwidth.egress_.
The values are in Mbps and apply to each gateway instance.

:::note
If _.spec.replicas_ is greater than 1, each replica gets the configured limit.
:::

The following example limits the selected namespaces to 100 Mbps ingress bandwidth and 200 Mbps egress bandwidth:

```yaml
apiVersion: kubeovn.io/v1
kind: VpcEgressGateway
metadata:
name: gateway-bandwidth
namespace: default
spec:
replicas: 1
internalSubnet: ovn-default
externalSubnet: underlay-ext
externalIPs:
- 172.17.0.21
bandwidth:
ingress: 100
egress: 200
selectors:
- namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values:
- ecommerce
- ecommerce-test
```

In this example, the direction is relative to the selected namespaces or workloads: `ingress` limits traffic from the external network to them, and `egress` limits traffic from them to the external network.

### Optional: Enable BFD-based High Availability

BFD-based failover depends on the VPC BFD LRP.
Expand Down Expand Up @@ -615,6 +658,25 @@ Session 1
If all gateway instances are down, egress traffic handled by the VPC Egress Gateway is dropped.
:::

## Validated Capacity Guidance

The following results were validated on ACP 4.3.2 with Kube-OVN v1.15.13, two replicas for each VPC Egress Gateway, and bfdd CPU request/limit `50m/200m`.
Use them as sizing guidance, not as a hard product limit.

:::note
The validated capacity is for reference only. Actual capacity can vary with hardware performance and cluster load.
:::

| BFD setting | Validated scale | Recommendation |
| --- | --- | --- |
| `100ms / 100ms x2` | 100 gateways / 200 gateway Pods | Use only for smaller deployments that require aggressive detection. |
| `100ms / 100ms x5` | 150 gateways / 300 gateway Pods | Use when 100 ms probing is required and a larger multiplier is acceptable. |
| `200ms / 200ms x5` | 300 gateways / 600 gateway Pods | Suitable for medium scale. |
| `300ms / 300ms x5` | 400 gateways / 800 gateway Pods | Keep capacity headroom when using this setting. |
| `500ms / 500ms x3` | 500 gateways / 1000 gateway Pods | Recommended for 500-gateway deployments. |

For dedicated Egress Gateway nodes with about 32 vCPUs and 96 GiB memory, the same `500ms / 500ms x3` BFD setting and bfdd `50m/200m` CPU request/limit validated up to about 200 gateway Pods per node.

## Operations That May Interrupt Traffic

The following operations may briefly interrupt egress traffic because they delete or recreate gateway instances:
Expand Down
3 changes: 3 additions & 0 deletions docs/en/networking/how_to/kube_ovn/multiple_networks.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ Example: `network-attachment-definition.yml`
:::note
The provider format in config is `<NAME>.<NAMESPACE>.ovn`, where `<NAME>` and `<NAMESPACE>` are the name and namespace of this NetworkAttachmentDefinition CR respectively.

The NetworkAttachmentDefinition name MUST NOT contain a dot (`.`).
Each NetworkAttachmentDefinition can be used by only one subnet. Do not reuse the same NetworkAttachmentDefinition across multiple subnets.

If you use an underlay secondary network, set `spec.vlan` in the Subnet to the target VLAN CR name.
:::

Expand Down
55 changes: 43 additions & 12 deletions docs/en/networking/observability/operator-deployment.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,15 +78,23 @@ metadata:
spec:
configuration:
files:
disable-trace-log: |
99-disable-system-logs.xml: | # [!code callout]
<clickhouse>
<trace_log remove="remove"/>
<query_thread_log remove="remove"/>
<query_log remove="remove"/>
<text_log remove="remove"/>
<metric_log remove="remove"/>
<asynchronous_metric_log remove="remove"/>
<part_log remove="remove"/>
<trace_log remove="1"/>
<text_log remove="1"/>
<query_log remove="1"/>
<query_thread_log remove="1"/>
<part_log remove="1"/>
<metric_log remove="1"/>
<asynchronous_metric_log remove="1"/>
<processors_profile_log remove="1"/>
<query_metric_log remove="1"/>
<error_log remove="1"/>
<crash_log remove="1"/>
<opentelemetry_span_log remove="1"/>
<session_log remove="1"/>
<transactions_info_log remove="1"/>
<backup_log remove="1"/>
</clickhouse>
clusters:
- layout:
Expand All @@ -98,7 +106,7 @@ spec:
- 127.0.0.1
- 0.0.0.0/0
- ::/0
clickhouse_root/password_sha256_hex: <SHA256_HEX_PASSWORD>
clickhouse_root/password_sha256_hex: <SHA256_HEX_PASSWORD> # [!code callout]
defaults:
templates:
dataVolumeClaimTemplate: default
Expand Down Expand Up @@ -152,6 +160,26 @@ spec:
storage: 15Gi
```

<Callouts>

1. Additional ClickHouse configuration file for disabling system logs that are not required by NetObserv. The `99-` prefix makes the file load after the default configuration, and `remove="1"` removes the corresponding log configuration entries.
2. SHA-256 hash of the password for the ClickHouse user `clickhouse_root`. Generate the hash from the original plaintext password and replace `<SHA256_HEX_PASSWORD>` with the command output.

</Callouts>

For example, to use `NetObserv@123` as the ClickHouse password, run:

```bash
printf '%s' 'NetObserv@123' | sha256sum | awk '{print $1}'
```

Use the generated hash in `clickhouse_root/password_sha256_hex`.
When you create the `ck-auth` secret later, use `clickhouse_root` as the username and the original plaintext password as the password.

If you want to use a different ClickHouse username, replace `clickhouse_root` in both `clickhouse_root/networks/ip` and `clickhouse_root/password_sha256_hex`.
For example, for a user named `netobserv_user`, use `netobserv_user/networks/ip` and `netobserv_user/password_sha256_hex`.
Then use the same username in the `ck-auth` secret.

After the ClickHouse instance is ready, get the service endpoint by inspecting field `.status.endpoint` of the ClickHouseInstallation instance.
You can also run the following command to get the endpoint:

Expand Down Expand Up @@ -190,10 +218,13 @@ This secret is referenced by `.spec.clickhouse.authSecret.name` in the FlowColle
1. Create a local file named _ck-auth.env_ with the following content:

```ini
username=<USERNAME>
password=<PASSWORD>
username=clickhouse_root
password=<PLAINTEXT_PASSWORD>
```

If you use the ClickHouseInstallation example in this section, set `password` to the original plaintext password that was used to generate `<SHA256_HEX_PASSWORD>`.
If you use a different ClickHouse username, set `username` to that value.

2. Create the Kubernetes secret:

```bash
Expand Down Expand Up @@ -281,4 +312,4 @@ spec:

If you are using an existing ClickHouse server, make sure that the FlowCollector can reach the server address and that the referenced secret contains valid credentials.

After the FlowCollector instance is created, the eBPF agents and the flow logs pipeline are deployed to the namespace specified by `.spec.namespace`.
After the FlowCollector instance is created, the eBPF agents and the flow logs pipeline are deployed to the namespace specified by `.spec.namespace`.