Skip to content

performance-testing.md

Latest commit

 

History

History
154 lines (119 loc) · 4.83 KB

performance-testing.md

File metadata and controls

154 lines (119 loc) · 4.83 KB
Raw

FerroGate Performance Testing

This document defines the manual performance-report workflow for high-load gateway validation. The regular CI job keeps fast smoke coverage; this workflow is for dedicated load-test hosts because 10k connections and 20k-100k RPS can exhaust developer laptops or shared CI runners.

What It Measures

The report workflow covers the scenarios that matter for gateway capacity planning:

  • 10k open connections against a FerroGate HTTP endpoint.
  • Staged target request rates: 20k, 50k, and 100k requests per second.
  • Five-minute sustained stages for stability observation.
  • FerroGate process CPU and RSS sampling during the full run.
  • Host network RX/TX throughput sampling during the full run.
  • Per-second latency curves with p50, p95, p99, and max latency.
  • Ready-to-share SVG charts embedded in the generated Markdown summary.
  • Raw Vegeta result files for deeper post-processing.

Required Tools

Install these on the load-test host:

vegeta version
jq --version
awk --version

Raise file descriptor limits before opening 10k connections:

ulimit -n 1048576

On Linux load-test machines, also review ephemeral port and backlog settings before running 50k-100k RPS from a single host:

sysctl net.ipv4.ip_local_port_range
sysctl net.ipv4.tcp_tw_reuse
sysctl net.core.somaxconn

Reverse Proxy Report

Start a local upstream service, start FerroGate with a route to that upstream, then run:

scripts/perf-gateway-report.sh \
  --url http://127.0.0.1:8088/local/proxy-check \
  --pid "$(pgrep -n ferrogate)" \
  --net-device lo \
  --connections 10000 \
  --rates 20000,50000,100000 \
  --duration 5m

For a lightweight validation of the script itself:

scripts/perf-gateway-report.sh \
  --url http://127.0.0.1:8088/local/proxy-check \
  --rates 10,25 \
  --connections 16 \
  --duration 10s

AI Gateway Report

Create a request body:

cat > /tmp/ferrogate-chat-body.json <<'JSON'
{"model":"fast-chat","messages":[{"role":"user","content":"hello"}]}
JSON

Run the staged report:

scripts/perf-gateway-report.sh \
  --url http://127.0.0.1:8080/v1/chat/completions \
  --method POST \
  --header 'Authorization: Bearer dev-secret' \
  --header 'Content-Type: application/json' \
  --body-file /tmp/ferrogate-chat-body.json \
  --pid "$(pgrep -n ferrogate)" \
  --net-device lo \
  --connections 10000 \
  --rates 20000,50000,100000 \
  --duration 5m

Use a local provider stub for this test. Do not point this profile at paid external model providers unless the budget impact is intentional.

Report Artifacts

By default reports are written under perf-reports/<UTC timestamp>/.

Committed report snapshots live under docs/performance-reports/. The local workstation baseline for the first date-based release is available at docs/performance-reports/v2026.05.05-local/. The local 100k-connection stress attempt and client-port limit finding report is available at docs/performance-reports/v2026.05.05-local-100k/.

Important files:

  • summary.md: human-readable stage summary.
  • overview.svg: target RPS vs achieved RPS with p99 labels.
  • resource-usage.svg: CPU and RSS curves.
  • network-io.svg: RX/TX Mbps curves for the selected network device.
  • process-metrics.csv: timestamped CPU, RSS, RX, and TX samples.
  • stages.csv: stage index with paths to generated artifacts.
  • rps-*.aggregate.json: aggregate Vegeta metrics for each stage.
  • rps-*.latency-curve.csv: per-second p50/p95/p99/max latency curve.
  • rps-*.latency.svg: per-stage latency curve image.
  • rps-*.histogram.txt: latency histogram.
  • rps-*.plot.html: Vegeta HTML plot.
  • rps-*.bin: raw Vegeta binary results.

Interpreting Results

Treat a run as suspicious when:

  • Achieved RPS is materially below target while CPU is saturated.
  • Success rate drops below 1.0.
  • p99 or max latency trends upward across the 5-minute stage.
  • RSS grows monotonically and does not stabilize after warmup.
  • CPU climbs over time at a fixed RPS without a matching throughput increase.
  • RX/TX throughput diverges unexpectedly from achieved RPS or keeps rising after traffic stops.

Open summary.md first. It embeds overview.svg, resource-usage.svg, network-io.svg, and each per-stage rps-*.latency.svg. Use the latency CSVs when you need to feed the same data into external dashboards. A stable stage should show mostly flat p95/p99 after warmup. A rising p99 curve under fixed RPS is a stronger regression signal than a single aggregate p99 number.

Why This Is Not In CI

The repository CI still runs:

cargo test -p ferrogate-cli --test runtime_perf --test ai_proxy_perf -- --nocapture

Those tests are fast performance smoke checks. The 10k-connection, 100k RPS, five-minute report is intentionally manual because it requires dedicated host capacity, kernel tuning, and a controlled network path.