Skip to content

Enable static-tls on all frameworks that both support static and json-tls - #1108

Merged
MDA2AV merged 6 commits into
MDA2AV:mainfrom
Kaliumhexacyanoferrat:static-tls-multi
Aug 13, 2026
Merged

Enable static-tls on all frameworks that both support static and json-tls#1108
MDA2AV merged 6 commits into
MDA2AV:mainfrom
Kaliumhexacyanoferrat:static-tls-multi

Conversation

@Kaliumhexacyanoferrat

Copy link
Copy Markdown
Collaborator

Description


PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run every test the framework subscribes to
/benchmark -f <framework> -t <test> Run one test only
/benchmark -f <framework> --save Run and save results (updates the leaderboard on merge)
/benchmark -f <framework> -t <test> --save Run one test and save results
/benchmark -f <framework> --compare <other> Measure the deltas against another framework instead of this one
/benchmark-multiple -f <fw1>,<fw2>,... Benchmark several frameworks in one run — takes -t and --save too; saved results land in a single commit
/benchmark-multiple --save No -f needed: benchmark and save every framework the PR touches
/benchmark-test -t <test> Benchmark all enabled frameworks subscribed to <test> and save the results

For /benchmark, always specify -f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory — one table per framework on multi runs. A new benchmark comment while a run is in flight queues behind it (one deep) instead of cancelling it. For multi-framework PRs (dependency bumps, same-language refactors) prefer /benchmark-multiple, which runs everything in a single job and commits all saved results together, so no run overwrites another. --compare works on single-framework runs only.

What the deltas are measured against. By default, this framework's own results published on main - answering "did this change help?". When you are tuning a variant or a successor entry, --compare re-bases them on another entry instead:

/benchmark -f genhttp-11 --compare genhttp

The reply states which baseline it used, and profiles the other framework does not run show n/a rather than a delta.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@Kaliumhexacyanoferrat

Copy link
Copy Markdown
Collaborator Author

/benchmark-multiple -t static-tls --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Frameworks: 31 | Test: static-tls

actix

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 55,882 6220.4% 4.6GiB NEW NEW
static-tls 4096 41,672 5858.6% 6.2GiB NEW NEW
static-tls 6800 47,327 5505.0% 11.0GiB NEW NEW

aleph

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 92,285 6471.1% 17.8GiB NEW NEW
static-tls 4096 84,868 6469.3% 18.2GiB NEW NEW
static-tls 6800 81,174 6449.0% 17.7GiB NEW NEW

aspnet-minimal

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 243,329 5212.8% 462MiB NEW NEW
static-tls 4096 255,222 6065.0% 922MiB NEW NEW
static-tls 6800 251,639 6195.6% 1.2GiB NEW NEW

blackbull

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 169,440 6400.0% 1.8GiB NEW NEW
static-tls 4096 164,491 6295.3% 3.0GiB NEW NEW
static-tls 6800 160,996 6234.0% 4.1GiB NEW NEW

bottle

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 9,110 2775.9% 2.6GiB NEW NEW
static-tls 4096 8,567 2477.5% 2.5GiB NEW NEW
static-tls 6800 7,930 2181.5% 2.5GiB NEW NEW

fastapi

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 23,393 6572.2% 5.2GiB NEW NEW
static-tls 4096 21,410 6482.2% 6.1GiB NEW NEW
static-tls 6800 20,124 6717.6% 7.7GiB NEW NEW

fastendpoints

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 253,173 5471.5% 475MiB NEW NEW
static-tls 4096 258,863 5992.3% 884MiB NEW NEW
static-tls 6800 254,583 6094.9% 1.2GiB NEW NEW

fastpysgi-asgi

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 655,490 6549.8% 580MiB NEW NEW
static-tls 4096 517,407 6552.0% 1.9GiB NEW NEW
static-tls 6800 469,239 6547.9% 3.0GiB NEW NEW

fastpysgi-wsgi

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 855,993 6482.6% 547MiB NEW NEW
static-tls 4096 555,960 6433.5% 1.9GiB NEW NEW
static-tls 6800 514,313 6417.0% 5.1GiB NEW NEW

fishcake

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 349,722 4874.3% 16.8GiB NEW NEW
static-tls 4096 324,823 4833.9% 16.8GiB NEW NEW
static-tls 6800 317,492 5003.5% 16.5GiB NEW NEW

fletch

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 158,730 6462.5% 3.5GiB NEW NEW
static-tls 4096 147,832 6469.8% 3.7GiB NEW NEW
static-tls 6800 141,565 6417.7% 3.9GiB NEW NEW

genhttp-11-kestrel

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 336,798 5650.6% 570MiB NEW NEW
static-tls 4096 334,065 6197.1% 955MiB NEW NEW
static-tls 6800 351,794 6026.3% 1.4GiB NEW NEW

genhttp-kestrel

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 199,349 6200.9% 602MiB NEW NEW
static-tls 4096 188,161 6085.1% 1.1GiB NEW NEW
static-tls 6800 186,526 6146.5% 1.5GiB NEW NEW

ktor

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 204,709 6407.0% 3.8GiB NEW NEW
static-tls 4096 238,340 6060.9% 5.0GiB NEW NEW
static-tls 6800 226,856 5788.5% 6.0GiB NEW NEW

ktor-ghost

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 191,799 6430.3% 3.3GiB NEW NEW
static-tls 4096 238,995 6151.4% 5.3GiB NEW NEW
static-tls 6800 229,300 5873.5% 5.8GiB NEW NEW

mq-bridge

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 415,105 6547.0% 255MiB NEW NEW
static-tls 4096 407,866 6436.1% 800MiB NEW NEW
static-tls 6800 404,021 6523.1% 1.3GiB NEW NEW

mq-bridge-py

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 298,762 6633.1% 3.8GiB NEW NEW
static-tls 4096 316,050 6495.2% 7.5GiB NEW NEW
static-tls 6800 301,755 6438.6% 10.6GiB NEW NEW

pyronova

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 214,118 6466.3% 2.3GiB NEW NEW
static-tls 4096 193,498 6410.1% 4.6GiB NEW NEW
static-tls 6800 187,432 6416.6% 6.2GiB NEW NEW

rails

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 103,733 6529.3% 10.9GiB NEW NEW
static-tls 4096 99,197 6455.9% 11.6GiB NEW NEW
static-tls 6800 99,183 6420.1% 11.7GiB NEW NEW

ring-http-exchange

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 190,477 6390.8% 5.0GiB NEW NEW
static-tls 4096 192,014 6341.7% 5.2GiB NEW NEW
static-tls 6800 184,649 6426.8% 12.2GiB NEW NEW

roadrunner

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 87,911 4700.0% 368MiB NEW NEW
static-tls 4096 28,174 2615.6% 290MiB NEW NEW
static-tls 6800 16,693 1769.8% 268MiB NEW NEW

roda

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 115,566 6554.4% 6.5GiB NEW NEW
static-tls 4096 113,709 6553.8% 9.5GiB NEW NEW
static-tls 6800 108,212 6547.1% 9.3GiB NEW NEW

servicestack

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 257,085 5299.1% 470MiB NEW NEW
static-tls 4096 259,789 5995.7% 857MiB NEW NEW
static-tls 6800 261,910 5839.3% 1.3GiB NEW NEW

simplew

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 66,475 4748.3% 1.1GiB NEW NEW
static-tls 4096 68,310 5243.6% 2.0GiB NEW NEW
static-tls 6800 68,328 5240.3% 2.9GiB NEW NEW

sinatra

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 21,773 6508.7% 7.2GiB NEW NEW
static-tls 4096 21,485 6453.9% 7.4GiB NEW NEW
static-tls 6800 21,187 6516.4% 7.4GiB NEW NEW

spring-boot

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 143,403 4835.5% 3.5GiB NEW NEW
static-tls 4096 134,714 4573.0% 4.3GiB NEW NEW
static-tls 6800 127,705 4554.6% 4.3GiB NEW NEW

trillium

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 25,319 3095.0% 366MiB NEW NEW
static-tls 4096 22,942 4244.9% 497MiB NEW NEW
static-tls 6800 19,470 2442.6% 722MiB NEW NEW

trillium-tuned

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 576,602 6426.9% 361MiB NEW NEW
static-tls 4096 520,694 6340.1% 550MiB NEW NEW
static-tls 6800 523,155 6325.0% 749MiB NEW NEW

true-async-server

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 683,087 5984.9% 378MiB NEW NEW
static-tls 4096 523,669 5903.4% 818MiB NEW NEW
static-tls 6800 479,908 5795.2% 1.2GiB NEW NEW

userver

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 156,652 6583.0% 588MiB NEW NEW
static-tls 4096 164,141 6401.9% 1.3GiB NEW NEW
static-tls 6800 161,969 6413.9% 1.8GiB NEW NEW

uvicorn

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 407,217 6457.2% 3.4GiB NEW NEW
static-tls 4096 344,477 6372.9% 3.9GiB NEW NEW
static-tls 6800 320,764 6253.9% 4.3GiB NEW NEW

@Kaliumhexacyanoferrat

Copy link
Copy Markdown
Collaborator Author

@MDA2AV ready when you are

@MDA2AV
MDA2AV merged commit 669e97e into MDA2AV:main Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants