Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 61 additions & 32 deletions BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ change and compares.
This document is the reference for what to run, when to run it, and
how to compare results.

Production cross-framework measurements live in the standalone
[microsoft/webui-benchmarks](https://github.com/microsoft/webui-benchmarks)
repository. This document covers the implementation benchmarks maintained with
WebUI itself.

## Quick reference

| Bench | Layer | Wall time | What it measures | Use when |
Expand All @@ -24,8 +29,8 @@ how to compare results.

## The before/after workflow

All benches support **named baselines**. The flag pattern is
identical across criterion, example, and Playwright benches:
Criterion, example, and Playwright benches support **named before/after
baselines**:

```bash
# 1. Snapshot current numbers as 'before'
Expand All @@ -44,8 +49,7 @@ Baselines are stored at `target/bench-baselines/`:
* `browser-<name>.json` — browser metrics table
* `browser-lazy-hydration-<name>.json` — offscreen rendering/hydration matrix
* `node-addon-<name>.json` — Node/V8/N-API latency table
* `target/criterion/<bench>/<name>` — criterion's native baseline
directory tree
* `target/criterion/<bench>/<name>` — criterion's native baseline directory tree

The compare phase prints a Δ%-table for every row. Negative Δ% =
improvement; positive = regression.
Expand All @@ -71,12 +75,37 @@ cargo xtask bench all --save-baseline before
The target list lives in `CRITERION_BENCHES` in `xtask/src/main.rs`. Add new
`benches/*.rs` harnesses there so `bench all` and its baselines pick them up.

### Updating the homepage benchmark snapshot

The docs homepage consumes only
`docs/.webui-press/state/benchmark-summary.json`. From the
`microsoft/webui-benchmarks` repository root, export the validated compact DTO
after matching progressive SSR, complete SSR, and headed browser runs:

```bash
pnpm run export:summary -- \
--input results/ssr-todo-<official-name>.json \
--complete-input results/ssr-todo-<official-name>-complete.json \
--browser-input results/ssr-todo-browser-outcomes-<official-name>.json \
--output results/benchmark-summary.json
```

Every `results/` path above belongs to `microsoft/webui-benchmarks`. The
exporter rejects quick, incomplete, noncanonical, or mismatched captures and
publishes exactly five selectors in this order: **No Streaming RPS**,
**Streaming RPS**, **LCP**, **JS Heap**, and **Renderer Private MB**. **No
Streaming RPS** is selected by default. Copy only the generated
`results/benchmark-summary.json` to
`webui/docs/.webui-press/state/benchmark-summary.json`. Keep full raw captures
and machine evidence in `webui-benchmarks` or its CI artifacts; do not copy
them into the WebUI documentation tree.

### Threshold guidance

| Source | Treat as noise | Treat as signal |
|---|---|---|
| criterion (well-isolated wall-clock) | < ±2% | > ±5% |
| streaming-resource (alloc count) | exact any change matters | any non-zero |
| streaming-resource (alloc count) | exact; any change matters | any non-zero |
| streaming-resource (bytes, CPU) | < ±2% | > ±5% |
| streaming-e2e-ttfb (loopback) | < ±10% | > ±20% |
| streaming-browser (real Chromium) | < ±5% | > ±15% |
Expand All @@ -95,10 +124,10 @@ Standard criterion harnesses. Each crate has its own `benches/` dir:
fused-streaming, and async-resumable split-path comparisons
* `crates/webui-expressions/benches/expressions_bench.rs`
* `crates/webui-state/benches/state_bench.rs`
* `crates/webui/benches/contact_book_bench.rs` end-to-end render
* `crates/webui/benches/streaming_bench.rs` writer-path wall-clock + TTFB
* `crates/webui/benches/component_assets_bench.rs` static asset graph rendering
* `crates/webui/benches/server_request_bench.rs` router-aware full HTML and JSON requests
* `crates/webui/benches/contact_book_bench.rs`: end-to-end render
* `crates/webui/benches/streaming_bench.rs`: writer-path wall-clock + TTFB
* `crates/webui/benches/component_assets_bench.rs`: static asset graph rendering
* `crates/webui/benches/server_request_bench.rs`: router-aware full HTML and JSON requests

These integrate with criterion's HTML reports
(`target/criterion/report/index.html`) and native baseline support
Expand All @@ -116,11 +145,11 @@ run a clean process where every `alloc` we observe came from the code
under test (or its dependencies).

Reports per (path × scale):
- **allocs/run** — exact count from the custom allocator
- **bytes/run** — exact bytes requested from the allocator
- **wall µs/run** — `Instant::elapsed()` per iteration
- **user µs/run** — `getrusage(RUSAGE_SELF).ru_utime` delta
- **process RSS** — `ru_maxrss` high-water mark
- **allocs/run:** exact count from the custom allocator
- **bytes/run:** exact bytes requested from the allocator
- **wall µs/run:** `Instant::elapsed()` per iteration
- **user µs/run:** `getrusage(RUSAGE_SELF).ru_utime` delta
- **process RSS:** `ru_maxrss` high-water mark

This is the **only** bench in the suite that gives you exact
allocation numbers. Use it to verify "zero per-write allocation"
Expand All @@ -145,11 +174,11 @@ with its own actix server and a Playwright spec that drives Chromium
through `PerformanceObserver`. Reports the **only** browser-perceived
metrics in the suite:

- **TTFB** — `responseStart - requestStart` from `PerformanceNavigationTiming`
- **FCP** — first-contentful-paint from `PerformanceObserver`
- **LCP** — largest-contentful-paint from `PerformanceObserver`
- **DCL** — `domContentLoadedEventEnd - startTime`
- **load** — `loadEventEnd - startTime`
- **TTFB:** `responseStart - requestStart` from `PerformanceNavigationTiming`
- **FCP:** first-contentful-paint from `PerformanceObserver`
- **LCP:** largest-contentful-paint from `PerformanceObserver`
- **DCL:** `domContentLoadedEventEnd - startTime`
- **load:** `loadEventEnd - startTime`

This is the bench that answers "does streaming actually help users
see the page faster?" The HTTP-level benches prove the bytes get to
Expand All @@ -167,14 +196,14 @@ that loads the release `microsoft-webui-node` artifact through the
public `@microsoft/webui` API. Unlike a Rust benchmark, it includes
V8/N-API string and callback crossings:

- **Protocol construction** — Node `Buffer` to protobuf decode/index after the addon is loaded
- **JSON-string render** — N-API conversion, JSON parse, Rust render,
- **Protocol construction:** Node `Buffer` to protobuf decode/index after the addon is loaded
- **JSON-string render:** N-API conversion, JSON parse, Rust render,
and the returned UTF-8 Node `Buffer`
- **Object render** — the same path plus public-wrapper
- **Object render:** the same path plus public-wrapper
`JSON.stringify`
- **Streaming first callback** — state conversion and JSON parse,
- **Streaming first callback:** state conversion and JSON parse,
followed by rendering until JavaScript receives the first 16 KiB chunk
- **Streaming total** — all callback crossings and complete render
- **Streaming total:** all callback crossings and complete render

It uses the same Contact Book fixture and 10/100/1000 scales as the
Rust end-to-end benchmark, rendering `/contacts` so output grows with
Expand Down Expand Up @@ -246,10 +275,10 @@ noise threshold`).

## Where the data lives

* **Stdout** — every bench prints a human-readable table.
* **JSON snapshots** — non-criterion benches write to
* **Stdout:** every bench prints a human-readable table.
* **JSON snapshots:** non-criterion benches write to
`target/bench-baselines/`.
* **Criterion HTML** — `target/criterion/report/index.html` for full
* **Criterion HTML:** `target/criterion/report/index.html` for full
PDF/CDF plots and per-baseline violin plots.

## Why so many benches?
Expand All @@ -269,14 +298,14 @@ catches one third of them.

## Reproducibility tips

* **Close other applications** — CPU-intensive background work adds
* **Close other applications:** CPU-intensive background work adds
noise.
* **Plug in to power** (laptops) battery savers throttle the CPU.
* **Pin to release builds** — `cargo bench` and `cargo xtask bench`
* **Plug in to power** (laptops): battery savers throttle the CPU.
* **Pin to release builds:** `cargo bench` and `cargo xtask bench`
always use release; debug builds are not representative.
* **Run on the same machine** — cross-machine baselines are not
* **Run on the same machine:** cross-machine baselines are not
meaningful.
* **Compare medians (P50)**, not means robust against thermal
* **Compare medians (P50)**, not means; medians are robust against thermal
spikes.
* **Re-run if Dev% > 15%** in any criterion row.

Expand Down
11 changes: 0 additions & 11 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ members = [
"crates/*",
"xtask",
"examples/integration/rust",
"examples/integration/ssr-performance-showdown",
"examples/integration/streaming-browser-bench/server",
"examples/app/commerce/server",
"examples/demo/server",
Expand Down
4 changes: 2 additions & 2 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -4980,13 +4980,13 @@ WebUI SSR marker formats are:
| Raw HTML start | `<!--wN-->` | Opens raw range `N` owned by a triple-brace binding |
| Raw HTML end | `<!--/wN-->` | Closes the same raw range `N` |

The WebUI handler plugin emits these seven comment marker roles. Escaped text bindings, attribute bindings, and event handlers are resolved from compiled pre-order element indices at hydration time - no DOM attribute markers are needed. Raw HTML is the exception because its rendered value can contain any number of top-level nodes and therefore needs explicit ownership boundaries. Raw markers carry a decimal pair identifier so adjacent bindings cannot claim each other's ranges. Exact `<!--wN-->` / `<!--/wN-->` comments are framework-reserved and trusted raw HTML must not emit a marker matching its surrounding range. During hydration the framework keeps `<!--wr-->` as the repeat anchor, keeps `<!--wc-->` only for an absent conditional body, retains `<!--wN-->` / `<!--/wN-->` for sibling-safe reactive replacement, and removes visible-condition starts, `<!--/wr-->`, `<!--/wc-->`, and `<!--wi-->`. A visible conditional creates an empty anchor only if it later becomes absent, and removes that anchor when content is restored.
The WebUI handler plugin emits these seven comment marker roles. Escaped text bindings, attribute bindings, and event handlers are resolved from compiled pre-order element indices at hydration time - no DOM attribute markers are needed. The handler only emits structural markers in active child scopes; the root page scope remains marker-free. Raw HTML is the exception because its rendered value can contain any number of top-level nodes and therefore needs explicit ownership boundaries. Raw markers carry a decimal pair identifier so adjacent bindings cannot claim each other's ranges. Exact `<!--wN-->` / `<!--/wN-->` comments are framework-reserved and trusted raw HTML must not emit a marker matching its surrounding range. During hydration the framework keeps `<!--wr-->` as the repeat anchor, keeps `<!--wc-->` only for an absent conditional body, retains `<!--wN-->` / `<!--/wN-->` for sibling-safe reactive replacement, and removes visible-condition starts, `<!--/wr-->`, `<!--/wc-->`, and `<!--wi-->`. A visible conditional creates an empty anchor only if it later becomes absent, and removes that anchor when content is restored.

WebUI Framework hydration assumes the SSR DOM, hydration markers, and compiled metadata were generated by the same trusted WebUI compiler/handler version. Hand-authored or partially modified marker streams are unsupported; missing structural closing markers are invalid input, not a recoverable runtime condition.

### Runtime contract

`@microsoft/webui-framework` consumes the metadata object above plus the SSR markers emitted by `WebUIHydrationPlugin`. This follows an Islands Architecture approach: the server delivers fully-rendered HTML, authored Web Components hydrate on startup or explicitly opt into visibility-driven activation, and compiler-owned scriptless hosts remain dormant until browser code actually writes state.
`@microsoft/webui-framework` consumes the metadata object above plus the SSR markers emitted by `WebUIHydrationPlugin`. This follows an Islands Architecture approach: the server delivers fully-rendered HTML, authored Web Components hydrate on startup or explicitly opt into visibility-driven activation, and compiler-owned scriptless hosts remain dormant until browser code actually writes state. An empty compiler-owned template still registers its tag for soft navigation but uses a minimal `HTMLElement` host rather than allocating dormant `TemplateElement` state.

- SSR hydration performs one pre-order walk per component that pairs each template element with the server-rendered element it hydrates and collects structural markers and raw HTML ranges in document order. Because compiler metadata and server output share source order, each block and raw range is unambiguous. Bindings then resolve by lookup rather than by rescanning, keeping hydration linear in subtree size instead of proportional to bindings times sibling count. The walk skips complete conditional, repeat, and raw HTML ranges - their rendered elements are not static children owned by the enclosing section - and stops at child components, which contribute no children to the parent's `h`. `<!--wi-->`, structural closing markers, and starts for visible conditions are removed afterwards; raw HTML boundaries remain for targeted updates.
- Reactive triple-brace updates delete only the nodes between the binding's retained
Expand Down
1 change: 1 addition & 0 deletions crates/webui/benches/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ For the cross-bench picture and recommended workflow, see
| `cargo xtask bench streaming-resource` | run the resource-counting example |
| `cargo xtask bench streaming-e2e-ttfb` | run the HTTP-level TTFB example |
| `cargo xtask bench streaming-browser` | run the Playwright browser-metrics test |
| `cargo xtask bench lazy-hydration` | run the Chromium offscreen work matrix |
| `cargo xtask bench node-addon` | run the Node/V8/N-API addon benchmark |
| `cargo xtask bench full` | run all four streaming-related benches in sequence |
| `cargo xtask bench all` | run every criterion bench in the workspace |
Expand Down
Loading
Loading