Skip to content

feat: agent ready docs#410

Merged
lazarv merged 1 commit into
mainfrom
feat/agent-ready-docs
Apr 28, 2026
Merged

feat: agent ready docs#410
lazarv merged 1 commit into
mainfrom
feat/agent-ready-docs

Conversation

@lazarv
Copy link
Copy Markdown
Owner

@lazarv lazarv commented Apr 28, 2026

Summary

Brings the docs site to the public agent-readiness bar (level 0 → level 5 on isitagentready.com) and ships the runtime support that other react-server applications need to do the same. Closes the gap that any pre-rendered react-server site has today: there is no way for an agent asking for Accept: text/markdown on the canonical URL to receive markdown, because the static layer always wins.

The work splits cleanly into three buckets. The docs site grows a set of agent-facing surfaces (well-known endpoints, link headers, content negotiation, an MCP server). The runtime grows the primitives those surfaces depend on (Accept-aware static deferral across every adapter, header propagation across short-circuiting middlewares, a generic fix for rolldown's CJS-imports-JSON wrapping bug). The same primitives let any other application following this PR's patterns reach the same readiness level with a few hundred lines of glue.

Runtime changes

Accept-aware static deferral

Every adapter in @lazarv/react-server previously gave static files unconditional priority over the worker. That makes browsers fast — and silently breaks content negotiation, because middleware never runs for paths whose pre-rendered HTML happens to match. This PR adds a shared shouldDeferToServer(request) / isHtmlRoute(url) pair under adapters/shared/accept.mjs and applies them at every static-first site so that browser navigation and asset traffic keep the fast path while agent traffic with Accept: text/markdown (or any concrete non-HTML media type) flows to the worker.

The Cloudflare adapter checks both predicates before calling env.ASSETS.fetch. The Node-mode static handler in lib/handlers/static.mjs now defers when it would have served text/html and the client clearly prefers something else — covering Bun, Deno, Docker, Azure Functions, AWS Lambda, Firebase, and the singlefile adapter at once. The Vercel adapter gains a has-conditioned route that runs before { handle: "filesystem" } and routes bare paths whose Accept header omits text/html to the function. The Netlify adapter drops the unconditional preferStatic: true from the function config so the in-process static handler can do the per-request decision; users who don't need content negotiation can opt back into the fast path via adapterOptions.functions.config.preferStatic = true. AWS Lambda's tryServeStatic and Docker's bespoke static-first server gain the same isHtmlRoute && shouldDeferToServer guard.

The deferral predicate is deliberately narrow: it requires the URL to look like an HTML route (no extension, or .html/.htm) and the client to explicitly prefer a concrete non-HTML media type at higher q-value than text/html and */*. Browsers always list text/html and never trigger it; image/CSS/JSON requests are excluded by the URL-extension filter; curl with the default */* is treated as "anything is fine" and gets the static reply.

Header propagation across short-circuiting middlewares

react-server now supports running multiple middlewares in sequence — for example, an agent-discovery middleware that sets a Link header followed by a content-negotiation middleware that returns a Response directly. The old behaviour silently dropped headers set on the HTTP context whenever a later middleware short-circuited. A new shared helper at lib/http/middleware-response.mjs#mergeContextHeaders is now invoked from both lib/start/ssr-handler.mjs and lib/dev/ssr-handler.mjs and merges setHeader / appendHeader / headers() output onto the returned Response (the Response's own headers win on conflict, so middlewares that explicitly set Content-Type or Cache-Control retain authority).

Generic fix for the rolldown CJS-imports-JSON wrapping bug

@lazarv/react-server/mcp pulls in @modelcontextprotocol/sdk, which transitively pulls in Express's CJS dependency tree (statuses, mime-types, finalhandler, http-errors). Several of those packages do var data = require('./codes.json') and then iterate Object.keys(data).forEach(k => data[k].toLowerCase()). The bundler's CJS-from-ESM interop wraps imported JSON as { __esModule: true, default: <getter> } and never unwraps it back, so the iteration crashes on the boolean __esModule with r.toLowerCase is not a function. The bug reproduces in examples/mcp on main and is the reason any react-server project trying to host an MCP server has been broken in production.

Docs site changes

Discovery surface

/robots.txt is now a proper file with a User-agent: * block, a Content-Signal: search=yes, ai-input=yes, ai-train=yes directive, and the sitemap. As an open-source documentation site, react-server.dev wants to be indexed and used by AI tools — the signal is intentionally permissive.

A new (agent-discovery).middleware.mjs serves four well-known endpoints with the right content types: /.well-known/api-catalog (RFC 9727 linkset format), /.well-known/agent-skills/index.json (Agent Skills v0.2 index), /.well-known/agent-skills/react-server/SKILL.md (the canonical skill body, imported via ?raw from the monorepo's skills/react-server/SKILL.md), and /.well-known/mcp/server-card.json. The same middleware sets RFC 8288 Link headers on every documentation page advertising the api-catalog, the MCP endpoint, llms.txt, the sitemap, and the agent-skills index.

A new (content-negotiation).middleware.mjs handles Accept: text/markdown by rewriting requests to the existing /md/[...slug] route. The homepage has no /md/ slug entry, so it returns the canonical llms.txt summary as text/markdown instead — that is the right "what is this site" answer for an agent landing at /. Vary: Accept is set on the response so HTML and markdown caches don't poison each other.

The original (i18n).middleware.mjs was reduced back to pure locale resolution. With three middlewares now living side by side (alphabetically ordered: agent-discovery, content-negotiation, i18n), each file does one thing.

MCP server and version source-of-truth

The docs site now hosts a live MCP server at /mcp, dogfooding @lazarv/react-server/mcp. It exposes a search_docs tool (free-text query against the page index), a read_doc tool (fetches any docs page as markdown via the public URL — works on every runtime including Cloudflare Workers without filesystem access), a templated docs-page resource that lists every page, and an explain-topic prompt that orchestrates the two tools.

A new docs/src/version.mjs strips the react-server/ prefix off the package's namespaced version export, giving callers a clean semver. The /mcp server, the MCP server card, and the agent-skills index now all read from this single source — there is no longer a hardcoded 1.0.0 to drift out of sync with the package.

WebMCP browser tools

A "use client" WebMCP.jsx component mounts once in the root layout and registers search_docs and get_docs_page via navigator.modelContext.registerTool. Any in-browser agent (Claude, Cursor, ChatGPT Atlas, Cloudflare Browser-Use) interacting with a docs page can now search the docs and fetch any page as markdown without scraping HTML.

@cloudflare-workers-and-pages
Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
react-server-docs 8391d19 Apr 28 2026, 11:10 PM

@github-actions
Copy link
Copy Markdown

⚡ Flight Protocol Benchmark

Commit: fb7f4a0

Serialization (renderToReadableStream)

Scenario @lazarv/rsc webpack vs webpack
react: minimal element 264.3K 45.8K 🟢 +477.6%
react: shallow wide (1000) 2.7K 447 🟢 +506.5%
react: deep nested (100) 21.8K 7.9K 🟢 +176.9%
react: product list (50) 8.4K 2.6K 🟢 +227.7%
react: large table (500x10) 341 106 🟢 +222.5%
data: primitives 227.9K 49.6K 🟢 +359.9%
data: large string (100KB) 9.6K 8.8K 🟢 +9.6%
data: nested objects (20) 79.1K 35.3K 🟢 +123.9%
data: large array (10K) 151 158 🔴 -4.6%
data: Map & Set 14.7K 7.6K 🟢 +92.9%
data: Date/BigInt/Symbol 212.1K 46.9K 🟢 +351.9%
data: typed arrays 46.5K 16.7K 🟢 +178.4%
data: mixed payload 11.9K 5.2K 🟢 +130.4%

Prerender (prerender)

Scenario @lazarv/rsc ops/s mean
react: minimal element 310.0K 3.2 µs
react: shallow wide (1000) 2.6K 381.3 µs
react: deep nested (100) 20.1K 49.9 µs
react: product list (50) 7.8K 127.8 µs
react: large table (500x10) 337 2.97 ms
data: primitives 241.9K 4.1 µs
data: large string (100KB) 845 1.18 ms
data: nested objects (20) 78.3K 12.8 µs
data: large array (10K) 156 6.41 ms
data: Map & Set 15.4K 64.9 µs
data: Date/BigInt/Symbol 234.8K 4.3 µs
data: typed arrays 851 1.17 ms
data: mixed payload 10.8K 92.9 µs

Deserialization (createFromReadableStream)

Scenario @lazarv/rsc webpack vs webpack
react: minimal element 220.6K 194.9K 🟢 +13.2%
react: shallow wide (1000) 24.1K 2.1K 🟢 +1070.5%
react: deep nested (100) 119.8K 25.5K 🟢 +369.9%
react: product list (50) 61.3K 16.4K 🟢 +273.4%
react: large table (500x10) 4.8K 2.8K 🟢 +73.1%
data: primitives 174.4K 169.9K 🟢 +2.7%
data: large string (100KB) 46.5K 46.9K ⚪ -0.9%
data: nested objects (20) 104.6K 91.8K 🟢 +13.9%
data: large array (10K) 363 320 🟢 +13.5%
data: Map & Set 21.4K 19.2K 🟢 +11.1%
data: Date/BigInt/Symbol 177.8K 155.7K 🟢 +14.2%
data: typed arrays 74.3K 52.9K 🟢 +40.4%
data: mixed payload 28.8K 19.4K 🟢 +48.9%

Roundtrip (serialize + deserialize)

Scenario @lazarv/rsc webpack vs webpack
react: minimal element 157.8K 34.4K 🟢 +358.5%
react: shallow wide (1000) 2.4K 365 🟢 +550.7%
react: deep nested (100) 19.2K 5.8K 🟢 +233.7%
react: product list (50) 7.4K 2.2K 🟢 +238.4%
react: large table (500x10) 334 101 🟢 +230.8%
data: primitives 122.1K 47.7K 🟢 +156.1%
data: large string (100KB) 8.5K 8.9K 🔴 -4.6%
data: nested objects (20) 49.6K 29.1K 🟢 +70.3%
data: large array (10K) 110 109 ⚪ +1.0%
data: Map & Set 9.0K 5.7K 🟢 +56.7%
data: Date/BigInt/Symbol 119.9K 44.9K 🟢 +167.2%
data: typed arrays 36.8K 14.7K 🟢 +150.4%
data: mixed payload 8.7K 4.3K 🟢 +100.0%
Legend & methodology

Indicators: 🟢 > 1% faster | 🔴 > 1% slower | ⚪ within noise margin

vs webpack: compares @lazarv/rsc against react-server-dom-webpack within the same run.
vs baseline: compares @lazarv/rsc against the previous main branch run.

Values shown are operations/second (higher is better). Each scenario runs for at least 100 iterations with warmup.

Benchmarks run on GitHub Actions runners (shared infrastructure) — expect ~5% variance between runs. Consistent directional changes across multiple scenarios are more meaningful than any single number.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 28, 2026

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
956 1 955 19
View the top 3 failed test(s) by shortest run time
__test__/deno.spec.mjs > create-react-server: deno runtime (npm) > preset: nextjs > starts in production mode
Stack Traces | 0.00051s run time
AssertionError: production start should work: expected false to be true // Object.is equality

- Expected
+ Received

- true
+ false

 ❯ __test__/deno.spec.mjs:74:66
__test__/deno.spec.mjs > create-react-server: deno runtime (npm) > preset: nextjs > builds the app
Stack Traces | 0.000726s run time
AssertionError: build should succeed: expected false to be true // Object.is equality

- Expected
+ Received

- true
+ false

 ❯ __test__/deno.spec.mjs:70:58
__test__/deno.spec.mjs > create-react-server: deno runtime (npm) > preset: router > starts in production mode
Stack Traces | 0.00126s run time
AssertionError: production start should work: expected false to be true // Object.is equality

- Expected
+ Received

- true
+ false

 ❯ __test__/deno.spec.mjs:74:66
__test__/deno.spec.mjs > create-react-server: deno runtime (npm) > preset: router > builds the app
Stack Traces | 0.00926s run time
AssertionError: build should succeed: expected false to be true // Object.is equality

- Expected
+ Received

- true
+ false

 ❯ __test__/deno.spec.mjs:70:58
__test__/scroll-restoration.spec.mjs > scroll restoration: back navigation restores scroll position
Stack Traces | 24.3s run time
AssertionError: expected 0 to be greater than 700
 ❯ __test__/scroll-restoration.spec.mjs:126:21
__test__/scroll-restoration.spec.mjs > scroll restoration: multiple back/forward preserves positions
Stack Traces | 27.6s run time
AssertionError: expected 0 to be greater than 400
 ❯ __test__/scroll-restoration.spec.mjs:158:25

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@github-actions
Copy link
Copy Markdown

⚡ Benchmark Results

PR 8391d19 main 5353c12
Config 50 connections, 10s/test 50 connections, 10s/test
Benchmark Req/s vs main Avg Latency vs main P99 Latency Throughput
minimal 1445 🟢 +2.7% 33.98 ms 🟢 -2.8% 73 ms 1.0 MB/s
small 1469 🟢 +2.9% 33.41 ms 🟢 -2.6% 58 ms 1.5 MB/s
medium 416 🟢 +3.5% 118.16 ms 🟢 -3.0% 192 ms 6.2 MB/s
large 47 🔴 -3.7% 995.83 ms 🔴 +4.4% 1891 ms 4.7 MB/s
deep 932 ⚪ -0.5% 52.94 ms ⚪ +0.4% 90 ms 3.2 MB/s
wide 72 🟢 +2.6% 672.51 ms 🟢 -1.8% 1255 ms 4.0 MB/s
cached 3606 🟢 +1.1% 13.33 ms 🟢 -1.1% 26 ms 53.1 MB/s
client-min 545 🟢 +3.5% 90.68 ms 🟢 -2.8% 146 ms 2.4 MB/s
client-small 560 ⚪ -0.1% 88.16 ms ⚪ -0.3% 138 ms 2.6 MB/s
client-med 406 ⚪ -0.2% 122.41 ms ⚪ +0.8% 178 ms 7.5 MB/s
client-large 93 🟢 +4.0% 510.41 ms 🟢 -5.3% 856 ms 9.8 MB/s
client-deep 536 🟢 +4.9% 92.46 ms 🟢 -4.4% 135 ms 3.8 MB/s
client-wide 150 🔴 -1.2% 325.52 ms 🔴 +1.6% 548 ms 8.8 MB/s
rsc-client-large 1242 🟢 +1.1% 39.61 ms 🟢 -1.1% 62 ms 3.2 MB/s
rsc-client-wide 1233 🔴 -1.5% 39.91 ms 🔴 +1.4% 60 ms 3.2 MB/s
static-json 10532 🔴 -1.5% 4.18 ms 🔴 +5.0% 14 ms 4.4 MB/s
static-js 10589 🟢 +1.0% 4.29 ms ⚪ -0.7% 14 ms 6.3 MB/s
404-miss 5571 🟢 +1.8% 8.49 ms 🟢 -3.9% 18 ms 0.7 MB/s
hybrid-min 564 🟢 +5.6% 87.23 ms 🟢 -5.9% 133 ms 2.7 MB/s
hybrid-small 528 🟢 +1.9% 93.91 ms 🟢 -1.7% 150 ms 3.1 MB/s
hybrid-medium 266 🟢 +2.7% 186.36 ms 🟢 -2.4% 262 ms 11.4 MB/s
hybrid-large 41 🔴 -7.6% 1130.56 ms 🔴 +5.4% 2065 ms 13.2 MB/s
hybrid-deep 410 ⚪ -0.2% 120.67 ms ⚪ -0.1% 170 ms 5.7 MB/s
hybrid-wide 63 🔴 -4.1% 756.5 ms 🔴 +3.0% 1258 ms 12.3 MB/s
hybrid-cached 2847 🔴 -4.5% 16.96 ms 🔴 +4.4% 32 ms 121.2 MB/s
hybrid-client-min 574 🟢 +2.2% 86.23 ms 🟢 -2.0% 130 ms 2.6 MB/s
hybrid-client-small 572 🟢 +3.8% 86.67 ms 🟢 -3.5% 130 ms 2.8 MB/s
hybrid-client-medium 414 ⚪ +0.7% 119.04 ms ⚪ -0.4% 175 ms 7.7 MB/s
hybrid-client-large 91 🟢 +3.8% 539.16 ms 🟢 -1.4% 1006 ms 9.6 MB/s
hybrid-client-deep 525 🟢 +3.1% 94.06 ms 🟢 -3.1% 155 ms 3.8 MB/s
hybrid-client-wide 157 🟢 +2.1% 315.07 ms 🟢 -1.1% 563 ms 9.2 MB/s
Legend

🟢 > 1% improvement | 🔴 > 1% regression | ⚪ within noise margin

Benchmarks run on GitHub Actions runners (shared infrastructure) — expect ~5% variance between runs. Consistent directional changes across multiple routes are more meaningful than any single number.

@lazarv lazarv merged commit 19706b5 into main Apr 28, 2026
153 of 156 checks passed
@lazarv lazarv deleted the feat/agent-ready-docs branch April 28, 2026 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants