Add llms.txt for LLM support

[phmatray]

Description

Publish an llms.txt index at https://docs.fallout.build/llms.txt so an LLM can find Fallout's documentation without crawling and guessing.

The 37 pages under docs/website/ are the source of truth for docs.fallout.build, which is built by the separate Fallout-build/docs.fallout.build Docusaurus repo. Today a model answering a question about Fallout has three bad options: crawl the rendered HTML and spend most of its tokens on navigation chrome, fall back on what it learned about NUKE and get the namespaces and the tool name wrong, or guess. The rebrand makes the third option actively harmful, because a model's training data almost certainly holds Nuke.* and nuke :setup rather than Fallout.* and fallout :setup.

llms.txt is a small markdown file at the site root: a title, a one-line summary, and grouped lists of links with descriptions. It costs one fetch and gives a model an accurate map of what exists and where.

The file must not be hand-maintained. This repo already has the right pattern for a generated artifact: GenerateTools writes the .Generated.cs wrappers and VerifyGeneratedTools fails CI when a spec was edited without regenerating. llms.txt should work the same way, so a page added to docs/website/ cannot silently go missing from the index.

Usage Example

An AI assistant answering "how do I install Fallout" fetches one file:

GET https://docs.fallout.build/llms.txt

# Fallout

> A C#-first build automation framework for .NET, the hard-fork successor to NUKE.
> Write CI/CD pipelines in plain C#, debug them locally, and share build steps
> across repositories.

## Getting Started

- [Installation](https://docs.fallout.build/getting-started/installation): Install the Fallout global tool.
- [Build Setup](https://docs.fallout.build/getting-started/setup): Set up a build project in a repository.
- [Build Execution](https://docs.fallout.build/getting-started/execution): Run and debug targets locally.

## Fundamentals

- [Targets](https://docs.fallout.build/fundamentals/targets): Declare targets and dependencies.
...

It then fetches only the one page it needs, and answers with dotnet tool install Fallout.GlobalTool --global instead of the NUKE command it was trained on.

A contributor regenerates the file the same way they regenerate a tool wrapper:

./build.ps1 GenerateLlmsTxt

Alternative

Today there is none. A model has to crawl docs.fallout.build page by page, or answer from stale NUKE knowledge. Neither is reliable, and the second one is wrong in exactly the places the rebrand changed.

Hand-writing the file in the Docusaurus repo is the obvious shortcut, and it is what this issue argues against. See the Brainstorm below.

Could you help with a pull-request?

Yes

Related: #648 (AI-native build interface epic), #646 (MCP server), #240 (project-local AI tooling config), #104 (AI-friendly structured output modes)

These cover making the build engine machine-readable. This issue covers making the documentation machine-readable. They are neighbours, not the same job: nothing in #648's scope table would produce an llms.txt, and resolving #648 would not resolve this.

🧠 Brainstorm

Problem and context

• docs/website/ holds 37 markdown pages. Every one carries a title: in its frontmatter; some also carry a description:.
• Directory and file names are numbered for ordering (01-getting-started/01-installation.md). Docusaurus strips the NN- prefixes, so that page is served at https://docs.fallout.build/getting-started/installation. The README already links that exact URL, which confirms the mapping.
• The markdown lives here. The site that serves it lives in Fallout-build/docs.fallout.build. Whatever this issue produces has to cross that boundary.
• AGENTS.md says the build is itself a C# console app and is the canonical example of consuming the framework. Generating this file with a Fallout target dogfoods the framework rather than adding a second toolchain.

Approaches

A. Hand-write a static llms.txt in the Docusaurus repo.

Cheapest to start: one file, no build changes, done in an afternoon.

It rots on the first docs PR. The person adding a page to docs/website/ in this repo has no reason to open the site repo, so the index silently falls behind the docs it indexes. A stale map is worse than no map, because a model trusts it and then fetches a URL that moved. This is also the one option that contradicts the repo's own convention for generated artifacts.

B. Generate from docs/website/ frontmatter with a Fallout target, commit the result, gate it in CI.

Mirrors GenerateTools / VerifyGeneratedTools exactly, which is the pattern this repo already uses for exactly this problem. The generator reads the frontmatter that is already there, so it needs no new metadata and no new dependency. The artifact is committed, so it shows up in a diff and a reviewer can read what changed. VerifyLlmsTxt fails the build when someone edits a page without regenerating, which is the property option A cannot have.

Costs: a checked-in generated file, and one target more in the build. The site repo still needs a one-line change to serve the file at the root.

C. A Docusaurus plugin in the site repo that emits llms.txt at site build time.

Always matches what is actually published, and nothing is checked in.

The logic ends up in the repo that does not own the content, written in a different language, invisible to the people editing the docs. It cannot be reviewed in the same diff as the page that changed, and it makes the site repo, currently a thin renderer, into a place with its own behaviour to maintain. It also fails the AGENTS.md preference for dogfooding.

Recommendation

Option B.

The deciding argument is the drift, not the effort. All three options produce a correct file on day one, and only B is still correct on day one hundred. B is also the option that already has a working precedent in this repo, so it introduces a target and a gate that a reviewer here recognises on sight rather than a new mechanism they have to evaluate.

The known cost of B is the cross-repo hop: the generated file has to reach the site root. That is one static-asset copy in the Docusaurus repo and it is called out as a non-goal below, with a companion issue to file.

📋 Spec

Goal

https://docs.fallout.build/llms.txt serves a valid llmstxt.org index of every published page, regenerated by ./build.ps1 GenerateLlmsTxt and guarded by a CI gate that fails when the file is out of sync with docs/website/.

Scope

• A GenerateLlmsTxt target that walks docs/website/**/*.md and writes docs/llms.txt.
• A VerifyLlmsTxt target that fails when the committed file is stale, in the shape of VerifyGeneratedTools.
• The generated docs/llms.txt, committed.
• Wiring VerifyLlmsTxt into the CI build so the gate actually runs.

Non-goals

• llms-full.txt (the whole corpus inlined). Worth having, and cheap once the walk exists, but it is a different file with a different size budget. Separate issue.
• Serving the file. static/llms.txt in Fallout-build/docs.fallout.build is a one-line change in a repo this issue does not touch. File a companion issue there and link it; this issue is not done in the user-visible sense until that lands, and the PR description should say so.
• Indexing anything outside docs/website/. docs/adr/, docs/agents/ and AGENTS.md are contributor-facing and are not published on the site. Linking a URL that 404s is worse than omitting it.
• A new metadata field in the frontmatter. The generator uses what is there.

Pipeline

flowchart LR
    A["docs/website/**/*.md<br/>37 pages"] --> B["Parse frontmatter<br/>title, description"]
    B --> C["Derive URL<br/>strip NN- prefixes"]
    C --> D["Group by section<br/>from directory"]
    D --> E["Render llms.txt"]
    E --> F["docs/llms.txt<br/>committed"]
    F -. "companion issue" .-> G["docs.fallout.build<br/>static/llms.txt"]
    F --> H["VerifyLlmsTxt<br/>CI gate"]

Loading

URL derivation

Given a path relative to docs/website/, strip the NN- prefix from every segment, drop the .md extension, and prefix https://docs.fallout.build/.

Source path	URL
01-getting-started/01-installation.md	.../getting-started/installation
05-cicd/03-github-actions.md	.../cicd/github-actions
badge.md	.../badge

Section grouping and ordering

Sections come from the top-level directory, title-cased from the slug with the numeric prefix stripped: 01-getting-started becomes Getting Started. Sections and the pages inside them keep the numeric order the prefixes encode, so the file reads in the same order as the site sidebar. Root-level pages (badge.md) go in a trailing ## Optional section, which is the part of the llmstxt.org format meant for lower-priority links.

Page entries

- [{title}]({url}): {description} where title is the frontmatter title. When description is absent, use the page's first non-empty prose line, trimmed to roughly 200 characters. Skip Docusaurus import lines and MDX components when picking that line, since several pages open with one.

Header

The H1 is # Fallout. The blockquote summary reuses docs/website/introduction.md's frontmatter description, which already reads as a one-line pitch, so the summary has exactly one source and cannot disagree with the site.

Edge cases

• introduction.md has sidebar_position: 0 and may be served at / rather than /introduction. Confirm against the live site before generating, and special-case it if so.
• A page with no title: should fail the target loudly rather than emit a blank link. That is a docs bug worth surfacing.
• Line endings. The repo is on Windows with CRLF in the working tree. Write the file with the same convention the other generated artifacts use, or VerifyLlmsTxt will report drift on every clean checkout.

Assumptions

• docs.fallout.build serves static/ at the site root, which is the Docusaurus default.
• 37 pages produce a file in the low tens of kilobytes, well inside any practical context budget, so no truncation is needed.
• There is no test project for build/, so the verification model is the Generate plus Verify target pair this repo already uses for .Generated.cs, not a unit-test suite. The plan below is shaped accordingly: each task ends by running the target and reading the output, and the CI gate is the durable regression guard. Adding a tests/Build.Specs project to unit-test the generator would be a defensible alternative, but it is a bigger change than this issue warrants and it should be its own decision.

🛠️ Implementation plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal. Generate and CI-gate an llmstxt.org index of docs/website/, following the GenerateTools / VerifyGeneratedTools pattern.

Architecture. The build is a C# console app in build/. New targets go in a partial of Build in their own build/Build.*.cs file, matching Build.CodeGeneration.cs.

Tech stack. .NET 10 (global.json, rollForward: latestMinor), Fallout's own AbsolutePath / GlobFiles / Assert helpers.

Global constraints.

• No conventional commits. AGENTS.md rule 7 forbids feat:, fix:, chore: and the rest. Commit messages are functional descriptions, for example Generate an llms.txt index from the docs frontmatter.
• No per-file license headers (AGENTS.md rule 6).
• Terse plain English in commits and comments (AGENTS.md rule 8). One idea per sentence, no idioms.
• Central package versions only. If a dependency is needed, it goes in Directory.Packages.props with no inline Version=. Prefer no new dependency: the frontmatter here is simple enough to parse without a YAML library, and Fallout.Utilities.Text.Yaml already exists if it is not.
• Non-breaking, targets develop. Additive only, no public API change.
• Verify with ./build.ps1 Compile before each commit.

Task 1: Walk the docs and derive URLs

Files: create build/Build.Documentation.cs; no other file changes yet.

Interfaces: Target GenerateLlmsTxt; private helpers AbsolutePath DocsWebsiteDirectory, string ToPublicUrl(AbsolutePath page), string ToSectionTitle(string directorySlug).

• Step 1: Create build/Build.Documentation.cs as a partial of Build, with DocsWebsiteDirectory => RootDirectory / "docs" / "website".
• Step 2: Implement ToPublicUrl: strip the NN- prefix from each path segment, drop .md, prefix https://docs.fallout.build/.
• Step 3: Implement ToSectionTitle: strip the numeric prefix and title-case the slug, so 01-getting-started becomes Getting Started.
• Step 4: Add a temporary GenerateLlmsTxt target that globs the pages and logs each source path next to its derived URL.
• Step 5: Run ./build.ps1 GenerateLlmsTxt. Confirm 37 pages are found, and that 01-getting-started/01-installation.md maps to https://docs.fallout.build/getting-started/installation, which is the URL the README already links.
• Step 6: Spot-check three of the logged URLs against the live site, including one from 05-cicd/ and the root-level badge.md.
• Step 7: Commit: Walk the docs website and derive public URLs.

Task 2: Parse frontmatter into title and description

Files: modify build/Build.Documentation.cs.

Interfaces: sealed record DocPage(string Title, string Description, string Url, string Section, int Order); DocPage ReadPage(AbsolutePath file).

• Step 1: Implement ReadPage: read the leading --- frontmatter block and pull title and description.
• Step 2: Fall back to the first non-empty prose line when description is missing. Skip import lines and MDX component tags, since several pages open with one. Trim to roughly 200 characters on a word boundary.
• Step 3: Make a missing title fail the target with Assert, naming the offending file. A blank link is a docs bug worth surfacing.
• Step 4: Extend the target to log the parsed DocPage for every file.
• Step 5: Run ./build.ps1 GenerateLlmsTxt. Confirm every page has a non-empty title, and check the fallback description on a page that has no description:, for example 01-getting-started/01-installation.md.
• Step 6: Confirm the target fails with a useful message when a title: is temporarily removed from a page, then restore it.
• Step 7: Commit: Parse the docs frontmatter into page metadata.

Task 3: Render and write docs/llms.txt

Files: modify build/Build.Documentation.cs; create docs/llms.txt.

Interfaces: string RenderLlmsTxt(IReadOnlyList<DocPage> pages).

• Step 1: Render the header: # Fallout, then a blockquote from docs/website/introduction.md's frontmatter description.
• Step 2: Group pages by section, order sections and pages by their numeric prefixes, and emit ## {Section} followed by - [{title}]({url}): {description} lines.
• Step 3: Emit root-level pages such as badge.md under a trailing ## Optional section.
• Step 4: Write the result to docs/llms.txt, matching the line-ending convention the other generated artifacts use so the gate in Task 4 does not report drift on a clean checkout.
• Step 5: Run ./build.ps1 GenerateLlmsTxt. Read docs/llms.txt in full and confirm it is a valid llmstxt.org document: one H1, one blockquote, ## sections, one link line per page, 37 links.
• Step 6: Confirm introduction.md resolves to the URL the live site actually serves. If it is served at / rather than /introduction, special-case it and note the reason in a comment.
• Step 7: Run ./build.ps1 GenerateLlmsTxt a second time and confirm git status is clean, which proves the output is deterministic. A non-deterministic generator would make the Task 4 gate flap.
• Step 8: Commit: Generate an llms.txt index from the docs frontmatter, including the generated docs/llms.txt.

Task 4: Gate the generated file in CI

Files: modify build/Build.Documentation.cs; modify the CI target list wherever VerifyGeneratedTools is wired in.

Interfaces: Target VerifyLlmsTxt, with .Requires(() => GitHasCleanWorkingCopy()).DependsOn(GenerateLlmsTxt).

• Step 1: Add VerifyLlmsTxt modelled on VerifyGeneratedTools in build/Build.CodeGeneration.cs, including the same explanatory comment about why Requires fires before the plan runs.
• Step 2: Make the failure message point at the fix: docs/llms.txt is out of sync with docs/website. Run './build.ps1 GenerateLlmsTxt' locally and commit the result.
• Step 3: Find where VerifyGeneratedTools is invoked in CI and wire VerifyLlmsTxt in the same place, so the gate actually runs on a PR.
• Step 4: Run ./build.ps1 VerifyLlmsTxt on a clean tree and confirm it passes.
• Step 5: Edit a page's title:, re-run ./build.ps1 VerifyLlmsTxt, and confirm it fails with the message from Step 2. Restore the page.
• Step 6: Run ./build.ps1 Compile and confirm exit 0 with no new warnings.
• Step 7: Commit: Fail the build when llms.txt is out of sync with the docs.

Task 5: Hand off the serving half

Files: none in this repo.

• Step 1: Confirm Fallout-build/docs.fallout.build serves static/ at the site root, which is the Docusaurus default.
• Step 2: File a companion issue on Fallout-build/docs.fallout.build: copy docs/llms.txt from this repo into static/llms.txt during the site build, so the file is served at https://docs.fallout.build/llms.txt.
• Step 3: Link that issue here, and say plainly in the PR description that this repo's half generates the file while the site repo's half serves it. Neither half is user-visible alone.
• Step 4: Once both have landed, fetch https://docs.fallout.build/llms.txt and confirm it serves as text/plain or text/markdown and matches the committed file.

[phmatray]

Couldn't apply labels from my account (no write access on this repo). Suggested: enhancement, documentation, target/vCurrent.

Note that half of this lands in Fallout-build/docs.fallout.build: this repo generates docs/llms.txt, the site repo serves it at static/llms.txt. Neither half is user-visible alone. I'll file the companion issue there once the shape here is agreed.

[phmatray]

Serving half filed: Fallout-build/docs.fallout.build#14.

This repo generates docs/llms.txt (#661); that repo copies it to static/llms.txt during the site build. https://docs.fallout.build/llms.txt stays 404 until both land, which is why the last box on Task 5 is left unchecked here.