Skip to content

Commit f56154f

Browse files
authored
docs: publish a documentation site on GitHub Pages (#812)
## Summary ### Why? SubmitQueue's docs exist only as Markdown in the repo. A public site with a landing page, navigation and search (like cadenceworkflow.io) makes the project easier to discover and the RFCs easier to browse. ### What? - `tool/docsite/`: MkDocs Material config that renders `doc/` in place, so docs are not copied and the nav picks up new RFCs automatically. - Bazelified: MkDocs runs on a hermetic Python 3.13 toolchain with hash-locked deps (`pip.parse` hub `docsite_pip`; the default Python toolchain is unchanged). `//tool/docsite:site_test` runs the strict build, so `make test` and the required CI test job fail on broken links between docs. - `hooks/repo_links.py` points links that leave `doc/` (code, package READMEs, `AGENTS.md`) at GitHub, so `--strict` stays on and still fails on broken links between pages. - `hooks/nav_titles.py` titles the sections Guides and Design (RFCs) and puts Quickstart first. - `doc/index.md` + `overrides/home.html`: landing page with a hero, feature cards and a quickstart snippet; green brand theme with light and dark modes. - `.github/workflows/docs.yml`: builds the site on PRs that touch `doc/**` or `tool/docsite/**`, and deploys to https://uber.github.io/submitqueue/ from `main`. It is not a required check. - `make docs-serve` / `make docs-build` run MkDocs through Bazel; `docs-serve` binds a free localhost port, like the local service containers (`DOCS_PORT=` pins one); `bazel run //tool/docsite:requirements.update` regenerates the lock. Pages is already enabled on the repo (source: GitHub Actions, deploys restricted to `main`), so merging publishes the site. ## Screenshots <img width="1505" height="2000" alt="docsite-home-dark" src="https://github.com/user-attachments/assets/9e4278ed-e804-4695-b901-b4c485325fad" /> <img width="1689" height="2000" alt="docsite-home-light" src="https://github.com/user-attachments/assets/3fbe3ae2-0eab-459b-8cda-b3899bf5d2de" /> <img width="2000" height="1154" alt="docsite-rfc-dark" src="https://github.com/user-attachments/assets/e69358ff-e86f-44e2-ae5c-376eb276bafa" /> ## Test Plan ✅ `make docs-build` (strict) and `make docs-serve` work through Bazel ✅ `make test`: 133 tests pass, including `//tool/docsite:site_test`; a deliberately broken link makes it fail ✅ `make check-tidy`, `make gazelle` and `make fmt` leave the tree unchanged; zizmor 1.25.2 and actionlint clean on `docs.yml` ✅ Local preview: checked landing page, tabs, sidebar order, links into code, light and dark modes, and an 845px-wide window 🤖 Generated with [Claude Code](https://claude.com/claude-code)
1 parent d5752e6 commit f56154f

20 files changed

Lines changed: 1135 additions & 1 deletion

‎.github/workflows/docs.yml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: Docs
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
paths:
8+
- doc/**
9+
- tool/docsite/**
10+
- .github/workflows/docs.yml
11+
- MODULE.bazel
12+
pull_request:
13+
paths:
14+
- doc/**
15+
- tool/docsite/**
16+
- .github/workflows/docs.yml
17+
- MODULE.bazel
18+
workflow_dispatch:
19+
20+
permissions:
21+
contents: read
22+
23+
jobs:
24+
build:
25+
name: Build site
26+
runs-on: ubuntu-latest
27+
steps:
28+
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
29+
with:
30+
persist-credentials: false
31+
- uses: ./.github/actions/setup
32+
33+
- name: Build site
34+
run: make docs-build
35+
36+
- uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0
37+
with:
38+
path: tool/docsite/build
39+
40+
deploy:
41+
name: Deploy to GitHub Pages
42+
if: ${{ github.event_name != 'pull_request' && github.ref == 'refs/heads/main' }}
43+
needs: build
44+
runs-on: ubuntu-latest
45+
permissions:
46+
pages: write
47+
id-token: write
48+
environment:
49+
name: github-pages
50+
url: ${{ steps.deployment.outputs.page_url }}
51+
# Never cancel a deploy in progress; a newer run queues behind it.
52+
concurrency:
53+
group: pages
54+
cancel-in-progress: false
55+
steps:
56+
- id: deployment
57+
uses: actions/deploy-pages@368f82528645a54fb793d4d04e342629a3f51346 # v5.0.1

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,3 +23,7 @@ bin/
2323

2424
# Make completion cache
2525
.make_targets_cache
26+
27+
# Documentation site build output
28+
/tool/docsite/build/
29+
/tool/docsite/hooks/__pycache__/

‎.yamlfmt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,5 @@ formatter:
33
indent: 2
44
retain_line_breaks_single: true
55
include_document_start: false
6+
exclude:
7+
- tool/docsite/build

‎MODULE.bazel‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,3 +73,19 @@ use_repo(
7373
"org_uber_go_yarpc",
7474
"org_uber_go_zap",
7575
)
76+
77+
# Python toolchain and PyPI packages for the documentation site (//tool/docsite).
78+
# Its targets pin this version, so the default toolchain other Python targets use
79+
# is unchanged. Regenerate the lock with: bazel run //tool/docsite:requirements.update
80+
DOCSITE_PYTHON_VERSION = "3.13"
81+
82+
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
83+
python.toolchain(python_version = DOCSITE_PYTHON_VERSION)
84+
85+
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
86+
pip.parse(
87+
hub_name = "docsite_pip",
88+
python_version = DOCSITE_PYTHON_VERSION,
89+
requirements_lock = "//tool/docsite:requirements_lock.txt",
90+
)
91+
use_repo(pip, "docsite_pip")

‎Makefile‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,7 @@ define assert_clean
148148
fi
149149
endef
150150

151-
.PHONY: build build-all-linux build-runway-linux build-submitqueue-gateway-client build-submitqueue-gateway-linux build-submitqueue-gateway-server build-submitqueue-orchestrator-linux build-stovepipe-linux build-stovepipe-linux-debug check-gazelle check-mocks check-tidy clean clean-proto demo-requests deps e2e-test fmt gazelle integration-test integration-test-submitqueue-consumer integration-test-extensions integration-test-submitqueue-gateway integration-test-submitqueue-orchestrator license-fix lint lint-binary lint-fmt lint-license local-init-runway-queue-schema local-init-stovepipe-schemas local-runway-start local-runway-stop local-submitqueue-stop local-submitqueue-clean local-submitqueue-gateway-start local-submitqueue-gateway-stop local-init-submitqueue-schemas local-submitqueue-logs local-submitqueue-orchestrator-start local-submitqueue-orchestrator-stop local-submitqueue-ps local-submitqueue-restart local-submitqueue-start local-stop local-stovepipe-debug-start local-stovepipe-logs local-stovepipe-start local-stovepipe-stop mocks proto query-deps query-targets run-client-runway run-client-submitqueue-gateway run-client-submitqueue-orchestrator run-client-stovepipe run-queue-admin test test-no-cache test-race tidy tidy-bazel tidy-go help
151+
.PHONY: build build-all-linux build-runway-linux build-submitqueue-gateway-client build-submitqueue-gateway-linux build-submitqueue-gateway-server build-submitqueue-orchestrator-linux build-stovepipe-linux build-stovepipe-linux-debug check-gazelle check-mocks check-tidy clean clean-proto demo-requests deps docs-build docs-serve e2e-test fmt gazelle integration-test integration-test-submitqueue-consumer integration-test-extensions integration-test-submitqueue-gateway integration-test-submitqueue-orchestrator license-fix lint lint-binary lint-fmt lint-license local-init-runway-queue-schema local-init-stovepipe-schemas local-runway-start local-runway-stop local-submitqueue-stop local-submitqueue-clean local-submitqueue-gateway-start local-submitqueue-gateway-stop local-init-submitqueue-schemas local-submitqueue-logs local-submitqueue-orchestrator-start local-submitqueue-orchestrator-stop local-submitqueue-ps local-submitqueue-restart local-submitqueue-start local-stop local-stovepipe-debug-start local-stovepipe-logs local-stovepipe-start local-stovepipe-stop mocks proto query-deps query-targets run-client-runway run-client-submitqueue-gateway run-client-submitqueue-orchestrator run-client-stovepipe run-queue-admin test test-no-cache test-race tidy tidy-bazel tidy-go help
152152

153153

154154
build: ## Build all services and examples
@@ -263,6 +263,12 @@ demo-requests: ## Create N changes, enqueue each as it is created, and watch (PR
263263
deps: tidy-go ## Download and tidy Go dependencies
264264
@echo "Dependencies installed!"
265265

266+
docs-build: ## Build the documentation site into tool/docsite/build (fails on broken links)
267+
@$(BAZEL) run //tool/docsite:mkdocs -- build --strict --config-file $(CURDIR)/tool/docsite/mkdocs.yml --site-dir $(CURDIR)/tool/docsite/build
268+
269+
docs-serve: ## Serve the documentation site with live reload on a free localhost port (DOCS_PORT=8000 to pin one)
270+
@$(BAZEL) run //tool/docsite:mkdocs -- serve --config-file $(CURDIR)/tool/docsite/mkdocs.yml $(if $(DOCS_PORT),--dev-addr 127.0.0.1:$(DOCS_PORT))
271+
266272
e2e-git-test: ## Run the hermetic git E2E (real merger against a bare repo; no credentials)
267273
@echo "Running hermetic git end-to-end tests..."
268274
@$(BAZEL) test //test/e2e/submitqueue:go_default_test --test_output=errors \

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# SubmitQueue
22

33
[![CI](https://github.com/uber/submitqueue/actions/workflows/ci.yml/badge.svg)](https://github.com/uber/submitqueue/actions/workflows/ci.yml)
4+
[![Docs](https://img.shields.io/badge/docs-uber.github.io%2Fsubmitqueue-blue)](https://uber.github.io/submitqueue/)
45
[![Go Version](https://img.shields.io/github/go-mod/go-version/uber/submitqueue)](go.mod)
56
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
67
[![Slack](https://img.shields.io/badge/Slack-join%20the%20community-4A154B?logo=slack&logoColor=white)](https://join.slack.com/t/submitqueue/shared_invite/zt-46gkqj682-7zcQphxm2pYqkjDo9lbmYA)

‎doc/BUILD.bazel‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
filegroup(
2+
name = "all_files",
3+
srcs = glob(
4+
["**"],
5+
exclude = ["BUILD.bazel"],
6+
),
7+
visibility = ["//tool/docsite:__pkg__"],
8+
)

‎doc/index.md‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
---
2+
title: Home
3+
template: home.html
4+
hide:
5+
- navigation
6+
- toc
7+
---
8+
9+
<div class="sq-github-only" markdown>
10+
11+
# SubmitQueue
12+
13+
A high-performance speculative submission queue that keeps your trunk consistently green at scale.
14+
15+
</div>
16+
17+
SubmitQueue does not validate changes one at a time. It speculatively rebases and validates many changes in parallel against predicted future states of HEAD. Changes whose validations pass land automatically. When a validation fails, SubmitQueue isolates the offending change and retries the rest, with no human involved. It is designed for large monorepos and fast-moving teams, where concurrent changes can introduce subtle conflicts and destabilize builds.
18+
19+
## Why SubmitQueue
20+
21+
<div class="grid cards sq-features" markdown>
22+
23+
- :material-source-branch-check:{ .lg .middle } **Speculative validation**
24+
25+
---
26+
27+
SubmitQueue builds a tree of possible future HEADs and validates the paths most likely to land, in parallel.
28+
29+
[:octicons-arrow-right-24: Speculation](rfc/submitqueue/speculation.md)
30+
31+
- :material-shield-check-outline:{ .lg .middle } **Isolates failures**
32+
33+
---
34+
35+
A failing change is isolated and rejected while the rest of its batch carries on to land.
36+
37+
[:octicons-arrow-right-24: Orchestrator workflow](rfc/submitqueue/workflow.md)
38+
39+
- :material-puzzle-outline:{ .lg .middle } **Pluggable extensions**
40+
41+
---
42+
43+
Build runners, change providers, storage, queues and scorers are vendor-agnostic interfaces with swappable implementations.
44+
45+
[:octicons-arrow-right-24: Extension contract](rfc/submitqueue/extension-contract.md)
46+
47+
- :material-tray-full:{ .lg .middle } **Durable, queue-driven pipeline**
48+
49+
---
50+
51+
Every stage is an idempotent consumer on an at-least-once message queue, with optimistic locking and no distributed transactions.
52+
53+
[:octicons-arrow-right-24: SQL-based queue](rfc/sql-queue-rfc.md)
54+
55+
</div>
56+
57+
## Components
58+
59+
<div class="grid cards" markdown>
60+
61+
- :material-call-merge:{ .lg .middle } **SubmitQueue**
62+
63+
---
64+
65+
The gateway and orchestrator that accept, batch, speculate on, build and land changes.
66+
67+
[:octicons-arrow-right-24: Workflow](rfc/submitqueue/workflow.md)
68+
69+
- :material-airplane-landing:{ .lg .middle } **Runway**
70+
71+
---
72+
73+
The landing service. It owns VCS operations, conflict checks and merges, on SubmitQueue's behalf.
74+
75+
[:octicons-arrow-right-24: Workflow](rfc/runway/workflow.md)
76+
77+
- :material-check-decagram-outline:{ .lg .middle } **Stovepipe**
78+
79+
---
80+
81+
The post-land pipeline. It validates landed commits and tracks the last green revision.
82+
83+
[:octicons-arrow-right-24: Workflow](rfc/stovepipe/workflow.md)
84+
85+
</div>
86+
87+
## Try it in a minute
88+
89+
You need only Docker. No repository, account or token is required.
90+
91+
```bash
92+
make local-submitqueue-start # Gateway + Orchestrator + Runway + MySQL
93+
make demo-requests # create changes, enqueue them, watch them land
94+
make local-submitqueue-stop
95+
```
96+
97+
The [Quickstart](howto/QUICKSTART.md) goes from a fake provider to a local git repository to real GitHub pull requests. Questions? Join the [Slack community](https://join.slack.com/t/submitqueue/shared_invite/zt-46gkqj682-7zcQphxm2pYqkjDo9lbmYA).

‎tool/README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,10 @@ bazel build //...
2323

2424
The Bazel version is controlled by `.bazelversion` at the repository root. Update that file to change the Bazel version used by the wrapper.
2525

26+
## Documentation site
27+
28+
`tool/docsite` holds the MkDocs Material configuration that renders `doc/` as the site published at https://uber.github.io/submitqueue/. The site reads `doc/` in place; `doc/index.md` is its landing page. Links that leave `doc/` (code, package READMEs, `AGENTS.md`) are rewritten to GitHub by `hooks/repo_links.py`, so the strict build still fails on a broken link between pages. The nav is generated from the directory tree; `hooks/nav_titles.py` maps directory names to section titles (`howto` → Guides, `rfc` → Design (RFCs)), and `overrides/home.html` renders the landing-page hero. MkDocs runs under Bazel with a hermetic Python toolchain and locked dependencies: `make docs-serve` previews the site locally, `make docs-build` runs the strict build into `tool/docsite/build`, and `//tool/docsite:site_test` runs that strict build as part of `make test`. To change dependency versions, edit `requirements.txt` and run `bazel run //tool/docsite:requirements.update` to regenerate `requirements_lock.txt`. `.github/workflows/docs.yml` builds the site on pull requests and deploys it from `main`.
29+
2630
## Git sandbox
2731

2832
`tool/gitsandbox` creates the bare repository that `make local-submitqueue-start PROVIDER=git` merges into. It runs before the stack starts, because Runway clones that repository at boot and fails if the target does not already exist. The result is one seed commit on the target branch. Running it again leaves an existing repository unchanged, so a restart keeps commits that earlier runs landed. The Makefile invokes it; `bazel run //tool/gitsandbox -- -sandbox-dir <dir>` is the direct form.

‎tool/docsite/BUILD.bazel‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
load("@docsite_pip//:requirements.bzl", "requirement")
2+
load("@rules_python//python:defs.bzl", "py_binary", "py_test")
3+
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
4+
5+
DOCSITE_PYTHON_VERSION = "3.13"
6+
7+
compile_pip_requirements(
8+
name = "requirements",
9+
src = "requirements.txt",
10+
python_version = DOCSITE_PYTHON_VERSION,
11+
requirements_txt = "requirements_lock.txt",
12+
# The generated lock-freshness test downloads from PyPI; keep it out of //... runs.
13+
tags = ["manual"],
14+
)
15+
16+
filegroup(
17+
name = "site_sources",
18+
srcs = [
19+
"mkdocs.yml",
20+
"//doc:all_files",
21+
] + glob(
22+
[
23+
"hooks/*.py",
24+
"overrides/**",
25+
],
26+
),
27+
)
28+
29+
py_binary(
30+
name = "mkdocs",
31+
srcs = ["mkdocs_main.py"],
32+
legacy_create_init = 0,
33+
main = "mkdocs_main.py",
34+
python_version = DOCSITE_PYTHON_VERSION,
35+
deps = [
36+
requirement("mkdocs"),
37+
requirement("mkdocs-material"),
38+
],
39+
)
40+
41+
py_test(
42+
name = "site_test",
43+
size = "small",
44+
srcs = ["site_test.py"],
45+
data = [":site_sources"],
46+
legacy_create_init = 0,
47+
main = "site_test.py",
48+
python_version = DOCSITE_PYTHON_VERSION,
49+
deps = [
50+
requirement("mkdocs"),
51+
requirement("mkdocs-material"),
52+
],
53+
)

0 commit comments

Comments
 (0)