Skip to content

docs: enforce Sphinx -W in CI (Sphinx cleanup PR 3)#413

Merged
igerber merged 4 commits into
mainfrom
docs/sphinx-warnings-as-errors
May 11, 2026
Merged

docs: enforce Sphinx -W in CI (Sphinx cleanup PR 3)#413
igerber merged 4 commits into
mainfrom
docs/sphinx-warnings-as-errors

Conversation

@igerber
Copy link
Copy Markdown
Owner

@igerber igerber commented May 10, 2026

Summary

Adds a `sphinx-build` job to `.github/workflows/docs-tests.yml` that runs `make -C docs html SPHINXOPTS="-W"` so any Sphinx warning becomes a build failure. This locks in the 0-warning state PR #410 achieved across the 3-PR Sphinx cleanup wave (PR #405: 86 ERRORs → 0; PR #407: 2,170 → 635 WARNINGs; PR #410: 635 → 0).

The job mirrors the RTD build at `.readthedocs.yaml`: Python 3.11, `pandoc` apt package for nbsphinx, and the same docs extras pinned in `pyproject.toml [project.optional-dependencies.docs]`. Skips the maturin/Rust build (docs/conf.py imports diff_diff via sys.path from checked-out source, matching RTD).

Also adds `tutorials/21_had_pretest_workflow` to the `Tutorials: Business Applications` toctree at `docs/index.rst:84`. PR #409 landed this notebook without a toctree entry; the local smoke test of `-W` caught it before push.

Methodology references

  • N/A - CI workflow + one-line toctree addition; no methodology changes.

Validation

  • Tests added/updated: No test changes. New `sphinx-build` job validates the docs build itself.
  • Backtest / simulation / notebook evidence: Local smoke `make -C docs html SPHINXOPTS="-W"` succeeded (initial run failed only on the T21 toctree gap, fixed in this PR).

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

@github-actions
Copy link
Copy Markdown

Overall Assessment

Looks good — no unmitigated P0/P1 findings. One non-blocking P2 CI coverage gap is noted.

Executive Summary

  • No estimator, math, variance/SE, weighting, assumptions, or defaults are changed.
  • Methodology registry check found the linked T21 tutorial is already documented as shipped in PR Tutorial 21: HAD pre-test workflow (composite QUG + Stute + Yatchew) #409.
  • The new Sphinx job matches the RTD Python version, pandoc dependency, and docs/runtime dependency set.
  • The toctree entry points to an existing notebook path.
  • P2: .readthedocs.yaml is a mirrored docs-build surface but is not included in docs-tests.yml path filters.

Methodology

Finding: No methodology defect
Severity: P3 informational
Impact: The PR changes docs CI and navigation only. docs/methodology/REGISTRY.md:L2556 already documents docs/tutorials/21_had_pretest_workflow.ipynb; no new estimator behavior or paper-adherence issue is introduced.
Concrete fix: None.

Code Quality

Finding: No code quality defect
Severity: P3 informational
Impact: The workflow is straightforward and consistent with the existing ready-for-CI gating pattern.
Concrete fix: None.

Performance

Finding: No performance defect
Severity: P3 informational
Impact: The added Sphinx job will add CI time, but that is expected for enforcing warning-free docs.
Concrete fix: None.

Maintainability

Finding: No blocking maintainability defect
Severity: P3 informational
Impact: The install command is duplicated between .github/workflows/docs-tests.yml:L85-L95 and .readthedocs.yaml:L13-L21, but this is intentional in the PR and mirrors the existing RTD setup.
Concrete fix: Optional future cleanup only: centralize docs dependency installation if drift becomes frequent.

Tech Debt

Finding: No new untracked blocker
Severity: P3 informational
Impact: No new deferred correctness or methodology debt is introduced.
Concrete fix: None.

Security

Finding: No security issue found
Severity: P3 informational
Impact: No secrets, credentials, or elevated token permissions are introduced; workflow permissions remain contents: read.
Concrete fix: None.

Documentation/Tests

Finding: RTD config-only changes will not trigger the new docs build
Severity: P2
Impact: The new Sphinx job explicitly mirrors .readthedocs.yaml, but the workflow path filters include docs/**, diff_diff/**, pyproject.toml, and the workflow file only; .readthedocs.yaml is omitted. A future PR changing only RTD Python version, apt packages, or post-install docs dependencies would skip this GitHub docs-build guard. See .github/workflows/docs-tests.yml:L6-L26 and .readthedocs.yaml:L6-L25.
Concrete fix: Add .readthedocs.yaml to both push.paths and pull_request.paths in .github/workflows/docs-tests.yml.

Validation note: I did not run make -C docs html SPHINXOPTS="-W" because this review environment is read-only and the Sphinx build writes under docs/_build. Static audits covered the changed workflow, RTD mirror, docs extras, toctree target, methodology registry, TODO carve-outs, sibling workflow surfaces, and pattern-wide inference grep.

igerber added a commit that referenced this pull request May 10, 2026
…ters

Reviewer caught that the new sphinx-build job mirrors .readthedocs.yaml
(Python version, pandoc apt package, post_install docs deps) but the
workflow's path filters omitted .readthedocs.yaml. A future PR changing
only RTD config (e.g., Python version bump or new apt package) would
skip this docs-build guard, allowing drift between CI's mirror and what
RTD actually does.

Added '.readthedocs.yaml' to both push.paths and pull_request.paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igerber
Copy link
Copy Markdown
Owner Author

igerber commented May 10, 2026

/ai-review

@github-actions
Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 35ad65be46cf8b9c0e5f0376762d4e69cc60277d


Overall Assessment

Looks good — no unmitigated P0/P1 findings. The previous P2 path-filter gap is resolved.

Executive Summary

  • Re-review finding addressed: .readthedocs.yaml now triggers the docs workflow on both push and PR events.
  • No estimator, math, weighting, SE/variance, assumptions, or defaults changed.
  • The Sphinx job mirrors RTD’s Python version, pandoc dependency, and install command.
  • The new toctree entry points to an existing tutorial, already documented in the methodology registry.
  • Completeness audits covered sibling push/PR filters, RTD mirror dependencies, workflow guard symmetry, transitive docs-build files, and an inference-pattern grep; no blocking issues found.

Methodology

Finding: No methodology defect
Severity: P3 informational
Impact: This PR changes docs CI and docs navigation only. The added T21 tutorial entry is already registered in docs/methodology/REGISTRY.md:L2556, and no estimator or cited-method implementation surface is modified.
Concrete fix: None.

Code Quality

Finding: Workflow structure is consistent with existing CI gating
Severity: P3 informational
Impact: The new sphinx-build job uses the same ready-for-ci PR guard pattern as the existing docs snippet job at .github/workflows/docs-tests.yml:L70-L75. A dry-run confirmed make -C docs html SPHINXOPTS="-W" expands to sphinx-build -M html "." "_build" -W.
Concrete fix: None.

Performance

Finding: Expected CI runtime increase from Sphinx build
Severity: P3 informational
Impact: The new job at .github/workflows/docs-tests.yml:L70-L101 adds a docs build, but that is the intended enforcement mechanism for warning-free docs.
Concrete fix: None.

Maintainability

Finding: RTD mirror is now covered by workflow triggers
Severity: P3 informational
Impact: The prior gap is fixed: .readthedocs.yaml is included in both push.paths and pull_request.paths at .github/workflows/docs-tests.yml:L16-L18 and .github/workflows/docs-tests.yml:L29-L31. The install command matches .readthedocs.yaml:L13-L21 and pyproject.toml:L47-L75.
Concrete fix: None.

Tech Debt

Finding: No new untracked tech debt
Severity: P3 informational
Impact: No new deferred methodology, correctness, or docs-build limitation is introduced. The pre-existing unrelated-label CI retrigger issue remains tracked in TODO.md:L121-L123.
Concrete fix: None for this PR.

Security

Finding: No security issue found
Severity: P3 informational
Impact: The workflow keeps permissions: contents: read at .github/workflows/docs-tests.yml:L38-L39; no secrets, credential handling, or elevated token permissions are added.
Concrete fix: None.

Documentation/Tests

Finding: Prior docs-build coverage gap resolved
Severity: P3 informational
Impact: RTD config-only changes now trigger the workflow. The added toctree entry at docs/index.rst:L75-L84 points to an existing docs/tutorials/21_had_pretest_workflow.ipynb file; I verified existence without loading notebook JSON.
Concrete fix: None.

Validation note: I did not run the full Sphinx build because this review workspace is read-only and the real build writes under docs/_build; static and dry-run audits passed.

igerber and others added 2 commits May 10, 2026 20:55
Adds a sphinx-build job to .github/workflows/docs-tests.yml that runs
make -C docs html SPHINXOPTS="-W" so warnings become build failures.
This locks in the 0-warning state PR #410 achieved and blocks any
future warning from sneaking in.

The job mirrors the RTD build (.readthedocs.yaml): Python 3.11, pandoc
apt package for nbsphinx, and the same docs extras pinned in
pyproject.toml [project.optional-dependencies.docs].

Also adds tutorials/21_had_pretest_workflow to the Tutorials: Business
Applications toctree at docs/index.rst:84. PR #409 landed this notebook
without a toctree entry; -W enforcement caught it on the local smoke
test before push.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ters

Reviewer caught that the new sphinx-build job mirrors .readthedocs.yaml
(Python version, pandoc apt package, post_install docs deps) but the
workflow's path filters omitted .readthedocs.yaml. A future PR changing
only RTD config (e.g., Python version bump or new apt package) would
skip this docs-build guard, allowing drift between CI's mirror and what
RTD actually does.

Added '.readthedocs.yaml' to both push.paths and pull_request.paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igerber igerber force-pushed the docs/sphinx-warnings-as-errors branch from 35ad65b to 86cc916 Compare May 11, 2026 00:56
@igerber
Copy link
Copy Markdown
Owner Author

igerber commented May 11, 2026

/ai-review

@github-actions
Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 86cc916b21f7a48d2307765bb5f2289fc05d399a


Overall Assessment

Looks good — no unmitigated P0/P1/P2 findings in the loaded diff. The prior path-filter gap remains addressed: .readthedocs.yaml is included in both push and pull_request path filters.

Executive Summary

  • No estimator, methodology, weighting, variance/SE, identification assumption, or default behavior changes are introduced.
  • The new sphinx-build workflow job enforces make -C docs html SPHINXOPTS="-W" and is gated consistently with the existing PR ready-for-ci pattern.
  • The previous re-review concern about RTD config drift triggering docs CI is resolved in the visible workflow diff.
  • The added docs/index.rst toctree entry for tutorial 21 is consistent with the Methodology Registry’s documented T21 HAD pretest workflow tutorial.
  • I could not run Sphinx or verify files outside the prompt; review is limited to the provided diff, PR context, prior review, and registry content.

Methodology

Finding: No methodology-impacting code changed

Severity: P3 informational
Impact: The PR modifies docs CI and a docs toctree entry only. It does not change an estimator, mathematical formula, weighting, variance/SE logic, identification assumptions, or defaults. The new tutorial entry is already described in the registry under the HeterogeneousAdoptionDiD Phase 5 wave 2 T21 tutorial note.
Location: .github/workflows/docs-tests.yml, docs/index.rst
Concrete fix: None.


Code Quality

Finding: Sphinx job is structurally clear and scoped

Severity: P3 informational
Impact: The new sphinx-build job has a descriptive name, installs pandoc before building nbsphinx docs, pins Python to 3.11 per the PR’s stated RTD mirror intent, and runs make -C docs html SPHINXOPTS="-W". The PR guard matches the existing ready-for-ci job-level pattern.
Location: .github/workflows/docs-tests.ymlsphinx-build job
Concrete fix: None.

Finding: Previous path-filter gap is addressed

Severity: P3 informational
Impact: .readthedocs.yaml is included in both the push.paths and pull_request.paths filters, so changes to RTD configuration should trigger the docs workflow. This addresses the prior P2-style coverage gap noted in earlier review history.
Location: .github/workflows/docs-tests.yml near on.push.paths and on.pull_request.paths
Concrete fix: None.


Performance

Finding: CI runtime increases by design

Severity: P3 informational
Impact: The new Sphinx HTML build adds CI work, including apt install and dependency installation. This is the intended tradeoff for enforcing zero-warning docs.
Location: .github/workflows/docs-tests.ymlsphinx-build job
Concrete fix: None.


Maintainability

Finding: Manual docs dependency list must remain synchronized

Severity: P3 informational
Impact: The workflow uses an explicit pip install ... command rather than installing a named extras group. This is acceptable in the visible diff, and comments explicitly call out the need to keep it in sync with pyproject.toml and .readthedocs.yaml. I cannot verify exact dependency parity because those files’ contents are not included in the loaded prompt.
Location: .github/workflows/docs-tests.yml — “Install docs dependencies” step
Concrete fix: Optional future improvement: if feasible, replace the manual list with the same install command RTD uses, e.g. a docs extras install, to reduce sync drift. No action required for this PR based on visible context.


Tech Debt

Finding: No new untracked tech debt visible

Severity: P3 informational
Impact: The diff does not add TODOs or deferred correctness limitations. The only durable maintenance concern is dependency-sync drift, which is documented inline in the workflow comments.
Location: .github/workflows/docs-tests.yml
Concrete fix: None.


Security

Finding: No security issue found in loaded diff

Severity: P3 informational
Impact: The workflow adds package installation and docs building only. No secrets, credential handling, token-permission expansion, or untrusted code execution beyond normal repository docs build steps is introduced in the visible diff.
Location: .github/workflows/docs-tests.yml
Concrete fix: None.


Documentation/Tests

Finding: Docs navigation now includes T21 tutorial

Severity: P3 informational
Impact: tutorials/21_had_pretest_workflow is added to the business applications toctree. This aligns with the registry’s note that the T21 HAD pretest workflow tutorial has landed. I cannot verify notebook existence from the prompt beyond the registry/PR context.
Location: docs/index.rst — Tutorials: Business Applications toctree
Concrete fix: None.

Finding: Sphinx warning enforcement is implemented as CI validation

Severity: P3 informational
Impact: No separate unit tests are added, but the shipped behavior is itself a workflow job that runs Sphinx with -W. For a CI-only docs enforcement change, this is an appropriate validation surface. I did not run the build in this review.
Location: .github/workflows/docs-tests.yml — “Build docs with warnings as errors” step
Concrete fix: None.

@igerber igerber added the ready-for-ci Triggers CI test workflows label May 11, 2026
igerber and others added 2 commits May 11, 2026 17:22
CI's new Sphinx HTML build (-W) failed with "Pygments lexer name
'ipython3' is not known [misc.highlighting_failure]" warnings on every
.ipynb in docs/tutorials/. nbsphinx renders notebook code cells with the
ipython3 lexer; that lexer ships with the IPython package and is not
included in our docs extras. My local venv had IPython transitively
(common dev dep), masking the gap. RTD has been emitting these warnings
silently since it builds without -W.

Added "ipython>=8.0" to all three sync'd surfaces:
- pyproject.toml [project.optional-dependencies.docs]
- .readthedocs.yaml post_install pip install
- .github/workflows/docs-tests.yml sphinx-build job

Verified locally with python3 -c "from pygments.lexers import
get_lexer_by_name; print(get_lexer_by_name('ipython3'))" — returns
<pygments.lexers.IPython3> when IPython is installed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Companion to commit 51022b5 which added ipython to the workflow only.
Per the workflow comment "Keep in sync with pyproject.toml
[project.optional-dependencies.docs] and .readthedocs.yaml post_install",
add the same dep to the other two surfaces so RTD builds (which run
without -W) also stop emitting silent highlighting_failure warnings on
notebook code cells.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igerber
Copy link
Copy Markdown
Owner Author

igerber commented May 11, 2026

/ai-review

@github-actions
Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: ade6429f6bb552178fe89d998e11fccdb95cf0a7


Overall Assessment

Looks good — no unmitigated P0/P1/P2 findings in the loaded diff.

Executive Summary

  • No estimator, methodology, weighting, variance/SE, identification assumption, or default behavior changes are introduced.
  • The Sphinx CI job enforces make -C docs html SPHINXOPTS="-W" and follows the existing ready-for-ci PR-label gate.
  • The prior docs-workflow trigger gap remains addressed: .readthedocs.yaml is included in both push and pull request path filters.
  • The docs dependency surfaces are now synchronized for the newly required ipython>=8.0 across CI, RTD, and pyproject.toml.
  • The T21 HAD pretest tutorial is added to the docs toctree, consistent with the Methodology Registry’s note that this tutorial landed.
  • I did not run Sphinx or inspect files outside the prompt; review is limited to the provided diff, PR context, prior review, and registry content.

Methodology

Finding: No methodology-impacting code changed

Severity: P3 informational
Impact: The PR changes docs CI configuration, RTD docs dependencies, docs optional dependencies, and a docs toctree entry only. It does not alter any estimator, mathematical formula, weighting scheme, inference/SE logic, identification assumption, or default estimator behavior.
Location: .github/workflows/docs-tests.yml:L14-L29, .github/workflows/docs-tests.yml:L65-L103, .readthedocs.yaml:L16-L19, docs/index.rst:L79-L84, pyproject.toml:L71-L80
Concrete fix: None.

Finding: T21 tutorial toctree addition aligns with registry

Severity: P3 informational
Impact: The registry documents the T21 HAD pretest workflow tutorial as landed under HeterogeneousAdoptionDiD Phase 5 wave 2. Adding tutorials/21_had_pretest_workflow to the Business Applications toctree is consistent with that documented surface. I cannot verify the notebook file’s presence from the loaded prompt, but the new Sphinx -W job should catch a missing toctree target when it runs.
Location: docs/index.rst:L79-L84
Concrete fix: None.


Code Quality

Finding: Sphinx warning enforcement is clearly scoped

Severity: P3 informational
Impact: The new sphinx-build job is straightforward: checkout, install pandoc, set Python 3.11, install docs dependencies, then run Sphinx with warnings as errors. The job-level PR label gate matches the existing docs snippet-test pattern, so it does not unexpectedly run on unlabelled PRs.
Location: .github/workflows/docs-tests.yml:L65-L103
Concrete fix: None.

Finding: Dependency synchronization improved across docs surfaces

Severity: P3 informational
Impact: The PR adds ipython>=8.0 to all visible docs dependency surfaces: CI install command, RTD post-install command, and pyproject.toml docs extras. This resolves the visible drift risk introduced by nbsphinx needing the ipython3 Pygments lexer.
Location: .github/workflows/docs-tests.yml:L87-L94, .readthedocs.yaml:L16-L19, pyproject.toml:L71-L80
Concrete fix: None.

Finding: Previous path-filter concern remains resolved

Severity: P3 informational
Impact: .readthedocs.yaml is included in both push.paths and pull_request.paths, so RTD config changes should trigger the docs workflow. This addresses the prior review concern about RTD-drift coverage.
Location: .github/workflows/docs-tests.yml:L14-L29
Concrete fix: None.


Performance

Finding: Docs CI runtime increases by design

Severity: P3 informational
Impact: The new job adds a full Sphinx HTML build plus apt/pip installation. This is the expected tradeoff for enforcing a zero-warning documentation build.
Location: .github/workflows/docs-tests.yml:L65-L103
Concrete fix: None.


Maintainability

Finding: Manual dependency list still requires synchronization

Severity: P3 informational
Impact: The workflow and RTD config use explicit pip install ... commands rather than installing a single extras target. The PR mitigates this by updating all visible surfaces and adding comments telling maintainers to keep them synchronized. This is acceptable for this PR.
Location: .github/workflows/docs-tests.yml:L87-L94, .readthedocs.yaml:L16-L19, pyproject.toml:L71-L80
Concrete fix: Optional future improvement: install a shared docs extras target in CI/RTD if feasible, to reduce manual sync burden. No action required for this PR.


Tech Debt

Finding: No new untracked tech debt visible

Severity: P3 informational
Impact: The PR does not add deferred correctness limitations or TODOs. The remaining maintenance consideration is the explicit docs dependency duplication, which is documented inline and kept synchronized in the visible diff.
Location: .github/workflows/docs-tests.yml:L87-L94, .readthedocs.yaml:L16-L19, pyproject.toml:L71-L80
Concrete fix: None.


Security

Finding: No security issue found in loaded diff

Severity: P3 informational
Impact: The workflow adds normal docs-build dependency installation and Sphinx execution. No secrets, credential handling, permissions expansion, or suspicious external script execution is introduced in the visible diff.
Location: .github/workflows/docs-tests.yml:L65-L103
Concrete fix: None.


Documentation/Tests

Finding: CI job is the appropriate validation surface for Sphinx warning enforcement

Severity: P3 informational
Impact: No separate unit tests are added, but the shipped behavior is itself a CI validation job that builds documentation with -W. For a docs-CI enforcement PR, this is an appropriate test surface. I did not run the build in this review.
Location: .github/workflows/docs-tests.yml:L98-L103
Concrete fix: None.

Finding: RTD/CI docs dependency parity for ipython is visible

Severity: P3 informational
Impact: The PR’s claim that ipython is needed for nbsphinx notebook highlighting is reflected consistently in CI, RTD, and pyproject.toml docs extras. This reduces the risk that CI passes while RTD fails, or vice versa, for notebook highlighting warnings.
Location: .github/workflows/docs-tests.yml:L87-L94, .readthedocs.yaml:L16-L19, pyproject.toml:L71-L80
Concrete fix: None.

@igerber igerber merged commit 0022528 into main May 11, 2026
24 checks passed
@igerber igerber deleted the docs/sphinx-warnings-as-errors branch May 11, 2026 23:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant