Skip to content

Spec v0.6.0: five new checks, /spec/web/ restructure, prose linting - #33

Merged
dacharyc merged 8 commits into
mainfrom
spec-v0.6.0
Sep 13, 2026
Merged

dacharyc merged 8 commits into
mainfrom
spec-v0.6.0

Conversation

@dacharyc

@dacharyc dacharyc commented Sep 13, 2026 •

Copy link
Copy Markdown
Member

Spec v0.6.0

Adds five checks (23 → 28), all grounded in production observations:

  • bot-protection-interference (Cat 7): CDN/WAF enforcement modes that fail invisibly for agents — challenge interstitials served as 200, tarpits, volume-triggered blocking. Evaluated run-wide as a byproduct of a normal scan.
  • page-size-transfer (Cat 3): served byte size of the HTML document, distinct from post-conversion size. Measured properties shipped 75–84% of bytes as serialized framework payloads (40:1–200:1 served-to-content ratios) while scoring perfectly on page-size-html (Spec gap: raw transfer size vs converted size are distinct failure modes #32).
  • single-fetch-completeness (Cat 3): pagination in markdown variants, with machine-followable continuation requirements. Observed: a catalog serving 100 of 102 entries whose advertised continuation URL returned an empty body.
  • markdown-link-portability (Cat 4): links in served markdown must be absolute and resolve to the representation they promise (content-type + soft-404 verification, not just status codes). Observed: 100+ generated links soft-404ing as SPA shells at .md URLs, all returning 200.
  • embedded-data-serialization (Cat 4): attributes page size to machine-generated bulk (a measured reference page: 302KB HTML, 64% table markup, ~17K chars of actual prose).

Plus: two interaction effects (Bot Protection Degrading Scan Reliability; Dynamic Content Rendered Statically), a Serving RAG Ingestion Pipelines section distinguishing web vs source ingestion (resolves #3), and a Related Surfaces scope statement naming planned companion specs (content composition, repo-local docs).

Site restructure

The spec outgrew the 100K truncation threshold its own checks warn about (124K chars), so it now practices its own progressive disclosure recommendation: served as per-category pages under /spec/web/ (largest page ~21K), with /spec/ becoming a landing page for the spec family. SPEC.md remains the canonical single source; site/split_spec.py splits at build time with absolute cross-page links per markdown-link-portability. The URL break from /spec/ is deliberate, buying a namespace that accommodates future companion specs.

Prose linting

Vale with AI-writing tells (adapted from tbhb/vale-ai-tells, MIT) now gates commits via check_prose_style; existing prose cleaned up.

Deploy notes

  • CI tests the live site, so checks here reflect prod until this deploys. Merge and deploy together: the merged README/llms.txt reference /spec/web/ URLs that 404 until the site ships.
  • afdocs (0.21.0) does not yet implement the five new checks; implementation follows separately.

🤖 Generated with Claude Code

dacharyc and others added 7 commits September 13, 2026 17:01
Adds bot-protection-interference, page-size-transfer,
single-fetch-completeness, markdown-link-portability, and
embedded-data-serialization (23 -> 28 checks), two interaction effects,
the Related Surfaces scope statement, and the Serving RAG Ingestion
Pipelines section.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The full document outgrew the 100K truncation threshold its own checks
warn about. SPEC.md stays the canonical source; split_spec.py splits it
at build time with absolute cross-page links. /spec/ becomes a landing
page for the spec family.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ports the DC style (partly adapted from tbhb/vale-ai-tells, MIT) with
two repo-specific tweaks: 'friendly' excepted from adverb density, and
all adverbial 'silently' uses reworded rather than exempted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Attributes the 28 checks to the Web Documentation Delivery Spec and
names the planned companion specs, matching the /spec/ landing page.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds bot-protection exemption as its own item, folds generated-markdown
verification into the serve-markdown item, and adds generated data
tables to the page-size item.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Names the distinction: splitting creates self-contained units reached
by navigation; pagination creates obligatory windows over one unit.
Split by meaning, don't window by size.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- llms-txt-links-resolve now requires representation verification
  (soft-404 heuristics, content-type for markdown links) instead of
  bare status codes, and recommends absolute URLs in llms.txt
- markdown-link-portability exempts same-document fragment links and
  covers llms.txt-discovered markdown
- single-fetch-completeness and markdown-link-portability dependency
  widened to include the llms-txt-links-markdown path
- embedded-data-serialization scoped to conversion-surviving bulk,
  with script payloads assigned to page-size-transfer
- Conventions now classifies Interaction Effects, Related Surfaces,
  and the RAG section

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Community vocabulary proposals exist and some platforms serve
frontmatter in markdown variants, but whether live agents benefit or
just spend truncation budget is unproven. Tracked in #34; change
detection signals tracked in #35.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@dacharyc
dacharyc marked this pull request as ready for review September 13, 2026 22:14
@dacharyc
dacharyc merged commit 5b34700 into main Sep 13, 2026
1 check passed
@dacharyc
dacharyc deleted the spec-v0.6.0 branch September 13, 2026 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Observations from Building a Documentation Agent (Consumer Perspective)

1 participant