readme derivation: a prose interval no longer opens a link, and pin the generator

.github/workflows/ci.yml

@@ -49,16 +49,28 @@ jobs:
       - name: Install nightly toolchain (merged doctests and fuzz build)
         uses: dtolnay/rust-toolchain@nightly
 
+      # rust-src is here for the trybuild suite: several `.stderr` snapshots
+      # quote std source (core's `panic!` expansion, `BitOr::bitor`,
+      # `Into::into`), and rustc prints those lines only when the sources are
+      # installed. Without it three cases mismatch on the quoted lines alone,
+      # which invites a blessing that would degrade the snapshots rather than
+      # supply what they need.
       - name: Install stable toolchain (default)
         uses: dtolnay/rust-toolchain@stable
         with:
-          components: clippy, rustfmt
+          components: clippy, rustfmt, rust-src
           targets: wasm32-unknown-unknown
 
+      # cargo-rdme carries a version because it is the only tool here whose
+      # output is a committed artifact compared byte for byte: readme-check
+      # holds the READMEs against what it emits. Tracking the newest release
+      # would let an upstream change redefine that expectation with no commit
+      # in the repo, turning the sweep red on a tree nobody touched. The others
+      # report findings rather than generate bytes, so they track latest.
       - name: Install just, cargo-nextest, cargo-rdme, cargo-fuzz, and wasm-pack
         uses: taiki-e/install-action@v2
         with:
-          tool: just,cargo-nextest,cargo-rdme,cargo-fuzz,wasm-pack
+          tool: just,cargo-nextest,cargo-rdme@2.1.0,cargo-fuzz,wasm-pack
 
       # The viz bundle (`just viz`) runs `npm install`, a strict TypeScript
       # typecheck, and an esbuild bundle.

AGENTS.md

@@ -40,11 +40,14 @@ it fully clean before every commit.
 ## Contributing a change
 
 One-time setup: everything `just gate` shells out to is a stable Rust
-toolchain (1.85 or later, for edition 2024) with clippy and rustfmt, a
-nightly toolchain (merged doctests), `just`, `cargo-nextest`, `cargo-rdme`,
-and python3 with bash (the `tools/` linters). `just ci` additionally wants
-the `wasm32-unknown-unknown` target, `wasm-pack`, `cargo-fuzz`, and
-node/npm.
+toolchain (1.85 or later, for edition 2024) with clippy, rustfmt, and
+rust-src, a nightly toolchain (merged doctests), `just`, `cargo-nextest`,
+`cargo-rdme`, and python3 with bash (the `tools/` linters). `just ci`
+additionally wants the `wasm32-unknown-unknown` target, `wasm-pack`,
+`cargo-fuzz`, and node/npm. rust-src is what lets rustc quote std source,
+which `before`'s trybuild snapshots do: without the sources installed, three
+compile-fail cases mismatch on the quoted lines alone while the guarantees
+they pin are perfectly intact.
 
 1. Iterate with the inner loop: `just check`, `just test <filter>`,
    `just clippy`, `just fmt`.

crates/before/README.md

@@ -47,9 +47,9 @@ freely `Clone`able.
 The insight of the original ITC paper is that a `Party` can be represented
 as a *tree* denoting a non-empty set of subintervals of `[0, 1)`, giving
 both compact representation and dynamic membership. The initial `Party`,
-`Party::seed`, is `{ 0, 1) }`; a [`fork` splits an interval
-in half, so the first fork yields `{ 0, 1/2) }` and `{ [1/2, 1) }`.
-Disjoint interval sets are [`join`ed by set union, merging
+`Party::seed`, is `{ [0, 1) }`; a `fork` splits an interval
+in half, so the first fork yields `{ [0, 1/2) }` and `{ [1/2, 1) }`.
+Disjoint interval sets are `join`ed by set union, merging
 adjacent intervals: `{ [0, 1/2), [5/8, 3/4) }` ∪ `{ [3/4, 1) }` = `{ [0,
 1/2), [5/8, 1) }`. Parties can therefore be minted and recycled freely while
 their representations stay small.

crates/before/tests/compile_fail.rs

@@ -24,6 +24,16 @@
 //! failure is unchanged (still a missing `Clone` / a use-after-move / a missing
 //! `BitOr` / the `N >= 1` const assert).
 //!
+//! Reproducing them needs the `rust-src` component. Several snapshots quote std
+//! source — `core`'s `panic!` expansion, `BitOr::bitor`, `Into::into` — and
+//! rustc can only print those lines when the sources are installed; lacking
+//! them, `clock_into_empty_array`, `party_into_empty_array`, and
+//! `clock_use_after_move` mismatch on exactly those quotations, each collapsing
+//! to a bare `$RUST` path or a `= note:` line. Reach for
+//! `rustup component add rust-src`, never `TRYBUILD=overwrite`: blessing a run
+//! that cannot see std trades these diagnostics for weaker ones and leaves any
+//! machine that does have the sources failing.
+//!
 //! [`From<Party>`]: before::Party
 //! [`From<Clock>`]: before::Clock
 

justfile

@@ -17,6 +17,15 @@ set shell := ["bash", "-euo", "pipefail", "-c"]
 
 nightly_toolchain := "nightly"
 
+# The triple the fuzz recipes build for. cargo-fuzz defaults `--target` to the
+# triple it was itself built for, not the host's, so a statically linked
+# prebuilt (what the CI installer ships) aims the sanitizer build at
+# `*-linux-musl`: a target whose std is absent, and whose static libc the
+# sanitizer refuses outright. Naming the host keeps the recipes indifferent to
+# how cargo-fuzz arrived.
+
+host_triple := `rustc -vV | sed -n 's/^host: //p'`
+
 # Default fuzz smoke duration per target, in seconds (matches the guidance in
 # crates/before/fuzz/Cargo.toml).
 
@@ -98,14 +107,19 @@ testdoc:
 # derived, never hand-edited: after editing crate-level rustdoc, run
 # `just readme`. `readme-check` re-derives the READMEs into scratch copies
 # and diffs, the same no-rot contract as fmt-check, so a rustdoc edit can't
-# silently desync the README. Needs cargo-rdme: `cargo install cargo-rdme`.
+# silently desync the README. It leads with the stripper's own self-test, which
+# pins the link forms rewritten and the forms preserved: a stripping bug then
+# names itself here instead of arriving as unexplained drift in a derived file,
+# or as corruption that a regeneration quietly commits.
+# Needs cargo-rdme: `cargo install cargo-rdme`.
 
 # Regenerate every crate's README from its crate-level rustdoc.
 readme:
     ./tools/readme write
 
 # Verify every README is in sync with its rustdoc (the gate variant of `readme`).
 readme-check:
+    ./tools/readme self-test
     ./tools/readme check
 
 # The dependency list is the ordering: build-free lints first for fast
@@ -170,16 +184,16 @@ bench-build:
 # Build the libFuzzer targets (nightly).
 [working-directory("crates/before/fuzz")]
 fuzz-build:
-    {{ justfile_directory() }}/tools/memwatch cargo +{{ nightly_toolchain }} fuzz build
+    {{ justfile_directory() }}/tools/memwatch cargo +{{ nightly_toolchain }} fuzz build --target {{ host_triple }}
 
 # The decode invariant (accepted input re-encodes stably and decodes back to
 # itself) is asserted inline in the targets, so any hit is a crash.
 
 # Short fuzz smoke: run each libFuzzer target for `secs` seconds.
 [working-directory("crates/before/fuzz")]
 fuzz secs=fuzz_smoke_secs:
-    cargo +{{ nightly_toolchain }} fuzz run fuzz_decode -- -max_total_time={{ secs }}
-    cargo +{{ nightly_toolchain }} fuzz run fuzz_decode_ops -- -max_total_time={{ secs }}
+    cargo +{{ nightly_toolchain }} fuzz run --target {{ host_triple }} fuzz_decode -- -max_total_time={{ secs }}
+    cargo +{{ nightly_toolchain }} fuzz run --target {{ host_triple }} fuzz_decode_ops -- -max_total_time={{ secs }}
 
 # ── the formal tier (formal/lean; needs elan) ────────────────────────────────
 # The proofs are kernel-checked by `lake build` (pins, negative controls,

proptest-regressions/tree/mirror/streaming/remote/proxy/tests/failures.txt

@@ -9,3 +9,4 @@ cc 346207da645ff7c84b2e6326c75960b05998807258d7d75cd6fbddca62ea94b5 # shrinks to
 cc 462a46e83620d8b297e0e857ad66eb85bba789cc9d243209cf292add3b2ceb27 # shrinks to (left, right) = (Root { ceiling: (0, (0, 0, 1), 0), root: Some(Node { prefix: "fdc83aee274cd72b2d7b70887b9315b8d98e13c1f15388d05cbedbf5c155bd43", children: Leaf { version: (0, (0, 0, 1), 0), message: () } }) }, Root { ceiling: 0, root: None }), fail_left = true, operation = Read, after = 9, bytes = false, chunk = 1, delays = [], buffered = false
 cc f8d2e756fe6035a30841561df8b200273f51ce30ebe432c98ed793f16bf49ff5 # shrinks to (left, right) = (Root { ceiling: (0, (0, 0, 1), 1), root: Some(Node { prefix: "", children: Branch { ceiling: OnceLock(<uninit>), floor: OnceLock(<uninit>), leaves: 2, children: {32: Node { prefix: "4d3be44e01d4d26c35fefb7ccc1825c9e1b643e4653608e7ba1d50ec07740a", children: Leaf { version: (0, 0, 1), message: () } }, 126: Node { prefix: "533e5b8b9c5ee2bdaaa92177495d3e8705b3fca0a69494ea7bb9f3f11ee9fd", children: Leaf { version: (0, (0, 0, 1), 1), message: () } }} } }) }, Root { ceiling: (0, (0, (0, 0, 1), 0), 1), root: None }), fail_left = false, operation = Read, after = 6, bytes = false, chunk = 1, delays = [0, 0, 0, 0, 0, 0, 0], buffered = false
 cc 6c26bc02d9ac5995ddb02e8f184e5d9145ed5b0b8035b7f1e42e3f6a3346f0f0 # shrinks to (left, right) = (Root { ceiling: (0, (0, 0, 3), 0), root: Some(Node { prefix: "", children: Branch { ceiling: OnceLock(<uninit>), floor: OnceLock(<uninit>), leaves: 3, version_bytes: OnceLock(<uninit>), children: {67: Node { prefix: "fdc83aee274cd72b2d7b70887b9315b8d98e13c1f15388d05cbedbf5c155bd", children: Leaf { version: (0, (0, 0, 1), 0), message: () } }, 89: Node { prefix: "44148b8e30a0a81ada41377d65e21b91070d89ed7a3279d4228c6632ca2e13", children: Leaf { version: (0, (0, 0, 3), 0), message: () } }, 198: Node { prefix: "626b476ab1ed9f7c6ff800bca9fc9b7ae062b100e85ac4999505d17a9edbe8", children: Leaf { version: (0, (0, 0, 2), 0), message: () } }} } }) }, Root { ceiling: (0, (0, (0, 0, 1), 0), 0), root: Some(Node { prefix: "09cabf1e315343f1e40384237d2eeddc6919a3dbed8e64014638926442a28e50", children: Leaf { version: (0, (0, (0, 0, 1), 0), 0), message: () } }) }), fail_left = false, operation = Accept, after = 0, bytes = true, chunk = 8, delays = [1, 1, 2, 1, 2, 1, 0, 2, 2, 0, 0, 0, 2, 2, 1, 0, 0, 0, 1, 0, 0, 1, 1, 2, 2, 2, 2, 0, 2, 2, 0], buffered = false
+cc f7b5859c1967d3b3b6f9ac3afd501a854a01c1864b59cc60ea1e7e863efae2bd # shrinks to (left, right) = (Root { ceiling: (0, (0, 0, 2), 3), root: Some(Node { prefix: "", children: Branch { ceiling: OnceLock(<uninit>), floor: OnceLock(<uninit>), leaves: 5, version_bytes: OnceLock(<uninit>), children: {13: Node { prefix: "b9bc6c0254ae5e131aac9c4ff0d10a18ab41bc4913a3bbcfe8a73c09a79439", children: Leaf { version: (0, 0, 3), message: () } }, 22: Node { prefix: "e840b5f788f4c660a52976907c35698fa434ccfb1ac8c625bc3041b74681ee", children: Leaf { version: (0, (0, 0, 1), 3), message: () } }, 32: Node { prefix: "4d3be44e01d4d26c35fefb7ccc1825c9e1b643e4653608e7ba1d50ec07740a", children: Leaf { version: (0, 0, 1), message: () } }, 207: Node { prefix: "8afe2f6c1e74cca8b8a80a907bded2cf809fd0305789e5efcf84e9dee567bb", children: Leaf { version: (0, (0, 0, 2), 3), message: () } }, 224: Node { prefix: "cd3e3252fa8aa03361796d1b1d2a9449261c84d35d357607b43b1d71102e14", children: Leaf { version: (0, 0, 2), message: () } }} } }) }, Root { ceiling: (0, (0, (0, 0, 3), 0), 3), root: Some(Node { prefix: "", children: Branch { ceiling: OnceLock(<uninit>), floor: OnceLock(<uninit>), leaves: 6, version_bytes: OnceLock(<uninit>), children: {13: Node { prefix: "b9bc6c0254ae5e131aac9c4ff0d10a18ab41bc4913a3bbcfe8a73c09a79439", children: Leaf { version: (0, 0, 3), message: () } }, 32: Node { prefix: "4d3be44e01d4d26c35fefb7ccc1825c9e1b643e4653608e7ba1d50ec07740a", children: Leaf { version: (0, 0, 1), message: () } }, 145: Node { prefix: "fa3e12c159d0b2da3a236250646402d9e75f877875d200d697324cefd0bfed", children: Leaf { version: (0, (0, (0, 0, 1), 0), 3), message: () } }, 166: Node { prefix: "804f3a0166209487f1514297735bb1b87e57a10ff2a33d1a617c38a695b56a", children: Leaf { version: (0, (0, (0, 0, 3), 0), 3), message: () } }, 204: Node { prefix: "6ef43154ec1e291df0aa528f7bfad263bc0a8e1148ad6bb07b480103c31449", children: Leaf { version: (0, (0, (0, 0, 2), 0), 3), message: () } }, 224: Node { prefix: "cd3e3252fa8aa03361796d1b1d2a9449261c84d35d357607b43b1d71102e14", children: Leaf { version: (0, 0, 2), message: () } }} } }) }), fail_left = true, operation = Read, after = 74, bytes = false, chunk = 2, delays = [0, 0, 0, 1], buffered = false

tools/readme

@@ -12,12 +12,15 @@ GitHub and crates.io. cargo-rdme's own `--intralinks-strip-links` doesn't
 recognize these forms either, so we strip them here, down to the link *text*,
 leaving real (http/https) links untouched.
 
-    tools/readme write    regenerate every crate's README in place
-    tools/readme check    re-derive into a scratch copy and diff; nonzero on drift
+    tools/readme write      regenerate every crate's README in place
+    tools/readme check      re-derive into a scratch copy and diff; nonzero on drift
+    tools/readme self-test  pin the link forms the stripper rewrites and preserves
 
 `check` never touches the committed files — it generates into a temp copy of
 each README (preserving that README's exact preamble) and diffs. The justfile
-exposes both as `just readme` / `just readme-check`; the latter rides in the gate.
+exposes both as `just readme` / `just readme-check`; the latter rides in the
+gate, led by `self-test` so a stripping bug surfaces as a tool failure rather
+than as drift in a derived file.
 """
 
 from __future__ import annotations
@@ -79,7 +82,15 @@ RUST_PATH = r"[A-Za-z_]\w*(?:::[A-Za-z_]\w*)*"
 # path (`Link#what-a-session-promises`). Text is kept verbatim with its own
 # markup (it may be code, *italic*, or plain), so only the link wrapper is
 # removed. A bare in-page anchor (`(#section)`) has no leading path and stays.
-WITH_DEST = re.compile(r"\[([^\]]+)\]\((" + RUST_PATH + r"(?:#[\w.-]+)?)\)")
+#
+# The link text admits no `[`, so that a half-open interval in the surrounding
+# prose (`{ [0, 1) }`) cannot open a link that closes at a later `](path)`:
+# such a run spans the interval and everything up to the true link's `]`, and
+# substituting the text for it swallows the interval's bracket. Every inline
+# link brings its own `[`, so excluding the character always resettles the
+# match onto the real opening bracket. Newlines remain legal — rustdoc wraps
+# prose, so link text may straddle a line break.
+WITH_DEST = re.compile(r"\[([^\][]+)\]\((" + RUST_PATH + r"(?:#[\w.-]+)?)\)")
 
 # `[`code`]` shortcut link not continued by a destination `(` or another
 # reference `[`. Intra-doc shortcut links are always code-spanned, so keying
@@ -155,10 +166,54 @@ def check() -> int:
     return 1 if drift else 0
 
 
+def self_test() -> int:
+    """Pin the link forms the derivation strips and the ones it must preserve."""
+    cases = [
+        # A shortcut link goes down to its code-spanned text.
+        ("a [`Peer`] joins", "a `Peer` joins"),
+        # So does an inline link whose destination is a Rust path, with or
+        # without a rustdoc heading anchor.
+        ("call [`send`](Rumors::send) now", "call `send` now"),
+        (
+            "the [contract](Link#what-a-session-promises) holds",
+            "the contract holds",
+        ),
+        # Link text keeps its own markup, and may straddle a line break.
+        ("a [*fresh*](Party::seed) party", "a *fresh* party"),
+        ("the [session\ncontract](Link) holds", "the session\ncontract holds"),
+        # An interval in the prose is not an opening bracket for the link that
+        # follows it: both brackets survive, on the same line and across lines.
+        (
+            "is `{ [0, 1) }`; a [`fork`](Party::fork) splits",
+            "is `{ [0, 1) }`; a `fork` splits",
+        ),
+        (
+            "yields `{ [0, 1/2) }`.\nSets are [`join`](Party::join)ed by union",
+            "yields `{ [0, 1/2) }`.\nSets are `join`ed by union",
+        ),
+        # Real links, bare in-page anchors, and reference definitions stay put.
+        ("see [the paper](https://example.com/itc)", "see [the paper](https://example.com/itc)"),
+        ("jump to [the sketch](#a-conceptual-sketch)", "jump to [the sketch](#a-conceptual-sketch)"),
+        ("[`Peer`]: https://example.com/peer", "[`Peer`]: https://example.com/peer"),
+    ]
+    failures = 0
+    for source, expected in cases:
+        actual = strip_intra_doc_links(source, {})
+        if actual != expected:
+            failures += 1
+            print(
+                f"self-test: {source!r}\n  expected {expected!r}\n  got      {actual!r}",
+                file=sys.stderr,
+            )
+    return 1 if failures else 0
+
+
 def main(argv: list[str]) -> int:
-    if len(argv) != 2 or argv[1] not in ("write", "check"):
-        print(f"usage: {argv[0]} <write|check>", file=sys.stderr)
+    if len(argv) != 2 or argv[1] not in ("write", "check", "self-test"):
+        print(f"usage: {argv[0]} <write|check|self-test>", file=sys.stderr)
         return 2
+    if argv[1] == "self-test":
+        return self_test()
     return write() if argv[1] == "write" else check()