From f42999aae91130f44bd0e9072481ca9af01ba255 Mon Sep 17 00:00:00 2001 From: finch Date: Wed, 29 Jul 2026 11:17:52 -0400 Subject: [PATCH 1/4] readme derivation: a prose interval no longer opens a link, and the stripper self-tests The intra-doc link stripper matched `[text](Rust::path)` with a text run that admitted `[`, so a half-open interval in the surrounding prose could serve as the opening bracket for a link further on. In `before`'s conceptual sketch the run began at `{ [0, 1) }`, swallowed the prose up to the `]` of a real link two lines later, and substituted its text for the whole span: the interval lost its bracket and the link kept its own. Excluding `[` from the text resettles every match onto the true opening bracket, since an inline link always brings one. Newlines stay legal, as rustdoc wraps prose across lines. `tools/readme self-test` pins the forms rewritten and the forms preserved, including both shapes of this collision, and `readme-check` leads with it: a stripping bug now names itself as a tool failure instead of arriving as unexplained drift, or as corruption a regeneration quietly commits. `crates/before/README.md` returns to its pre-corruption bytes exactly. CI installed cargo-rdme at whatever version was newest. The derived READMEs are compared byte for byte against what it emits, so 2.1.0's arrival redefined the expected bytes and the sweep went red on trees nobody had touched; the version is now pinned. With the stripper fixed, 1.5.1 and 2.1.0 render both READMEs identically, so the pin buys reproducibility rather than a required upgrade. --- .github/workflows/ci.yml | 8 ++++- crates/before/README.md | 6 ++-- justfile | 7 ++++- tools/readme | 67 ++++++++++++++++++++++++++++++++++++---- 4 files changed, 77 insertions(+), 11 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index efea6297..2dc6415c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -55,10 +55,16 @@ jobs: components: clippy, rustfmt targets: wasm32-unknown-unknown + # cargo-rdme carries a version because it is the only tool here whose + # output is a committed artifact compared byte for byte: readme-check + # holds the READMEs against what it emits. Tracking the newest release + # would let an upstream change redefine that expectation with no commit + # in the repo, turning the sweep red on a tree nobody touched. The others + # report findings rather than generate bytes, so they track latest. - name: Install just, cargo-nextest, cargo-rdme, cargo-fuzz, and wasm-pack uses: taiki-e/install-action@v2 with: - tool: just,cargo-nextest,cargo-rdme,cargo-fuzz,wasm-pack + tool: just,cargo-nextest,cargo-rdme@2.1.0,cargo-fuzz,wasm-pack # The viz bundle (`just viz`) runs `npm install`, a strict TypeScript # typecheck, and an esbuild bundle. diff --git a/crates/before/README.md b/crates/before/README.md index d09fae9f..674dfbc8 100644 --- a/crates/before/README.md +++ b/crates/before/README.md @@ -47,9 +47,9 @@ freely `Clone`able. The insight of the original ITC paper is that a `Party` can be represented as a *tree* denoting a non-empty set of subintervals of `[0, 1)`, giving both compact representation and dynamic membership. The initial `Party`, -`Party::seed`, is `{ 0, 1) }`; a [`fork` splits an interval -in half, so the first fork yields `{ 0, 1/2) }` and `{ [1/2, 1) }`. -Disjoint interval sets are [`join`ed by set union, merging +`Party::seed`, is `{ [0, 1) }`; a `fork` splits an interval +in half, so the first fork yields `{ [0, 1/2) }` and `{ [1/2, 1) }`. +Disjoint interval sets are `join`ed by set union, merging adjacent intervals: `{ [0, 1/2), [5/8, 3/4) }` ∪ `{ [3/4, 1) }` = `{ [0, 1/2), [5/8, 1) }`. Parties can therefore be minted and recycled freely while their representations stay small. diff --git a/justfile b/justfile index 329edbfd..b06d4f5f 100644 --- a/justfile +++ b/justfile @@ -98,7 +98,11 @@ testdoc: # derived, never hand-edited: after editing crate-level rustdoc, run # `just readme`. `readme-check` re-derives the READMEs into scratch copies # and diffs, the same no-rot contract as fmt-check, so a rustdoc edit can't -# silently desync the README. Needs cargo-rdme: `cargo install cargo-rdme`. +# silently desync the README. It leads with the stripper's own self-test, which +# pins the link forms rewritten and the forms preserved: a stripping bug then +# names itself here instead of arriving as unexplained drift in a derived file, +# or as corruption that a regeneration quietly commits. +# Needs cargo-rdme: `cargo install cargo-rdme`. # Regenerate every crate's README from its crate-level rustdoc. readme: @@ -106,6 +110,7 @@ readme: # Verify every README is in sync with its rustdoc (the gate variant of `readme`). readme-check: + ./tools/readme self-test ./tools/readme check # The dependency list is the ordering: build-free lints first for fast diff --git a/tools/readme b/tools/readme index a46ff8b4..cf7a46e6 100755 --- a/tools/readme +++ b/tools/readme @@ -12,12 +12,15 @@ GitHub and crates.io. cargo-rdme's own `--intralinks-strip-links` doesn't recognize these forms either, so we strip them here, down to the link *text*, leaving real (http/https) links untouched. - tools/readme write regenerate every crate's README in place - tools/readme check re-derive into a scratch copy and diff; nonzero on drift + tools/readme write regenerate every crate's README in place + tools/readme check re-derive into a scratch copy and diff; nonzero on drift + tools/readme self-test pin the link forms the stripper rewrites and preserves `check` never touches the committed files — it generates into a temp copy of each README (preserving that README's exact preamble) and diffs. The justfile -exposes both as `just readme` / `just readme-check`; the latter rides in the gate. +exposes both as `just readme` / `just readme-check`; the latter rides in the +gate, led by `self-test` so a stripping bug surfaces as a tool failure rather +than as drift in a derived file. """ from __future__ import annotations @@ -79,7 +82,15 @@ RUST_PATH = r"[A-Za-z_]\w*(?:::[A-Za-z_]\w*)*" # path (`Link#what-a-session-promises`). Text is kept verbatim with its own # markup (it may be code, *italic*, or plain), so only the link wrapper is # removed. A bare in-page anchor (`(#section)`) has no leading path and stays. -WITH_DEST = re.compile(r"\[([^\]]+)\]\((" + RUST_PATH + r"(?:#[\w.-]+)?)\)") +# +# The link text admits no `[`, so that a half-open interval in the surrounding +# prose (`{ [0, 1) }`) cannot open a link that closes at a later `](path)`: +# such a run spans the interval and everything up to the true link's `]`, and +# substituting the text for it swallows the interval's bracket. Every inline +# link brings its own `[`, so excluding the character always resettles the +# match onto the real opening bracket. Newlines remain legal — rustdoc wraps +# prose, so link text may straddle a line break. +WITH_DEST = re.compile(r"\[([^\][]+)\]\((" + RUST_PATH + r"(?:#[\w.-]+)?)\)") # `[`code`]` shortcut link not continued by a destination `(` or another # reference `[`. Intra-doc shortcut links are always code-spanned, so keying @@ -155,10 +166,54 @@ def check() -> int: return 1 if drift else 0 +def self_test() -> int: + """Pin the link forms the derivation strips and the ones it must preserve.""" + cases = [ + # A shortcut link goes down to its code-spanned text. + ("a [`Peer`] joins", "a `Peer` joins"), + # So does an inline link whose destination is a Rust path, with or + # without a rustdoc heading anchor. + ("call [`send`](Rumors::send) now", "call `send` now"), + ( + "the [contract](Link#what-a-session-promises) holds", + "the contract holds", + ), + # Link text keeps its own markup, and may straddle a line break. + ("a [*fresh*](Party::seed) party", "a *fresh* party"), + ("the [session\ncontract](Link) holds", "the session\ncontract holds"), + # An interval in the prose is not an opening bracket for the link that + # follows it: both brackets survive, on the same line and across lines. + ( + "is `{ [0, 1) }`; a [`fork`](Party::fork) splits", + "is `{ [0, 1) }`; a `fork` splits", + ), + ( + "yields `{ [0, 1/2) }`.\nSets are [`join`](Party::join)ed by union", + "yields `{ [0, 1/2) }`.\nSets are `join`ed by union", + ), + # Real links, bare in-page anchors, and reference definitions stay put. + ("see [the paper](https://example.com/itc)", "see [the paper](https://example.com/itc)"), + ("jump to [the sketch](#a-conceptual-sketch)", "jump to [the sketch](#a-conceptual-sketch)"), + ("[`Peer`]: https://example.com/peer", "[`Peer`]: https://example.com/peer"), + ] + failures = 0 + for source, expected in cases: + actual = strip_intra_doc_links(source, {}) + if actual != expected: + failures += 1 + print( + f"self-test: {source!r}\n expected {expected!r}\n got {actual!r}", + file=sys.stderr, + ) + return 1 if failures else 0 + + def main(argv: list[str]) -> int: - if len(argv) != 2 or argv[1] not in ("write", "check"): - print(f"usage: {argv[0]} ", file=sys.stderr) + if len(argv) != 2 or argv[1] not in ("write", "check", "self-test"): + print(f"usage: {argv[0]} ", file=sys.stderr) return 2 + if argv[1] == "self-test": + return self_test() return write() if argv[1] == "write" else check() From 06eaf4fb19a6e142c3cb972fafecdfe5e265da59 Mon Sep 17 00:00:00 2001 From: finch Date: Wed, 29 Jul 2026 11:56:46 -0400 Subject: [PATCH 2/4] CI installs rust-src, which the trybuild snapshots need to render Three compile-fail snapshots quote std source: core's `panic!` expansion in the zero-length-split cases, and `BitOr::bitor` / `Into::into` in the use-after-move case. rustc prints those lines only when the sources are installed, so on a toolchain without rust-src the quotations collapse to a bare `$RUST` path or a `= note:` line and `clock_into_empty_array`, `party_into_empty_array`, and `clock_use_after_move` mismatch. The CI toolchain asked for clippy and rustfmt alone, so it had never rendered them. The component is the cause, not the compiler version: on one 1.97.1 toolchain and one tree, the suite fails 3 of 7 without rust-src and passes 7 of 7 with it. The committed snapshots are correct as they stand, and hold byte-identical across 1.96.1 and 1.97.1 once the sources are present. The trap this closes is the blessing: `TRYBUILD=overwrite` on a machine that cannot see std would turn the sweep green by trading these diagnostics for weaker ones, and would then fail on every machine that can. The test's module doc and the setup list in AGENTS.md both name the requirement so the next mismatch routes to the component instead. --- .github/workflows/ci.yml | 8 +++++++- AGENTS.md | 13 ++++++++----- crates/before/tests/compile_fail.rs | 10 ++++++++++ 3 files changed, 25 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2dc6415c..e1afd365 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -49,10 +49,16 @@ jobs: - name: Install nightly toolchain (merged doctests and fuzz build) uses: dtolnay/rust-toolchain@nightly + # rust-src is here for the trybuild suite: several `.stderr` snapshots + # quote std source (core's `panic!` expansion, `BitOr::bitor`, + # `Into::into`), and rustc prints those lines only when the sources are + # installed. Without it three cases mismatch on the quoted lines alone, + # which invites a blessing that would degrade the snapshots rather than + # supply what they need. - name: Install stable toolchain (default) uses: dtolnay/rust-toolchain@stable with: - components: clippy, rustfmt + components: clippy, rustfmt, rust-src targets: wasm32-unknown-unknown # cargo-rdme carries a version because it is the only tool here whose diff --git a/AGENTS.md b/AGENTS.md index 402485e1..cfb08a77 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -40,11 +40,14 @@ it fully clean before every commit. ## Contributing a change One-time setup: everything `just gate` shells out to is a stable Rust -toolchain (1.85 or later, for edition 2024) with clippy and rustfmt, a -nightly toolchain (merged doctests), `just`, `cargo-nextest`, `cargo-rdme`, -and python3 with bash (the `tools/` linters). `just ci` additionally wants -the `wasm32-unknown-unknown` target, `wasm-pack`, `cargo-fuzz`, and -node/npm. +toolchain (1.85 or later, for edition 2024) with clippy, rustfmt, and +rust-src, a nightly toolchain (merged doctests), `just`, `cargo-nextest`, +`cargo-rdme`, and python3 with bash (the `tools/` linters). `just ci` +additionally wants the `wasm32-unknown-unknown` target, `wasm-pack`, +`cargo-fuzz`, and node/npm. rust-src is what lets rustc quote std source, +which `before`'s trybuild snapshots do: without the sources installed, three +compile-fail cases mismatch on the quoted lines alone while the guarantees +they pin are perfectly intact. 1. Iterate with the inner loop: `just check`, `just test `, `just clippy`, `just fmt`. diff --git a/crates/before/tests/compile_fail.rs b/crates/before/tests/compile_fail.rs index 1fd9339d..69c047fd 100644 --- a/crates/before/tests/compile_fail.rs +++ b/crates/before/tests/compile_fail.rs @@ -24,6 +24,16 @@ //! failure is unchanged (still a missing `Clone` / a use-after-move / a missing //! `BitOr` / the `N >= 1` const assert). //! +//! Reproducing them needs the `rust-src` component. Several snapshots quote std +//! source — `core`'s `panic!` expansion, `BitOr::bitor`, `Into::into` — and +//! rustc can only print those lines when the sources are installed; lacking +//! them, `clock_into_empty_array`, `party_into_empty_array`, and +//! `clock_use_after_move` mismatch on exactly those quotations, each collapsing +//! to a bare `$RUST` path or a `= note:` line. Reach for +//! `rustup component add rust-src`, never `TRYBUILD=overwrite`: blessing a run +//! that cannot see std trades these diagnostics for weaker ones and leaves any +//! machine that does have the sources failing. +//! //! [`From`]: before::Party //! [`From`]: before::Clock From 7e7d6730b78c96d0546a23ed6bc47ff056060a5b Mon Sep 17 00:00:00 2001 From: finch Date: Wed, 29 Jul 2026 12:12:54 -0400 Subject: [PATCH 3/4] proxy failures suite: a second witness for the hash-seed-sensitive terminal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sampling the suite while diagnosing the sweep turned up a counterexample distinct from the pinned one: a two-sided fan where the left supply fails on a Read 74 operations in, chunked at 2. It exercised the same terminal-ordering defect and passes under the current rule, so it rides along as an independent witness — the pinned pair now covers an Accept-time failure with a long delay train and a Read-time failure deep into a byte-chunked stream. --- .../tree/mirror/streaming/remote/proxy/tests/failures.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/proptest-regressions/tree/mirror/streaming/remote/proxy/tests/failures.txt b/proptest-regressions/tree/mirror/streaming/remote/proxy/tests/failures.txt index fb963c21..03c1f88f 100644 --- a/proptest-regressions/tree/mirror/streaming/remote/proxy/tests/failures.txt +++ b/proptest-regressions/tree/mirror/streaming/remote/proxy/tests/failures.txt @@ -9,3 +9,4 @@ cc 346207da645ff7c84b2e6326c75960b05998807258d7d75cd6fbddca62ea94b5 # shrinks to cc 462a46e83620d8b297e0e857ad66eb85bba789cc9d243209cf292add3b2ceb27 # shrinks to (left, right) = (Root { ceiling: (0, (0, 0, 1), 0), root: Some(Node { prefix: "fdc83aee274cd72b2d7b70887b9315b8d98e13c1f15388d05cbedbf5c155bd43", children: Leaf { version: (0, (0, 0, 1), 0), message: () } }) }, Root { ceiling: 0, root: None }), fail_left = true, operation = Read, after = 9, bytes = false, chunk = 1, delays = [], buffered = false cc f8d2e756fe6035a30841561df8b200273f51ce30ebe432c98ed793f16bf49ff5 # shrinks to (left, right) = (Root { ceiling: (0, (0, 0, 1), 1), root: Some(Node { prefix: "", children: Branch { ceiling: OnceLock(), floor: OnceLock(), leaves: 2, children: {32: Node { prefix: "4d3be44e01d4d26c35fefb7ccc1825c9e1b643e4653608e7ba1d50ec07740a", children: Leaf { version: (0, 0, 1), message: () } }, 126: Node { prefix: "533e5b8b9c5ee2bdaaa92177495d3e8705b3fca0a69494ea7bb9f3f11ee9fd", children: Leaf { version: (0, (0, 0, 1), 1), message: () } }} } }) }, Root { ceiling: (0, (0, (0, 0, 1), 0), 1), root: None }), fail_left = false, operation = Read, after = 6, bytes = false, chunk = 1, delays = [0, 0, 0, 0, 0, 0, 0], buffered = false cc 6c26bc02d9ac5995ddb02e8f184e5d9145ed5b0b8035b7f1e42e3f6a3346f0f0 # shrinks to (left, right) = (Root { ceiling: (0, (0, 0, 3), 0), root: Some(Node { prefix: "", children: Branch { ceiling: OnceLock(), floor: OnceLock(), leaves: 3, version_bytes: OnceLock(), children: {67: Node { prefix: "fdc83aee274cd72b2d7b70887b9315b8d98e13c1f15388d05cbedbf5c155bd", children: Leaf { version: (0, (0, 0, 1), 0), message: () } }, 89: Node { prefix: "44148b8e30a0a81ada41377d65e21b91070d89ed7a3279d4228c6632ca2e13", children: Leaf { version: (0, (0, 0, 3), 0), message: () } }, 198: Node { prefix: "626b476ab1ed9f7c6ff800bca9fc9b7ae062b100e85ac4999505d17a9edbe8", children: Leaf { version: (0, (0, 0, 2), 0), message: () } }} } }) }, Root { ceiling: (0, (0, (0, 0, 1), 0), 0), root: Some(Node { prefix: "09cabf1e315343f1e40384237d2eeddc6919a3dbed8e64014638926442a28e50", children: Leaf { version: (0, (0, (0, 0, 1), 0), 0), message: () } }) }), fail_left = false, operation = Accept, after = 0, bytes = true, chunk = 8, delays = [1, 1, 2, 1, 2, 1, 0, 2, 2, 0, 0, 0, 2, 2, 1, 0, 0, 0, 1, 0, 0, 1, 1, 2, 2, 2, 2, 0, 2, 2, 0], buffered = false +cc f7b5859c1967d3b3b6f9ac3afd501a854a01c1864b59cc60ea1e7e863efae2bd # shrinks to (left, right) = (Root { ceiling: (0, (0, 0, 2), 3), root: Some(Node { prefix: "", children: Branch { ceiling: OnceLock(), floor: OnceLock(), leaves: 5, version_bytes: OnceLock(), children: {13: Node { prefix: "b9bc6c0254ae5e131aac9c4ff0d10a18ab41bc4913a3bbcfe8a73c09a79439", children: Leaf { version: (0, 0, 3), message: () } }, 22: Node { prefix: "e840b5f788f4c660a52976907c35698fa434ccfb1ac8c625bc3041b74681ee", children: Leaf { version: (0, (0, 0, 1), 3), message: () } }, 32: Node { prefix: "4d3be44e01d4d26c35fefb7ccc1825c9e1b643e4653608e7ba1d50ec07740a", children: Leaf { version: (0, 0, 1), message: () } }, 207: Node { prefix: "8afe2f6c1e74cca8b8a80a907bded2cf809fd0305789e5efcf84e9dee567bb", children: Leaf { version: (0, (0, 0, 2), 3), message: () } }, 224: Node { prefix: "cd3e3252fa8aa03361796d1b1d2a9449261c84d35d357607b43b1d71102e14", children: Leaf { version: (0, 0, 2), message: () } }} } }) }, Root { ceiling: (0, (0, (0, 0, 3), 0), 3), root: Some(Node { prefix: "", children: Branch { ceiling: OnceLock(), floor: OnceLock(), leaves: 6, version_bytes: OnceLock(), children: {13: Node { prefix: "b9bc6c0254ae5e131aac9c4ff0d10a18ab41bc4913a3bbcfe8a73c09a79439", children: Leaf { version: (0, 0, 3), message: () } }, 32: Node { prefix: "4d3be44e01d4d26c35fefb7ccc1825c9e1b643e4653608e7ba1d50ec07740a", children: Leaf { version: (0, 0, 1), message: () } }, 145: Node { prefix: "fa3e12c159d0b2da3a236250646402d9e75f877875d200d697324cefd0bfed", children: Leaf { version: (0, (0, (0, 0, 1), 0), 3), message: () } }, 166: Node { prefix: "804f3a0166209487f1514297735bb1b87e57a10ff2a33d1a617c38a695b56a", children: Leaf { version: (0, (0, (0, 0, 3), 0), 3), message: () } }, 204: Node { prefix: "6ef43154ec1e291df0aa528f7bfad263bc0a8e1148ad6bb07b480103c31449", children: Leaf { version: (0, (0, (0, 0, 2), 0), 3), message: () } }, 224: Node { prefix: "cd3e3252fa8aa03361796d1b1d2a9449261c84d35d357607b43b1d71102e14", children: Leaf { version: (0, 0, 2), message: () } }} } }) }), fail_left = true, operation = Read, after = 74, bytes = false, chunk = 2, delays = [0, 0, 0, 1], buffered = false From c6fac5c7dc40a015389459b6d6d4217e6671c1d1 Mon Sep 17 00:00:00 2001 From: finch Date: Wed, 29 Jul 2026 13:07:12 -0400 Subject: [PATCH 4/4] fuzz recipes name the host triple, so the sanitizer build lands on real std MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit cargo-fuzz defaults `--target` to the triple it was itself built for rather than the host's. A statically linked prebuilt — what the CI installer ships — therefore aimed the build at `x86_64-unknown-linux-musl`, and it failed twice over: the sanitizer rejects a statically linked libc outright, and that target's std was never installed, so `core` was missing. A source-built cargo-fuzz hides this entirely, its own triple being the host's. Both recipes now name the host, leaving them indifferent to how cargo-fuzz arrived. Verified with a build and a one-second smoke of each target. --- justfile | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/justfile b/justfile index b06d4f5f..bbe31c60 100644 --- a/justfile +++ b/justfile @@ -17,6 +17,15 @@ set shell := ["bash", "-euo", "pipefail", "-c"] nightly_toolchain := "nightly" +# The triple the fuzz recipes build for. cargo-fuzz defaults `--target` to the +# triple it was itself built for, not the host's, so a statically linked +# prebuilt (what the CI installer ships) aims the sanitizer build at +# `*-linux-musl`: a target whose std is absent, and whose static libc the +# sanitizer refuses outright. Naming the host keeps the recipes indifferent to +# how cargo-fuzz arrived. + +host_triple := `rustc -vV | sed -n 's/^host: //p'` + # Default fuzz smoke duration per target, in seconds (matches the guidance in # crates/before/fuzz/Cargo.toml). @@ -175,7 +184,7 @@ bench-build: # Build the libFuzzer targets (nightly). [working-directory("crates/before/fuzz")] fuzz-build: - {{ justfile_directory() }}/tools/memwatch cargo +{{ nightly_toolchain }} fuzz build + {{ justfile_directory() }}/tools/memwatch cargo +{{ nightly_toolchain }} fuzz build --target {{ host_triple }} # The decode invariant (accepted input re-encodes stably and decodes back to # itself) is asserted inline in the targets, so any hit is a crash. @@ -183,8 +192,8 @@ fuzz-build: # Short fuzz smoke: run each libFuzzer target for `secs` seconds. [working-directory("crates/before/fuzz")] fuzz secs=fuzz_smoke_secs: - cargo +{{ nightly_toolchain }} fuzz run fuzz_decode -- -max_total_time={{ secs }} - cargo +{{ nightly_toolchain }} fuzz run fuzz_decode_ops -- -max_total_time={{ secs }} + cargo +{{ nightly_toolchain }} fuzz run --target {{ host_triple }} fuzz_decode -- -max_total_time={{ secs }} + cargo +{{ nightly_toolchain }} fuzz run --target {{ host_triple }} fuzz_decode_ops -- -max_total_time={{ secs }} # ── the formal tier (formal/lean; needs elan) ──────────────────────────────── # The proofs are kernel-checked by `lake build` (pins, negative controls,