diff --git a/README.md b/README.md index e6d3443d1..43495410a 100644 --- a/README.md +++ b/README.md @@ -2120,9 +2120,9 @@ wrong, and it has. These are the results that say so, all in-tree, all published ### In the tests
-729 gate scripts, five contracts no unit test can hold, and the house rule: write the gate before the code it measures +730 gate scripts, five contracts no unit test can hold, and the house rule: write the gate before the code it measures -`test/regression.sh` names **729 gate scripts** and is the authoritative list; +`test/regression.sh` names **730 gate scripts** and is the authoritative list; `python3 test/pargates.py . ./build/ripwire -j 6` runs the same set in parallel. On top of them sit the contracts that do not fit a unit test: two runs byte-identical, warm output identical to cold, output that pipes clean through `xmllint --noout`, a sanitizer build with `-fno-sanitize-recover=all`, and a diff --git a/changelog.d/433-git-log-framing.md b/changelog.d/433-git-log-framing.md new file mode 100644 index 000000000..d4c9203b1 --- /dev/null +++ b/changelog.d/433-git-log-framing.md @@ -0,0 +1,37 @@ +### Fixed — a committed filename can no longer be read as commit metadata (`git log` framing) + +Every `git log --name-only` consumer in the history miners framed its stream with a line-based reader and a +`__C__` marker matched as a line prefix, so a repository containing a file whose name begins with the marker +bytes had that file read as commit metadata. Measured on a hostile fixture (files named `__C__`, +`__C__x|1694000000.cpp`, `__C__ 1694000000.cpp`, `ab.cpp`, `q"uote.cpp`, `sp ace.cpp`, `back\slash.cpp`, +`pi|pe.cpp`, `--dashdash.cpp`): `--owners` analysed 8 of 11 files, five of them owned by an author email that +appears in no commit, one file's top owner flipped to a different real author, and the newline-named file lost +its churn entirely. + +One reader now parses the stream for all five consumers (`gitLogFileSets`, `gitLogNameOnlyRaw`, +`gitFileCommitCountsInDayWindow`, `gitLogDecayedFileMining`, `gitFileAuthors`): +`git log -z --name-only --format=%x00__C__%x00%ae%x00%at%x00%ct`, with the marker a whole-field sentinel a path +can never spell and one field of lookahead so a merge's prefix empty is not mistaken for a record boundary. +Any POSIX-legal filename — newlines, quotes, backslashes, pipes, leading dashes, the marker itself — +round-trips byte-exact, and ownership answers with the author, never the committer. On trees with ordinary +names the answers are byte-identical to before, pinned by a golden recorded from the pre-change binary over +`--owners`, `--hotspots` and `--cochange`. + +The history-walk cache's `kQChurnCacheScheme` moves 3 → 4: a blob written by a pre-lane binary holds the old +reader's stream, which for the repositories this fix targets is the corrupted one, and no key component can +tell that blob from a fresh read. Every repository's history is walked once more after the upgrade; on trees +with ordinary names the walk produces the same stream, so the cost is one cold walk and no changed answer. + +**The walkers' stop contract rides the field read.** The line readers this replaces checked the abandon flag +before every line; the first cut of the field reader checked it only between fields, so a stream whose fields +never arrive — a git answering in its default format, or a shim streaming bare lines — held the read inside one +unterminated field until end-of-stream instead of stopping at the next unit. The flag is now polled once per +byte, and the memory-guard fixture that measures it refuses in about a second where the narrowed read blocked +for the whole stream. + +**One case is disclosed, not fixed:** a merge whose FIRST combined path is a file literally named `__C__` +still reads as pathless — its prefix empty is followed by the marker-spelled path, which the boundary rule +cannot tell from a real record boundary, so that merge's files lose their attribution (a pathless phantom +record is emitted that every consumer ignores). The full fix is a format sentinel change, owned by a +follow-on spec. `test/gitframingcheck.sh` pins both the fix and this collision's exact observable, including +where the two churn miners disagree by design. diff --git a/changelog.d/versions.md b/changelog.d/versions.md index 19cb7525c..7f9c32f2c 100644 --- a/changelog.d/versions.md +++ b/changelog.d/versions.md @@ -17,4 +17,4 @@ written by 0.6.8 is re-parsed once. `kCacheVersion` 36 → 46: a Rust binding ty (`&dyn T`, `impl T`, a bound), a Rust `impl T for X` block and a JS/TS binding written after its declaration are new binding records, as is a JS/TS module-scope object literal's binding name, a Kotlin call's trailing-lambda byte (#155) widens the reference record, and a file's parse-health record gains the Rust re-parse count, so a cache written before them is rejected and rebuilt (the branches that added them used 37, 38, 39, 41, 42 and 45). The quality-snapshot scheme moves 22 → 24 (#228: Django and -unittest hooks and closure self-dispatch change the dead set; old snapshots are not used; the branch used 23). The session legend dictionary is `dictv=05b48ed93c807ee2 entries=981`. +unittest hooks and closure self-dispatch change the dead set; old snapshots are not used; the branch used 23). The session legend dictionary is `dictv=05b48ed93c807ee2 entries=981`. The history-walk cache's `kQChurnCacheScheme` moves 3 → 4 (#433: a blob written by a pre-fix binary holds the old line-frame reader's stream, which for the repositories the framing fix targets is the corrupted one, and no key component can tell that blob from a fresh read; every repository's history is walked once more after the upgrade, and on trees with ordinary names the walk produces the same stream, so the cost is one cold walk and no changed answer). diff --git a/docs/EVALS.md b/docs/EVALS.md index 6d03c8161..845ffaadc 100644 --- a/docs/EVALS.md +++ b/docs/EVALS.md @@ -21,7 +21,7 @@ section, and it is not an afterthought. | **Co-change / known-item evals** | `--eval`, `--eval-retrieval` (see `bench/ANSWERQUALITY.md`) | Whether the tool surfaces the other files a real historical commit touched; and known-item retrieval across four rankers. | | **Ensemble calibration harness** | `bench/ensemblecal/` | Whether `--ensemble`'s four evidence families are actually orthogonal, how often each fires, how stable each is across commits — and the preset ladder derived from that (§9). | | **Differential argv harness** | `test/argvdiffcheck.sh` | That a refactor changed *nothing observable*: two binaries, every argv vector, stdout + stderr + exit code byte-identical. | -| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 729 gate scripts plus the determinism, cache-transparency and golden contracts. | +| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 730 gate scripts plus the determinism, cache-transparency and golden contracts. | | **`--quality-delta`** | `src/quality.h` | Ten measured code-quality failure modes, reported only where a change made them worse. | ### The labeling protocol (why the held-out eval is allowed to disagree with the ranker) @@ -5888,7 +5888,7 @@ copy here would be exactly the dialect divergence that gate exists to catch. Com tags, wrap, stable-order defaults), seven individually invoked standalone gates (`g1freshcheck`, `skillscan`, `htmlexport`, `compresscheck`, `handoffcheck`, `releaseinstallcheck`, `taskroutecheck`), and a single loop -naming **729 gate scripts**, all of which exist on disk. +naming **730 gate scripts**, all of which exist on disk. `python3 test/pargates.py . ./build/ripwire -j 6` runs the same scripts in parallel so a full verification fits in one sitting. It does not modify `regression.sh`. @@ -7005,7 +7005,7 @@ Listed because the reason is more useful than the silence. shipped**. See `bench/locbench/anchorhop_calib.json`. The mention anchor's reproducible numbers are the ablations in §4. - **A single round gate-count.** Two in-tree numbers disagree (`test/pargates.py`'s docstring says - ~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 729. The + ~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 730. The loop is the authority; the stale docstrings are a known drift. Since 2026-09-10 the number is not written by hand anywhere: `docs/gatecount_build.py` derives it from the loop and rewrites every published site, `test/gatecountcheck.sh` fails if any of them drifts, and `test/manifestcheck.sh` diff --git a/present/deck5_ripwire_build.js b/present/deck5_ripwire_build.js index 36a2ab66a..241f10921 100644 --- a/present/deck5_ripwire_build.js +++ b/present/deck5_ripwire_build.js @@ -1214,7 +1214,7 @@ function storyCards(s, { kick, head, stories, footText }){ kicker(s, "// how it stays true", AMBER); title(s, "Proven, not promised"); const cards = [ - ["729 gate scripts", "the suite runs on every push — plus determinism, cache-transparency and golden contracts; the gate count itself is gated against the runner's own loop"], // gatecount + ["730 gate scripts", "the suite runs on every push — plus determinism, cache-transparency and golden contracts; the gate count itself is gated against the runner's own loop"], // gatecount ["byte-identical, always", "two runs over the same tree produce the same bytes; warm equals cold. Enforced in CI, twice — Release AND a plain flavour, because NDEBUG once blinded a whole class of checks"], ["differential refactoring", "a refactor must prove it changed nothing observable: two binaries, hundreds of argv vectors, stdout + stderr + exit codes byte-identical"], ["held-out labels, authored blind", "eval labels were written by reading source before the ranker ever ran on them — so the eval is allowed to say the ranker is wrong. It has."], @@ -1238,7 +1238,7 @@ function storyCards(s, { kick, head, stories, footText }){ title(s, "Claims you can trust, because we publish what failed", { size: 32 }); card(s, MX, 1.72, 3.86, 1.72); - stat(s, "729", "gate scripts named by test/regression.sh — and the COUNT itself is gated against the runner's own loop, so it cannot go stale quietly", // gatecount + stat(s, "730", "gate scripts named by test/regression.sh — and the COUNT itself is gated against the runner's own loop, so it cannot go stale quietly", // gatecount MX+0.15, 1.86, 3.56, CYAN, { bsize: 42, bh: 0.66, lsize: 9.5 }); card(s, 4.68, 1.72, 3.86, 1.72, CARD2); stat(s, "8", "registered NEGATIVES — changes built, gated green, measured against a band written before the code, and reverted rather than tuned", @@ -1540,7 +1540,7 @@ function storyCards(s, { kick, head, stories, footText }){ ["193 long flags · 36 slides", "bash test/deckclaimcheck.sh"], ["every --flag named here exists", "bash test/deckcheck.sh"], ["74.7% fewer element bytes", "bash test/showcasecapturecheck.sh"], - ["729 gate scripts", "bash test/manifestcheck.sh"], // gatecount + ["730 gate scripts", "bash test/manifestcheck.sh"], // gatecount ["49 repos · 71 papers · 237 surveyed","bash test/readmedriftcheck.sh"], ["the ten moments, any row", "ripwire . --callers=SYM | wc -c"], ["the head-to-head table", "bench/headtohead/r4-2026-08-06/"], diff --git a/src/gitmine.h b/src/gitmine.h index 3297a6bc9..0fc70f5f7 100644 --- a/src/gitmine.h +++ b/src/gitmine.h @@ -735,6 +735,265 @@ inline GitCommandLines gitCommandLines( const std::string& cmd ) return out; } +// ── the ONE git-log record frame (audit finding #4): every --name-only consumer reads through here ── +// +// The old frame was a LINE frame: `--format=tformat:__C__…` printed a marker LINE, and the five +// consumers below each re-derived it with their own test (exact match, rfind prefix, 5-byte prefix). A +// line marker is ambiguous three ways at once — a committed file named `__C__` IS the marker line, a +// file named `__C__x|` prefix-matches it, and a filename containing '\n' splits into phantom +// records — and the ownership walk's `__C__%ae|%at` header let one forged name attribute every path +// after it to a forged author. `git log -z` closes all three: fields and paths are NUL-terminated and +// RAW (no C-quoting), and the marker is a WHOLE-FIELD sentinel a path can never spell. +// +// The pinned byte layout (specs/git-porcelain-framing/evidence/layout-pin.md; the prototype command +// that produced it is in the spec's Implementation Notes; re-pinned by test/gitframingcheck.sh): +// +// \0 __C__ \0 \0 \0 [ \0 ] [ \n | ] \0 \0 … +// +// Every record opens with an empty field, then the marker, then its three metadata fields. A commit +// with paths follows the metadata with a '\n'-prefixed first path (git's separator); a path-less +// commit stops after its metadata. A MERGE follows with one MORE empty field — the prefix empty — +// and then BARE paths with no '\n', which is what a combined diff emits; a clean merge stops after +// its prefix empty. The next record's opening empty field is the record boundary. A field that ends +// at end-of-stream without its delimiter still counts; readNulSafeField returns it. +// +// The boundary rule, with ONE field of lookahead: an empty field ends the current record and opens +// the next iff the field after it is the marker (or the stream ends). Any other empty field is a +// merge prefix and is skipped — that is what keeps a merge's combined-diff files, the ones the first +// migration draft dropped, attached to the merge that introduced them. The one collision left is the +// same class the line frame had: a merge whose FIRST combined path is a file literally named `__C__` +// reads as a pathless record, because its prefix empty is followed by the marker-spelled path. That +// boundary is disclosed here and pinned in the gate rather than papered over. +struct GitLogRecord +{ + std::string authorEmail; + std::int64_t authorTs = 0; + std::int64_t committerTs = 0; + std::vector paths; +}; + +// The walk's exit status, carried so the K51 contract (a prefix is not an answer) survives the +// migration: `unread` — git would not start, or died part-way, so the records are empty or a prefix; +// `notRun` — git never ran at all (no popen, not on PATH, not executable), so there is no prefix either. +// The visitor form streams: there is no records vector to keep, which is the point — one record in +// memory at a time, the whole-history walk costs what it always cost. +struct GitLogWalk +{ + bool unread = false; + bool notRun = false; + int exitCode = 0; + // The degraded-epoch count — records whose author or committer epoch field was empty or + // unparsable (parseEpochField nullopt): their epochs read 0, so a window cut on them excluded + // those commits and its count is a FLOOR. This field is the sink a consumer reads to disclose + // that; folding it away would be the silent-floor the honesty rules forbid. + int epochUnparsed = 0; +}; + +// One field of lookahead over a NUL-framed stream: `next` is readNulSafeField or the held field, which +// the boundary rule in forEachGitLogRecord parks here while it decides what an empty field means. +// `abandon` rides into the field read itself — the walkers' stop-at-the-next-field contract, polled +// once per byte, so a stream whose fields never arrive (a git answering in its default line format, or +// a shim streaming bare lines) cannot park the reader inside one field until end-of-stream. +struct NulFieldStream +{ + std::FILE* pipe = nullptr; + std::string held; + bool hasHeld = false; + const std::atomic* abandon = nullptr; + + bool next( std::string& field ) + { + if( hasHeld ) { field = std::move( held ); hasHeld = false; return true; } + return readNulSafeField( pipe, field, abandon ); + } +}; + +// The marker as a whole field. A path may carry these bytes; only a field that IS this string opens a record. +inline constexpr const char* kGitLogMarker = "__C__"; + +// An epoch field: git's decimal seconds. nullopt when the field is empty, carries non-digits, or +// leaves bytes unconsumed — a DEGRADED read the walk counts (GitLogWalk::epochUnparsed) instead of +// folding it into a genuine zero, because epoch 0 is 1970 and can be real, while a failed parse +// means a commit whose window position is unknown: it falls below every cutoff, and any count that +// excluded it is a floor until the consumer says so. PR #433 review. +inline std::optional parseEpochField( const std::string& field ) noexcept +{ + if( field.empty() ) { return std::nullopt; } + char* end = nullptr; + const std::int64_t v = std::strtoll( field.c_str(), &end, 10 ); + return ( end != nullptr && end != field.c_str() && *end == '\0' ) ? std::optional{ v } : std::nullopt; +} + +// A pclose()/wait status as an exit code: WEXITSTATUS on a normal exit, -1 for a signal or an unreadable status. +inline int decodedExitCode( int status ) noexcept +{ + return WIFEXITED( status ) ? WEXITSTATUS( status ) : -1; +} + +// The one command every --name-only consumer shares. `extraArgs` is the per-consumer tail: the window +// args (`--since=…`) and, for the single-file ownership query, the trailing `-- ` pathspec. +// The diff-program hardening (--no-ext-diff --no-textconv, upstream train 28) and the +// combined-diff flag ride HERE, once, instead of on five command lines that could drift apart. +// +// extraArgs rides at the END, after every option, and that position is LOAD-BEARING: a pathspec is only +// a pathspec when it comes last, and everything after `--` is read as a path or revision. Put the +// pathspec first and git consumes the `-z --name-only --format=…` that follows it as path arguments — +// the flags never apply, git answers in its DEFAULT format, and the reader sees a stream with no marker +// field in it. Measured: `--owners=SYM` answered "git unavailable / no history" on a healthy one-commit +// repo while the all-files form beside it was correct. A `--since=` window is an option and rides +// anywhere; both forms end in a space, which is what keeps the redirect from gluing on. +inline std::string gitLogRecordsCmd( const std::string& root, const std::string& extraArgs ) +{ + return gitCmd( " -c core.quotepath=false -C " ) + shSingleQuote( root ) + + " log --no-ext-diff --no-textconv " + kMergeDiffArgs + + "-z --name-only --format=%x00__C__%x00%ae%x00%at%x00%ct " + extraArgs + "2>/dev/null"; +} + +// Stream every record of one `git log --name-only` walk to `visit`, one record in memory at a time, so +// the whole-history walk costs what it always cost. Returns the walk's exit status; the records arrive +// through the visitor in the stream's order (newest commit first, git's order). +template +inline GitLogWalk forEachGitLogRecord( const std::string& root, const std::string& extraArgs, Visitor&& visit ) +{ + GitLogWalk walk; + const std::string cmd = gitLogRecordsCmd( root, extraArgs ); + std::FILE* pipe = os::popen( cmd.c_str(), "r" ); + if( !pipe ) + { + walk.unread = true; + walk.notRun = true; + walk.exitCode = -1; + return walk; + } + + // The walkers' abandon contract rides into the field read: the flag is polled per byte, so a + // stream whose fields never arrive (the default-format degrade, a test shim's bare lines) stops + // the read where the old line reader stopped it — at its next unit — instead of blocking to + // end-of-stream inside one unterminated field. + NulFieldStream fields{ pipe, {}, false, &gGitWalksAbandoned }; + std::string field; + std::string lookahead; + bool haveLookahead = false; + bool atEnd = false; + + GitLogRecord cur; + bool inRecord = false; // a record is open (its opening empty field was seen) + bool sawMarker = false; // ...and its marker was read: a record without one is never visited + bool wantMarker = false; // the next field must be the marker + int metaLeft = 0; // metadata fields still to read for this record + int empties = 0; // empty fields since this record's metadata (0 = none, 1 = a merge prefix) + bool havePath = false; // this record has taken its first path + + const auto closeRecord = [ & ]() + { + if( inRecord && sawMarker ) { visit( cur ); } + cur = GitLogRecord{}; + sawMarker = false; + }; + + while( !atEnd ) + { + if( gitWalksAbandoned() ) { break; } // the walk readers' abandon contract: stop at the next field; pclose ends the child + // COPY, not move: the slot is a reusable buffer, and a copy hands the held value over while + // leaving the slot in a valid state for its next fill. No read ever sees a moved-from slot — + // the two writers below each assign a FRESH value before they set the flag — but a move here + // is what bugprone-use-after-move reads as a second use of a moved-from object: the + // reassignment that makes it safe happens in an earlier iteration than the next move, and no + // analyzer can prove the correlation between the flag and the writers. (A swap is no better: + // it reads the slot too, so the checker relocates the finding to the move at the path append + // below.) The copy runs once per record boundary — and once per a merge's prefix empty — + // against a walk that spawns a git child per run, so it costs nothing measurable, and it + // leaves the slot's lifetime readable to man and machine alike. + if( haveLookahead ) { field = lookahead; haveLookahead = false; } + else if( !fields.next( field ) ) { atEnd = true; break; } + + // An empty field's meaning is decided by ONE field of lookahead, and the pending-metadata + // state changes only the non-boundary reading: an empty followed by the MARKER (or the end + // of the stream) is a record boundary no matter what is pending — that is also how a + // truncated phantom record (the disclosed collision) is closed without swallowing the record + // after it. An empty followed by anything else is DATA while the record still owes its + // marker or metadata — git accepts an empty author email (--author="No Email <>"), so %ae + // can arrive legitimately empty, and eating it as a merge prefix shifted the whole record + // (email←the %at digits, epochs off by one, the first path's '\n' unstripped) into a + // pathless phantom. Only a record that owes no metadata reads a non-boundary empty as a + // merge prefix. PR #433 review. + if( field.empty() ) + { + // One field of lookahead decides what this empty IS: the record boundary (the next field is + // the marker, or the stream ends here) or a merge's prefix empty (anything else follows). + // The held-lookahead slot is ALWAYS empty here: the top of the loop consumes it (and clears the + // flag) before any field is examined, and the only writers — the two branches below — each + // assign a FRESH value before the next iteration. So the field after this empty is always the + // next one on the stream; there is no held field to prefer. A `haveLookahead ? … : …` here would + // be dead code, and its move of `lookahead` is also what made the slot's lifetime unreadable to + // bugprone-use-after-move (a second move in a later iteration that no reader can prove safe). + std::string next; + bool haveNext = fields.next( next ); + if( !haveNext || next == kGitLogMarker ) + { + closeRecord(); + inRecord = true; + wantMarker = true; + metaLeft = 3; + empties = 0; + havePath = false; + if( haveNext ) { lookahead = std::move( next ); haveLookahead = true; } + continue; + } + if( wantMarker || metaLeft > 0 ) + { + // DATA: park the lookahead and fall through — the empty field itself is delivered to + // the state machine below as the pending metadata field it is (the empty-%ae shape). + lookahead = std::move( next ); haveLookahead = true; + } + else + { + ++empties; // a merge prefix (or the record's own close): skipped, but counted + lookahead = std::move( next ); haveLookahead = true; + continue; + } + } + + if( wantMarker ) + { + wantMarker = false; + if( field == kGitLogMarker ) { sawMarker = true; } // anything else: a malformed stream — skip to the next boundary + continue; + } + if( metaLeft > 0 ) + { + if( metaLeft == 3 ) { cur.authorEmail = field; } + else + { + const std::optional epoch = parseEpochField( field ); + if( !epoch ) { ++walk.epochUnparsed; } + if( metaLeft == 2 ) { cur.authorTs = epoch.value_or( 0 ); } + else { cur.committerTs = epoch.value_or( 0 ); } + } + --metaLeft; + continue; + } + + // A path. The first path of a REGULAR commit arrives with git's '\n' separator (no empty was + // counted since the metadata); a merge's first path arrives bare after its prefix empty. + if( !havePath ) + { + havePath = true; + if( empties == 0 && !field.empty() && field.front() == '\n' ) { field.erase( 0, 1 ); } + } + cur.paths.push_back( std::move( field ) ); + } + closeRecord(); + + walk.exitCode = decodedExitCode( os::pclose( pipe ) ); + if( walk.exitCode != 0 ) + { + walk.unread = true; + walk.notRun = ( walk.exitCode == 127 || walk.exitCode == 126 ); + } + return walk; +} + // The repo TOPLEVEL enclosing `absDir` (absolute and already realpath'd), or empty when `absDir` is not inside // a git repo at all. realpath'd on the way out so it compares byte-for-byte with a realpath'd file path — on // this platform `/tmp` is a symlink to `/private/tmp`, and an unresolved toplevel would fail every prefix test @@ -1501,35 +1760,20 @@ inline std::vector> gitLogFileSets( const std::string // open --name-only pipe would leave a filled pipe buffer waiting on us while we wait on the probe. const GitPathIndex byGitPath = gitPathIndexOfFiles( ing, onlyRoot ); - const std::string cmd = gitCmd( " -c core.quotepath=false -C " ) + shSingleQuote( root ) + " log --no-ext-diff --no-textconv " + kMergeDiffArgs + windowArgs + "--name-only --format=tformat:__C__ 2>/dev/null"; - std::FILE* pipe = os::popen( cmd.c_str(), "r" ); - if( !pipe ) - { - return sets; - } - std::vector cur; const auto flush = [ & ]() { recordCommitFileSet( cur, maxFiles, sets, outCensus ); }; - std::string s; - while( readByteSafeLine( pipe, s ) ) // F6: THE line reader, not a char[4096] a long path can be split across + forEachGitLogRecord( root, windowArgs, [ & ]( const GitLogRecord& rec ) { - while( !s.empty() && ( s.back() == '\n' || s.back() == '\r' ) ) + for( const std::string& p : rec.paths ) { - s.pop_back(); - } - if( s == "__C__" ) { flush(); continue; } - if( s.empty() ) - { - continue; - } - const std::uint32_t f = resolveGitPath( byGitPath, s ); - if( f != UINT32_MAX ) - { - cur.push_back( f ); + const std::uint32_t f = resolveGitPath( byGitPath, p ); + if( f != UINT32_MAX ) + { + cur.push_back( f ); + } } - } - flush(); - os::pclose( pipe ); + flush(); + } ); return sets; } @@ -1595,70 +1839,21 @@ struct RawCommitStream int exitCode = 0; }; -// K51: mark a walk unread. `exitCode` is git's decoded exit (-1: it never started, or did not exit normally); the shell's 127 -// (not on PATH) and 126 (not executable) mean git never ran either, so `commits` is empty, not a prefix. -inline void markWalkUnread( RawCommitStream& out, int exitCode, bool neverStarted ) noexcept -{ - out.unread = true; - out.exitCode = exitCode; - out.notRun = neverStarted || exitCode == 127 || exitCode == 126; -} - -// A pclose()/wait status as an exit code: WEXITSTATUS on a normal exit, -1 for a signal or an unreadable status. -inline int decodedExitCode( int status ) noexcept -{ - return WIFEXITED( status ) ? WEXITSTATUS( status ) : -1; -} - -// K51: fold the walk's pclose() status into `out` — 0 is a clean read and changes nothing. -inline void noteWalkExit( RawCommitStream& out, int status ) noexcept -{ - if( status != 0 ) - { - markWalkUnread( out, decodedExitCode( status ), false ); - } -} - inline RawCommitStream gitLogNameOnlyRaw( const std::string& root, const std::string& coSince ) { PROFILE_SCOPE_DESCRIBE( "gitmine: gitLogNameOnlyRaw (git log --name-only popen)" ); RawCommitStream out; - const std::string cmd = gitCmd( " -c core.quotepath=false -C " ) + shSingleQuote( root ) - + " log --no-ext-diff --no-textconv " + kMergeDiffArgs + "--since=" + shSingleQuote( defaultWindowSince( root, coSince ) ) // F1: HEAD-anchored when `coSince` is a default month window - + " --name-only --format=tformat:__C__%x20%ct 2>/dev/null"; - std::FILE* pipe = os::popen( cmd.c_str(), "r" ); - if( !pipe ) - { - markWalkUnread( out, -1, true ); - return out; - } - - std::string s; - while( readWalkLine( pipe, s ) ) // F6: THE line reader, not a char[4096] a long path can be split across; stops when abandoned - { - while( !s.empty() && ( s.back() == '\n' || s.back() == '\r' ) ) - { - s.pop_back(); - } - if( s.rfind( "__C__", 0 ) == 0 ) // new commit marker: "__C__ " - { - const std::int64_t epoch = ( s.size() > 6 ) ? std::strtoll( s.c_str() + 6, nullptr, 10 ) : 0; - out.commits.push_back( RawCommitStream::Commit{ epoch, {} } ); - continue; - } - if( s.empty() ) - { - continue; - } - if( out.commits.empty() ) - { - out.commits.push_back( RawCommitStream::Commit {} ); // defensive: tformat always emits the marker first, but never crash on a malformed stream - } - out.commits.back().paths.push_back( std::move( s ) ); // readByteSafeLine clear()s its buffer first, so moving out of it is safe - } - // K51: git died (a missing or corrupt object, a bad ref) — the lines above are a prefix, not the window — or the shell - // never ran it (127: not on PATH, 126: not executable), and there are no lines at all. - noteWalkExit( out, os::pclose( pipe ) ); + // F1: HEAD-anchored when `coSince` is a default month window (defaultWindowSince resolves it) + const std::string extraArgs = "--since=" + shSingleQuote( defaultWindowSince( root, coSince ) ) + " "; + const GitLogWalk walk = forEachGitLogRecord( root, extraArgs, [ & ]( const GitLogRecord& rec ) + { + out.commits.push_back( RawCommitStream::Commit{ rec.committerTs, rec.paths } ); + } ); + // K51: git died (a missing or corrupt object, a bad ref) — the records above are a prefix, not the + // window — or the shell never ran it (127: not on PATH, 126: not executable), and there are none. + out.unread = walk.unread; + out.notRun = walk.notRun; + out.exitCode = walk.exitCode; return out; } @@ -1839,57 +2034,32 @@ inline std::vector gitFileCommitCountsInDayWindow( const std::str const GitPathIndex byGitPath = gitPathIndexOfFiles( ing ); const auto resolve = [ & ]( const std::string& gp ) { return resolveGitPath( byGitPath, gp ); }; - // Stream `git log --name-only` with each commit's epoch on its __C__ marker line: "__C__ ". - // A commit counts once per file it touches, iff its epoch is within [cutoff, headEpoch]. - const std::string cmd = gitCmd( " -c core.quotepath=false -C " ) + shSingleQuote( root ) - + " log --no-ext-diff --no-textconv " + kMergeDiffArgs + "--name-only --format=tformat:__C__%x20%ct 2>/dev/null"; - std::FILE* pipe = os::popen( cmd.c_str(), "r" ); - if( !pipe ) - { - return counts; - } - - bool inWindow = false; // is the commit we are currently reading files for in-window? - std::vector cur; // this commit's resolved fileIds (dedup before tally) - const auto flush = [ & ]() + // Stream `git log --name-only` through the shared NUL-framed reader: one record per commit, its + // committer epoch in hand. A commit counts once per file it touches, iff its epoch is within + // [cutoff, headEpoch]. + std::vector cur; // this commit's resolved fileIds (dedup before tally) + forEachGitLogRecord( root, "", [ & ]( const GitLogRecord& rec ) { - if( inWindow ) + if( rec.committerTs < cutoff || rec.committerTs > headEpoch ) { - std::sort( cur.begin(), cur.end() ); - cur.erase( std::unique( cur.begin(), cur.end() ), cur.end() ); - for( std::uint32_t f : cur ) - { - ++counts[f]; - } - } - cur.clear(); - }; - std::string s; - while( readByteSafeLine( pipe, s ) ) // F6: THE line reader, not a char[4096] a long path can be split across - { - while( !s.empty() && ( s.back() == '\n' || s.back() == '\r' ) ) - { - s.pop_back(); - } - if( s.rfind( "__C__", 0 ) == 0 ) // new commit marker: "__C__ " - { - flush(); - const std::int64_t epoch = ( s.size() > 6 ) ? std::strtoll( s.c_str() + 6, nullptr, 10 ) : 0; - inWindow = ( epoch >= cutoff && epoch <= headEpoch ); - continue; + return; // out of window: this commit's files do not count } - if( s.empty() || !inWindow ) + for( const std::string& gp : rec.paths ) { - continue; + const std::uint32_t f = resolve( gp ); + if( f != UINT32_MAX ) + { + cur.push_back( f ); + } } - const std::uint32_t f = resolve( s ); - if( f != UINT32_MAX ) + std::sort( cur.begin(), cur.end() ); + cur.erase( std::unique( cur.begin(), cur.end() ), cur.end() ); + for( std::uint32_t f : cur ) { - cur.push_back( f ); + ++counts[f]; } - } - flush(); - os::pclose( pipe ); + cur.clear(); + } ); return counts; } @@ -2070,20 +2240,25 @@ inline DecayedChurnMined gitLogDecayedFileMining( const std::string& root, const // built BEFORE the log pipe opens, for the reason gitLogFileSets states: it runs a git probe of its own. const GitPathIndex byGitPath = gitPathIndexOfFiles( ing, onlyRoot ); - const std::string cmd = gitCmd( " -c core.quotepath=false -C " ) + shSingleQuote( root ) + " log --no-ext-diff --no-textconv " + kMergeDiffArgs + windowArgs - + "--name-only --format=tformat:__C__%x20%ct 2>/dev/null"; - std::FILE* pipe = os::popen( cmd.c_str(), "r" ); - if( !pipe ) + // Stream `git log --name-only` through the shared NUL-framed reader: one record per commit, its + // committer epoch in hand, its paths resolved against the index built above. + bool anyCommit = false; + forEachGitLogRecord( root, windowArgs, [ & ]( const GitLogRecord& rec ) { - return m; - } + anyCommit = true; + const std::int64_t epoch = rec.committerTs; + const std::int64_t ageSec = ( epoch > 0 && headEpoch > epoch ) ? ( headEpoch - epoch ) : 0; // clamped: never > 1 + const double curWeight = std::pow( 0.5, ( double( ageSec ) / 86400.0 ) / kChurnDecayHalfLifeDays ); - bool anyCommit = false; - double curWeight = 0.0; // this commit's decayed weight - std::int64_t curEpoch = 0; // this commit's committer epoch (F3: the newest one per file is kept) - std::vector cur; // this commit's resolved fileIds (dedup before tally) - const auto flush = [ & ]() - { + std::vector cur; // this commit's resolved fileIds (dedup before tally) + for( const std::string& gp : rec.paths ) + { + const std::uint32_t f = resolveGitPath( byGitPath, gp ); + if( f != UINT32_MAX ) + { + cur.push_back( f ); + } + } std::sort( cur.begin(), cur.end() ); cur.erase( std::unique( cur.begin(), cur.end() ), cur.end() ); if( cur.size() >= 1 && cur.size() <= maxFiles ) @@ -2091,44 +2266,14 @@ inline DecayedChurnMined gitLogDecayedFileMining( const std::string& root, const for( std::uint32_t f : cur ) { weights[f] += curWeight; - if( curEpoch > m.lastEpoch[f] ) { m.lastEpoch[f] = curEpoch; } + if( epoch > m.lastEpoch[f] ) { m.lastEpoch[f] = epoch; } // F3: the newest epoch per file is kept } } else if( cur.size() > maxFiles ) { ++m.mergeBombsSkipped; // the rule fired: disclosed on , never silently absorbed } - cur.clear(); - }; - std::string s; - while( readByteSafeLine( pipe, s ) ) // F6: THE line reader, not a char[4096] a long path can be split across - { - while( !s.empty() && ( s.back() == '\n' || s.back() == '\r' ) ) - { - s.pop_back(); - } - if( s.rfind( "__C__", 0 ) == 0 ) // new commit marker: "__C__ " - { - flush(); - anyCommit = true; - const std::int64_t epoch = ( s.size() > 6 ) ? std::strtoll( s.c_str() + 6, nullptr, 10 ) : 0; - const std::int64_t ageSec = ( epoch > 0 && headEpoch > epoch ) ? ( headEpoch - epoch ) : 0; // clamped: never > 1 - curWeight = std::pow( 0.5, ( double( ageSec ) / 86400.0 ) / kChurnDecayHalfLifeDays ); - curEpoch = epoch; - continue; - } - if( s.empty() ) - { - continue; - } - const std::uint32_t f = resolveGitPath( byGitPath, s ); - if( f != UINT32_MAX ) - { - cur.push_back( f ); - } - } - flush(); - os::pclose( pipe ); + } ); m.anyHistory = anyCommit; return m; } @@ -2430,11 +2575,12 @@ inline std::vector gitFileAuthors( }; const std::string rootSlash = root + "/"; - // git log -c --name-only --format=tformat:__C__%ae|%at 2>/dev/null (optionally scoped to one path with - // -- ): one "__C__email|unix-ts" header per commit, then its changed files, one per line, - // until a blank line / the next header. Mirrors gitCommitFileSets' exact invocation style. - std::string cmd = gitCmd( " -c core.quotepath=false -C " ) + shSingleQuote( root ) - + " log --no-ext-diff --no-textconv " + kMergeDiffArgs + "--name-only --format=tformat:__C__%ae\\|%at"; + // git log -c -z --name-only --format='%x00__C__%x00%ae%x00%at%x00%ct' 2>/dev/null (optionally + // scoped to one path with `-- `): one record per commit through the shared NUL-framed + // reader, author email and author timestamp as their own fields. NO trailing %x00 — the last + // field runs to the stream's end, and the boundary rule reads the empty that FOLLOWS it. + // Mirrors every other consumer's single-pass invocation style. + std::string extraArgs; if( singleFile ) { const std::string& fp = ing.files[ onlyFileId ]; @@ -2443,65 +2589,49 @@ inline std::vector gitFileAuthors( { relPath = fp.substr( rootSlash.size() ); } - cmd += " -- " + shSingleQuote( relPath ); + // extraArgs rides at the END of the command (see gitLogRecordsCmd): a pathspec is only a pathspec + // when it comes last, and anything after `--` is read as a path or revision. The trailing space + // keeps the `2>/dev/null` redirect from gluing onto the quoted path. Both properties are + // load-bearing — without the first, git reads the following -z --name-only --format=… as path + // arguments and answers in its DEFAULT format, which the reader sees as a stream with no marker + // in it (measured as "git unavailable / no history" while the all-files form was correct). + extraArgs = " -- " + shSingleQuote( relPath ) + " "; } - cmd += " 2>/dev/null"; - std::FILE* pipe = os::popen( cmd.c_str(), "r" ); - if( !pipe ) + forEachGitLogRecord( root, extraArgs, [ & ]( const GitLogRecord& rec ) { - return {}; - } - - std::string curEmail; - std::int64_t curTs = 0; - bool haveCommit = false; - std::string s; - while( readByteSafeLine( pipe, s ) ) // F6: THE line reader, not a char[4096] a long path can be split across - { - while( !s.empty() && ( s.back() == '\n' || s.back() == '\r' ) ) + // The old line frame parsed `email|ts` out of one field and skipped the commit's files when + // either half was missing or the epoch did not parse. Under the NUL frame the epoch half of + // that guard stands alone: git never stamps a real commit at or before the epoch, and a + // header that forged one is exactly what this frame exists to make impossible. The EMAIL half + // had to go — git accepts an empty author email (--author="No Email <>", PR #433 review), and + // the frame now delivers such a record intact, so skipping on an empty email would discard + // real history; the ownership key for it is honestly empty. + if( rec.authorTs <= 0 ) { - s.pop_back(); + return; } - if( s.compare( 0, 5, "__C__" ) == 0 ) + std::string curEmail = rec.authorEmail; + for( char& c : curEmail ) { - const std::string_view rest( s.data() + 5, s.size() - 5 ); - const std::size_t pipePos = rest.find( '|' ); - if( pipePos == std::string_view::npos || pipePos == 0 || pipePos + 1 >= rest.size() ) + c = char( std::tolower( static_cast( c ) ) ); + } + const std::int64_t curTs = rec.authorTs; + for( const std::string& gp : rec.paths ) + { + const std::uint32_t f = resolve( gp ); + if( f == UINT32_MAX ) { - haveCommit = false; continue; } - curEmail = rest.substr( 0, pipePos ); - for( char& c : curEmail ) + perFile[ f ][ curEmail ].push_back( curTs ); + std::int64_t& newest = newestTsByFile[ f ]; + if( curTs > newest ) { - c = char( std::tolower( static_cast( c ) ) ); + newest = curTs; } - const std::string tsStr( rest.substr( pipePos + 1 ) ); - char* end = nullptr; - curTs = std::strtoll( tsStr.c_str(), &end, 10 ); - haveCommit = ( end != tsStr.c_str() ); - continue; } - if( s.empty() || !haveCommit ) - { - continue; // blank separator line, or malformed header — skip its files - } - - const std::uint32_t f = resolve( s ); - if( f == UINT32_MAX ) - { - continue; - } - - perFile[ f ][ curEmail ].push_back( curTs ); - std::int64_t& newest = newestTsByFile[ f ]; - if( curTs > newest ) - { - newest = curTs; - } - } - os::pclose( pipe ); + } ); std::vector result; result.reserve( perFile.size() ); diff --git a/src/infra/stdinline.h b/src/infra/stdinline.h index 214760293..1c187abdb 100644 --- a/src/infra/stdinline.h +++ b/src/infra/stdinline.h @@ -32,6 +32,7 @@ #include "Diagnostics.h" // ASSUME — the null-stream precondition +#include #include #include @@ -136,4 +137,40 @@ inline bool readByteSafeLineBounded( std::FILE* in, std::string& line, std::size return detail::readByteSafeLineCore( in, line, maxBytes, overflowed ); } +// The NUL-framed sibling of readByteSafeLine, for `git log -z` byte streams: -z terminates every +// --format field and every --name-only path with NUL instead of '\n', and an EMPTY field is the record +// boundary the parser must see — so, unlike the line reader, a zero-length field is a SUCCESSFUL read +// here, and only end-of-stream with nothing read returns false. Same parity contract as +// readByteSafeLine: one fgetc per byte, EOF as an int sentinel distinct from every byte value, the only +// narrowing explicit through `unsigned char`. Kept a separate function rather than a delimiter +// parameter on the line reader because the two delimiters answer different questions ("where does this +// line end" vs "is this field the record boundary") and a caller that confused them would read a +// hostile filename as commit metadata. +// +// `abandon`, when non-null, is polled once per byte: the git history walkers stop at their next FIELD, +// and a stream whose fields never arrive — a git answering in its default line format, or a test shim +// streaming bare lines — must still abandon INSIDE the field read rather than park the reader in one +// never-terminated field until end-of-stream. When it flips, the partial field is dropped and false is +// returned, which the caller reads as end-of-stream; it then closes the pipe, and that is what ends +// the child. false is ALSO end-of-stream, so a caller that must tell the two apart consults the flag +// itself — which is what the walkers already do at the top of their loop. +inline bool readNulSafeField( std::FILE* in, std::string& field, const std::atomic* abandon = nullptr ) +{ + ASSUME( in != nullptr ); + + field.clear(); + bool didReadAnyByte = false; + for( int byteOrEof = std::fgetc( in ); byteOrEof != EOF; byteOrEof = std::fgetc( in ) ) + { + if( abandon != nullptr && abandon->load( std::memory_order_relaxed ) ) { return false; } + didReadAnyByte = true; + if( byteOrEof == '\0' ) + { + return true; // delimiter consumed, not appended + } + field.push_back( static_cast( static_cast( byteOrEof ) ) ); + } + return didReadAnyByte; // EOF: a final unterminated field still counts; nothing read is the end +} + } // namespace rw diff --git a/src/quality.h b/src/quality.h index 18d9e5e77..a03acd9f3 100644 --- a/src/quality.h +++ b/src/quality.h @@ -5071,7 +5071,16 @@ computeWindowRefBodyHashes( const std::string& root, std::uint32_t days, // newest prefix of the window) as if it were the window, and the key cannot tell that blob from a real read. K51 never // stores one, but a blob already on disk would keep serving its silent zero or prefix, with no history_unread=, until HEAD // moved. The bump makes every pre-K51 blob a clean miss; the cost is one cold walk per repository. -constexpr std::uint32_t kQChurnCacheScheme = 3; +// SCHEME 4 (2026-10-23, the git-porcelain-framing lane): the walk's READER was rebuilt — the line-framed +// `__C__` marker parse became one NUL-framed reader with a whole-field sentinel and one field of lookahead +// (test/gitframingcheck.sh). On a repository whose committed paths spell the marker bytes, carry a +// newline, or sort after a path that does, the OLD reader's stream was corrupted (forged commit headers, +// swallowed paths), so a blob written by a pre-lane binary would keep serving that corrupted stream to a +// fixed one until HEAD moved — the same wrong-answer-from-cache as scheme 2, one layer down: there the +// COMMAND changed under an unchanged key, here the PARSE did. Neither is visible to the key, which is why +// this bump is manual. On ordinary trees the two readers' streams are byte-identical (the lane's golden +// arm over owners/hotspots/cochange), so the bump costs those repos one cold walk and changes no answer. +constexpr std::uint32_t kQChurnCacheScheme = 4; constexpr char kQChurnMagic[4] = { 'Q', 'C', 'H', 'N' }; inline std::string serializeRawCommitStream( const RawCommitStream& raw, const std::string& keyMat ) diff --git a/test/churnjoincheck.sh b/test/churnjoincheck.sh index e495c2cba..4c5602dbb 100755 --- a/test/churnjoincheck.sh +++ b/test/churnjoincheck.sh @@ -849,14 +849,20 @@ fi # and those are safe BY SHAPE", and pinned the COUNT of readByteSafeLine adoptions at five. Both halves were the # trap. Counting adoptions cannot see a survivor (that is trap #10 stated exactly), and the count itself rots # the moment a reader is legitimately added — as it did here. The bar is now: no fixed-buffer reader survives -# ANYWHERE in the file (§9c asserts that from source, comments excluded), and at LEAST the five path readers -# go through the byte-safe one. A lower bound cannot rot upward; a survivor still fails §9c. +# ANYWHERE in the file (§9c asserts that from source, comments excluded), and the git-pipe readers go through a +# BYTE-SAFE reader — of EITHER family, because the git-porcelain-framing lane (test/gitframingcheck.sh) moved +# the five `--name-only` consumers onto the shared NUL-framed reader forEachGitLogRecord (which reads fields +# through readNulSafeField), and left the byte-safe LINE reader (readByteSafeLine, behind readWalkLine) +# carrying gitCommandLines' own output. A lower bound cannot rot upward; a survivor still fails §9c. LEFTOVER="$( grep -c 'char line\[ 4096 \]' "$ROOT/src/gitmine.h" || true )" SAFEREADERS="$( grep -c 'readByteSafeLine( pipe' "$ROOT/src/gitmine.h" || true )" +NULREADERS="$( grep -c 'forEachGitLogRecord( root' "$ROOT/src/gitmine.h" || true )" [ "${LEFTOVER:-1}" = 0 ] && ok "F6/completeness: no 'char line[ 4096 ]' path reader survives in gitmine.h" \ || no "F6/completeness: $LEFTOVER fixed-buffer path reader(s) still in gitmine.h — a long path can still be split there" -[ "${SAFEREADERS:-0}" -ge 5 ] && ok "F6/completeness: all five git-pipe PATH readers go through readByteSafeLine ($SAFEREADERS call sites: gitCommandLines, gitLogFileSets, gitLogNameOnlyRaw, gitFileCommitCountsInDayWindow, gitFileAuthors, + any later adopter)" \ - || no "F6/completeness: only $SAFEREADERS readByteSafeLine call site(s) in gitmine.h, expected at least the five path readers" +[ "${SAFEREADERS:-0}" -ge 1 ] && ok "F6/completeness: the line-shaped git-pipe reader keeps its byte-safe core (readByteSafeLine behind readWalkLine, $SAFEREADERS site — gitCommandLines' own output)" \ + || no "F6/completeness: no readByteSafeLine core site in gitmine.h — a git-pipe line reader lost its byte-safe read" +[ "${NULREADERS:-0}" -ge 5 ] && ok "F6/completeness: all five git-pipe PATH readers go through the shared byte-safe NUL reader forEachGitLogRecord ($NULREADERS call sites: gitLogFileSets, gitLogNameOnlyRaw, gitFileCommitCountsInDayWindow, gitLogDecayedFileMining, gitFileAuthors, + any later adopter)" \ + || no "F6/completeness: only $NULREADERS forEachGitLogRecord call site(s) in gitmine.h, expected at least the five path readers" # ── 9c. G3 — the migration claim is re-derived FROM SOURCE, not read from the comment that makes it ── # F6 recorded "every git-pipe reader in this file uses readByteSafeLine"; it was true at one of five. The fix diff --git a/test/crashsweepcheck.sh b/test/crashsweepcheck.sh index 7cee7aa4a..636770c06 100755 --- a/test/crashsweepcheck.sh +++ b/test/crashsweepcheck.sh @@ -305,13 +305,9 @@ diffhunks.h gitDiffHunksVsHead popen 1 closes continue-only loop; pclose after i docparse.h openRegularFileStream fdopen 1 owned stored straight into pathguard::NoFollowRead, whose destructor fcloses it docparse.h openRegularFileStream open 1 owned fdopen'd into NoFollowRead; ::close only when fdopen declined it docparse.h runMarkitdown popen 1 closes no exit between popen and pclose +gitmine.h forEachGitLogRecord popen 1 closes the shared --name-only reader: the walk loop leaves by stream end or the abandon break; one pclose after it, on the only exit past the failed open gitmine.h gitCommandLines popen 1 closes status = pclose after the loop; no exit between -gitmine.h gitFileAuthors popen 1 closes continue-only loop; pclose after it -gitmine.h gitFileCommitCountsInDayWindow popen 1 closes continue-only loop; pclose after it -gitmine.h gitLogDecayedFileMining popen 1 closes continue-only loop; pclose after the flush infra/os_win32.cpp system popen 1 closes Windows body of os::system (compiled only for Windows): the drain loop has no exit; pclose after it -gitmine.h gitLogFileSets popen 1 closes continue-only loop; pclose after the flush -gitmine.h gitLogNameOnlyRaw popen 1 closes continue-only loop; pclose after it gitmine.h popenTrimmed popen 1 closes no exit between popen and pclose gitoracle.h loadOracleCache fopen 1 closes returns only on a failed open; fclose after the read loop gitoracle.h saveOracleCache fdopen 1 closes adopts ExclTempFile's released fd; fclose on its own line; a failed fdopen ::closes the fd diff --git a/test/deckcheck_allowlist.txt b/test/deckcheck_allowlist.txt index eb6e22286..15b5aee91 100644 --- a/test/deckcheck_allowlist.txt +++ b/test/deckcheck_allowlist.txt @@ -140,3 +140,4 @@ --no-ext-diff git's own flag, named in the CHANGELOG's git-subprocess hardening entry as an option ripwire passes to its own git diff/log/show/blame children, not a ripwire flag --no-textconv git's own flag, same CHANGELOG hardening entry as --no-ext-diff above, not a ripwire flag --ignore-submodules git's own flag ("--ignore-submodules=dirty" on status/diff), same CHANGELOG hardening entry as --no-ext-diff above, not a ripwire flag +--dashdash not a flag at all — the leading bytes of the hostile fixture's FILENAME `--dashdash.cpp` (test/gitframingcheck.sh), quoted in the CHANGELOG's git-log framing entry where that fixture's file names are enumerated; a filename, never a ripwire flag diff --git a/test/gitframingcheck.sh b/test/gitframingcheck.sh new file mode 100755 index 000000000..b92a403e3 --- /dev/null +++ b/test/gitframingcheck.sh @@ -0,0 +1,530 @@ +#!/usr/bin/env bash +# gitframingcheck.sh — the git-log FRAMING gate: a committed filename can never be read as commit +# metadata, and the NUL frame that guarantees it stays pinned. +# +# THE DEFECT (audit finding #4, Medium). Every `git log --name-only` consumer in src/gitmine.h framed +# its stream with a LINE-based reader and a `__C__` marker matched as a line PREFIX — three spellings +# across five sites — so a path whose name begins with the marker bytes was read as commit metadata. +# Measured on the hostile fixture below with the PRE-CHANGE binary: +# files="8" of 11 indexed — the four marker-prefixed names (`__C__`, `__C__ 1694000000.cpp`, +# `__C__.cpp`, `__C__x|1694000000.cpp`) read as forged headers instead of paths +# five files owned by top="x", an author email that appears in NO commit +# (`__C__x|1694000000.cpp` sorts before lowercase a..z, so every path after it inherits the +# forgery; `--dashdash.cpp` and `Alpha.cpp`, sorting before it, keep their true author) +# zeta.cpp carrying x as a co-author (authors=3), its top owner flipped to quiet2@x.com — the +# forgery changes the ANSWER, not a detail +# +# THE FIX (lane/git-porcelain-framing-v2). One reader parses the stream for all five consumers: +# `git log -z --name-only --format=%x00__C__%x00%ae%x00%at%x00%ct`. Fields and paths are +# NUL-terminated and RAW (core.quotepath=false), and the marker is a WHOLE-FIELD sentinel a path can +# never spell. An empty field — which only the format can emit, because paths are never empty — is +# the record boundary, read with ONE field of lookahead so a merge's prefix empty is not mistaken +# for one. The five consumers (gitLogFileSets, gitLogNameOnlyRaw, gitFileCommitCountsInDayWindow, +# gitLogDecayedFileMining, gitFileAuthors) became visitors over forEachGitLogRecord. +# +# THE PINNED LAYOUT (specs/git-porcelain-framing/evidence/layout-pin.md, derived empirically — the +# '\n' before a commit's first path, the merge prefix empty and the end-of-stream unterminated field +# are NOT documented git behaviour): +# \0 __C__ \0 \0 \0 [ \0 ] [ \n | ] \0 \0 ... +# regular commit: metadata, then a '\n'-prefixed first path. empty commit: metadata only. +# merge: metadata, a prefix empty, then BARE paths. clean merge: metadata + prefix empty. +# MINIMUM GIT 2.43.0 (arm 8 refuses below it): a git that changes any of the above fails arm 2 +# loudly instead of silently corrupting the first path. +# +# ARMS +# (a) the spoof is neutralized — every --owners row on the hostile fixture names its TRUE author, +# and no forged-author row exists anywhere. +# (b) the newline filename round-trips — ab.cpp is exactly one path, churn=1, owned. +# (c) the exact-marker file is a path — a file named exactly `__C__` never opens a record; the +# indexed file after it in byte order keeps its attribution. +# (c2) the single-file pathspec seam — --owners=SYM appends a pathspec to the shared command, and a +# pathspec glued to -z answers EMPTY, which reads as "git unavailable / no history" rather than +# as a broken command. The pin for the one space that separates them. +# (f) an empty METADATA field is data, not framing — git accepts an empty author email +# (--author="No Email <>"), and the boundary rule must not eat a pending-metadata empty as a +# merge prefix: pre-fix the whole record shifted (email←the %at digits, epochs off by one, the +# first path's '\n' unstripped) and the commit was left PATHLESS, its file unattributed. +# (d) the honest tree is byte-identical to the golden RECORDED FROM THE PRE-CHANGE BINARY — the +# "no behavior change on honest trees" claim, over --owners/--hotspots/--cochange. +# (e) the marker literal lives ONLY inside the shared helper — five consumer bodies, zero hits. +# 2. the raw-stream digest + structural count (11 records, 13 marker-spelled byte sequences). +# 3. the merge shapes: an ordinary name-bearing merge's files are ATTACHED (the lookahead is +# load-bearing — a parser without it drops them); the DISCLOSED collision is pinned. +# 4. author-not-committer: ownership answers with the AUTHOR, never the committer. +# 5. non-vacuity: the fixture is what the arms claim (12 tracked, 11 indexed, the extension-less +# `__C__` unindexed BY DESIGN per layout-pin Finding 1). +# 6. determinism: two runs byte-identical. +# 7. the oracle: the tool's numbers re-derive from its own command (trap #12). +# 8. the git floor, and a shimmed git that rejects -z degrades honestly — no fabricated rows. +# +# THE DISCLOSED COLLISION (not papered over; full fix = a format sentinel change, follow-on spec +# material): a merge whose FIRST combined path is a file literally named `__C__` reads as pathless — +# its prefix empty is followed by the marker-spelled path, which the boundary rule cannot tell from +# a real record boundary. The fixture carries exactly such a merge (MERGE, resolving `__C__` and +# `__C__.cpp`), and arm 3 pins its observable: the merge's files lose their attribution, and a +# pathless phantom record (whose "author" is the marker-spelled path) is emitted that every consumer +# no-ops on. Note the two miners disagree there BY DESIGN of the disclosure: --for's churn (the +# migrated reader) reads 3 for `__C__.cpp`, while --hotspots' churn rides the unmigrated +# gitCommandLines miner and reads 4. +# +# EXPIRY: like test/mergechurncheck.sh this fixture hardcodes 2026-06 dates; the churn verbs mine a +# HEAD-anchored 12-month window, so the numbers are stable, but a future reader moving the dates +# must re-derive every literal here. The digest pins the exact stream. +# +# Usage: test/gitframingcheck.sh # build/ripwire +# test/gitframingcheck.sh asan/ripwire # positional seam +# RIPWIRE_BIN=asan/ripwire test/gitframingcheck.sh # env seam +set -u +ROOT="$( cd "$( dirname "$0" )/.." && pwd )" +. "$ROOT/test/lib/clean-env.sh" +# (5) THE TIMEZONE. The fixture's dates are naive strings ("2026-06-01T12:00:00"), and git reads a +# naive GIT_AUTHOR_DATE in the MACHINE'S zone — so every %at/%ct in the stream moves with the host, +# and with them the digest pin below and the golden. Measured: the same fixture, the same command, +# the same git, reads a823b914… under WEST and 9a112638… under UTC — CI's runners are UTC, a dev Mac +# is not, and the gate failed on exactly that. Pin it HERE rather than in clean-env.sh: that file is +# sourced by gates whose own fixtures (mergechurncheck and its 2026-06 dates) were derived on a +# non-UTC machine, and shifting them is a separate change with its own re-derivation. +export TZ=UTC +BIN="${1:-${RIPWIRE_BIN:-$ROOT/build/ripwire}}" # BOTH seams — positional and env +[ "${BIN#/}" = "$BIN" ] && BIN="$ROOT/$BIN" +GOLD="$ROOT/test/gitframingcheck_golden.txt" +SRC="$ROOT/src/gitmine.h" +fail=0 +ok(){ printf ' PASS %s\n' "$*" || { fail=1; printf ' FAIL could not write the PASS line for: %s\n' "$*"; }; return 0; } +no(){ printf ' FAIL %s\n' "$*"; fail=1; } + +[ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first (cmake --build build -j)"; exit 2; } +command -v git >/dev/null 2>&1 || { echo "git required"; exit 2; } +REAL_GIT="$( command -v git )" +echo "gitframingcheck: BIN=$BIN ROOT=$ROOT git=$( git --version )" + +TMP="$( mktemp -d )"; H="$( mktemp -d )"; G="$( mktemp -d )"; E="$( mktemp -d )" +trap 'rm -rf "$TMP" "$H" "$G" "$E"' EXIT +# The fixtures are FRESH mktemp paths every run, and that is load-bearing beyond tidiness: the tool +# memoizes its history walks under $TMPDIR/ripwire keyed on (repo, HEAD, window) — the repo key is +# path-derived — so a fixed fixture path would let a previous run's cached churn answer this run's +# questions. A fresh path is a cold cache: every number below is computed, never remembered. + +mkfn(){ printf 'int %s( int x )\n{\n if( x > 1 ) return x + 1;\n return x - 1;\n}\n' "$1"; } +D(){ export GIT_AUTHOR_DATE="$1" GIT_COMMITTER_DATE="$1"; } +ID(){ export GIT_AUTHOR_NAME="$1" GIT_AUTHOR_EMAIL="$2" GIT_COMMITTER_NAME="$3" GIT_COMMITTER_EMAIL="$4"; } +UC(){ unset GIT_AUTHOR_DATE GIT_COMMITTER_DATE GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL GIT_COMMITTER_NAME GIT_COMMITTER_EMAIL; } +# attr_of FILE PATHSUFFIX ATTR — the ATTR value of the row whose p= ends with PATHSUFFIX ("" if no row). +# p= is XML-escaped in the output, so ab.cpp spells p="a b.cpp" and q"uote.cpp spells +# p="q"uote.cpp"; callers pass the ESCAPED spelling, which is also the stable one. +psel(){ awk -v s="$1" '{ if( match( $0, /p="[^"]*"/ ) ) { v = substr( $0, RSTART + 3, RLENGTH - 4 ); + if( v == s || ( length(v) > length(s) && substr( v, length(v) - length(s) ) == "/" s ) ) print } }'; } +attr_of(){ tr '>' '\n' < "$1" | psel "$2" | grep -oE "$3=\"[^\"]*\"" | head -1 | sed 's/^[^"]*"//; s/"$//'; } +churn_of_fn(){ tr '>' '\n' < "$1" | grep -F "n=\"$2\"" | grep -oE 'churn="[0-9]+"' | head -1 | tr -cd '0-9'; } +# code_only FILE — FILE with comment-only lines dropped, for "the literal appears in CODE" counts. +# This codebase's block-comment interior lines start with '*', and both code sites carrying the marker +# are plain code lines, so the count is exact for the pins below. +code_only(){ awk '{ l=$0; sub(/^[ \t]+/,"",l); if( l ~ /^\/\// || l ~ /^\*/ || l ~ /^\/\*/ ) next; print }' "$1"; } + +# ── THE HOSTILE FIXTURE ───────────────────────────────────────────────────────────────────────── +# History, read commit by commit (these readings ARE the literals asserted below — never computed +# from the tool, never from the same git flag the code uses): +# c1 Real 2026-06-01 creates the 12-file battery: Alpha.cpp, zeta.cpp, and the +# hostile names `__C__`, `__C__ 1694000000.cpp`, `__C__.cpp`, +# `__C__x|1694000000.cpp`, `q"uote.cpp`, `sp ace.cpp`, `back\slash.cpp`, +# `pi|pe.cpp`, `--dashdash.cpp`, `ab.cpp +# c2 author Real, COMMITTER Committer 2026-06-03 — appends zeta.cpp +# (the author-not-committer arm: author date 06-02, committer date 06-03) +# s1 Sidedev 2026-06-04 on branch `side`: modifies `__C__` and `__C__.cpp` +# m1 Real 2026-06-05 on main: modifies the same two differently (a conflict on both) +# MERGE author Real, COMMITTER Mergecommitter 2026-06-07 — resolves BOTH +# with content neither parent had, so the combined diff is +# [`__C__`, `__C__.cpp`] and its FIRST path is the marker-spelled file: +# THE DISCLOSED COLLISION. The merge reads as pathless. +# c3 Real 2026-06-08 --allow-empty: the path-less record +# q1 Quietdev 2026-06-09 on branch `quiet`: modifies Alpha.cpp +# m2 Real 2026-06-10 on main: modifies Alpha.cpp the other way +# q2 Quietdev 2026-06-11 on branch `quiet2`: appends zeta.cpp +# CLEAN Real 2026-06-12 `git merge -s ours --no-ff quiet2`: a real two-parent merge whose +# combined diff is EMPTY (the clean-merge shape) +# M2 author Real, COMMITTER Mergecommitter 2026-06-13 — merges `quiet`, resolves Alpha.cpp +# with new content: a name-bearing merge whose first combined path is an +# ORDINARY file. The lookahead attaches it; a parser without one drops it. +mkdir -p "$H" +git -C "$H" init -q -b main +C(){ git -C "$H" add -A >/dev/null; git -C "$H" commit -qm "$1"; } + +# layout-pin Finding 1: an unindexed hostile name is invisible to every miner, so the forgery is only +# observable where a hostile name sorts BEFORE an indexed file. `__C__` (0x5F) sorts before lowercase +# a..z; `Alpha.cpp` (0x41) sorts before it. The extension-less `__C__` is unindexed BY DESIGN — its +# path-level survival is pinned by the digest (arm 2), and its danger through the files after it. +ID Real real@x.com Real real@x.com; D 2026-06-01T12:00:00 +mkfn alphaFn > "$H/Alpha.cpp" +mkfn zetaFn > "$H/zeta.cpp" +mkfn cFn > "$H/__C__" +mkfn spFn > "$H/__C__ 1694000000.cpp" +mkfn dotFn > "$H/__C__.cpp" +mkfn spoofFn > "$H/__C__x|1694000000.cpp" +mkfn qFn > "$H/q\"uote.cpp" +mkfn spFn2 > "$H/sp ace.cpp" +mkfn bsFn > "$H/back\\slash.cpp" +mkfn piFn > "$H/pi|pe.cpp" +mkfn ddFn > "$H/--dashdash.cpp" +mkfn nlFn > "$H/a"$'\n'"b.cpp" +C c1 + +ID Real real@x.com Committer committer@x.com +D 2026-06-02T12:00:00; export GIT_COMMITTER_DATE="2026-06-03T09:00:00" # author 06-02, committer 06-03 +printf '// c2 edit\n' >> "$H/zeta.cpp" +C c2 +unset GIT_COMMITTER_DATE + +git -C "$H" checkout -q -b side +ID Sidedev side@x.com Sidedev side@x.com; D 2026-06-04T12:00:00 +printf 'int cFn( int x )\n{\n return x + 2;\n}\n' > "$H/__C__" +printf 'int dotFn( int x )\n{\n return x + 3;\n}\n' > "$H/__C__.cpp" +C s1 + +git -C "$H" checkout -q main +ID Real real@x.com Real real@x.com; D 2026-06-05T12:00:00 +printf 'int cFn( int x )\n{\n return x + 4;\n}\n' > "$H/__C__" +printf 'int dotFn( int x )\n{\n return x + 5;\n}\n' > "$H/__C__.cpp" +C m1 + +ID Real real@x.com Mergecommitter mergecommitter@x.com; D 2026-06-06T12:00:00 +export GIT_COMMITTER_DATE="2026-06-07T08:00:00" +git -C "$H" merge --no-commit --no-ff side >/dev/null 2>&1 || true # conflicts on both by construction +printf 'int cFn( int x )\n{\n return x + 6;\n}\n' > "$H/__C__" +printf 'int dotFn( int x )\n{\n if( x > 1 ) return x + 7;\n return x - 7;\n}\n' > "$H/__C__.cpp" +C "merge side (collision: first combined path is the file named __C__)" +unset GIT_COMMITTER_DATE + +ID Real real@x.com Real real@x.com; D 2026-06-08T12:00:00 +git -C "$H" commit -q --allow-empty -m c3 + +git -C "$H" checkout -q -b quiet +ID Quietdev quiet@x.com Quietdev quiet@x.com; D 2026-06-09T12:00:00 +printf 'int alphaFn( int x )\n{\n return x + 8;\n}\n' > "$H/Alpha.cpp" +C q1 + +git -C "$H" checkout -q main +ID Real real@x.com Real real@x.com; D 2026-06-10T12:00:00 +printf 'int alphaFn( int x )\n{\n return x + 9;\n}\n' > "$H/Alpha.cpp" +C m2 + +git -C "$H" checkout -q -b quiet2 +ID Quietdev quiet2@x.com Quietdev quiet2@x.com; D 2026-06-11T12:00:00 +printf '// q2 edit\n' >> "$H/zeta.cpp" +C q2 +git -C "$H" checkout -q main +ID Real real@x.com Real real@x.com; D 2026-06-12T12:00:00 +git -C "$H" merge -s ours --no-commit --no-ff quiet2 >/dev/null 2>&1 || true # clean: main's tree wins +C "merge quiet2 (clean: -s ours, empty combined diff)" + +ID Real real@x.com Mergecommitter mergecommitter@x.com; D 2026-06-13T12:00:00 +git -C "$H" merge --no-commit --no-ff quiet >/dev/null 2>&1 || true # conflicts on Alpha.cpp by construction +printf 'int alphaFn( int x )\n{\n if( x > 1 ) return x + 10;\n return x - 10;\n}\n' > "$H/Alpha.cpp" +C "merge quiet (name-bearing: resolves Alpha.cpp, first combined path is ordinary)" +UC + +# ── THE HONEST FIXTURE (arm d) ───────────────────────────────────────────────────────────────────── +# A tree with NO hostile names. The claim under test: the new reader changes NOTHING here — the golden +# was recorded from the PRE-CHANGE binary over exactly this history. The fixture is byte-deterministic +# (every date, identity and message pinned), so its HEAD sha — and the golden's at= — are stable; a +# reader who moves a date must re-record the golden, which is the EXPIRY note above made concrete. +# b1 Basedev 2026-06-01 Alpha.cpp, zeta.cpp, shared.cpp +# s1 Sidedev 2026-06-02 on `side`: adds sidefile.cpp, appends shared.cpp +# m1 Basedev 2026-06-03 on main: appends shared.cpp differently (a conflict on both sides) +# MERGE author Mergeauthor, COMMITTER Mergecommitter 2026-06-04/05 — a name-bearing merge: resolves +# the conflict with new content and adds mergeonly.cpp; its first combined path is ORDINARY +# c2 Basedev 2026-06-06 appends Alpha.cpp and zeta.cpp +mkdir -p "$G" +git -C "$G" init -q -b main +CG(){ git -C "$G" add -A >/dev/null; git -C "$G" commit -qm "$1"; } +ID Basedev base@x.com Basedev base@x.com; D 2026-06-01T12:00:00 +mkfn baseFn > "$G/Alpha.cpp" +mkfn zetaFn > "$G/zeta.cpp" +mkfn sharedFn > "$G/shared.cpp" +CG b1 +git -C "$G" checkout -q -b side +ID Sidedev side@x.com Sidedev side@x.com; D 2026-06-02T12:00:00 +mkfn branchFn > "$G/sidefile.cpp" +printf '// s1\n' >> "$G/shared.cpp" +CG s1 +git -C "$G" checkout -q main +ID Basedev base@x.com Basedev base@x.com; D 2026-06-03T12:00:00 +printf '// m1\n' >> "$G/shared.cpp" +CG m1 +ID Mergeauthor mergeauthor@x.com Mergecommitter mergecommitter@x.com; D 2026-06-04T12:00:00 +export GIT_COMMITTER_DATE="2026-06-05T08:00:00" # author 06-04, committer 06-05 +git -C "$G" merge --no-commit --no-ff side >/dev/null 2>&1 || true +mkfn mergeOnlyFn > "$G/mergeonly.cpp" +mkfn sharedFn > "$G/shared.cpp"; printf '// resolved in the merge itself\n' >> "$G/shared.cpp" +CG "merge side (name-bearing)" +unset GIT_COMMITTER_DATE +ID Basedev base@x.com Basedev base@x.com; D 2026-06-06T12:00:00 +printf '// c2\n' >> "$G/Alpha.cpp"; printf '// c2\n' >> "$G/zeta.cpp" +CG c2 +UC + +# ── THE TOOL'S ANSWERS ON THE HOSTILE FIXTURE ────────────────────────────────────────────────────── +"$BIN" --owners --detail=1 --limit=60 "$H" > "$TMP/owners.xml" 2>"$TMP/owners.err"; owners_rc=$? +"$BIN" --hotspots --limit=60 "$H" > "$TMP/hotspots.xml" 2>"$TMP/hotspots.err" +for fn in alphaFn dotFn zetaFn nlFn spoofFn; do + "$BIN" --for=$fn "$H" > "$TMP/for-$fn.xml" 2>/dev/null +done + +# ── ARM (a): the spoof is neutralized ────────────────────────────────────────────────────────────── +echo "arm a: the spoof is neutralized on the hostile fixture" +if [ "$owners_rc" = 0 ]; then ok "--owners exits 0 on the hostile fixture" +else no "--owners exits $owners_rc: $( head -1 "$TMP/owners.err" )"; fi +# rows start with `' '\n' < "$TMP/owners.xml" | grep 'b.cpp has churn=1 — one path, one commit (a line frame splits it into 'a' and 'b.cpp')" \ + || no "ab.cpp churn='$nl_churn', want 1" +nl_top=$( attr_of "$TMP/owners.xml" 'a b.cpp' top ) +[ "$nl_top" = "real@x.com" ] && ok "ab.cpp is owned by its true author" \ + || no "ab.cpp is owned by '$nl_top'" +halves=$( tr '>' '\n' < "$TMP/owners.xml" | grep -cE '-z --name-only ...`, so ANY extraArgs a consumer appends +# must carry its own trailing space. gitFileAuthors' --owners=SYM form appends a pathspec (`-- `); +# without that space the shell reads the pathspec and the -z as ONE word (`-- 'shared.cpp'-z`), git matches +# a file that does not exist, and the walk answers EMPTY — which surfaces as "git unavailable / no history", +# not as a broken command. Measured on the seam-broken build: --owners=sharedFn exited 1 with exactly that +# sentence while the all-files form beside it answered normally. This arm is the pin. +echo "arm c2: the single-file pathspec seam — --owners=SYM must reach its file" +"$BIN" --owners=sharedFn --detail=1 --no-cache "$G" > "$TMP/owners-sym.xml" 2>"$TMP/owners-sym.err"; sym_rc=$? +[ "$sym_rc" = 0 ] && ok "--owners=sharedFn exits 0 (the seam-broken build exited 1: 'git unavailable / no history')" \ + || no "--owners=sharedFn exits $sym_rc: $( head -1 "$TMP/owners-sym.err" )" +grep -q 'git unavailable' "$TMP/owners-sym.err" \ + && no "--owners=SYM answered 'git unavailable / no history' — the pathspec never reached git" \ + || ok "no 'git unavailable' on the SYM form" +grep -q ']*of="sharedFn"' "$TMP/owners-sym.xml" \ + && ok 'the SYM form echoes its selector in of=' \ + || no 'the SYM form does not echo of="sharedFn"' +sym_p=$( tr '>' '\n' < "$TMP/owners-sym.xml" | grep -oE ' "$E/plain.cpp" +git -C "$E" add -A >/dev/null +git -C "$E" commit -qm "e1: empty-email author" --author="No Email <>" >/dev/null +UC +"$BIN" --owners=emptyMailFn --detail=1 --no-cache "$E" > "$TMP/owners-empty.xml" 2>"$TMP/owners-empty.err"; e_rc=$? +[ "$e_rc" = 0 ] && ok "--owners=emptyMailFn exits 0" \ + || no "--owners=emptyMailFn exits $e_rc: $( head -1 "$TMP/owners-empty.err" )" +e_p=$( tr '>' '\n' < "$TMP/owners-empty.xml" | grep -oE ' "$TMP/for-empty.xml" 2>/dev/null +e_churn=$( churn_of_fn "$TMP/for-empty.xml" "emptyMailFn" ) +[ "$e_churn" = "1" ] \ + && ok 'plain.cpp churn=1 — the empty-email commit counts for its own file' \ + || no "plain.cpp churn='$e_churn', want 1 (pre-fix: no row, no churn)" + +# ── ARM (d): the honest tree is byte-identical to the pre-change golden ──────────────────────────── +echo "arm d: the honest tree is byte-identical to the golden recorded from the PRE-CHANGE binary" +{ + echo "===== owners"; "$BIN" --owners --detail=1 --limit=60 "$G"; echo + echo "===== hotspots"; "$BIN" --hotspots --limit=60 "$G"; echo + echo "===== cochange"; "$BIN" --cochange "$G"; echo +} | sed 's/root="[^"]*"/root="FIXTURE"/' > "$TMP/golden.new" +if cmp -s "$GOLD" "$TMP/golden.new"; then + ok "owners+hotspots+cochange byte-identical to test/gitframingcheck_golden.txt (root= normalized)" +else + no "honest-tree drift from the golden:"; diff "$GOLD" "$TMP/golden.new" | head -10 +fi + +# ── ARM (e): the marker literal lives ONLY inside the shared helper ──────────────────────────────── +echo "arm e: the marker literal lives ONLY inside the shared helper" +code_hits=$( code_only "$SRC" | grep -o '__C__' | wc -l | tr -d ' ' ) +[ "$code_hits" = 2 ] \ + && ok "exactly 2 non-comment __C__ occurrences in src/gitmine.h" \ + || no "$code_hits non-comment __C__ occurrences in src/gitmine.h, want 2" +const_line=$( code_only "$SRC" | grep -c 'kGitLogMarker = "__C__"' ) +fmt_line=$( code_only "$SRC" | grep -c 'format=%x00__C__%x00%ae%x00%at%x00%ct' ) +if [ "$const_line" = 1 ] && [ "$fmt_line" = 1 ]; then + ok "the two sites are the kGitLogMarker constant and the shared command's --format= — no consumer body carries the marker" +else + no "the marker's code sites moved: constant=$const_line format=$fmt_line" +fi +tformat_hits=$( code_only "$SRC" | grep -o 'tformat:' | wc -l | tr -d ' ' ) +[ "$tformat_hits" = 0 ] && ok "the old line-frame spellings (tformat:__C__…) are gone from the code" \ + || no "$tformat_hits tformat: marker(s) remain in code" + +# ── ARM 2: the raw stream is byte-pinned ─────────────────────────────────────────────────────────── +echo "arm 2: the raw stream the reader parses is byte-pinned" +# The reader's contract is a BYTE layout, so the layout itself is pinned: the exact command +# gitLogRecordsCmd builds (its prefix flags — --no-optional-locks, the fsmonitor/signature pins and the +# driver pins — govern the index and the worktree, never `log` output, so plain git reproduces it). +"$REAL_GIT" -c core.quotepath=false -C "$H" log --no-ext-diff --no-textconv -c \ + -z --name-only --format='%x00__C__%x00%ae%x00%at%x00%ct' 2>/dev/null > "$TMP/raw.z" +digest=$( python3 -c 'import hashlib,sys; print(hashlib.sha256(open(sys.argv[1],"rb").read()).hexdigest())' "$TMP/raw.z" ) +[ "$digest" = 9a11263860e97452932460b72c9eca9020a6ad01b1d29ceba917f1db1167796c ] \ + && ok "the stream is byte-identical to the pinned digest" \ + || no "raw-stream digest drifted: $digest" +# Fields that spell the marker EXACTLY (NUL-delimited, so a '\n'-prefixed first path does not count): +# 11 record openers + 2 paths — c1's mid-list `__C__` (not first in byte order, hence no '\n' prefix) +# and the collision merge's bare first path. +fields=$( python3 -c ' +import sys +d = open( sys.argv[1], "rb" ).read() +needle = b"\x00__C__\x00" +n = 0 +i = d.find( needle ) +while i >= 0: + n += 1 + i = d.find( needle, i + 1 ) +print( n )' "$TMP/raw.z" ) +[ "$fields" = 13 ] \ + && ok "13 NUL-delimited fields spell the marker: 11 record openers + 2 marker-spelled paths" \ + || no "marker-spelled fields: $fields, want 13" + +# ── ARM 3: the merge shapes — the lookahead is load-bearing, the collision is pinned ──────────────── +echo "arm 3: the merge shapes" +alpha_churn=$( churn_of_fn "$TMP/for-alphaFn.xml" alphaFn ) +[ "$alpha_churn" = 4 ] \ + && ok "M2's name-bearing merge ATTACHES Alpha.cpp (churn=4; a parser without the one-field lookahead reads the merge's prefix empty as a boundary and drops its files — 3)" \ + || no "alphaFn churn='$alpha_churn', want 4 — the lookahead is not attaching merge paths" +dot_churn=$( churn_of_fn "$TMP/for-dotFn.xml" dotFn ) +[ "$dot_churn" = 3 ] \ + && ok "the disclosed collision is pinned: the collision merge's __C__.cpp is dropped (churn=3, not 4)" \ + || no "dotFn churn='$dot_churn', want 3 — the collision's observable moved" +hs_dot=$( attr_of "$TMP/hotspots.xml" '__C__.cpp' churn ) +[ "$hs_dot" = 4 ] \ + && ok "the two miners disagree there BY DESIGN: --hotspots rides the unmigrated gitCommandLines miner and still counts the collision merge's file (4)" \ + || no "--hotspots __C__.cpp churn='$hs_dot', want 4 — the documented disagreement moved, re-derive" + +# ── ARM 4: ownership answers with the AUTHOR, never the committer ────────────────────────────────── +echo "arm 4: author-not-committer" +zeta_top=$( attr_of "$TMP/owners.xml" 'zeta.cpp' top ) +[ "$zeta_top" = "real@x.com" ] \ + && ok "zeta.cpp is owned by its author real@x.com — c2's committer was Committer and the merges' Mergecommitter " \ + || no "zeta.cpp is owned by '$zeta_top'" +committers=$( grep -oE 'committer@x\.com|mergecommitter@x\.com' "$TMP/owners.xml" | wc -l | tr -d ' ' ) +[ "$committers" = 0 ] && ok "no committer email appears as an owner anywhere" \ + || no "$committers committer-email leak(s) in the owners output" +zeta_authors=$( attr_of "$TMP/owners.xml" 'zeta.cpp' authors ) +[ "$zeta_authors" = 2 ] \ + && ok "zeta.cpp has exactly 2 authors (real + quiet2; the committers of c2 and the merges do not count)" \ + || no "zeta.cpp authors='$zeta_authors', want 2" + +# ── ARM 5: non-vacuity — the fixture is what the arms claim ──────────────────────────────────────── +echo "arm 5: non-vacuity — the fixture is what the arms claim" +tracked=$( git -C "$H" ls-files | wc -l | tr -d ' ' ) +[ "$tracked" = 12 ] \ + && ok "12 tracked files: the 11 indexed plus the extension-less __C__ (unindexed BY DESIGN, layout-pin Finding 1)" \ + || no "tracked files: $tracked, want 12" +marker_tracked=$( git -C "$H" ls-files | grep -cx '__C__' ) +[ "$marker_tracked" = 1 ] && ok "a file named exactly __C__ is tracked — the name the frame must survive" \ + || no "the marker-named file is not tracked ($marker_tracked)" +files_n=$( grep -oE 'files="[0-9]+"' "$TMP/owners.xml" | head -1 | grep -oE '[0-9]+' ) +[ "$files_n" = 11 ] \ + && ok "--owners analysed 11 files (files=11)" \ + || no "owners files=$files_n, want 11" +ranked=$( grep -oE 'ranked="[0-9]+"' "$TMP/hotspots.xml" | head -1 | grep -oE '[0-9]+' ) +unranked=$( grep -oE 'unranked_no_complexity="[0-9]+"' "$TMP/hotspots.xml" | head -1 | grep -oE '[0-9]+' ) +[ "$ranked" = 11 ] && [ "$unranked" = 0 ] \ + && ok "--hotspots ranked all 11 files (every fixture file carries a branch, so nothing is unranked_no_complexity)" \ + || no "--hotspots ranked=$ranked unranked_no_complexity=$unranked, want 11/0" + +# ── ARM 6: determinism ───────────────────────────────────────────────────────────────────────────── +echo "arm 6: determinism — the reader's answers do not move between runs" +"$BIN" --owners --detail=1 --limit=60 "$H" > "$TMP/owners2.xml" 2>/dev/null +"$BIN" --hotspots --limit=60 "$H" > "$TMP/hotspots2.xml" 2>/dev/null +if cmp -s "$TMP/owners.xml" "$TMP/owners2.xml" && cmp -s "$TMP/hotspots.xml" "$TMP/hotspots2.xml"; then + ok "two runs byte-identical (owners + hotspots)" +else + no "the answers moved between runs" +fi + +# ── ARM 7: the oracle — the tool's number re-derived from git's own stream ───────────────────────── +echo "arm 7: the oracle" +# Trap #12: a number the tool agrees with itself about is not evidence. Re-derive Alpha.cpp's churn +# straight from git's own --name-only stream: every combined diff that lists it. (tr turns each NUL +# into a line end, which also splits git's '\n'-prefixed first paths — exactly the fields counted.) +oracle=$( "$REAL_GIT" -c core.quotepath=false -C "$H" log --no-ext-diff --no-textconv -c \ + -z --name-only --format='%x00__C__%x00' 2>/dev/null | tr '\0' '\n' | grep -cx 'Alpha.cpp' ) +[ "$oracle" = "$alpha_churn" ] \ + && ok "git's own stream lists Alpha.cpp $oracle times = the --for churn of $alpha_churn" \ + || no "oracle=$oracle, tool=$alpha_churn — the tool's number does not re-derive from git" + +# ── ARM 8: the git floor, and a git that cannot frame degrades honestly ──────────────────────────── +echo "arm 8: the git floor, and a git that rejects -z degrades honestly" +git_ver=$( git --version | grep -oE '[0-9]+\.[0-9]+(\.[0-9]+)?' | head -1 ) +gmaj=${git_ver%%.*}; gmin=${git_ver#*.}; gmin=${gmin%%.*} +if [ "$gmaj" -gt 2 ] || { [ "$gmaj" -eq 2 ] && [ "$gmin" -ge 43 ]; }; then + ok "git $git_ver >= 2.43.0 — the layout-pin floor (the '\n' separator, the merge prefix empty and the end-of-stream unterminated field are empirical; below the floor the pinned layout is unverified)" +else + no "git $git_ver is below the 2.43.0 floor — the pinned layout is unverified on this git" +fi +# A git that refuses -z: the reader must degrade HONESTLY — no fabricated rows, no silent half-answer. +SHIM="$TMP/shim"; mkdir -p "$SHIM" +cat > "$SHIM/git" <<'SHIM' +#!/usr/bin/env bash +for a in "$@"; do + if [ "$a" = "-z" ]; then + echo "git: the -z flag is not supported here" >&2 + exit 129 + fi +done +exec "$REAL_GIT" "$@" +SHIM +chmod +x "$SHIM/git" +PATH="$SHIM:$PATH" "$BIN" --owners --detail=1 --limit=60 "$H" > "$TMP/shim.xml" 2>"$TMP/shim.err"; shim_rc=$? +[ "$shim_rc" != 0 ] && ok "a git that rejects -z: --owners exits $shim_rc (non-zero)" \ + || no "a git that rejects -z: --owners exited 0 — a silent half-answer" +shim_rows=$( tr '>' '\n' < "$TMP/shim.xml" | grep -c '; bf=1 = one person holds it. window: shown= total= capped= has_more= next_offset= offset= limit= (capped=1 cut; next_offset= pastes as offset=). at=: commit+dirty+shallow. root=: p= relative to it. files=N: files analysed. --> +===== hotspots + +===== cochange + diff --git a/test/regression.sh b/test/regression.sh index fead89513..083bd6234 100755 --- a/test/regression.sh +++ b/test/regression.sh @@ -278,7 +278,7 @@ else fi # retired: forspinecheck — the --for caller spine (Change A) was dropped by its own offline kill rule: a spine of up to 2 caller hops from the top 3 seeds named a missed rubric item on 1 of 7 in-sample rows, so the gate that pinned it went with it # retired: cacheexclkeycheck — the per-configuration auto-cache key it pinned is a registered NEGATIVE (docs/EVALS.md, "The auto-cache key ignores --exclude", RUN 2026-09-03: a 158K-file root with >= 12 gate configurations thrashed the 2 GiB sweep); the retry design keeps ONE superset blob per root and will bring its own gate -for _g in a9disclosurecheck abicheck accessshapecheck ackonlycheck adaptivecheck adaptivecutshapecheck affectedcheck agentloopclaudecheck agentloopcodexcheck agentloopeditsuitecheck agentloopfollowupcheck agentloopgradercheck agentlooplockcheck agentloopopencodecheck agentsurfacecheck agenttablecheck aiderbytescheck anchorbodycheck anchorcheck archcheck archmetricscheck argvdiffcheck arisefollowupcheck ariseshimcheck aritycheck artifactcheck astqueryregexcheck astrocheck atcheck atomscheck attribmultidefcheck attrvocabcheck baselinecheck baselinedirtycheck baselineportcheck bashsourcecheck batchcheck binoverridecheck blindspotcheck bm25boundcheck bm25check bodiesshowncheck bodydialectcheck budgetpolicycheck buildtypestampcheck builtinbindcheck bundleidcheck cachefuzzcheck cachehashcheck cacheidentitycheck cacheisolationcheck cachelintcheck cacheoffsetcheck cachereservecheck cachesplitcheck callerscheck callformcheck callsrankordercheck candheadcheck candidatescheck canoncheck capdisclosurecheck capsweepcheck ccheck ccjsoncheck ceilingverdictcheck chacheck chaconecheck chainguardcheck chainidcheck changelogfragcheck childwalkscalecheck churndecaycheck churnjoincheck churnjsonstampcheck claudeconfigdircheck clicheck clojurecheck clonebandcheck clonecachecheck clonededupcheck cloneidiomcheck clonelexcheck clsrecvcheck cochangeboostcheck cochangecliocheck cochangesurprisecheck codexinstallhonestycheck codexplugincheck codexwrapcheck collectioncapcheck columnarattrcheck columnarcheck columnarcommacheck commentcoherencecheck commenttokencheck communitydrillcheck communitylabelcheck compactlegendcheck compactroutecheck completecheck composedhopscheck composelangcheck connectcheck connectcorecheck connectjoincheck constcheck contextratiocheck coplintcheck countfloorcheck cppbenchcheck cppoperatorcheck cppqualcheck cpptmplscopecheck crashsweepcheck crawlescapecheck crossdirincludecheck crossrefcheck crossrefdegradecheck csharpcheck csharpcondcheck cudacheck cyclecutcheck danglingwarncheck dartcheck deadcheck deadfiltercheck deadprecisioncheck deckcheck deckclaimcheck declinecheck declinedlistcheck declinedsitecheck decltodefcheck deeptailcheck defaultceilingcheck defectshapecheck defoverdeclcheck defslookupcheck degradedhintcheck dependencypincheck deplangscheck depsprecisecheck detailcheck diagnoticecheck didyoumeancheck dispatchordercheck dmmcheck docanchorcheck docdemotecheck docdriftcheck docdriftcommentcheck docmdcachecheck docmentioncheck docscommandscheck doctorcheck donelegendcheck droppedpositivecheck duprowcheck dynmapsimdcheck editcheckanswercheck editcheckcheck editcheckdeclcheck editchecknotecheck edithandlehintcheck editpayloadbinarycheck editplancheck editplanpayloadconfinecheck editplanrecheckcheck editplanrollbackmsgcheck editpreviewcheck editroundtripcheck edittargetfileabscheck eliximportcheck elixircheck elixirnamearitycheck elixirsemanticcheck emitescapecheck emittertruthcheck emptycorpuscheck emptyvaluerefusecheck ensembleavailcheck ensemblecheck enumtablecheck epmanifestcheck essentialcxcheck estchargecheck evalcheck evictioncheck exemplarcheck exemplarconfcheck exercisescheck expandbodyfirstcheck expandcallscheck expandmodecheck expandrangecheck expandsibscheck expandtokencheck expandtopk0check extentcheck externalvetocheck falseedgecheck fcallforwardcheck fficheck fieldaffinitycheck fieldidcheck fieldnarrowcheck fieldusescheck filerootcheck fileselectorrefusecheck fillordercheck fixedbufsweep flagscheck flagsnoisecheck flagsurfacecheck flagtablecheck flipcheck floormarkcheck fnliteralcheck fnptrcheck fnslotcheck forautobodycheck forblowupcheck forbodyfoldcheck forbodyslotcheck forbudgetmonotoncheck forcalibfactscheck forcallerhopcheck forcompresscheck forctorhopcheck fordisclosurecheck forhdrshapecheck forheadrrcheck forinitslotscheck forlenscheck formatgatecheck formaxtokenscheck formembernamescheck fornotesbudgetcheck fornotesjsoncheck forrankordercheck forrootlegendcheck forroutecheck forsectioncollapsecheck forsigspancheck forsoftbudgetcheck forwidencheck freshclonecheck freshnesscheck g1configcheck gateabilitycheck gatecountcheck gateexitcheck gdscriptcheck genrecallcheck gitchildoptscheck gitenvhermeticcheck githardencheck gitignorecheck gitquotepathcheck gitstampcheck goinstcheck gointerfacecheck gokindcheck goscopecheck graphlegendbudgetcheck graphqueryrefusecheck grepanchorcheck grepandcheck grepbytescheck grepcheck grepcontextcheck grepcorpuscheck grepfastcheck grepfollowupcheck grepignorecheck grepscancheck grepseamcheck greptiercheck guardmsgcheck handlerrulecheck hasacheck hazardpatterncheck headbinstagecheck headsnapcachecheck helpbudgetcheck hermesinstallcheck historyoraclecheck honestlexcheck hookcheck hop1check hopbodycheck hostilecheck hotspotsincecheck htmlcolorcheck htmlhostcheck htmlrendercheck identitycheck ifacedispatchcheck impactdepthcheck impactimportcheck impactpartitioncheck impacttypecheck importcapcheck importnarrowcheck includeanglecheck includeprecisecheck indexoutcheck infraportcheck isolateprovenancecheck javamethodrefcheck javarubycheck jsdefnamingcheck jslangcheck jsmetricscheck jsnestedcheck jsoncheck jsonlangcheck jsonparitycheck jsonredactcheck jsonrefusallegendcheck jsonwalkcheck jsshapecheck jsverbscheck jsxcallcheck knownitemcheck kotlincheck landingcheck langcensuscheck langcheck langidentitycheck layerquerycheck layoutcheck lb3namecheck legendcostcheck legendcoveragecheck legenddriftcheck legendrefcheck legobundlecheck legocheck legotransitivecheck liftdisclosurecheck limitstablecheck lintbudgetcheck lintcatalogcheck lintcheck lintdedupcheck lintpayloadcapcheck lintprecisioncheck lintrulescheck lintscopecheck lintselectcheck listingpagingcheck localdeclcheck localitycheck localscountcheck loopconservationcheck lpincheck luacheck luarequirecheck macroedgecheck macroreparsecheck manifestcheck mapdatasectioncheck mapdiffcheck mapinventorycheck mapstemcheck matchcapturecheck matchgrammarcheck maxfilesizecheck mcpattrparitycheck mcpaudit4hardencheck mcpclidiffcheck mcpcodexmetacheck mcpcontractcheck mcpdegradedhintcheck mcpeditcheck mcpeditkindcheck mcpeditmodecheck mcpeditpresencecheck mcpeditracecheck mcpflagshipcheck mcpforparitycheck mcpframehonestycheck mcpgrepdegradedcheck mcphandlecheck mcpincrementalcheck mcpmanifestcheck mcprangeedgecheck mcpreadloopcheck mcpredactcheck mcpreloadasancheck mcpreloadcheck mcpremotecheck mcprobustcheck mcpslicecheck mcpstalecheck mcpstdiolinecapcheck mcpstrictschemacheck mcptoolprunecheck mcptoolsubsetcheck mcptranchecheck mcptwinclaimscheck mcpverbscheck mcpw2fixcheck mcpw3fixcheck mcpwatchercheck mdembedcheck mdsectioncheck memguardcheck mentioncapcheck mentioncheck mentionsverbcheck mergechurncheck mergescoutcheck mergescoutlonglinecheck metalcheck meterdisclosurecheck metricscheck modifierguardcheck moduleconstcheck morecontractcheck mrowalkcheck multirootcheck multiswecheck namedfileinputcheck nameinfocheck namekeyhopcheck namingcalibrationcheck namingconsistencycheck naminglenscheck naminglocalscheck narrowcheck narrowlangcheck neighbourcapcheck nestedimportcheck nestedqualcheck nestprofilecheck nextverbcheck noaliascheck nodekindcheck nongitqmetricscheck nonlocalstatecheck notecanoncheck notescheck notesdegradecheck nsfiltercheck nulbytecheck numericrefusecheck objcfieldcheck objcsniffcheck opencodewrapcheck optremarkscheck optremarkshotcheck ordercheck osswitchcheck oswin32logiccheck outlinecheck overbudgetcommentcheck ownerhopcheck ownerscheck packcallersharecheck packtaskcheck packtaskmonotoncheck packtaskquotacheck padscalecheck paginationcheck pagingsweepcheck panellegendcheck pargatescheck parsehealthcheck partitioncheck pathgapcheck pathownercheck pathpaircheck patterncheck perfharnesscheck phpcheck pincensuscheck planlanescheck planlintcheck pmccheck portablebuildcheck portablecachecheck postingscheck ppaltcheck ppdeadrolescheck pranchorcheck prbudgetcheck prcheck prcontextcheck prconvergecheck precedencecheck preproccondcheck preprocdeadscalecheck prmaskanchorcheck prnestedcapcheck probecheck propcostcheck prrefsafecheck prrenamecheck pyaliascallcheck pyimportprecisecheck pymodulealiascheck pyshapecheck qackconcurrencycheck qackorigincheck qbaselineproducercheck qchurncheck qchurnmemocheck qddegradecheck qddialscheck qdrefpaircheck qextractionkeycheck qoriginoraclecheck qrevtokencheck qrowlocatorcheck qschemetripcheck qsnapcachecheck qsnapprefetchcheck qsnapproducercheck qualifiedresolvecheck qualitycheck qualitycrosslangcheck qualityexcludecheck qualitykeycheck qualitykindscheck qualityorigincheck qualitypanelcheck qualityscopecheck qualitysignalcheck qualitystalecheck qualitysymcheck qualnamecheck qualnewcheck querycheck queryfilescancheck queuehopcheck racymtimecheck radixsimdcheck rangecomposecheck rankbycheck reachcheck readabilitycheck readmedriftcheck readmeexamplecheck recallanchorcheck recallboundarycheck recallbudgetcheck recallbufcheck recallevalcheck recallparitycheck recallpassagecheck recallrankdepthcheck recallrelcheck recallshapecheck recalltablecheck recalltotalcheck receiptpostcheck receiverevidencecheck receiverfixescheck recentscopecheck redactcheck redactfixcheck refusaltailcheck regexbombcheck regexcheck regexguardcheck regexrefusecheck registermacrocheck relevancefloorcheck relinkcheck reportcheck resolvecheck resolverhonestycheck retrievalqualitycheck reusefirstworkflowcheck ripwirepubliccheck rootrelcheck rootrelemitcheck rootspellingcheck routecheck routeedgecheck routehookcheck routeoncecheck routingreportcheck rubyargcheck rubyattrscheck rubybarecallcheck rubyclassrecvcheck rubyconstcheck rubydeclrefcheck rubydescribedclasscheck rubyinheritcheck rubymetricscheck rubyrakejbuildercheck rubyreachcheck rubyrecvcheck rubyrecvnarrowcheck rubyrequirecheck rubyrspectargetcheck rubyschemacheck rubyscopecheck rubysettercheck rubytypedrecvcheck runhintcheck runtracecheck rustanccheck rustimportprecisecheck rustqualcheck safedeletecheck sarifcheck savecachecheck scipcheck scipjoincheck scorecardcheck scoutheadconflictcheck scoutkeycheck scroundtripcheck seedboundscheck selectorchaincheck selectorhonestycheck selectorrefusecheck selectorscopecheck selfcheckcheck selfcontainedcheck shadowcheck shallowhistorycheck shapingflagcheck shellgateindexcheck showcasecapturecheck sibliftcheck sidecarsymlinkcheck sigredactcheck sincecheck sincecochangecheck sincewindowcheck singledefcheck situdiffcheck situhunkcheck situshapecheck skilldescbudgetcheck skillevalcheck skillevalsplitcheck skillinstallcheck skillroutingjudgedcheck skillscanreadcheck skilltruthcheck skipclassifycheck skippedcheck skipreasoncheck slicecheck slicediffcheck sliceflowcheck sliceflowsenscheck slotjoincheck spectimingcheck staleackcheck statgatecheck stdmembergatecheck stdqualcheck strkerncheck structlayoutcheck sublistcountcheck subscriptopcheck substrfiltercheck subtokencheck svectorcheck swiftcheck swiftmemberscheck swiftshapecheck taskechocheck tempfilesymlinkcheck termmargincheck testedreachcheck testgatecheck testgatehunkcheck testgatelegendbudgetcheck testgatepagecheck testgaterefusecheck testmacrocheck testonlydeadcheck testrowruncheck testscopecheck textdocscheck timsortcheck tokenbudgetcheck tomllangcheck toolcallroutecheck tornreadcheck traceasanlinearcheck tracecheck tracehandoffcapcheck tracehopcheck traceminecheck treecheck truncvocabcheck tsimportprecisecheck tsshapecheck type3check type3clonecheck typerefcheck unreachablecheck unresolvedcheck usescheck usesliteralcheck usesselectorcheck usingdeclcheck utf8scrubcheck vendoredassetcheck vendoredbundlecheck vendorpatchcheck verifycheck versioncheck w2verbscheck w3fixbudgetcheck w3fixlegendcheck weaksignalcheck withgraphcheck withprofilecheck worktreeleakcheck wrapverbscheck writetargetcheck writtenspellingcheck xmlwellformed yamllangcheck zonecheck zoneconsistencycheck zoomcheck; do +for _g in a9disclosurecheck abicheck accessshapecheck ackonlycheck adaptivecheck adaptivecutshapecheck affectedcheck agentloopclaudecheck agentloopcodexcheck agentloopeditsuitecheck agentloopfollowupcheck agentloopgradercheck agentlooplockcheck agentloopopencodecheck agentsurfacecheck agenttablecheck aiderbytescheck anchorbodycheck anchorcheck archcheck archmetricscheck argvdiffcheck arisefollowupcheck ariseshimcheck aritycheck artifactcheck astqueryregexcheck astrocheck atcheck atomscheck attribmultidefcheck attrvocabcheck baselinecheck baselinedirtycheck baselineportcheck bashsourcecheck batchcheck binoverridecheck blindspotcheck bm25boundcheck bm25check bodiesshowncheck bodydialectcheck budgetpolicycheck buildtypestampcheck builtinbindcheck bundleidcheck cachefuzzcheck cachehashcheck cacheidentitycheck cacheisolationcheck cachelintcheck cacheoffsetcheck cachereservecheck cachesplitcheck callerscheck callformcheck callsrankordercheck candheadcheck candidatescheck canoncheck capdisclosurecheck capsweepcheck ccheck ccjsoncheck ceilingverdictcheck chacheck chaconecheck chainguardcheck chainidcheck changelogfragcheck childwalkscalecheck churndecaycheck churnjoincheck churnjsonstampcheck claudeconfigdircheck clicheck clojurecheck clonebandcheck clonecachecheck clonededupcheck cloneidiomcheck clonelexcheck clsrecvcheck cochangeboostcheck cochangecliocheck cochangesurprisecheck codexinstallhonestycheck codexplugincheck codexwrapcheck collectioncapcheck columnarattrcheck columnarcheck columnarcommacheck commentcoherencecheck commenttokencheck communitydrillcheck communitylabelcheck compactlegendcheck compactroutecheck completecheck composedhopscheck composelangcheck connectcheck connectcorecheck connectjoincheck constcheck contextratiocheck coplintcheck countfloorcheck cppbenchcheck cppoperatorcheck cppqualcheck cpptmplscopecheck crashsweepcheck crawlescapecheck crossdirincludecheck crossrefcheck crossrefdegradecheck csharpcheck csharpcondcheck cudacheck cyclecutcheck danglingwarncheck dartcheck deadcheck deadfiltercheck deadprecisioncheck deckcheck deckclaimcheck declinecheck declinedlistcheck declinedsitecheck decltodefcheck deeptailcheck defaultceilingcheck defectshapecheck defoverdeclcheck defslookupcheck degradedhintcheck dependencypincheck deplangscheck depsprecisecheck detailcheck diagnoticecheck didyoumeancheck dispatchordercheck dmmcheck docanchorcheck docdemotecheck docdriftcheck docdriftcommentcheck docmdcachecheck docmentioncheck docscommandscheck doctorcheck donelegendcheck droppedpositivecheck duprowcheck dynmapsimdcheck editcheckanswercheck editcheckcheck editcheckdeclcheck editchecknotecheck edithandlehintcheck editpayloadbinarycheck editplancheck editplanpayloadconfinecheck editplanrecheckcheck editplanrollbackmsgcheck editpreviewcheck editroundtripcheck edittargetfileabscheck eliximportcheck elixircheck elixirnamearitycheck elixirsemanticcheck emitescapecheck emittertruthcheck emptycorpuscheck emptyvaluerefusecheck ensembleavailcheck ensemblecheck enumtablecheck epmanifestcheck essentialcxcheck estchargecheck evalcheck evictioncheck exemplarcheck exemplarconfcheck exercisescheck expandbodyfirstcheck expandcallscheck expandmodecheck expandrangecheck expandsibscheck expandtokencheck expandtopk0check extentcheck externalvetocheck falseedgecheck fcallforwardcheck fficheck fieldaffinitycheck fieldidcheck fieldnarrowcheck fieldusescheck filerootcheck fileselectorrefusecheck fillordercheck fixedbufsweep flagscheck flagsnoisecheck flagsurfacecheck flagtablecheck flipcheck floormarkcheck fnliteralcheck fnptrcheck fnslotcheck forautobodycheck forblowupcheck forbodyfoldcheck forbodyslotcheck forbudgetmonotoncheck forcalibfactscheck forcallerhopcheck forcompresscheck forctorhopcheck fordisclosurecheck forhdrshapecheck forheadrrcheck forinitslotscheck forlenscheck formatgatecheck formaxtokenscheck formembernamescheck fornotesbudgetcheck fornotesjsoncheck forrankordercheck forrootlegendcheck forroutecheck forsectioncollapsecheck forsigspancheck forsoftbudgetcheck forwidencheck freshclonecheck freshnesscheck g1configcheck gateabilitycheck gatecountcheck gateexitcheck gdscriptcheck genrecallcheck gitchildoptscheck gitenvhermeticcheck gitframingcheck githardencheck gitignorecheck gitquotepathcheck gitstampcheck goinstcheck gointerfacecheck gokindcheck goscopecheck graphlegendbudgetcheck graphqueryrefusecheck grepanchorcheck grepandcheck grepbytescheck grepcheck grepcontextcheck grepcorpuscheck grepfastcheck grepfollowupcheck grepignorecheck grepscancheck grepseamcheck greptiercheck guardmsgcheck handlerrulecheck hasacheck hazardpatterncheck headbinstagecheck headsnapcachecheck helpbudgetcheck hermesinstallcheck historyoraclecheck honestlexcheck hookcheck hop1check hopbodycheck hostilecheck hotspotsincecheck htmlcolorcheck htmlhostcheck htmlrendercheck identitycheck ifacedispatchcheck impactdepthcheck impactimportcheck impactpartitioncheck impacttypecheck importcapcheck importnarrowcheck includeanglecheck includeprecisecheck indexoutcheck infraportcheck isolateprovenancecheck javamethodrefcheck javarubycheck jsdefnamingcheck jslangcheck jsmetricscheck jsnestedcheck jsoncheck jsonlangcheck jsonparitycheck jsonredactcheck jsonrefusallegendcheck jsonwalkcheck jsshapecheck jsverbscheck jsxcallcheck knownitemcheck kotlincheck landingcheck langcensuscheck langcheck langidentitycheck layerquerycheck layoutcheck lb3namecheck legendcostcheck legendcoveragecheck legenddriftcheck legendrefcheck legobundlecheck legocheck legotransitivecheck liftdisclosurecheck limitstablecheck lintbudgetcheck lintcatalogcheck lintcheck lintdedupcheck lintpayloadcapcheck lintprecisioncheck lintrulescheck lintscopecheck lintselectcheck listingpagingcheck localdeclcheck localitycheck localscountcheck loopconservationcheck lpincheck luacheck luarequirecheck macroedgecheck macroreparsecheck manifestcheck mapdatasectioncheck mapdiffcheck mapinventorycheck mapstemcheck matchcapturecheck matchgrammarcheck maxfilesizecheck mcpattrparitycheck mcpaudit4hardencheck mcpclidiffcheck mcpcodexmetacheck mcpcontractcheck mcpdegradedhintcheck mcpeditcheck mcpeditkindcheck mcpeditmodecheck mcpeditpresencecheck mcpeditracecheck mcpflagshipcheck mcpforparitycheck mcpframehonestycheck mcpgrepdegradedcheck mcphandlecheck mcpincrementalcheck mcpmanifestcheck mcprangeedgecheck mcpreadloopcheck mcpredactcheck mcpreloadasancheck mcpreloadcheck mcpremotecheck mcprobustcheck mcpslicecheck mcpstalecheck mcpstdiolinecapcheck mcpstrictschemacheck mcptoolprunecheck mcptoolsubsetcheck mcptranchecheck mcptwinclaimscheck mcpverbscheck mcpw2fixcheck mcpw3fixcheck mcpwatchercheck mdembedcheck mdsectioncheck memguardcheck mentioncapcheck mentioncheck mentionsverbcheck mergechurncheck mergescoutcheck mergescoutlonglinecheck metalcheck meterdisclosurecheck metricscheck modifierguardcheck moduleconstcheck morecontractcheck mrowalkcheck multirootcheck multiswecheck namedfileinputcheck nameinfocheck namekeyhopcheck namingcalibrationcheck namingconsistencycheck naminglenscheck naminglocalscheck narrowcheck narrowlangcheck neighbourcapcheck nestedimportcheck nestedqualcheck nestprofilecheck nextverbcheck noaliascheck nodekindcheck nongitqmetricscheck nonlocalstatecheck notecanoncheck notescheck notesdegradecheck nsfiltercheck nulbytecheck numericrefusecheck objcfieldcheck objcsniffcheck opencodewrapcheck optremarkscheck optremarkshotcheck ordercheck osswitchcheck oswin32logiccheck outlinecheck overbudgetcommentcheck ownerhopcheck ownerscheck packcallersharecheck packtaskcheck packtaskmonotoncheck packtaskquotacheck padscalecheck paginationcheck pagingsweepcheck panellegendcheck pargatescheck parsehealthcheck partitioncheck pathgapcheck pathownercheck pathpaircheck patterncheck perfharnesscheck phpcheck pincensuscheck planlanescheck planlintcheck pmccheck portablebuildcheck portablecachecheck postingscheck ppaltcheck ppdeadrolescheck pranchorcheck prbudgetcheck prcheck prcontextcheck prconvergecheck precedencecheck preproccondcheck preprocdeadscalecheck prmaskanchorcheck prnestedcapcheck probecheck propcostcheck prrefsafecheck prrenamecheck pyaliascallcheck pyimportprecisecheck pymodulealiascheck pyshapecheck qackconcurrencycheck qackorigincheck qbaselineproducercheck qchurncheck qchurnmemocheck qddegradecheck qddialscheck qdrefpaircheck qextractionkeycheck qoriginoraclecheck qrevtokencheck qrowlocatorcheck qschemetripcheck qsnapcachecheck qsnapprefetchcheck qsnapproducercheck qualifiedresolvecheck qualitycheck qualitycrosslangcheck qualityexcludecheck qualitykeycheck qualitykindscheck qualityorigincheck qualitypanelcheck qualityscopecheck qualitysignalcheck qualitystalecheck qualitysymcheck qualnamecheck qualnewcheck querycheck queryfilescancheck queuehopcheck racymtimecheck radixsimdcheck rangecomposecheck rankbycheck reachcheck readabilitycheck readmedriftcheck readmeexamplecheck recallanchorcheck recallboundarycheck recallbudgetcheck recallbufcheck recallevalcheck recallparitycheck recallpassagecheck recallrankdepthcheck recallrelcheck recallshapecheck recalltablecheck recalltotalcheck receiptpostcheck receiverevidencecheck receiverfixescheck recentscopecheck redactcheck redactfixcheck refusaltailcheck regexbombcheck regexcheck regexguardcheck regexrefusecheck registermacrocheck relevancefloorcheck relinkcheck reportcheck resolvecheck resolverhonestycheck retrievalqualitycheck reusefirstworkflowcheck ripwirepubliccheck rootrelcheck rootrelemitcheck rootspellingcheck routecheck routeedgecheck routehookcheck routeoncecheck routingreportcheck rubyargcheck rubyattrscheck rubybarecallcheck rubyclassrecvcheck rubyconstcheck rubydeclrefcheck rubydescribedclasscheck rubyinheritcheck rubymetricscheck rubyrakejbuildercheck rubyreachcheck rubyrecvcheck rubyrecvnarrowcheck rubyrequirecheck rubyrspectargetcheck rubyschemacheck rubyscopecheck rubysettercheck rubytypedrecvcheck runhintcheck runtracecheck rustanccheck rustimportprecisecheck rustqualcheck safedeletecheck sarifcheck savecachecheck scipcheck scipjoincheck scorecardcheck scoutheadconflictcheck scoutkeycheck scroundtripcheck seedboundscheck selectorchaincheck selectorhonestycheck selectorrefusecheck selectorscopecheck selfcheckcheck selfcontainedcheck shadowcheck shallowhistorycheck shapingflagcheck shellgateindexcheck showcasecapturecheck sibliftcheck sidecarsymlinkcheck sigredactcheck sincecheck sincecochangecheck sincewindowcheck singledefcheck situdiffcheck situhunkcheck situshapecheck skilldescbudgetcheck skillevalcheck skillevalsplitcheck skillinstallcheck skillroutingjudgedcheck skillscanreadcheck skilltruthcheck skipclassifycheck skippedcheck skipreasoncheck slicecheck slicediffcheck sliceflowcheck sliceflowsenscheck slotjoincheck spectimingcheck staleackcheck statgatecheck stdmembergatecheck stdqualcheck strkerncheck structlayoutcheck sublistcountcheck subscriptopcheck substrfiltercheck subtokencheck svectorcheck swiftcheck swiftmemberscheck swiftshapecheck taskechocheck tempfilesymlinkcheck termmargincheck testedreachcheck testgatecheck testgatehunkcheck testgatelegendbudgetcheck testgatepagecheck testgaterefusecheck testmacrocheck testonlydeadcheck testrowruncheck testscopecheck textdocscheck timsortcheck tokenbudgetcheck tomllangcheck toolcallroutecheck tornreadcheck traceasanlinearcheck tracecheck tracehandoffcapcheck tracehopcheck traceminecheck treecheck truncvocabcheck tsimportprecisecheck tsshapecheck type3check type3clonecheck typerefcheck unreachablecheck unresolvedcheck usescheck usesliteralcheck usesselectorcheck usingdeclcheck utf8scrubcheck vendoredassetcheck vendoredbundlecheck vendorpatchcheck verifycheck versioncheck w2verbscheck w3fixbudgetcheck w3fixlegendcheck weaksignalcheck withgraphcheck withprofilecheck worktreeleakcheck wrapverbscheck writetargetcheck writtenspellingcheck xmlwellformed yamllangcheck zonecheck zoneconsistencycheck zoomcheck; do [ -f "$ROOT/test/$_g.sh" ] || continue if RIPWIRE_BIN="$BIN" bash "$ROOT/test/$_g.sh" >/dev/null 2>&1; then ok "absorb gate ($_g.sh)"