Skip to content

the measurement can run against an engine that has not been released - #452

Merged
Sunrisepeak merged 2 commits into
mainfrom
the-measurement-can-run-against-an-unreleased-engine
Sep 20, 2026
Merged

Sunrisepeak merged 2 commits into
mainfrom
the-measurement-can-run-against-an-unreleased-engine

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

What this adds

MCPP_SOURCE_REF — non-empty, the measurement builds mcpp from that ref and runs the thirty member graphs against it instead of the released pin. Every ecosystem repository mcpp reaches already honours this variable; this workflow was the gap, and it is the one that matters most: mcpp CI cannot see a defect that appears only in a real dependency graph, and this repository holds thirty of them.

The cost of the gap is measured, not hypothetical

2026.9.20.1 shipped on 2026-09-20. Within a day, the four members that stop at #include <windows.h> through a borrowed __CYGWIN__archive, sqlite3, mimalloc, c-ares — made a second release necessary.

That reading was available from these same thirty graphs before the first release merged. Nothing could produce it, because this job could only install a published engine.

A release is not one build: a tag, four platform builds, two mirrors, a GET check per asset, an xim-pkgindex bump, an index artifact, a consumer-side confirmation. What this closes is a second pass down that entire chain.

Shape

  • The released engine is still installed — mcpp builds itself and is its own bootstrap.
  • The build from the ref replaces it on PATH; MCPP_HOME follows.
  • .xlings.json is removed from the clone first: it pins the mcpp that compiles mcpp, does not move when mcpp is released, and would otherwise ask the index for a version it may no longer carry.
  • The version under review is printed, because a PATH entry that does not win looks exactly like one that does until something has been measured with the wrong engine.

The pins-agree check is unaffected: it compares this workflow pin with pins.toml, and both still name the released version. A source ref is an override on top of that agreement, not a replacement for it.

Every ecosystem repository mcpp's changes reach already honours
`MCPP_SOURCE_REF`: non-empty, it builds the engine from that ref and asserts
that the binary on PATH is the one it built. This workflow was the gap, and it
is the one that matters most --- mcpp's own CI cannot see a defect that appears
only in a real dependency graph, and this repository holds thirty of them.

THE COST OF THE GAP IS MEASURED, NOT HYPOTHETICAL. 2026.9.20.1 shipped on
2026-09-20. Within a day the four members that stop at `#include <windows.h>`
through a borrowed `__CYGWIN__` --- archive, sqlite3, mimalloc, c-ares ---
made a second release necessary. That reading was available from these same
thirty graphs before the first release merged; nothing could produce it,
because this job could only install a published engine.

A release is not one build. It is a tag, four platform builds, two mirrors,
a GET check per asset, an xim-pkgindex bump, an index artifact, and a
consumer-side confirmation. What this closes is a second pass down that whole
chain.

Shape: the released engine the pin names is still installed, because mcpp
builds itself and is its own bootstrap; the build from the ref then replaces it
on PATH, and `MCPP_HOME` follows it. `.xlings.json` is removed from the clone
first --- it pins the mcpp that compiles mcpp, does not move when mcpp is
released, and would otherwise ask the index for a version it may no longer
carry. The version under review is printed, because a PATH entry that does not
win looks exactly like one that does until something has been measured with
the wrong engine.

The pins-agree check is unaffected: it compares this workflow's pin with
pins.toml, and both still name the released version. A source ref is an
override on top of that agreement, not a replacement for it.
The first revision prepended the freshly built `mcpp` to PATH beside its own
`target/` directory. The measurement then stopped at

    error: xlings binary not found

A released archive carries a BUNDLED xlings that mcpp resolves relative to its
own location; a binary sitting in a build tree has no such neighbour, and
moving `MCPP_HOME` to that tree moved the search with it.

Overwriting `bin/mcpp` inside the released root instead keeps everything about
that root as the release left it --- the bundled xlings, the layout,
`MCPP_HOME`, the PATH entry already exported --- and changes only the one file
under review. The version is reported from the path the measurement will
actually run, because a copy that did not land looks exactly like one that did
until something has been measured with the wrong engine.

Found by using this mechanism on its first real cross-verification, which is
where a plumbing defect is cheap.
@Sunrisepeak
Sunrisepeak merged commit ab0e5ba into main Sep 20, 2026
2 checks passed
@Sunrisepeak
Sunrisepeak deleted the the-measurement-can-run-against-an-unreleased-engine branch September 20, 2026 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant