fix: make the build.mcpp host helper self-contained on musl and MinGW (#298)

* fix: statically link musl build helpers

* ci: use self-built mcpp in ARM source builds

* fix: make the build.mcpp host helper self-contained on musl AND MinGW (2026.7.28.2)

The compiled build.mcpp helper is exec'd by the host OS, outside anything
mcpp controls — no wrapper, no injected LD_LIBRARY_PATH, no PATH guarantee.
Whether it is actually runnable is a per-platform mechanism, and only two of
the four were closed:

  Linux + glibc payload  ok  — host_base_flags bakes absolute payload paths
                               into -Wl,-rpath
  Linux + musl (#295)    bad — rpath cannot reach PT_INTERP, an absolute
                               /lib/ld-musl-<arch>.so.1 no payload installs
                               and no glibc distro ships -> execve ENOENT,
                               surfacing as a bare exit 127
  Windows PE (#299)      bad — PE has no rpath at all, so the helper resolves
                               libstdc++-6 / libgcc_s / libwinpthread via the
                               process PATH -> STATUS_DLL_NOT_FOUND
  macOS                  ok  — the system libc++ is always present

Both gaps are the same defect (build_program.cppm re-derives the helper link
policy instead of reusing the main build's, cf. prepare.cppm's musl ->
linkage="static" default), so they get ONE decision point rather than two
parallel predicates: a single staticHostHelper in run_build_program, with the
chosen policy folded into the build-program cache identity so pre-fix helpers
are rebuilt instead of reused. `-static` is appended to the final link argv
only — never to `base`, which also feeds the bundled module's compile and
precompile commands.

Also:
- process.cppm: keep the posix_spawnp error code and message in the captured
  output instead of collapsing to an unexplained exit 127 (both capture_exec
  and capture_exec_deadline). All three consumers of RunResult::output gate on
  exit_code first, so nothing downstream changes.
- host_base_flags: stop emitting -Wl,-rpath on PE, where it is inert.
- tests/e2e/168 (was 167 — taken on main by 167_build_defines_module_scan):
  discovers the installed musl-gcc payload instead of hardcoding 15.1.0, so it
  runs on the aarch64 host where the regression actually lives; asserts static
  + no PT_INTERP. Verified by negative control: reverting the -static push
  reproduces the exact #295 failure.
- tests/e2e/97: hold the build.mcpp helper to the same standalone bar as the
  produced exe (import table + run with the toolchain bin off PATH). This runs
  under the real MinGW GCC 16.1.0 on Windows CI, which is #299's configuration.
- ci-aarch64-fresh-install: append a PR regression gate that builds this source
  and runs e2e 168 against it. The checkout must come AFTER the fresh-install
  steps: .xlings.json declares an `mcpp` workspace pin, so a checkout present
  in $GITHUB_WORKSPACE makes `xlings install mcpp` workspace-scoped — which
  both voids the fresh-install charter (validating the pinned version, not the
  published one) and leaves bare `mcpp` unresolvable outside the workspace.
  Dropped the duplicated inline assertions, including
  `! readelf … | grep -q …`, which bash exempts from errexit and which could
  therefore never fail.

Closes #295
Closes #299
Supersedes #300

Co-authored-by: Zhe Feng <zhefeng7110@outlook.com>

---------

Co-authored-by: wellwei <1827104243@qq.com>
Co-authored-by: sunrisepeak <speakshen@163.com>
Co-authored-by: Zhe Feng <zhefeng7110@outlook.com>

Author: 4 people

.github/workflows/ci-aarch64-fresh-install.yml

@@ -16,11 +16,21 @@ on:
   workflow_dispatch:
   schedule:
     - cron: '0 6 * * 1'   # weekly Mon 06:00 UTC
+  pull_request:
+    branches: [ main ]
+    paths:
+      - 'src/build/build_program.cppm'
+      - 'src/platform/process.cppm'
+      - 'tests/e2e/168_build_mcpp_musl_host_static.sh'
+      - '.github/workflows/ci-aarch64-fresh-install.yml'
   push:
     branches: [ main ]
     paths:
       - '.github/workflows/ci-aarch64-fresh-install.yml'
 
+permissions:
+  contents: read
+
 concurrency:
   group: ci-${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
@@ -34,6 +44,8 @@ jobs:
       # Verbose every mcpp invocation — cold bootstrap path (src/cli.cppm).
       MCPP_VERBOSE: "1"
     steps:
+      # NB: the checkout deliberately comes LAST, after every fresh-install
+      # step below — see the comment above it.
       - name: System info
         run: |
           uname -a
@@ -79,7 +91,7 @@ jobs:
           mcpp build
           out=$(mcpp run 2>/dev/null || true)
           echo "program output: $out"
-          bin=$(find target -type f -name hello | head -1)
+          bin=$(find target -type f -path '*/bin/hello' | head -1)
           file "$bin"
           file "$bin" | grep -q "ARM aarch64" || { echo "expected aarch64 ELF"; exit 1; }
 
@@ -91,12 +103,39 @@ jobs:
           cd /tmp/mcpp-src
           mcpp self config --mirror GLOBAL 2>/dev/null || true
           mcpp build --target aarch64-linux-musl
-          m=$(find target/aarch64-linux-musl -type f -name mcpp | head -1)
+          m=$(find target/aarch64-linux-musl -type f -path '*/bin/mcpp' | head -1)
           file "$m" | grep -q "ARM aarch64" || { echo "expected aarch64 mcpp"; exit 1; }
           "$m" --version
           git clone --depth 1 https://github.com/openxlings/xlings /tmp/xlings-src
           cd /tmp/xlings-src
           mcpp build --target aarch64-linux-musl
-          x=$(find target/aarch64-linux-musl -type f -name xlings | head -1)
+          x=$(find target/aarch64-linux-musl -type f -path '*/bin/xlings' | head -1)
           file "$x" | grep -q "ARM aarch64" || { echo "expected aarch64 xlings"; exit 1; }
           "$x" --version
+
+      # ── PR regression gate ────────────────────────────────────────────────
+      # Everything above is the fresh-install charter: it must run exactly as a
+      # new user's machine does. Check out only NOW — this repo's .xlings.json
+      # declares an `mcpp` WORKSPACE pin, so with the checkout present in
+      # $GITHUB_WORKSPACE `xlings install mcpp` installs workspace-scoped
+      # instead of globally: the steps above would silently validate the pinned
+      # version rather than the freshly published one, and bare `mcpp` stops
+      # resolving anywhere outside the workspace.
+      - uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+
+      # The PR's own source, then the musl host-helper regression against it.
+      # This is the only runner where a musl toolchain is the NATIVE one, so it
+      # is the only place #295 can actually be reproduced.
+      - name: Build current mcpp source for native regression tests
+        run: |
+          mcpp build --target aarch64-linux-musl
+          self=$(find target/aarch64-linux-musl -type f -path '*/bin/mcpp' | head -1)
+          test -x "$self"
+          self=$(realpath "$self")
+          "$self" --version
+          echo "MCPP_SELF=$self" >> "$GITHUB_ENV"
+
+      - name: "Regression: build.mcpp host helper is self-contained (#295)"
+        run: MCPP="$MCPP_SELF" bash tests/e2e/168_build_mcpp_musl_host_static.sh

mcpp.toml

@@ -1,6 +1,6 @@
 [package]
 name        = "mcpp"
-version     = "2026.7.28.1"
+version     = "2026.7.28.2"
 description = "Modern C++ build & package management tool"
 license     = "Apache-2.0"
 authors     = ["mcpp-community"]

src/build/build_program.cppm

@@ -18,7 +18,7 @@ import mcpp.platform;
 import mcpp.platform.process;
 import mcpp.toolchain.fingerprint;   // hash_file / hash_string (FNV-1a, 16 hex)
 import mcpp.toolchain.linkmodel;     // shared C-library / clang-cfg-bypass model
-import mcpp.toolchain.model;         // Toolchain, PayloadPaths, is_clang/is_musl_target
+import mcpp.toolchain.model;         // Toolchain, PayloadPaths, is_clang/is_musl_target/is_mingw_target
 import mcpp.toolchain.registry;      // archive_tool
 import mcpp.toolchain.triple;        // host_triple (MCPP_HOST contract value)
 import mcpp.ui;
@@ -160,8 +160,10 @@ std::string env_value(const std::string& name) {
 
 // The host subset of flags.cppm's sysroot/runtime handling — enough to compile +
 // link a freestanding host program on a fresh sandbox (where bare `g++ file -o x`
-// can't find crt/libc). build.mcpp is host-only (skipped under cross), so we need
-// only the native cases; these are passed as separate argv tokens (no shell).
+// can't find crt/libc). `tc` is always a HOST-targeting toolchain: under a cross
+// --target, prepare.cppm resolves the spec a second time without the target axis
+// (host_tc_for_build_program) and passes that here, so the native cases are the
+// only ones needed. Passed as separate argv tokens (no shell).
 std::vector<std::string> host_base_flags(const mcpp::toolchain::Toolchain& tc) {
     std::vector<std::string> f;
     const auto lm = mcpp::toolchain::resolve_link_model(tc);
@@ -228,9 +230,14 @@ std::vector<std::string> host_base_flags(const mcpp::toolchain::Toolchain& tc) {
         if (!ar.empty()) f.push_back("-B" + ar.parent_path().string());
     }
     // Runtime lib dirs so the produced program can load private libs in-tree.
+    // -L is link-time and wanted everywhere; rpath is an ELF-only concept —
+    // this is the one host_base_flags branch a PE target reaches, where the
+    // flag is inert and the self-containment answer is the static link in
+    // run_build_program instead (#299).
     for (auto& d : tc.linkRuntimeDirs) {
         f.push_back("-L" + d.string());
-        f.push_back("-Wl,-rpath," + d.string());
+        if constexpr (mcpp::platform::supports_rpath)
+            f.push_back("-Wl,-rpath," + d.string());
     }
     return f;
 }
@@ -585,8 +592,37 @@ std::expected<void, std::string> run_build_program(
     auto childEnv = contract_env(root, outDir, env);
     std::string ctxHash = contract_hash(childEnv);
 
+    // ── Helper self-containment (the single decision point) ─────────────────
+    // The compiled helper is exec'd by the host OS, outside anything mcpp
+    // controls — no wrapper, no injected LD_LIBRARY_PATH, no PATH guarantee.
+    // Making it runnable is a per-platform mechanism, and only two of the four
+    // need a static link:
+    //   Linux + glibc payload — host_base_flags bakes absolute payload paths
+    //     into -Wl,-rpath; that already closes it, so leave it dynamic.
+    //   Linux + musl (#295)   — rpath cannot reach PT_INTERP, an absolute
+    //     /lib/ld-musl-<arch>.so.1 that no payload installs and glibc distros
+    //     do not ship. Only a static link removes the interpreter entirely.
+    //   Windows PE (#299)     — PE has no rpath, so a dynamic helper resolves
+    //     libstdc++-6 / libgcc_s / libwinpthread through the process PATH and
+    //     dies with STATUS_DLL_NOT_FOUND unless the user put the toolchain bin
+    //     there by hand. A static link is the only PATH-independent answer.
+    //   macOS                 — the system libc++ is always present.
+    // Keep this the ONLY place that decides; the flag is appended to the final
+    // link argv further down, never to `base`.
+    const bool muslStaticHelper  = mcpp::platform::supports_full_static
+                                && mcpp::toolchain::is_musl_target(tc);
+    const bool mingwStaticHelper = mcpp::toolchain::is_mingw_target(tc);
+    const bool staticHostHelper  = muslStaticHelper || mingwStaticHelper;
+
+    // Fold the policy into the compiler identity: a helper produced under an
+    // older link policy must be rebuilt, not reused from the cache.
+    std::string compilerIdentity = hostCompiler.string();
+    compilerIdentity += "\nbuild-program-link=";
+    compilerIdentity += muslStaticHelper  ? "musl-static-v1"
+                      : mingwStaticHelper ? "mingw-static-v1"
+                                          : "default-v1";
     std::string programHash  = mcpp::toolchain::hash_file(src);
-    std::string compilerHash = mcpp::toolchain::hash_string(hostCompiler.string());
+    std::string compilerHash = mcpp::toolchain::hash_string(compilerIdentity);
 
     // Fast path: declared inputs + contract unchanged → reapply cached
     // directives, no run.
@@ -641,6 +677,11 @@ std::expected<void, std::string> run_build_program(
         compileArgv.push_back("-x"); compileArgv.push_back("none");
         compileArgv.push_back((bdir / "mcpp.o").string());
     }
+    // Self-contained helper link — see the staticHostHelper doctrine above.
+    // Deliberately NOT in `base`: that also feeds the bundled module's
+    // compile/precompile commands, where a link flag has no business (and for
+    // Clang would perturb the default PIC/PIE codegen of mcpp.o).
+    if (staticHostHelper) compileArgv.push_back("-static");
     compileArgv.push_back("-o"); compileArgv.push_back(bin.string());
     mcpp::ui::info("build.mcpp", "compiling");
     // GCC resolves `import mcpp;` via gcm.cache/ relative to the compile cwd, so

src/platform/process.cppm

@@ -230,6 +230,11 @@ std::vector<std::string> merged_environ(
     }
     return out;
 }
+
+std::string spawn_failure(std::string_view program, int error) {
+    return std::format("posix_spawnp('{}') failed (error {}): {}\n",
+                       program, error, std::generic_category().message(error));
+}
 #else
 // Build a shell command line from an argv vector (Windows + residual non-POSIX
 // fallback only; Linux/macOS exec directly, #248). The first token (program)
@@ -444,7 +449,12 @@ RunResult capture_exec(
     int sp = ::posix_spawnp(&pid, cargv[0], &fa, nullptr, cargv.data(), envp.data());
     ::posix_spawn_file_actions_destroy(&fa);
     ::close(fds[1]);
-    if (sp != 0) { ::close(fds[0]); result.exit_code = 127; return result; }
+    if (sp != 0) {
+        ::close(fds[0]);
+        result.exit_code = 127;
+        result.output = spawn_failure(argv.front(), sp);
+        return result;
+    }
 
     std::array<char, 4096> buf{};
     ssize_t n;
@@ -547,7 +557,12 @@ RunResult capture_exec_deadline(
     int sp = ::posix_spawnp(&pid, cargv[0], &fa, nullptr, cargv.data(), envp.data());
     ::posix_spawn_file_actions_destroy(&fa);
     ::close(fds[1]);
-    if (sp != 0) { ::close(fds[0]); result.exit_code = 127; return result; }
+    if (sp != 0) {
+        ::close(fds[0]);
+        result.exit_code = 127;
+        result.output = spawn_failure(argv.front(), sp);
+        return result;
+    }
 
     ::fcntl(fds[0], F_SETFL, ::fcntl(fds[0], F_GETFL) | O_NONBLOCK);
 

src/toolchain/fingerprint.cppm

@@ -18,7 +18,7 @@ import mcpp.toolchain.detect;
 
 export namespace mcpp::toolchain {
 
-inline constexpr std::string_view MCPP_VERSION = "2026.7.28.1";
+inline constexpr std::string_view MCPP_VERSION = "2026.7.28.2";
 
 struct FingerprintInputs {
     Toolchain                       toolchain;

tests/e2e/168_build_mcpp_musl_host_static.sh

@@ -0,0 +1,103 @@
+#!/usr/bin/env bash
+# requires: elf
+# 168_build_mcpp_musl_host_static.sh — a build.mcpp is a HOST executable that
+# the OS execs directly. Under a musl host toolchain it must be fully static:
+# a dynamic musl binary carries PT_INTERP=/lib/ld-musl-<arch>.so.1, an absolute
+# path no toolchain payload installs and no glibc distro ships, so execve fails
+# with ENOENT and the build dies as an unexplained exit 127 (#295).
+#
+# Runs standalone too — the aarch64 fresh-install workflow invokes it directly:
+#   MCPP=/path/to/mcpp bash tests/e2e/168_build_mcpp_musl_host_static.sh
+set -euo pipefail
+
+if [[ "$(uname -s)" != "Linux" ]]; then
+    echo "SKIP: Linux ELF regression"
+    echo "OK"
+    exit 0
+fi
+
+: "${MCPP:?MCPP must point to the mcpp binary under test}"
+
+# Which musl toolchain to pin is a property of the machine, not of the test:
+# x86_64 dev boxes carry 15.1.0, the aarch64 runner resolves 16.1.0. Discover
+# the installed payload instead of hardcoding either — which is also why this
+# owns its own precondition (`requires: elf`, not `requires: musl`): run_all.sh's
+# musl capability probe is pinned to x86_64/15.1.0 and would false-skip on the
+# aarch64 host where this regression actually lives. Never auto-install here:
+# that would be a ~200 MB download per CI run (same rule as 28_target_static).
+find_musl_version() {
+    local root gxx
+    for root in "${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs/xim-x-musl-gcc" \
+                "$HOME/.xlings/data/xpkgs/xim-x-musl-gcc"; do
+        [[ -d "$root" ]] || continue
+        gxx=$(ls "$root"/*/bin/*-linux-musl-g++ 2>/dev/null | head -1 || true)
+        if [[ -n "$gxx" ]]; then
+            # <root>/<version>/bin/<triple>-g++ → <version>
+            basename "$(dirname "$(dirname "$gxx")")"
+            return 0
+        fi
+    done
+    return 1
+}
+
+MUSL_VERSION=$(find_musl_version || true)
+if [[ -z "$MUSL_VERSION" ]]; then
+    echo "SKIP: no musl-gcc payload installed — not testing the musl host helper"
+    echo "OK"
+    exit 0
+fi
+echo "musl toolchain: gcc@${MUSL_VERSION}-musl"
+
+TMP=$(mktemp -d)
+trap 'rm -rf "$TMP"' EXIT
+cd "$TMP"
+"$MCPP" new app > new.log 2>&1 || {
+    cat new.log
+    echo "FAIL: mcpp new failed"
+    exit 1
+}
+cd app
+
+cat >> mcpp.toml <<EOF
+
+[toolchain]
+default = "gcc@${MUSL_VERSION}-musl"
+EOF
+
+cat > build.mcpp <<'EOF'
+#include <cstdio>
+int main() {
+    FILE* marker = std::fopen("helper-ran", "w");
+    if (!marker) return 2;
+    std::fputs("ok\n", marker);
+    std::fclose(marker);
+    std::puts("mcpp:rerun-if-changed=build.mcpp");
+    return 0;
+}
+EOF
+
+"$MCPP" build > build.log 2>&1 || {
+    cat build.log
+    echo "FAIL: musl build.mcpp helper did not run on this host"
+    exit 1
+}
+
+helper=target/.build-mcpp/build.mcpp.bin
+test -f helper-ran || { cat build.log; echo "FAIL: build.mcpp did not run"; exit 1; }
+test -x "$helper" || { cat build.log; echo "FAIL: helper binary missing"; exit 1; }
+
+file "$helper"
+file "$helper" | grep -q 'statically linked' || {
+    echo "FAIL: musl host helper is not statically linked"
+    exit 1
+}
+# The property that actually decides whether execve succeeds. NB: an `if`, not
+# `! readelf ... | grep -q ...` — bash exempts !-inverted pipelines from
+# errexit, so the negated form would never fail the test.
+if readelf -lW "$helper" | grep -q 'Requesting program interpreter'; then
+    readelf -lW "$helper"
+    echo "FAIL: musl host helper contains PT_INTERP"
+    exit 1
+fi
+
+echo "OK"

tests/e2e/97_mingw_toolchain.sh

@@ -7,6 +7,7 @@
 #   - new → build → run works with import std + a named module (gcm pipeline)
 #   - the produced exe is portable: runs from a clean dir without the
 #     toolchain's bin on PATH (static libstdc++/libgcc defaults)
+#   - so is the build.mcpp HOST helper mcpp execs mid-build (#299)
 set -e
 
 CONF="${MCPP_HOME:-$HOME/.mcpp}/config.toml"
@@ -87,4 +88,39 @@ if [[ $iso_rc -ne 0 || "$iso_out" != *"mingw-ok"* ]]; then
     exit 1
 fi
 
-echo "PASS: mingw toolchain — install, default, modules build/run, standalone exe"
+# 5) the build.mcpp HOST helper is held to the same bar (#299). It is exec'd
+#    by mcpp mid-build, and PE has no rpath — a dynamic helper resolves
+#    libstdc++-6 / libgcc_s / libwinpthread through the process PATH and dies
+#    with STATUS_DLL_NOT_FOUND (exit -1073741511) unless the user added the
+#    toolchain bin to PATH by hand.
+cat > build.mcpp <<'EOF'
+#include <cstdio>
+int main() {
+    FILE* marker = std::fopen("helper-ran", "w");
+    if (!marker) return 2;
+    std::fputs("ok\n", marker);
+    std::fclose(marker);
+    std::puts("mcpp:rerun-if-changed=build.mcpp");
+    return 0;
+}
+EOF
+out=$("$MCPP" build 2>&1) || { echo "FAIL: build with build.mcpp: $out"; exit 1; }
+[[ -f helper-ran ]] || { echo "FAIL: build.mcpp did not run: $out"; exit 1; }
+HELPER="target/.build-mcpp/build.mcpp.exe"
+[[ -f "$HELPER" ]] || { echo "FAIL: no helper binary at $HELPER"; exit 1; }
+if [[ -x "$OBJDUMP" ]]; then
+    h_imports=$("$OBJDUMP" -p "$HELPER" 2>/dev/null | grep -i "DLL Name" || true)
+    echo "helper imports:"; echo "$h_imports"
+    bad=$(echo "$h_imports" | grep -iE "libstdc|libgcc|libwinpthread" || true)
+    [[ -z "$bad" ]] || { echo "FAIL: toolchain DLLs in helper import table: $bad"; exit 1; }
+fi
+HISO="$TMP/hiso"; mkdir -p "$HISO"
+cp "$HELPER" "$HISO/"
+h_rc=0
+(cd "$HISO" && PATH="/usr/bin:/c/Windows/System32" ./build.mcpp.exe >/dev/null 2>&1) || h_rc=$?
+[[ $h_rc -eq 0 ]] || {
+    echo "FAIL: build.mcpp helper does not run without the toolchain bin on PATH (rc=$h_rc)"
+    exit 1
+}
+
+echo "PASS: mingw toolchain — install, default, modules build/run, standalone exe + build.mcpp helper"

tests/unit/test_process_run_exec.cpp

@@ -44,6 +44,14 @@ TEST(RunExec, ReturnsErrorWhenProgramMissing) {
     EXPECT_NE(process::run_exec({"/no/such/program/mcpp-xyz"}), 0);
 }
 
+TEST(CaptureExec, MissingProgramReportsSpawnFailure) {
+    constexpr std::string_view missing = "/no/such/program/mcpp-capture-xyz";
+    auto r = process::capture_exec({std::string(missing)});
+    EXPECT_EQ(r.exit_code, 127);
+    EXPECT_NE(r.output.find(missing), std::string::npos) << r.output;
+    EXPECT_NE(r.output.find("error 2"), std::string::npos) << r.output;
+}
+
 TEST(CaptureExec, CapturesStdoutWithoutShell) {
     auto r = process::capture_exec({"/bin/echo", "hello world"});
     EXPECT_EQ(r.exit_code, 0);