Skip to content

Commit 6225db0

Browse files
fix: make the build.mcpp host helper self-contained on musl AND MinGW (2026.7.28.2)
The compiled build.mcpp helper is exec'd by the host OS, outside anything mcpp controls — no wrapper, no injected LD_LIBRARY_PATH, no PATH guarantee. Whether it is actually runnable is a per-platform mechanism, and only two of the four were closed: Linux + glibc payload ok — host_base_flags bakes absolute payload paths into -Wl,-rpath Linux + musl (#295) bad — rpath cannot reach PT_INTERP, an absolute /lib/ld-musl-<arch>.so.1 no payload installs and no glibc distro ships -> execve ENOENT, surfacing as a bare exit 127 Windows PE (#299) bad — PE has no rpath at all, so the helper resolves libstdc++-6 / libgcc_s / libwinpthread via the process PATH -> STATUS_DLL_NOT_FOUND macOS ok — the system libc++ is always present Both gaps are the same defect (build_program.cppm re-derives the helper link policy instead of reusing the main build's, cf. prepare.cppm's musl -> linkage="static" default), so they get ONE decision point rather than two parallel predicates: a single staticHostHelper in run_build_program, with the chosen policy folded into the build-program cache identity so pre-fix helpers are rebuilt instead of reused. `-static` is appended to the final link argv only — never to `base`, which also feeds the bundled module's compile and precompile commands. Also: - process.cppm: keep the posix_spawnp error code and message in the captured output instead of collapsing to an unexplained exit 127 (both capture_exec and capture_exec_deadline). All three consumers of RunResult::output gate on exit_code first, so nothing downstream changes. - host_base_flags: stop emitting -Wl,-rpath on PE, where it is inert. - tests/e2e/168 (was 167 — taken on main by 167_build_defines_module_scan): discovers the installed musl-gcc payload instead of hardcoding 15.1.0, so it runs on the aarch64 host where the regression actually lives; asserts static + no PT_INTERP. Verified by negative control: reverting the -static push reproduces the exact #295 failure. - tests/e2e/97: hold the build.mcpp helper to the same standalone bar as the produced exe (import table + run with the toolchain bin off PATH). This runs under the real MinGW GCC 16.1.0 on Windows CI, which is #299's configuration. - ci-aarch64-fresh-install: build the PR source and run e2e 168 against it, and leave the published-asset validation steps using the installed mcpp. The duplicated inline assertions are gone, along with `! readelf … | grep -q …`, which bash exempts from errexit and which therefore could never fail. Closes #295 Closes #299 Supersedes #300 Co-authored-by: Zhe Feng <zhefeng7110@outlook.com>
1 parent d0342db commit 6225db0

7 files changed

Lines changed: 212 additions & 113 deletions

File tree

.github/workflows/ci-aarch64-fresh-install.yml

Lines changed: 27 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ on:
2121
paths:
2222
- 'src/build/build_program.cppm'
2323
- 'src/platform/process.cppm'
24-
- 'tests/e2e/167_build_mcpp_musl_host_static.sh'
24+
- 'tests/e2e/168_build_mcpp_musl_host_static.sh'
2525
- '.github/workflows/ci-aarch64-fresh-install.yml'
2626
push:
2727
branches: [ main ]
@@ -84,57 +84,50 @@ jobs:
8484
git clone --depth 1 https://github.com/openxlings/xim-pkgindex "$idx"
8585
grep -n "skipping relocation\|os.isfile(path.join(bindir" "$idx/pkgs/m/musl-gcc.lua" | head -2 || true
8686
87-
- name: Build current mcpp source for native regression tests
88-
run: |
89-
mcpp build --target aarch64-linux-musl
90-
self=$(find target/aarch64-linux-musl -type f -name mcpp | head -1)
91-
test -x "$self"
92-
self=$(realpath "$self")
93-
"$self" --version
94-
echo "MCPP_SELF=$self" >> "$GITHUB_ENV"
95-
9687
- name: Native build + run an `import std` program
9788
run: |
9889
work=$(mktemp -d); cd "$work"
99-
"$MCPP_SELF" new hello
90+
mcpp new hello
10091
cd hello
101-
cat > build.mcpp <<'EOF'
102-
#include <cstdio>
103-
int main() {
104-
FILE* marker = std::fopen("helper-ran", "w");
105-
if (!marker) return 2;
106-
std::fputs("ok\n", marker);
107-
std::fclose(marker);
108-
return 0;
109-
}
110-
EOF
11192
# default src uses import std (C++23)
112-
"$MCPP_SELF" build
113-
test -f helper-ran
114-
helper=target/.build-mcpp/build.mcpp.bin
115-
file "$helper"
116-
file "$helper" | grep -q 'statically linked'
117-
! readelf -lW "$helper" | grep -q 'Requesting program interpreter'
118-
out=$("$MCPP_SELF" run 2>/dev/null)
93+
mcpp build
94+
out=$(mcpp run 2>/dev/null || true)
11995
echo "program output: $out"
120-
bin=$(find target -type f -name hello | head -1)
96+
bin=$(find target -type f -path '*/bin/hello' | head -1)
12197
file "$bin"
12298
file "$bin" | grep -q "ARM aarch64" || { echo "expected aarch64 ELF"; exit 1; }
12399
100+
# ── PR regression gate ────────────────────────────────────────────────
101+
# Everything above validates the PUBLISHED aarch64 assets. The two steps
102+
# below build the PR's own source and run the musl host-helper regression
103+
# against it — this is the only runner where a musl toolchain is the
104+
# NATIVE one, so it is the only place #295 can actually be reproduced.
105+
- name: Build current mcpp source for native regression tests
106+
run: |
107+
mcpp build --target aarch64-linux-musl
108+
self=$(find target/aarch64-linux-musl -type f -path '*/bin/mcpp' | head -1)
109+
test -x "$self"
110+
self=$(realpath "$self")
111+
"$self" --version
112+
echo "MCPP_SELF=$self" >> "$GITHUB_ENV"
113+
114+
- name: "Regression: build.mcpp host helper is self-contained (#295)"
115+
run: MCPP="$MCPP_SELF" bash tests/e2e/168_build_mcpp_musl_host_static.sh
116+
124117
- name: Self-host — build mcpp + xlings from source natively
125118
run: |
126119
# mcpp/xlings manifests pin a glibc default toolchain; on aarch64 the
127120
# musl-static target is the published path, so build with --target.
128121
git clone --depth 1 https://github.com/mcpp-community/mcpp /tmp/mcpp-src
129122
cd /tmp/mcpp-src
130-
"$MCPP_SELF" self config --mirror GLOBAL 2>/dev/null || true
131-
"$MCPP_SELF" build --target aarch64-linux-musl
132-
m=$(find target/aarch64-linux-musl -type f -name mcpp | head -1)
123+
mcpp self config --mirror GLOBAL 2>/dev/null || true
124+
mcpp build --target aarch64-linux-musl
125+
m=$(find target/aarch64-linux-musl -type f -path '*/bin/mcpp' | head -1)
133126
file "$m" | grep -q "ARM aarch64" || { echo "expected aarch64 mcpp"; exit 1; }
134127
"$m" --version
135128
git clone --depth 1 https://github.com/openxlings/xlings /tmp/xlings-src
136129
cd /tmp/xlings-src
137-
"$MCPP_SELF" build --target aarch64-linux-musl
138-
x=$(find target/aarch64-linux-musl -type f -name xlings | head -1)
130+
mcpp build --target aarch64-linux-musl
131+
x=$(find target/aarch64-linux-musl -type f -path '*/bin/xlings' | head -1)
139132
file "$x" | grep -q "ARM aarch64" || { echo "expected aarch64 xlings"; exit 1; }
140133
"$x" --version

mcpp.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "mcpp"
3-
version = "2026.7.28.1"
3+
version = "2026.7.28.2"
44
description = "Modern C++ build & package management tool"
55
license = "Apache-2.0"
66
authors = ["mcpp-community"]

src/build/build_program.cppm

Lines changed: 43 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ import mcpp.platform;
1818
import mcpp.platform.process;
1919
import mcpp.toolchain.fingerprint; // hash_file / hash_string (FNV-1a, 16 hex)
2020
import mcpp.toolchain.linkmodel; // shared C-library / clang-cfg-bypass model
21-
import mcpp.toolchain.model; // Toolchain, PayloadPaths, is_clang/is_musl_target
21+
import mcpp.toolchain.model; // Toolchain, PayloadPaths, is_clang/is_musl_target/is_mingw_target
2222
import mcpp.toolchain.registry; // archive_tool
2323
import mcpp.toolchain.triple; // host_triple (MCPP_HOST contract value)
2424
import mcpp.ui;
@@ -160,8 +160,10 @@ std::string env_value(const std::string& name) {
160160

161161
// The host subset of flags.cppm's sysroot/runtime handling — enough to compile +
162162
// link a freestanding host program on a fresh sandbox (where bare `g++ file -o x`
163-
// can't find crt/libc). build.mcpp is host-only (skipped under cross), so we need
164-
// only the native cases; these are passed as separate argv tokens (no shell).
163+
// can't find crt/libc). `tc` is always a HOST-targeting toolchain: under a cross
164+
// --target, prepare.cppm resolves the spec a second time without the target axis
165+
// (host_tc_for_build_program) and passes that here, so the native cases are the
166+
// only ones needed. Passed as separate argv tokens (no shell).
165167
std::vector<std::string> host_base_flags(const mcpp::toolchain::Toolchain& tc) {
166168
std::vector<std::string> f;
167169
const auto lm = mcpp::toolchain::resolve_link_model(tc);
@@ -228,9 +230,14 @@ std::vector<std::string> host_base_flags(const mcpp::toolchain::Toolchain& tc) {
228230
if (!ar.empty()) f.push_back("-B" + ar.parent_path().string());
229231
}
230232
// Runtime lib dirs so the produced program can load private libs in-tree.
233+
// -L is link-time and wanted everywhere; rpath is an ELF-only concept —
234+
// this is the one host_base_flags branch a PE target reaches, where the
235+
// flag is inert and the self-containment answer is the static link in
236+
// run_build_program instead (#299).
231237
for (auto& d : tc.linkRuntimeDirs) {
232238
f.push_back("-L" + d.string());
233-
f.push_back("-Wl,-rpath," + d.string());
239+
if constexpr (mcpp::platform::supports_rpath)
240+
f.push_back("-Wl,-rpath," + d.string());
234241
}
235242
return f;
236243
}
@@ -585,12 +592,35 @@ std::expected<void, std::string> run_build_program(
585592
auto childEnv = contract_env(root, outDir, env);
586593
std::string ctxHash = contract_hash(childEnv);
587594

588-
const bool staticHostHelper = mcpp::platform::supports_full_static
589-
&& mcpp::toolchain::is_musl_target(tc);
595+
// ── Helper self-containment (the single decision point) ─────────────────
596+
// The compiled helper is exec'd by the host OS, outside anything mcpp
597+
// controls — no wrapper, no injected LD_LIBRARY_PATH, no PATH guarantee.
598+
// Making it runnable is a per-platform mechanism, and only two of the four
599+
// need a static link:
600+
// Linux + glibc payload — host_base_flags bakes absolute payload paths
601+
// into -Wl,-rpath; that already closes it, so leave it dynamic.
602+
// Linux + musl (#295) — rpath cannot reach PT_INTERP, an absolute
603+
// /lib/ld-musl-<arch>.so.1 that no payload installs and glibc distros
604+
// do not ship. Only a static link removes the interpreter entirely.
605+
// Windows PE (#299) — PE has no rpath, so a dynamic helper resolves
606+
// libstdc++-6 / libgcc_s / libwinpthread through the process PATH and
607+
// dies with STATUS_DLL_NOT_FOUND unless the user put the toolchain bin
608+
// there by hand. A static link is the only PATH-independent answer.
609+
// macOS — the system libc++ is always present.
610+
// Keep this the ONLY place that decides; the flag is appended to the final
611+
// link argv further down, never to `base`.
612+
const bool muslStaticHelper = mcpp::platform::supports_full_static
613+
&& mcpp::toolchain::is_musl_target(tc);
614+
const bool mingwStaticHelper = mcpp::toolchain::is_mingw_target(tc);
615+
const bool staticHostHelper = muslStaticHelper || mingwStaticHelper;
616+
617+
// Fold the policy into the compiler identity: a helper produced under an
618+
// older link policy must be rebuilt, not reused from the cache.
590619
std::string compilerIdentity = hostCompiler.string();
591-
compilerIdentity += staticHostHelper
592-
? "\nbuild-program-link=musl-static-v1"
593-
: "\nbuild-program-link=default-v1";
620+
compilerIdentity += "\nbuild-program-link=";
621+
compilerIdentity += muslStaticHelper ? "musl-static-v1"
622+
: mingwStaticHelper ? "mingw-static-v1"
623+
: "default-v1";
594624
std::string programHash = mcpp::toolchain::hash_file(src);
595625
std::string compilerHash = mcpp::toolchain::hash_string(compilerIdentity);
596626

@@ -647,10 +677,10 @@ std::expected<void, std::string> run_build_program(
647677
compileArgv.push_back("-x"); compileArgv.push_back("none");
648678
compileArgv.push_back((bdir / "mcpp.o").string());
649679
}
650-
// A dynamic musl helper requires /lib/ld-musl-<arch>.so.1 on the host,
651-
// which glibc distributions do not provide. Keep the host build program in
652-
// the musl toolchain's documented static world. This is link-only: `base`
653-
// also feeds the bundled module's compile/precompile commands above.
680+
// Self-contained helper link — see the staticHostHelper doctrine above.
681+
// Deliberately NOT in `base`: that also feeds the bundled module's
682+
// compile/precompile commands, where a link flag has no business (and for
683+
// Clang would perturb the default PIC/PIE codegen of mcpp.o).
654684
if (staticHostHelper) compileArgv.push_back("-static");
655685
compileArgv.push_back("-o"); compileArgv.push_back(bin.string());
656686
mcpp::ui::info("build.mcpp", "compiling");

src/toolchain/fingerprint.cppm

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ import mcpp.toolchain.detect;
1818

1919
export namespace mcpp::toolchain {
2020

21-
inline constexpr std::string_view MCPP_VERSION = "2026.7.28.1";
21+
inline constexpr std::string_view MCPP_VERSION = "2026.7.28.2";
2222

2323
struct FingerprintInputs {
2424
Toolchain toolchain;

tests/e2e/167_build_mcpp_musl_host_static.sh

Lines changed: 0 additions & 63 deletions
This file was deleted.
Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
#!/usr/bin/env bash
2+
# requires: elf
3+
# 168_build_mcpp_musl_host_static.sh — a build.mcpp is a HOST executable that
4+
# the OS execs directly. Under a musl host toolchain it must be fully static:
5+
# a dynamic musl binary carries PT_INTERP=/lib/ld-musl-<arch>.so.1, an absolute
6+
# path no toolchain payload installs and no glibc distro ships, so execve fails
7+
# with ENOENT and the build dies as an unexplained exit 127 (#295).
8+
#
9+
# Runs standalone too — the aarch64 fresh-install workflow invokes it directly:
10+
# MCPP=/path/to/mcpp bash tests/e2e/168_build_mcpp_musl_host_static.sh
11+
set -euo pipefail
12+
13+
if [[ "$(uname -s)" != "Linux" ]]; then
14+
echo "SKIP: Linux ELF regression"
15+
echo "OK"
16+
exit 0
17+
fi
18+
19+
: "${MCPP:?MCPP must point to the mcpp binary under test}"
20+
21+
# Which musl toolchain to pin is a property of the machine, not of the test:
22+
# x86_64 dev boxes carry 15.1.0, the aarch64 runner resolves 16.1.0. Discover
23+
# the installed payload instead of hardcoding either — which is also why this
24+
# owns its own precondition (`requires: elf`, not `requires: musl`): run_all.sh's
25+
# musl capability probe is pinned to x86_64/15.1.0 and would false-skip on the
26+
# aarch64 host where this regression actually lives. Never auto-install here:
27+
# that would be a ~200 MB download per CI run (same rule as 28_target_static).
28+
find_musl_version() {
29+
local root gxx
30+
for root in "${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs/xim-x-musl-gcc" \
31+
"$HOME/.xlings/data/xpkgs/xim-x-musl-gcc"; do
32+
[[ -d "$root" ]] || continue
33+
gxx=$(ls "$root"/*/bin/*-linux-musl-g++ 2>/dev/null | head -1 || true)
34+
if [[ -n "$gxx" ]]; then
35+
# <root>/<version>/bin/<triple>-g++ → <version>
36+
basename "$(dirname "$(dirname "$gxx")")"
37+
return 0
38+
fi
39+
done
40+
return 1
41+
}
42+
43+
MUSL_VERSION=$(find_musl_version || true)
44+
if [[ -z "$MUSL_VERSION" ]]; then
45+
echo "SKIP: no musl-gcc payload installed — not testing the musl host helper"
46+
echo "OK"
47+
exit 0
48+
fi
49+
echo "musl toolchain: gcc@${MUSL_VERSION}-musl"
50+
51+
TMP=$(mktemp -d)
52+
trap 'rm -rf "$TMP"' EXIT
53+
cd "$TMP"
54+
"$MCPP" new app > new.log 2>&1 || {
55+
cat new.log
56+
echo "FAIL: mcpp new failed"
57+
exit 1
58+
}
59+
cd app
60+
61+
cat >> mcpp.toml <<EOF
62+
63+
[toolchain]
64+
default = "gcc@${MUSL_VERSION}-musl"
65+
EOF
66+
67+
cat > build.mcpp <<'EOF'
68+
#include <cstdio>
69+
int main() {
70+
FILE* marker = std::fopen("helper-ran", "w");
71+
if (!marker) return 2;
72+
std::fputs("ok\n", marker);
73+
std::fclose(marker);
74+
std::puts("mcpp:rerun-if-changed=build.mcpp");
75+
return 0;
76+
}
77+
EOF
78+
79+
"$MCPP" build > build.log 2>&1 || {
80+
cat build.log
81+
echo "FAIL: musl build.mcpp helper did not run on this host"
82+
exit 1
83+
}
84+
85+
helper=target/.build-mcpp/build.mcpp.bin
86+
test -f helper-ran || { cat build.log; echo "FAIL: build.mcpp did not run"; exit 1; }
87+
test -x "$helper" || { cat build.log; echo "FAIL: helper binary missing"; exit 1; }
88+
89+
file "$helper"
90+
file "$helper" | grep -q 'statically linked' || {
91+
echo "FAIL: musl host helper is not statically linked"
92+
exit 1
93+
}
94+
# The property that actually decides whether execve succeeds. NB: an `if`, not
95+
# `! readelf ... | grep -q ...` — bash exempts !-inverted pipelines from
96+
# errexit, so the negated form would never fail the test.
97+
if readelf -lW "$helper" | grep -q 'Requesting program interpreter'; then
98+
readelf -lW "$helper"
99+
echo "FAIL: musl host helper contains PT_INTERP"
100+
exit 1
101+
fi
102+
103+
echo "OK"

0 commit comments

Comments
 (0)