diff --git a/.agents/docs/2026-08-11-origin-precedence-implementation-plan.md b/.agents/docs/2026-08-11-origin-precedence-implementation-plan.md new file mode 100644 index 00000000..9f69ad56 --- /dev/null +++ b/.agents/docs/2026-08-11-origin-precedence-implementation-plan.md @@ -0,0 +1,240 @@ +# `$ORIGIN` 优先级 + 共享库运行时契约 —— 实施计划 + +配套分析:`2026-08-11-runtime-search-origin-precedence-analysis.md` +范围:**A + B + C2 + C1**,单 PR(2026.8.11.3) +状态:**已实施**(实测结果见 §2.2,与计划的偏差见 §2.1) + +--- + +## 0. 四个角度的取舍 + +### 优雅设计 —— 把「顺序」从字符串拼接提升为声明 + +缺陷的形状是:**一条链接命令行的顺序,由两个互不知情的生产者用 `+=` 决定。** +`flags.cppm` 把 farm 拼在全局 ldflags 末尾并注释「so it is LAST」;`plan.cppm` 把 +`$ORIGIN` 拼在 per-unit;`ninja_backend` 渲染成 `$ldflags $unit_ldflags`。 +三处都对,合起来是错的。 + +对策不是「换个地方拼」,而是新增 `src/build/link_line.cppm` +(`mcpp.build.link_line`):把 per-unit 尾部声明成**具名槽位**,相对顺序写在类型里, +由单测钉死。加一个新的调用方不再可能把顺序拧错 —— 它必须选一个槽。 + +### 架构稳定性 —— 消灭同一决策的第二处推导 + +`dist::default_contract(Role)` 自称「The role -> contract policy, in one place」, +**却没有任何生产调用方**(只有单测);真正的策略在 `flags.cppm:704-709` 独立算了 +一遍。这正是 `distribution.cppm` 开篇声讨的那类债(「used to be derived +independently in five places」),只是换了个位置复发。 + +本次把 `default_contract` 变成**活的唯一真源**,`flags.cppm` 调它。 + +### 兼容性 —— 只有一个平台的行为改变,且正是有缺陷的那个 + +| 目标格式 | 共享库契约 变化 | 产物字节 | +|---|---|---| +| **ELF** | SelfContained → **ToolchainCoupled** | 变(这是修复) | +| Mach-O | 不变(SelfContained) | **不变** | +| PE | 不变(SelfContained) | **不变** | + +`-Wl,-rpath,` 换槽位后在**没有共享库依赖的工程**上字节完全不变 +(槽位为空即不渲染);有共享库依赖的工程只有 rpath 次序变化。 + +用户逃生舱保留:`cxx_runtime = { shared = "self-contained" }` 可恢复旧行为, +且此时自动补 C1 护栏,不会重新打开符号泛滥。 + +**缓存**:`.so` 的链接边在工程自己的 ninja 图里(`build bin/libX11.so : cxx_shared +…`),不来自依赖缓存;ninja 按命令行变化重跑链接。契约只影响链接标志、不影响编译 +标志,因此对象缓存键无需改动。发版 bump 会改工程指纹 ⇒ 全量重建,不存在旧契约残留。 + +### 跨平台 —— 平台差异只准出现在已有的格式维度里 + +`distribution.cppm` 的机制表本来就是 `(contract × stdlib × format) → flags`。 +共享库的危害本身就是格式相关的,所以把 `Format` 提到 `default_contract` 的入参, +是让既有维度承担它,而不是新开一条平台分支: + +- **ELF**:一个全局符号命名空间,先加载的定义胜出 ⇒ 静态内嵌 libstdc++ 的 `.so` + 会把 2931 个 std 符号无版本导出(其中 **777 个是 GLOBAL 定义**,只可能来自 + `libstdc++.a`),可执行文件的 std 引用被绑到它身上 +- **Mach-O**:self-contained 的机制本就是 `-Wl,-load_hidden,`,符号是 + hidden,dyld 不会归一 ⇒ **危害不存在**;且 toolchain-coupled 在 macOS 是已知死路(#202) +- **PE**:没有全局命名空间,导入按 DLL 逐个按名解析 ⇒ **危害不存在**;`-static` + 是那里的标准约定 + +`link_line::UnitTail` 保持格式中立:槽位按**职责**命名,不按标志拼写。 +PE 的 `runtimeFallback`/`loaderTag` 天然为空,Mach-O 的 `dependencies` 装 +`@loader_path` —— 不需要任何 `if (platform)`。 + +--- + +## 1. 步骤拆分 + +每一步都有独立判据,可单独 revert。 + +### S1 — 新增 `mcpp.build.link_line` 协议模块 + +`src/build/link_line.cppm`,导出: + +```cpp +struct UnitTail { + std::string dependencies; // 1. -L/-l + $ORIGIN / @loader_path + std::string cxxRuntime; // 2. -static-libstdc++ / -load_hidden / --exclude-libs + std::string runtimeFallback; // 3. 兜底运行期搜索(今天=SubOS farm 视图) + std::string loaderTag; // 4. --disable-new-dtags,必须字面最后 + std::string render() const; // 按上述顺序连接,自动补分隔空格 + bool empty() const; +}; +``` + +每个槽位的注释必须写清**为什么在这个位置**,尤其: +`dependencies` 在最前,因为那些目录装的正是本次链接解析到的物理文件; +`runtimeFallback` 必须晚于它们,因为 farm 是 `xlings install` 会重写的视图; +`loaderTag` 字面最后,因为 ld 认最后一个 `--enable/--disable-new-dtags`。 + +**判据**:`tests/unit/test_link_line.cpp` 断言四槽顺序 + 空槽不产生多余空格 ++ 任意槽为空时其余顺序不变。 + +### S2 — `CompileFlags` 增加 `ldRuntimeFallback`,farm 不再进 `f.ld` + +`flags.cppm`:`farm_ld` 从 `f.ld`(`:996-999`)移出,赋给 `f.ldRuntimeFallback`。 +字段名取 provider 中立的 "runtime fallback",与 `link_line` 的槽位同名。 + +**判据**:`test_build_flags.cpp` 断言 `f.ld` 不含 farm 路径、`f.ldRuntimeFallback` 含。 + +### S3 — `ninja_backend` 改用 `UnitTail` 组装 `unit_ldflags` + +`ninja_backend.cppm:1400-1418` 的三次 `unit +=` 换成填槽 + `render()`。 + +**判据**:`test_ninja_backend.cpp` 新增用例 —— 带 `$ORIGIN` 的链接单元, +在 `ldflags + " " + unit_ldflags` 合成串里 `$ORIGIN` 位置 **早于** farm。 +(仅 Linux 分支产出 farm,其它宿主 `GTEST_SKIP`) + +### S4 — `Role` 拆出 `SharedLibrary`,`default_contract` 变 `(Role, Format)` 并成为唯一真源 + +1. `distribution.cppm`:`Role` **末尾**追加 `SharedLibrary`(不动既有枚举值, + 因为 `ldStdlibByRole` 按枚举值索引);导出 `kRoleCount` +2. `default_contract(Role, Format)`:`(SharedLibrary, Elf) → ToolchainCoupled`, + 其余一律 `SelfContained`(= 今天的行为) +3. `to_string(Role)` 补 `"shared-library"` +4. `flags.cppm:83-92` `role_of`:`LinkUnit::SharedLibrary → Role::SharedLibrary` +5. `flags.cppm:52,54`:`std::array<…, 3>` → `kRoleCount` +6. `flags.cppm:704-709`:`base` 改为调用 `default_contract`,并新增 + `sharedContract`;`mi.format` 的推导上移到契约计算之前 +7. `wantsArchives`(`:768`)把 `sharedContract` 纳入 + +**判据**:`test_distribution.cpp` 逐格断言 `(Role × Format) → Contract` 全表。 + +### S5 — C1 护栏:ELF 上显式 self-contained 的共享库必须隐藏归档符号 + +`distribution.cppm` ELF 分支,`Role::SharedLibrary` 且 `effective == +SelfContained` 时追加: + +- libstdc++:`-Wl,--exclude-libs,libstdc++.a` +- libc++:`-Wl,--exclude-libs,libc++.a -Wl,--exclude-libs,libc++abi.a` + (有 libunwind.a 时再加一条) + +用**归档基名**而非路径:`--exclude-libs` 按归档文件名匹配,GNU ld 与 lld 一致。 +显式列名而非 `ALL`,以免连用户自己的静态库一起隐藏。 + +**判据**:`test_distribution.cpp` 断言该单元格产出含 `--exclude-libs`, +且 `Role::Distributable` 不含。 + +### S6 — manifest:`cxx_runtime` 增加 `shared` 键 + +`types.cppm` 加 `cxxRuntimeShared`(`[build]` 与 `[target.]` 两处); +`toml.cppm:954-968` 的键白名单加 `"shared"`,错误文案同步。 + +**判据**:`test_manifest.cpp` 解析 `cxx_runtime = { default=…, tests=…, shared=… }`; +未知键仍报错。 + +### S7 — 测试:把假绿换成真判据 + +1. `tests/e2e/219_runtime_search_farm_is_last.sh`:断言 farm 是 DT_RPATH 的 + **字面**最后一项,删掉「last ABSOLUTE entry」放宽与那段理由 +2. **新增 e2e:行为不变量**。构造 `$ORIGIN` 与 farm 同名 SONAME 的局面, + 用 `LD_DEBUG=libs` 断言解析到 `$ORIGIN` 那一份。 + ⚠️ **不得依赖崩溃** —— C2 落地后崩溃会消失,依赖崩溃的断言会立刻假绿 +3. 新增 e2e:ELF 共享库**不得**导出 std 符号(`nm -D` 计数为 0) + +### S8 — 文档 + 版本 + pin + +- `docs/` 用户文档:`cxx_runtime` 的 `shared` 键、共享库默认契约按平台的说明 +- 版本 bump(`YYYY.M.D.N`,月日不补零) +- pin 最新 xlings(唯一真源 `src/platform/xlings/xlings.cppm::kXlingsVersion`, + 由 `check_version_pins.sh` 机器校验 16 处) + +--- + +## 2. 合并后的验证清单(缺一不可) + +| # | 判据 | 方法 | +|---|---|---| +| V1 | farm 是 DT_RPATH 字面最后一项 | `readelf -d` | +| V2 | `libX11.so.6` 解析到 `$ORIGIN` | `LD_DEBUG=libs`,**看行为不看形状** | +| V3 | `bin/libX11.so` 导出 std 符号数 = 0 | `nm -D \| grep -cE '_ZNSt\|_ZNKSt\|_ZSt'` | +| V4 | exe 不再有 `U _ZNKSt13runtime_error4whatEv` | `nm -D --undefined-only` | +| V5 | helloegui GUI 真的起来 | 本机跑,`timeout` 退 143 | +| V6 | **revert 掉 S2+S3 后 V2 必须重新变红** | 证明 C2 没吃掉 A 的判据 | +| V7 | PE / Mach-O 产物字节不变 | CI 对应 job | +| V8 | 生态:mcpp-index workspace 全绿 | compat 包全是 `.so`,是重灾区 | + +--- + +## 2.1 实施记录 —— 计划没写对的三处 + +**① `default_contract` 根本没有生产调用方。** +它自称「The role -> contract policy, in one place」,实际只有单测调用;真正的策略 +在 `flags.cppm:704-709` 独立算了第二遍。所以 S4 不只是"加一个入参",而是把这个 +函数**接回主路径**。这也是本次改动里架构收益最大的一处。 + +**② 「共享库导出零个 std 符号」这个判据一开始是错的。** +实测:一个用了 `std::string` 的 C++ 共享库,即使 toolchain-coupled,也会导出 **31 个** +std 符号 —— 全部是 `W`(weak/COMDAT 模板实例化,从头文件实例化进它自己的 TU), +GLOBAL 为 0。这些是 C++ ABI 的预期行为,进程内归一它们是**对的**。 + +真正的判据是 **GLOBAL 计数为 0**:`T` 符号只可能来自 `libstdc++.a`。 +对照数据:坏版本的共享库是 **713 GLOBAL**(+ weak),好版本是 **0 GLOBAL**。 +如果按"总数为零"写,断言不可满足,最后只会被删掉 —— 一条真不变量换成没有不变量。 + +**③ e2e 的工程形状换了两次才对。** +- `int main()`:DT_RPATH 里**根本没有 `$ORIGIN`**,整条断言链空转(这正是旧断言 + 能被写成那样的原因 —— 它从未在有 `$ORIGIN` 的产物上跑过) +- 同包内的 `kind = "shared"` target:mcpp 把该包的模块对象**直接链进可执行文件**, + 没有 `-l`、也没有 `$ORIGIN` +- **消费一个 path 依赖提供的共享库**:才产生 `-Lbin -Wl,-rpath,'$ORIGIN' -lgreetdep`, + 与出问题的真实产物同形 + +另外接口里写内联定义(`export int f() { return 7; }`)会让符号被实例化进消费者、 +依赖边消失,所以接口与实现必须分文件。 + +## 2.2 实测结果(helloegui:imgui + GLFW + X11) + +| 判据 | 修复前 | 修复后 | +|---|---|---| +| DT_RPATH 尾部 | `… : /lib : $ORIGIN` | `… : $ORIGIN : /lib` | +| `libX11.so.6` 解析到 | farm(`xim:libX11 1.8.10`) | `$ORIGIN`(farm 未被试到) | +| `bin/libX11.so` 导出 std 符号 | 2931(777 GLOBAL) | **0** | +| `bin/libXau.so` | 9 557 936 B | **39 368 B** | +| `bin/libXdmcp.so` | 9 561 504 B | **41 552 B** | +| exe 未定义 `runtime_error::what` | 有 | **无** | +| 运行 | `symbol lookup error` | **GUI 正常启动** | + +红测(两条新 e2e 对已发布 2026.8.11.2):219 报「farm is not the last entry」, +222 报「exports 713 GLOBAL standard-library symbols」—— 均以正确理由失败。 + +## 3. 已知风险 + +1. **C2 掩盖 A 的症状** —— 见 V6,这是本 PR 最大的假绿风险 +2. **本机 e2e 噪声** —— 共享 gcc specs 污染、`pipefail`+`grep -q` 的 SIGPIPE + flake;本机红必须逐条与已发布二进制比对,不可直接当回归 +3. ~~**`--exclude-libs` 的链接器覆盖面**~~ —— **已实测**。两条路径都验证过: + - libstdc++ 分支(`-lstdc++` 形式):e2e 222 不变量 4,GLOBAL 导出 713 → 0 + - libc++ 分支(**按完整路径**给归档):直接对拍 + `g++ -shared -nostdlib++ /libstdc++.a` ± `-Wl,--exclude-libs,libstdc++.a` + ⇒ `_ZNKSt13runtime_error4whatEv` 导出数 **1 → 0**,证明它按**基名**匹配、 + 与归档是 `-l` 还是完整路径给出无关 + + 若将来接入其它链接器,应在机制表里加格式/链接器维度,而不是加平台分支。 + +4. **`dist::CompileFlags::contractByRole` 只写不读**(既有,非本次引入)。 + 它与 `TargetEntry::cxxRuntimeTests`(既解析不了也不生效)是同一类死字段, + 本次没有顺手清理 —— 删一个公开结构体字段是另一件事,不该混进修缺陷的 PR。 diff --git a/.agents/docs/2026-08-11-runtime-search-origin-precedence-analysis.md b/.agents/docs/2026-08-11-runtime-search-origin-precedence-analysis.md new file mode 100644 index 00000000..01862979 --- /dev/null +++ b/.agents/docs/2026-08-11-runtime-search-origin-precedence-analysis.md @@ -0,0 +1,453 @@ +# `$ORIGIN` 被 SubOS farm 遮蔽 —— helloegui 运行期 undefined symbol 分析与修复方案 + +日期:2026-08-11 +起因:`mcpp run`(helloegui,依赖 imgui 0.0.6)链接成功、运行即死 +涉及版本:2026.8.11.2(PR #413 首次把 SubOS 库视图写进 DT_RPATH) +状态:**分析完成,方案已定,未实施** +本批 PR 范围:**A + B + C2 + C1**(决策记录见 §6.1) + +--- + +## 0. 一句话 + +PR #413 把 SubOS farm(`/registry/subos/default/lib`,一个 300 条目的 +平铺符号链接视图)加进了产物的 DT_RPATH,但它落在 **`$ORIGIN` 之前**;于是产物 +运行期加载的 `libX11.so.6` 不是链接时的那一个,而是 farm 里 xim 上游的另一份构建。 +两份 libX11 不可互换 —— mcpp 自建的那份因为 `-static-libstdc++` 把整个 libstdc++ +烙了进去并**对外导出**,链接期正是它满足了可执行文件的 `std::runtime_error::what()`。 + +模块自己写着「FARM LAST」这条不变量(`src/build/plan.cppm:660`),而 +`$ORIGIN` 由另一个生产者在另一条通道上发出,两者从未在同一个排序里相遇。 + +--- + +## 1. 现象 + +``` +$ mcpp run + Finished dev [unoptimized + debuginfo] in 3.26s + Running `target/x86_64-linux-gnu/eb46e2850893f013/bin/helloegui` + +.../bin/helloegui: symbol lookup error: .../bin/helloegui: + undefined symbol: _ZNKSt13runtime_error4whatEv +``` + +同时刷出 13 条 runtime closure 警告(`rule B inconclusive … has no loader path / +has no library directory`)。**这两件事无关**,见 §5。 + +复现:稳定,100%。第二次 `mcpp build`(指纹变为 `4ab95c7547486246`)警告消失, +**崩溃依旧** —— 崩溃与首次运行无关。 + +--- + +## 2. 根因链(每一环都有实测) + +### 环 1 — 产物的 DT_RPATH 里 `$ORIGIN` 排在 farm 之后 + +``` +$ readelf -d bin/helloegui | grep RPATH + RPATH: [ …/xim-x-glibc/2.44/lib64 + : …/xim-x-gcc/16.1.0/lib64 + : …/compat-x-glx-runtime/…/glx_runtime/lib + : /home/speak/.mcpp/registry/subos/default/lib ← farm + : $ORIGIN ] ← 载荷目录,最后 +``` + +**两个生产者,一条 ninja 命令行,没有共同的排序:** + +| 条目 | 生产者 | 落入 | +|---|---|---| +| glibc / gcc / glx_runtime | `flags.cppm` runtime_dirs | 全局 `$ldflags` | +| **farm** | `flags.cppm:508-526` → `f.ld`(`:996-999`) | 全局 `$ldflags`(末尾) | +| **`$ORIGIN`** | `plan.cppm:450-467` `shared_library_link_flags` | per-unit `$unit_ldflags` | + +链接规则(`ninja_backend.cppm:867-872`): + +``` +command = $cxx $in -o $out $ldflags $unit_ldflags +``` + +`$ldflags` 在前 ⇒ farm 在前 ⇒ **`$ORIGIN` 永远最后**。 + +`flags.cppm:509` 的注释写的是「appended after everything else so it is LAST in +the artifact's DT_RPATH」—— 它只在 `$ldflags` 内部为真,而 `unit_ldflags` 还在后面。 + +### 环 2 — 同一个 SONAME 在两个目录里都存在 + +「同名」指的是 **SONAME,不是包名**。本工程里同时存在两个物理文件: + +| | mcpp compat 包 | xim 上游包 | +|---|---|---| +| 包身份 | `compat.x11 v1.8.13`(源码包) | `xim:libX11@1.8.10` | +| 谁产出 | **mcpp 自己从源码编译**(`obj/compat_x11/src/*.o`) | xlings 装的预编译产物 | +| 落在哪 | `target/…/bin/libX11.so`(= `$ORIGIN`) | `…/xpkgs/xim-x-libX11/1.8.10/lib/`,经 farm 符号链接暴露为 `/lib/libX11.so.6` | +| **SONAME** | **`libX11.so.6`** | **`libX11.so.6`** | + +上游版本都不同(1.8.13 vs 1.8.10),SONAME 却相同 —— 而**加载器只按 SONAME 找**。 +farm 是 `xlings install` 每次都会重写的平铺视图,凡是 mcpp 有 compat 源码包、 +xim 又有同一个库的预编译包,就会撞上。这不是边缘情况,是常态。 + +**两条通道,两套顺序,从未被约束成一致:** + +| | 构建期通道 | 运行期通道 | +|---|---|---| +| xim 包 | `--sysroot=` ⇒ `/lib` 成为链接器默认目录 | farm 进 DT_RPATH(#413 新加) | +| mcpp compat 包 | `-Lbin` | `$ORIGIN` | + +- `-L` 序:`-L` `-L` **`-Lbin`** … sysroot 默认目录在最后 + ⇒ `-lX11` 选中 **`bin/libX11.so`** +- RPATH 序:`glibc : gcc : glx : /lib : $ORIGIN` + ⇒ 运行期选中 **farm 里的 xim 那份** + +即:**mcpp 拿 A 链接,却让产物去加载 B。** + +链接期用了哪一份不必推理,**产物自己证明了**:exe 里 +`_ZNKSt13runtime_error4whatEv` 是动态未定义且没有 `NEEDED libstdc++.so.6`, +而只有 `bin/libX11.so` 导出该符号(见环 3)。若链接期用的是 farm 那份,该符号 +要么链接期报错,要么从 `libstdc++.a` 拉入 —— 两种都产生不出现在这个产物。 + +`LD_DEBUG=libs` 实测(前 4 个 rpath 目录逐个 miss,farm 命中): + +``` +find library=libX11.so.6 [0]; searching + trying file=…/xim-x-glibc/2.44/lib64/libX11.so.6 ✗ + trying file=…/xim-x-gcc/16.1.0/lib64/libX11.so.6 ✗ + trying file=…/glx_runtime/lib/libX11.so.6 ✗ + trying file=…/registry/subos/default/lib/libX11.so.6 ✓ ← 命中 farm +``` + +`$ORIGIN` 根本没被走到。 + +### 环 3 — 两份 libX11 不可互换 + +| | `bin/libX11.so`(mcpp 自建) | `xim-x-libX11/1.8.10`(farm) | +|---|---|---| +| 导出 std 符号数 | **2931**(777 T + 2777 W) | **0** | +| `_ZNKSt13runtime_error4whatEv` | 导出 | 无 | +| 符号版本 | 无(`GLIBCXX_*` verdef 不存在) | — | + +mcpp 自建的那份为什么会导出整个标准库: + +1. `flags.cppm:86-90` 把 `SharedLibrary` 映射为 `dist::Role::Distributable` + (`distribution.cppm:47-51`:"Binary / SharedLibrary — leaves this machine") +2. `distribution.cppm:330-338`:ELF + libstdc++ + SelfContained ⇒ `-static-libstdc++` +3. **每个链接单元都带 `obj/std.o`**(`import std` 的模块对象)。实测 + `bin/libXau.so` 的输入是 8 个 `.o`(纯 C)**加一个 `obj/std.o`**, + 于是 `libstdc++.a` 被整个拖进来 —— libXau 从应有的 ~20KB 变成 **9.5MB** + +### 环 4 — 可执行文件的 `-static-libstdc++` 因此落空 + +链接行(`build.ninja` 第 1957 行的 unit_ldflags): + +``` +… -Lbin -Wl,-rpath,'$ORIGIN' -lX11 … -lXext -static-libstdc++ -Wl,--disable-new-dtags +``` + +`-lX11` 在驱动追加的 `-lstdc++`(此处即 `libstdc++.a`)**之前**。ld 处理到 +`-lX11` 时该符号已被这个**共享库**满足 ⇒ 归档成员从不被拉入 ⇒ 引用留在动态未定义: + +``` +$ nm -D --undefined-only bin/helloegui | grep runtime_error + U _ZNKSt13runtime_error4whatEv +$ readelf -d bin/helloegui | grep NEEDED + … libX11.so.6 … libm.so.6 libgcc_s.so.1 libc.so.6 ← 没有 libstdc++.so.6 +``` + +即:**helloegui 的 C++ 运行时事实上来自 `libX11.so`**。它声称的 +self-contained 是假的,而这份「假」在环 1 把 libX11 换成上游那份的瞬间变成硬崩溃。 + +### 因果验证(A/B,不是推理) + +| 实验 | 结果 | +|---|---| +| 原样运行 | `symbol lookup error` | +| `LD_PRELOAD=bin/libX11.so.6` | **正常启动,GUI 窗口出现**(挂起到被 kill) | +| 副本 patchelf,载荷目录提到 RPATH 首位 | **正常运行 8s 被 timeout 杀掉(exit 143)** | + +第三条是决定性的:**只调整 RPATH 顺序,不改任何代码,崩溃消失。** + +--- + +## 3. 为什么测试没拦住 —— 一个被实测结果反向驯化的断言 + +PR #413 新增了 `tests/e2e/219_runtime_search_farm_is_last.sh`,标题就叫 +"farm is last"。它绿着,而真实产物里 farm 不是最后一位。 + +`tests/e2e/219…sh:151-171`: + +```bash +# The farm must be the last ABSOLUTE entry — not literally the last entry. +# +# `$ORIGIN`-relative entries are a different kind: they address the artifact's +# own directory, not this machine, so they travel with it and their position +# says nothing about which machine-local directory wins. … +# (Measured on a real GLFW app, whose DT_RPATH ends `… : /lib : $ORIGIN`.) +RPATH_LAST_ABS="$(… [x for x in DT_RPATH.split(':') if x.startswith('/')] … [-1])" +``` + +作者**实测到了本文分析的这个真实顺序**,判定它是可接受的,于是把断言从 +「字面最后」放宽到「最后一个绝对路径条目」。 + +那条理由恰好反了:决定胜负的**正是**「同一个 SONAME 在 `$ORIGIN` 和 farm 里都有」, +而这在 mcpp 生态里是常态而非例外(见环 2)。 + +**准确地说,这条断言不是「把坏顺序钉成预期」,而是对它结构性失明** —— 因为 +`$ORIGIN` 不以 `/` 开头,被那行 python 过滤掉了: + +| | 绝对路径条目 | 最后一项 | 判定 | +|---|---|---|---| +| 未修 `… : /lib : $ORIGIN` | `[glibc, gcc, glx, /lib]` | farm | ✓ 通过 | +| 修好 `… : $ORIGIN : /lib` | `[glibc, gcc, glx, /lib]` | farm | ✓ 通过 | + +两者**完全一样**。所以它在坏产物和好产物上给出同一个结论,而它的名字叫 +"farm is last" —— 它在一个 farm 并非最后的二进制上报告「farm 最后」。 +这比「会在修复后变红」更糟:修复根本不会惊动它。 + +> 教训(与既有记录一致):断言被实测结果驯化时,要先证明「实测到的形态是对的」, +> 而不是把它当成基准。这条不变量的正确判据是**行为**(同名库解析到谁), +> 不是**形状**(哪一项排在末尾)—— 形状断言在这里天生不够,这正是 §4-B 必须补 +> 一条行为不变量的原因。 + +模型层面同样有缺口:`runtime_search.cppm` 的 `rank()`(`:82-92`)只有 +Payload / Package / SubosFarm / HostDefault 四档,**`$ORIGIN` 不在闭包里**。 +`plan.cppm:660` 声称「`search::ordered` is what enforces it」,但它管不到 +一个由别的通道发出的条目 —— 这个排序模型对最关键的目录是装饰性的。 + +--- + +## 4. 修复方案 + +### A(必须,修崩溃)— farm 移到整条链接行的尾部 + +**改动** + +1. `src/build/flags.cppm`:`CompileFlags` 新增 `std::string ldFarmTail;` + `farm_ld` 不再拼进 `f.ld`(`:996-999`),改为 `f.ldFarmTail = farm_ld;` +2. `src/build/ninja_backend.cppm:1400-1418`:组装 `unit_ldflags` 时, + 在 `flags.ldStdlibFor(role)` **之后**、`lu.loaderTagFlag` **之前**追加 + `flags.ldFarmTail`;`StaticLibrary` 跳过(与 `ldStdlibFor` 对 Intermediate + 返回空一致)。 + - `loaderTagFlag` 必须保持字面最后 —— 加载器标签由 ld 看到的最后一个 + `--enable/--disable-new-dtags` 决定(该处注释已写明) + +**结果** + +``` +RPATH: glibc/lib64 : gcc/lib64 : glx_runtime/lib : $ORIGIN : /lib +``` + +载荷目录仍在最前(libc/libstdc++ 来自钉住的载荷),`$ORIGIN` 次之(产物链接时 +看见的正是这些同级库),farm 真正兜底。 + +**否决的替代** + +- *把 `$ORIGIN` 提到全局 ldflags*:`-Lbin` 与 `-Wl,-rpath,'$ORIGIN'` 是成对的 + per-unit 事实(只有消费共享库的单元才需要),提成全局会给每个产物无条件加一条 +- *调换 ninja 规则里 `$ldflags` / `$unit_ldflags` 的次序*:会同时移动所有 `-L` + 搜索序与 `-specs`,影响面远超本问题 + +**风险**:低。`verify_hermetic_link`(`hermetic.cppm:103`)检查的是 `flags.ld` +解析出的库路径是否落在允许根内;farm 路径位于 `tc.sysroot`(= ``)之下, +本就在允许集合里,把它从被检字符串中移走不改变判定。 + +### A+(单独 PR,紧随本批)— 让闭包模型真正拥有这个顺序 + +`runtime_search.cppm` 增加 `Origin::Artifact`,`rank()` 置于 `Package` 与 +`SubosFarm` 之间;`runtime_search_closure`(`plan.cppm:667`)把产物输出目录 +(记为 `$ORIGIN`)纳入闭包。收益: + +- `resolution.json` 记录的闭包与 DT_RPATH 变得**逐项可比**(今天记录里没有 + `$ORIGIN`,e2e 219 的「记录 vs 产物」比对因此天生有个缺口) +- 「FARM LAST」不再靠两个生产者各自自觉 + +代价:`is_machine_local()` 需为 Artifact 定义语义(`$ORIGIN` 随产物走 ⇒ 非 +machine-local)。 + +**不涉及 `pack`(已核)**:`is_machine_local` 全仓只有一个生产消费方 +(`prepare.cppm:6417` → 写进 `resolution.json` → `doctor.cppm:703` 打 +`[machine-local]` 标签);`src/pack/pack.cppm` 不 import `mcpp.platform.runtime_search`, +它自己用 patchelf 把所有 RPATH 重写成 `$ORIGIN/../lib`(`pack.cppm:745-779`)。 + +A+ 有两个档位,建议只做 (i): + +- **(i) 轻**:产物输出目录以 `Origin::Artifact` 进入闭包**记录**。收益是记录与 + DT_RPATH 变得逐项可比,测试可以硬比对。改动 = enum + 3 处 switch + + `plan.cppm` 加一条 + doctor 显示。 +- **(ii) 重**:让闭包成为**唯一**的 rpath 生产者,即把 `$ORIGIN` 的发出也从 + `shared_library_link_flags` 挪进来。这才真正消灭「两个生产者」,但 `$ORIGIN` + 本质是 per-unit 的(只有消费共享库的单元才需要),挪进全局闭包意味着要给闭包 + 引入 per-unit 概念 —— 改动量与风险都明显更大。记 issue,不急。 + +### B(必须)— 把测试改回真不变量 + +1. `tests/e2e/219_runtime_search_farm_is_last.sh:151-171`:断言 farm 是 DT_RPATH + 的**字面最后一项**,删除「last ABSOLUTE entry」的放宽与那段理由 +2. **新增行为不变量**(比形状断言更硬):构造一个同时存在于 `$ORIGIN` 与 farm 的 + SONAME,断言 `LD_DEBUG=libs` 解析到 `$ORIGIN` 那一份。这是本次缺陷的直接判据 +3. 单测 `tests/unit/test_ninja_backend.cpp`:给带 `-Wl,-rpath,'$$ORIGIN'` 的 + 链接单元设置 `plan.runtimeSearch` 含一条 `Origin::SubosFarm`,断言在 + `ldflags + " " + unit_ldflags` 的合成串里 `$ORIGIN` 的位置 **早于** farm。 + (仅 Linux 分支产出 farm_ld,其它宿主 `GTEST_SKIP`) + +先写测试、确认变红,再改代码。 + +### C2(已批准,与 A 同一个 PR)— 共享库的 C++ 运行时契约改为 toolchain-coupled + +**今天**:mcpp 每建一个 `.so` 都按「要离开这台机器的成品」处理 ⇒ +`-static-libstdc++` ⇒ 把整个 libstdc++ 塞进这个 `.so` 并对外导出。 + +**C2**:`.so` 不再自带 std,而是 `NEEDED libstdc++.so.6`,运行期从 gcc 载荷目录 +解析 —— 那个目录本来就是 DT_RPATH 第 2 项,已经在了。 + +| | 今天(self-contained) | C2(toolchain-coupled) | +|---|---|---| +| 一个进程里几份 libstdc++ | exe 一份 + 每个 `.so` 一份 | 一份(见「残留」) | +| `bin/libXau.so` | 9.5 MB | ~20 KB(叠加 C3 后) | +| `.so` 单独拷走能不能跑 | 能(自带) | 不能,需带上 `libstdc++.so.6` | +| 跨 `.so` 边界抛 std 异常 | 有风险(两份 typeinfo) | 正常 | + +**改动点** + +1. `distribution.cppm:47-51` `Role` 拆出 `SharedLibrary` —— 今天 `Binary` 与 + `SharedLibrary` 共用 `Distributable`,注释就写着 "Binary / SharedLibrary — + leaves this machine",这一行正是本缺陷的策略源头 +2. `distribution.cppm:123` `default_contract`:新角色 → `Contract::ToolchainCoupled`; + `to_string(Role)`(`:97`)补一项 +3. `flags.cppm:83-92` `role_of`:`LinkUnit::SharedLibrary` 不再落到 `Distributable` +4. `flags.cppm:52,54`:`std::array<…, 3>` → `4` +5. manifest:`cxx_runtime` 已经是 role-aware 的表形式 + (`{ default = …, tests = … }`,`toml.cppm:933-949`),补一个 `shared` 键; + `types.cppm:421-426` 与 target 段的 `:621-622` 同步 +6. **机制表无需改动**:`distribution.cppm:330-338` 已把 ELF + libstdc++ + + ToolchainCoupled 处理成「不发任何标志」,驱动默认链 `libstdc++.so`,而 gcc + 载荷目录本来就在 `-L`/`-rpath` 里 + +**预期效果(可直接测)** + +- `nm -D --defined-only bin/libX11.so | grep -cE '_ZNSt|_ZNKSt|_ZSt'` 从 2931 → 0 +- exe 链接期 `-lX11` 不再满足 `runtime_error::what()` ⇒ 从 `libstdc++.a` 拉入 + ⇒ exe 的 `-static-libstdc++` **恢复为真** + +**⚠️ C2 会掩盖 A 的症状 —— 同 PR 时这是首要风险** + +C2 之后,即使 RPATH 顺序仍然是坏的,helloegui 也**不会再崩** —— 符号已经在 exe +内部。但产物加载的**仍然是 farm 里的 libX11 1.8.10,而不是链接时的 1.8.13**: +一次响亮的崩溃被换成一个静默的版本错配。 + +**所以 A 的回归测试绝不能依赖崩溃。** §4-B-2 那条行为不变量(同名 SONAME 必须 +解析到 `$ORIGIN`)不是锦上添花,它是 A 在 C2 之后唯一还能变红的判据。 +写测试的顺序必须是:先在未打 A 也未打 C2 的二进制上确认它红,再分别验证。 + +### C1(与 C2 同批)— 逃生舱的护栏 + +C2 之后,用户仍可显式写 `cxx_runtime = { shared = "self-contained" }` 让 `.so` +静态链 libstdc++。此时必须同时发 `-Wl,--exclude-libs,libstdc++.a` +(视情况含 `libsupc++.a` / `libgcc.a`),否则符号泛滥原样复现。 + +C1 不再是止血手段,而是让「非默认选项」不至于重新打开这个洞。 + +### C3(单独 PR,不阻塞本批)— 别把 `obj/std.o` 塞进不需要它的单元 + +**接缝已定位**:`ninja_backend.cppm:1362-1381` 对 `Binary` / `TestBinary` / +`SharedLibrary` **无条件**追加 `obj/std.o`,完全不看该单元是否真的 `import std`。 +实测 `bin/libXau.so` 的输入 = 8 个纯 C `.o` + 一个 `obj/std.o`。 + +C2 之后 `.so` 不再内嵌 libstdc++,但**仍然会因为 std.o 而 `NEEDED +libstdc++.so.6`** —— 纯 C 的 compat 包(libXau / libXdmcp / libX11)凭空多一条 +依赖。C3 把它摘掉,顺带让 libXau 回到 ~20KB、libX11 回到上游量级。 + +需要先确认的:`import std` 的传递性(依赖的 BMI 传递 import std 的历史坑见 +`dep-bmi-cache-cross-version-poisoning`),以及静态库单元的处理。 + +### 残留(记 issue,不在本批)— 两份 libstdc++ + +C2 之后,exe(SelfContained,静态)+ 真正用 C++ 的 `.so`(ToolchainCoupled, +动态)= 一个进程两份 std,跨 `.so` 边界抛 std 异常会失败(typeinfo 不同)。 +今天不会发生(compat 包都是纯 C,C3 还会把 std.o 摘掉),但这是 C2 引入的新形态。 + +顺带一个观察,值得写进那个 issue:本工程产物的 PT_INTERP 是 +`/xim-x-glibc/2.44/lib64/ld-linux-x86-64.so.2` —— **它本来就离不开载荷**, +所以 exe 上的 `-static-libstdc++` 在这个配置下几乎买不到东西。 +「exe 是否也该 ToolchainCoupled」才是那个 issue 的真正问题。 + +### D(低优先)— 首次运行 rule B 全线 inconclusive + +实测: + +| | `binding.loader` | `binding.library_dirs` | 警告 | +|---|---|---|---| +| 首次运行(同时安装工具链) | `""` | `[]` | 13 条 | +| 第二次 `mcpp build` | `…/xim-x-glibc/2.44/lib/ld-linux-x86-64.so.2` | `[…/xim-x-glibc/2.44/lib]` | 无 | + +`runtime_binding.cppm:337-380` 从 `/lib64`、`/lib` 里找 +`libc.so.6` 与唯一的 `ld-linux-*` 来填这两个字段;磁盘上二者都在(`lib/libc.so.6` +符号链接、唯一一个 `ld-linux-x86-64.so.2`),说明**首次运行时求值早于 farm 落盘**。 +精确接缝(binding 解析点 vs 载荷/farm 写入点)还需要一次探针确认,不要照着推理改。 + +影响:新用户的第一次构建看到 13 条自己无法处理的警告,而 rule B 恰在最该生效的 +那一次运行里失效。 + +**必须说清楚:即使 rule B 完全正常,它也抓不到本文的崩溃。** 它只比对 +libc / PT_INTERP 的同一性(`elf_runtime.cppm:761-801`),不管其它 SONAME 解析到谁。 +把 D 修好不能替代 A。 + +--- + +## 5. 两件事无关 + +首次运行的 13 条警告(D)与崩溃(A)在时间上同时出现,容易被读成一件事。 +第二次构建警告消失、崩溃照旧,已经把它们分开。 + +--- + +## 6. 实施顺序 + +**本批 PR = A + B + C2 + C1。** A+(i)、C3、D、以及「两份 libstdc++」各自独立。 + +1. **先写测试,并在未打任何补丁的二进制上确认全红** + - B-3 单测(`test_ninja_backend.cpp`):`$ORIGIN` 必须早于 farm + - B-1 e2e 219:断言 farm 是 DT_RPATH 的**字面**最后一项 + - B-2 e2e **行为**不变量:同名 SONAME 必须解析到 `$ORIGIN` + —— ⚠️ 这一条**不得依赖崩溃**,否则 C2 一落地它就假绿(见 §4-C2) +2. **A**:farm 移到 per-unit 尾部 → B-1 / B-3 转绿,B-2 转绿 +3. **C2 + C1**:角色拆分 + 契约改判 + `--exclude-libs` 护栏 + - 判据:`nm -D bin/libX11.so | grep -cE '_ZNSt|_ZNKSt|_ZSt'` = 0 + - 判据:exe 不再有 `U _ZNKSt13runtime_error4whatEv` +4. **合并验证(缺一不可)** + - helloegui 全链复现 ⇒ GUI 启动 + - 产物的 `libX11.so.6` 解析到 `$ORIGIN`(`LD_DEBUG=libs` 实证,不看形状) + - **把 A 单独 revert 掉,B-2 必须重新变红** —— 证明 C2 没有把 A 的判据吃掉 +5. 全量单测 + e2e(注意既有本机噪声:共享 gcc specs 污染、`pipefail`+`grep -q` + 的 SIGPIPE flake —— 本机红需逐条与已发布二进制比对,不可直接当回归) +6. 生态验证:C2 改的是 `.so` 的运行期契约,发版前必须在真实 mcpp-index + workspace 上跑一遍(compat 包全是 `.so` 的重灾区) +7. 独立开:**A+(i)**、**C3**、**D**、**两份 libstdc++** + +## 6.1 决策记录(2026-08-11) + +| 项 | 决定 | +|---|---| +| C2(共享库 → toolchain-coupled) | **采纳**,与 A 同一个 PR | +| C1(`--exclude-libs` 护栏) | 随 C2 一起 | +| A+ | 只做 (i) 轻档,**单独 PR**;(ii) 记 issue | +| C3(`std.o` 无条件追加) | 单独 PR,不阻塞 | +| D(首次运行 rule B inconclusive) | 单独 issue,需先探针定位接缝 | + +--- + +## 7. 附:关键实测命令 + +```bash +# 顺序 +readelf -d bin/helloegui | grep RPATH +# 谁被加载 +LD_DEBUG=libs <私有 loader> bin/helloegui 2>&1 | grep -A6 'find library=libX11' +# 两份库的差别 +nm -D --defined-only bin/libX11.so | grep -cE '_ZNSt|_ZNKSt|_ZSt' # 2931 +nm -D --defined-only /xim-x-libX11/1.8.10/lib/libX11.so.6 | grep -cE '_ZNSt|_ZNKSt|_ZSt' # 0 +# 决定性 A/B +patchelf --force-rpath --set-rpath ":<原有其余项>" ./helloegui-copy && timeout 8 ./helloegui-copy +``` + +> 注:本机 `readelf` / `ldd` 被 xlings shim 劫持(`readelf` 那条还指向一个已消失的 +> scratchpad 路径),诊断一律走 `/usr/bin/` 绝对路径。 diff --git a/CHANGELOG.md b/CHANGELOG.md index fa9e8db9..91e6c172 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,62 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.8.11.3] — 2026-08-11 + +### 修复 + +- **⚠️ 回归:产物加载的库不是它链接的那一份 —— `$ORIGIN` 被 SubOS 库视图遮蔽。** + + `2026.8.11.2`(PR #413)首次把 SubOS 库视图(farm)写进产物的 `DT_RPATH`,但它 + 落在 **`$ORIGIN` 之前**。于是 imgui/GLFW 应用链接的是 mcpp 从 `compat.x11` 源码 + 构建、部署到产物目录的 `libX11.so`,运行期加载的却是 farm 里 xlings 装的 + `xim:libX11` —— **链接期用 A,运行期加载 B**,程序在 main 之前就死: + + ``` + undefined symbol: _ZNKSt13runtime_error4whatEv + ``` + + 真因不是「放错了位置」,而是**一条链接命令行的顺序由两个互不知情的生产者用 + `+=` 决定**:`flags.cppm` 把 farm 拼进全局 ldflags(并注释「so it is LAST」), + `plan.cppm` 把 `$ORIGIN` 拼进 per-unit,而每条链接规则渲染的是 + `$ldflags $unit_ldflags`。三处各自都对,合起来是错的。 + + 新增 `mcpp.build.link_line`:把 per-unit 尾部声明成**具名槽位**,相对顺序写在 + 类型里、由单测钉死。新增一个生产者必须先选一个槽 —— 而"选"正是"在产物自己的 + 目录之前还是之后"这个问题被提出来的地方。 + +- **⚠️ 共享库不再把自己的 C++ 运行时导出给别人(ELF)。** + + `SharedLibrary` 此前与可执行文件共用 `Distributable` 角色,于是拿到同一份 + self-contained 契约:`-static-libstdc++`。在 ELF 上这不是"私有一份" —— 只有一个 + 全局符号命名空间,共享对象会导出它定义的每一个全局符号。一个**纯 C** 的 compat 包 + 因此导出了 777 个 GLOBAL 标准库符号(`libXau.so`:39KB 的 Xau + 9.5MB 的 libstdc++)。 + + 可执行文件链接时 `-lX11` 排在驱动的 `-lstdc++` 之前,ld 就用它满足了 + `std::runtime_error::what()`,归档成员从不拉入 —— **可执行文件的 + `-static-libstdc++` 变成空操作,它的 C++ 运行时事实上是那个 `.so`**。上一条的 + 库替换之所以致命,根源在这里。 + + 共享库默认契约改为**按目标格式分档**:ELF `toolchain-coupled`, + Mach-O / PE 维持 `self-contained`(两者都没有这个危害 —— Mach-O 的机制本就是 + `-load_hidden`,PE 没有全局命名空间)。显式 `cxx_runtime = { shared = "…" }` + 仍可选回自包含,此时自动补 `-Wl,--exclude-libs`,让内嵌的运行时留在动态符号表之外。 + + 实测(helloegui,imgui + GLFW + X11):`libXau.so` 9.5MB → 39KB, + `libX11.so` 导出 std 符号 2931 → 0,GUI 正常启动。 + +### 内部 + +- `dist::default_contract` 从**没有任何生产调用方**变成唯一真源:角色→契约的策略 + 此前在 `flags.cppm` 被第二次推导,而这正是 `distribution.cppm` 开篇声讨的那类债 + (「used to be derived independently in five places」)换个位置复发。 + +- e2e 219 的断言由「farm 是最后一个**绝对路径**条目」收紧为「**字面**最后一项」, + 并补一条**行为**不变量(`LD_DEBUG=libs` 实测同名 SONAME 解析到 `$ORIGIN`)。 + 旧断言把 `$ORIGIN` 过滤掉了,对坏顺序与好顺序给出同一个结论 —— 它在一个 farm + 并非最后的二进制上报告「farm is last」。测试工程也从 `int main()` 换成消费依赖 + 共享库,否则它连 `$ORIGIN` 都不产生,整条断言链是空转的。 + ## [2026.8.11.2] — 2026-08-11 ### 修复 diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index 5246c191..2286f9ee 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -285,8 +285,10 @@ cxx_runtime = "self-contained" # applies to every target (the default) # or, per role: [build.cxx_runtime] -default = "self-contained" # binaries and shared libraries +default = "self-contained" # executables tests = "host-coupled" # test binaries never leave this machine +shared = "self-contained" # shared libraries (see below — the + # default differs by target format) # or, per target triple — beside `linkage`, which is the same axis: [target.x86_64-linux-gnu] @@ -309,14 +311,31 @@ libc++.a/libc++abi.a/libunwind.a explicitly. A lower macOS floor (11–13) requi self-built libc++ archive (already verified to work, a data-level switch, available on request). +**Shared libraries are the one role whose default depends on the target format**, +because the hazard does. A `.so`/`.dylib`/`.dll` is not a small executable — it is +loaded *into* a process that already has a C++ runtime. + +| target | default for `kind = "shared"` | why | +|---|---|---| +| ELF (Linux, …) | `toolchain-coupled` | ELF has one global symbol namespace and the first definition loaded wins. A `.so` that statically embedded libstdc++ **exports** it, and the executable linking that library binds *its* `std::` references there — its own `self-contained` contract silently becomes a no-op, and its C++ runtime is whichever build of that library happens to load. | +| Mach-O | `self-contained` | the mechanism there is already `-load_hidden`, i.e. hidden visibility, so dyld never unifies those symbols; and toolchain-coupled is not available on macOS at all (see the note below). | +| PE (Windows) | `self-contained` | PE has no global symbol namespace — imports resolve per-DLL by name, so a DLL's private runtime cannot be picked up by anything else. | + +Setting `shared = "self-contained"` on ELF is supported and does exactly what it +says: the library embeds the runtime. mcpp additionally passes +`-Wl,--exclude-libs` for the standard-library archives, so the embedded copy stays +out of the library's dynamic symbol table and cannot be picked up by anything that +links it. Template instantiations your own code emits (weak/COMDAT `std::string` +symbols and the like) are still exported — that is the intended C++ ABI behaviour +and is not the leak this guards against. + +A project-wide `cxx_runtime = "…"` (or `static_stdlib = false`) applies to shared +libraries too: a human said what the whole project promises. The format-specific +default applies only when nobody said anything. + `static_stdlib` is the older spelling and still works: `true` means `self-contained`, `false` means `host-coupled`. An explicit `cxx_runtime` wins. -> **Current implementation limitation.** The parser recognizes `cxx_runtime`, -> but the current `[build]` unknown-key allowlist omits it. A normal build can -> therefore emit an unsupported-key warning, and `--strict` rejects the manifest. -> This is an implementation defect, not a different spelling or contract. - **A contract that cannot be honored is reported, never silently downgraded.** If a toolchain ships no `libc++.a`, or a contract has no mechanism on that platform (`self-contained` under the MSVC runtime would need `/MT`, which mcpp does not emit diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index ae91df70..b8c9c934 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -255,8 +255,9 @@ cxx_runtime = "self-contained" # 作用于所有目标(默认值) # 或者按角色分别指定: [build.cxx_runtime] -default = "self-contained" # 可执行文件与共享库 +default = "self-contained" # 可执行文件 tests = "host-coupled" # 测试二进制从不离开本机 +shared = "self-contained" # 共享库(见下 —— 默认值随目标格式而变) # 或者按目标三元组 —— 与 `linkage` 并列,因为它们是同一根轴: [target.x86_64-linux-gnu] @@ -277,13 +278,28 @@ libc++/libc++abi —— 系统 libc++ 会把实际可运行版本钉死在构建 libc++.a/libc++abi.a/libunwind.a。更低的 macOS floor(11–13)需自建 libc++ 归档(已验证可行,数据级切换,按需提供)。 +**共享库是唯一一个默认值随目标格式变化的角色**,因为危害本身随格式变化。 +`.so`/`.dylib`/`.dll` 不是一个小号可执行文件 —— 它被加载**进**一个已经有 +C++ 运行时的进程。 + +| 目标格式 | `kind = "shared"` 的默认契约 | 原因 | +|---|---|---| +| ELF(Linux 等) | `toolchain-coupled` | ELF 只有一个全局符号命名空间,先加载的定义胜出。静态内嵌了 libstdc++ 的 `.so` 会把它**导出**,链接该库的可执行文件于是把自己的 `std::` 引用绑到那里 —— 它自己的 `self-contained` 契约静默变成空操作,它的 C++ 运行时变成"碰巧加载的那一份该库"。 | +| Mach-O | `self-contained` | 那里的机制本来就是 `-load_hidden`(hidden 可见性),dyld 不会归一这些符号;而且 macOS 上根本没有 toolchain-coupled 这一档(见下文注)。 | +| PE(Windows) | `self-contained` | PE 没有全局符号命名空间 —— 导入按 DLL 逐个按名解析,一个 DLL 的私有运行时不可能被别人捡走。 | + +在 ELF 上显式写 `shared = "self-contained"` 是支持的,而且就是字面意思:库会内嵌 +运行时。此时 mcpp 会额外发 `-Wl,--exclude-libs`(针对标准库归档),让内嵌的那份 +留在库的动态符号表之外,链接它的任何东西都捡不走。你自己代码产生的模板实例化 +(`std::string` 之类的 weak/COMDAT 符号)仍然会导出 —— 那是 C++ ABI 的预期行为, +不是这里要防的泄漏。 + +工程级的 `cxx_runtime = "…"`(或 `static_stdlib = false`)同样作用于共享库: +有人写下了整个工程的承诺。只有在**没人写**的时候,随格式变化的默认值才生效。 + `static_stdlib` 是旧拼写,仍然有效:`true` 等价于 `self-contained`,`false` 等价于 `host-coupled`。显式写了 `cxx_runtime` 时以后者为准。 -> **当前实现限制。** 解析器能识别 `cxx_runtime`，但当前 `[build]` 未知键白名单漏了 -> 它。因此普通构建可能输出 unsupported-key warning，`--strict` 会拒绝该 manifest。 -> 这是实现缺陷，不是另一种拼写或不同的运行时契约。 - **兑现不了的契约会被报出来,绝不静默降级。** 若工具链不带 `libc++.a`,或某个 契约在该平台上没有对应机制(MSVC 运行时的 `self-contained` 需要 `/MT`,mcpp 目前不发射),构建会打印实际退到了哪一档,而不是悄悄交付一个与 manifest 所述 diff --git a/mcpp.toml b/mcpp.toml index 731c84df..ebdebc2f 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.11.2" +version = "2026.8.11.3" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/distribution.cppm b/src/build/distribution.cppm index fa41a379..bbc3e3ee 100644 --- a/src/build/distribution.cppm +++ b/src/build/distribution.cppm @@ -45,11 +45,28 @@ export namespace mcpp::build::dist { // a musl helper needs `-static` because of PT_INTERP, a PE helper because // DLLs resolve via PATH — not about the C++ runtime this module governs. enum class Role { - Distributable, // Binary / SharedLibrary — leaves this machine + Distributable, // Binary — leaves this machine as a program Test, // TestBinary — runs here, right now, then is discarded Intermediate, // StaticLibrary — carries no runtime, contract is vacuous + // SharedLibrary — loaded INTO a process that already has a C++ runtime. + // + // Split out of `Distributable` after a .so that carried its own static + // libstdc++ exported 2931 std symbols — 777 of them GLOBAL definitions + // that exist nowhere but libstdc++.a — and the executable linking it bound + // ITS std references to them, silently defeating the executable's own + // `-static-libstdc++`. "Leaves this machine" is true of both, but the + // HAZARD is not the same, and the hazard is what the contract is for. + // + // Appended rather than inserted: `CompileFlags::ldStdlibByRole` indexes by + // the enum value, so moving the existing three would silently re-map every + // stored contract. + SharedLibrary, }; +// One past the last role. The per-role arrays size themselves from this, so +// adding a role cannot leave a stale `3` behind. +inline constexpr std::size_t kRoleCount = 4; + // ---------------------------------------------------------------- Layer 2 // What the artifact promises about the machine that runs it. This is a @@ -99,6 +116,7 @@ std::string_view to_string(Role r) { case Role::Distributable: return "distributable"; case Role::Test: return "test"; case Role::Intermediate: return "intermediate"; + case Role::SharedLibrary: return "shared-library"; } return "distributable"; } @@ -110,7 +128,20 @@ std::optional parse_contract(std::string_view s) { return std::nullopt; } -// The role -> contract policy, in one place. +// The (role x format) -> default contract policy, in one place. +// +// THIS FUNCTION IS THE SOURCE. It used to have no caller at all — `flags.cppm` +// derived the same policy a second time from the manifest — which is the exact +// shape of debt this module's opening comment was written to retire, relocated +// rather than removed. `compute_flags` now asks here. +// +// WHY FORMAT IS AN INPUT. A default is a judgement about a hazard, and the +// hazard a shared library poses is format-specific. Folding it into the +// mechanism table instead would have to spell the difference as a DEGRADATION, +// and a degradation means "mcpp promised something it could not deliver" — it +// prints a diagnostic. There is nothing broken about a self-contained .dylib; +// it is simply the right answer there. Say so in the default rather than +// apologising for it later. // // Test binaries default to SelfContained rather than the HostCoupled that // their role alone would suggest, and that is deliberate: on macOS a test @@ -120,11 +151,41 @@ std::optional parse_contract(std::string_view s) { // role model makes that trade visible instead of hard-coding it in the // emitter; a project that wants the other side of it writes // `cxx_runtime = { tests = "host-coupled" }` and now actually gets it. -Contract default_contract(Role r) { +Contract default_contract(Role r, Format f) { switch (r) { case Role::Distributable: return Contract::SelfContained; case Role::Test: return Contract::SelfContained; case Role::Intermediate: return Contract::SelfContained; + case Role::SharedLibrary: + // ELF has ONE global symbol namespace and the first definition + // loaded wins. A .so that statically embedded libstdc++ exports + // those symbols UNVERSIONED, and the linker then resolves + // the executable's own std references against that .so — because + // `-lfoo` precedes the driver's `-lstdc++`, so the archive member + // is never pulled. The executable's `-static-libstdc++` becomes a + // no-op and its C++ runtime is, in fact, whichever build of that + // .so happens to be loaded. Swap the .so for another build of the + // same SONAME and `std::runtime_error::what()` simply is not + // there. Coupling to the toolchain's libstdc++.so is the only + // spelling under which the executable keeps its own contract. + // + // The other two formats do not have that hazard, and their + // current behaviour is therefore correct and unchanged: + // + // Mach-O the self-contained mechanism already IS hiding — + // `-Wl,-load_hidden,` gives the archive's + // symbols hidden visibility precisely so dyld cannot + // unify them (PR #117). ToolchainCoupled is also a + // documented dead end there (#202): LLVM's macOS + // libc++abi/libunwind dylibs upward-link /usr/lib/libc++ + // and a second libc++ loads alongside the toolchain's. + // + // PE no global symbol namespace at all — imports resolve + // per-DLL by name, so a DLL's private CRT cannot be + // picked up by anything else. `-static` is additionally + // the standalone-DLL convention there. + return f == Format::Elf ? Contract::ToolchainCoupled + : Contract::SelfContained; } return Contract::SelfContained; } @@ -197,6 +258,37 @@ namespace detail { inline bool is_libstdcxx(std::string_view id) { return id == "libstdc++"; } inline bool is_libcxx(std::string_view id) { return id == "libc++"; } +// Keep a statically linked standard library OUT of a shared object's dynamic +// symbol table. +// +// Only a SHARED LIBRARY needs this, and only when it actually embedded the +// runtime — which after `default_contract` happens on ELF exclusively through +// an explicit `cxx_runtime = { shared = "self-contained" }`. An executable's +// static libstdc++ is already local (ld exports only what a loaded object +// references, and mcpp passes no `-rdynamic`); a .so exports every global it +// defines, which is how a pure-C compat package came to publish 777 GLOBAL +// libstdc++ definitions and become the executable's de-facto C++ runtime. +// +// It does NOT hide the weak/COMDAT template instantiations the library's own +// code emits, and must not: unifying those across the process is the intended +// C++ ABI behaviour, not a leak. +// +// The escape hatch has to stay usable, so it is guarded rather than refused. +// +// Archive BASENAMES — that is what `--exclude-libs` matches, and GNU ld and +// lld agree on it. Listed by name rather than `ALL` so a user's own static +// library linked into their .so keeps its exports. +std::string hide_static_cxx_runtime(Role role, + std::initializer_list archives) { + if (role != Role::SharedLibrary) return {}; + std::string out; + for (auto archive : archives) { + out += " -Wl,--exclude-libs,"; + out += archive; + } + return out; +} + } // namespace detail // The one table. Total by construction: every return path sets `effective`, @@ -330,11 +422,18 @@ Mechanism resolve(const MechanismInput& in) { case Format::Elf: default: { if (detail::is_libstdcxx(in.stdlibId)) { - if (m.effective == Contract::SelfContained) + if (m.effective == Contract::SelfContained) { m.unitFlags = " -static-libstdc++"; + m.unitFlags += detail::hide_static_cxx_runtime( + in.role, {"libstdc++.a"}); + } // ToolchainCoupled and HostCoupled are the same emission on ELF // (no flag); they differ in the rpath the link already carries, - // which is the documented limit of this contract. + // which is the documented limit of this contract. For a shared + // library ToolchainCoupled is the DEFAULT (see `default_contract`) + // and "no flag" is the entire mechanism: the driver links + // libstdc++.so, and the toolchain's lib directory is already an + // `-L` and an rpath entry on this line. return m; } if (detail::is_libcxx(in.stdlibId)) { @@ -355,8 +454,12 @@ Mechanism resolve(const MechanismInput& in) { // is part of the mechanism, not an optional extra. m.unitFlags = " -nostdlib++ " + in.libcxxArchive + " " + in.libcxxAbiArchive; + m.unitFlags += detail::hide_static_cxx_runtime( + in.role, {"libc++.a", "libc++abi.a"}); if (!in.libunwindArchive.empty()) { m.unitFlags += " " + in.libunwindArchive; + m.unitFlags += detail::hide_static_cxx_runtime( + in.role, {"libunwind.a"}); } else { m.degraded = true; // effective stays SelfContained: the C++ // runtime IS embedded; the unwinder is not diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 08cec2e4..3948dc16 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -35,6 +35,13 @@ struct CompileFlags { std::string as; // asm-safe subset for .S/.s via the C driver std::string nasm; // NASM global flags (.asm; own spelling) std::string ld; // ldflags string + // The LAST-RESORT run-time search path (today: the SubOS library view). + // NOT part of `ld`, and that is the whole point: `ld` is rendered BEFORE + // the per-unit flags, and the per-unit flags are where the artifact's own + // directory (`$ORIGIN`) lives. Emitted here it outranked `$ORIGIN`, so an + // artifact loaded a different build of a library than it linked against. + // It reaches the line through `link_line::UnitTail::runtimeFallback`. + std::string ldRuntimeFallback; std::filesystem::path cxxBinary; // g++ / clang++ / cl.exe std::filesystem::path ccBinary; // gcc / clang (derived; cl.exe = same) std::filesystem::path arBinary; // ar / llvm-ar / lib.exe (empty → PATH) @@ -49,9 +56,10 @@ struct CompileFlags { // roles in one build may hold different contracts, which is precisely what // `static_stdlib = false` could not express for test binaries before #336. // Produced by exactly one call to `dist::resolve` per role. - std::array ldStdlibByRole{}; + std::array ldStdlibByRole{}; // The contract each role actually got (after any degradation). - std::array contractByRole{}; + std::array contractByRole{}; // macOS + self-contained: link units need the initializer-ordering shim // object prepended to their inputs (issue #336). bool needsStreamInitShim = false; @@ -84,8 +92,13 @@ constexpr mcpp::build::dist::Role role_of(LinkUnit::Kind k) { switch (k) { case LinkUnit::TestBinary: return mcpp::build::dist::Role::Test; case LinkUnit::StaticLibrary: return mcpp::build::dist::Role::Intermediate; - case LinkUnit::Binary: - case LinkUnit::SharedLibrary: break; + // A shared library leaves this machine too, but it is LOADED INTO a + // process that already has a C++ runtime rather than being one. That + // is a different contract, not a different flavour of the same one — + // sharing `Distributable` with executables is what let a .so publish + // a whole static libstdc++ and take over the executable's runtime. + case LinkUnit::SharedLibrary: return mcpp::build::dist::Role::SharedLibrary; + case LinkUnit::Binary: break; } return mcpp::build::dist::Role::Distributable; } @@ -506,8 +519,17 @@ CompileFlags compute_flags(const BuildPlan& plan) { render_link_intent_flags(plan.linkIntent, linkIntentFlavor); // The SubOS farm tail — the only origin in `plan.runtimeSearch` with no - // other producer, appended after everything else so it is LAST in the - // artifact's DT_RPATH (see `runtime_search_closure`). + // other producer, and the one that must be LAST in the artifact's + // DT_RPATH (see `runtime_search_closure`). + // + // IT DOES NOT GO INTO `f.ld`. That was the defect: `f.ld` is rendered as + // `$ldflags`, which every link rule places BEFORE `$unit_ldflags`, and + // `$unit_ldflags` is where `$ORIGIN` lives. "Appended last" inside `f.ld` + // is still ahead of the artifact's own directory, so a project with a + // shared-library dependency resolved `libX11.so.6` out of the mutable farm + // view instead of the `bin/` directory it had just been linked against. + // It now travels as `link_line::UnitTail::runtimeFallback`, which is + // after `$ORIGIN` by construction. // // RUNPATH ONLY, never `-L`. Link-time resolution already works: mcpp // passes `--sysroot=`, which makes `/lib` the linker's @@ -524,6 +546,12 @@ CompileFlags compute_flags(const BuildPlan& plan) { "-Wl,-rpath," + dir.path.string())); } } + // Assigned HERE, not at the end: several target branches below return + // early, and every one of them is PE (where `farm_ld` is empty anyway). + // Filling the slot at its point of definition makes that a fact rather + // than something the reader has to re-derive from the return paths. + f.ldRuntimeFallback = farm_ld; + std::filesystem::path binutilsBin; if (!isMuslTc && !isMingwTc && caps.stdlib_id == "libstdc++") { auto ar = mcpp::toolchain::archive_tool(plan.toolchain); @@ -698,15 +726,48 @@ CompileFlags compute_flags(const BuildPlan& plan) { namespace dist = mcpp::build::dist; auto const& bc = plan.manifest.buildConfig; + // The output FORMAT is resolved first because the role defaults + // depend on it: what a shared library should promise is a judgement + // about a hazard, and the hazard is format-specific (see + // `dist::default_contract`). + // + // Target-keyed, not host-keyed: a Linux-hosted MinGW cross build + // produces a PE and must take the PE answer. + const dist::Format format = [&] { + if (isMingwTc) return dist::Format::Pe; + if constexpr (mcpp::platform::needs_explicit_libcxx) + return dist::Format::MachO; + else if constexpr (mcpp::platform::is_windows) + return dist::Format::Pe; + else + return dist::Format::Elf; + }(); + // `static_stdlib` is a faithful alias of the two ends of the contract: // its documented meaning has always been exactly self-contained vs the // dynamic system runtime. An explicit `cxx_runtime` wins. + // + // The role defaults come from `dist::default_contract` rather than + // being spelled again here. They were spelled again here, and that + // second derivation is why `default_contract` sat with no caller while + // this file quietly disagreed with it about shared libraries. const dist::Contract base = dist::parse_contract(bc.cxxRuntime).value_or( - bc.staticStdlib ? dist::Contract::SelfContained - : dist::Contract::HostCoupled); + bc.staticStdlib + ? dist::default_contract(dist::Role::Distributable, format) + : dist::Contract::HostCoupled); const dist::Contract testsContract = dist::parse_contract(bc.cxxRuntimeTests).value_or(base); + // A project-wide statement (`cxx_runtime = "…"` or `static_stdlib = + // false`) applies to shared libraries too — a human said what the + // whole project promises. Only when nobody said anything does the + // role's own default apply, which is the case that changes on ELF. + const bool projectWideExplicit = !bc.cxxRuntime.empty() || !bc.staticStdlib; + const dist::Contract sharedContract = + dist::parse_contract(bc.cxxRuntimeShared).value_or( + projectWideExplicit + ? base + : dist::default_contract(dist::Role::SharedLibrary, format)); // Archive lookup. LLVM lays these out either directly under lib/ (the // macOS packages) or under lib// (the Linux ones), so try @@ -756,18 +817,12 @@ CompileFlags compute_flags(const BuildPlan& plan) { mi.fullStaticLibc = (f.linkage == "static"); mi.mingw = isMingwTc; mi.macosFloor = !macosDeploymentTarget.empty(); - if constexpr (mcpp::platform::needs_explicit_libcxx) { - mi.format = dist::Format::MachO; - } else if constexpr (mcpp::platform::is_windows) { - mi.format = dist::Format::Pe; - } - // Target-keyed, not host-keyed: a Linux-hosted MinGW cross build - // produces a PE and must take the PE mechanism. - if (isMingwTc) mi.format = dist::Format::Pe; + mi.format = format; const bool wantsArchives = (base == dist::Contract::SelfContained - || testsContract == dist::Contract::SelfContained) + || testsContract == dist::Contract::SelfContained + || sharedContract == dist::Contract::SelfContained) && caps.stdlib_id == "libc++"; if (wantsArchives) { auto libcxxA = find_archive("libc++.a"); @@ -789,13 +844,28 @@ CompileFlags compute_flags(const BuildPlan& plan) { // "Explicit" = a human wrote it down. `static_stdlib = false` counts: // nobody sets a flag to its default to get non-default behavior. - const bool explicitBase = !bc.cxxRuntime.empty() || !bc.staticStdlib; - const bool explicitTests = explicitBase || !bc.cxxRuntimeTests.empty(); + const bool explicitBase = projectWideExplicit; + const bool explicitTests = explicitBase || !bc.cxxRuntimeTests.empty(); + const bool explicitShared = explicitBase || !bc.cxxRuntimeShared.empty(); + + // Report a role's degradation only if this build HAS that role. + // + // The contract is still RESOLVED for every role — `ldStdlibByRole` is + // indexed on demand and must be total. What is gated is the WARNING: + // telling a project with no shared library what its shared libraries + // promise is not information, and with four roles an unconditional + // report turns one honest warning into a wall of them. + auto role_is_built = [&](dist::Role r) { + return std::ranges::any_of(plan.linkUnits, [&](auto const& lu) { + return role_of(lu.kind) == r; + }); + }; for (auto [role, requested, wasAsked] : { - std::tuple{dist::Role::Distributable, base, explicitBase}, - std::tuple{dist::Role::Test, testsContract, explicitTests}, - std::tuple{dist::Role::Intermediate, base, explicitBase}}) { + std::tuple{dist::Role::Distributable, base, explicitBase}, + std::tuple{dist::Role::Test, testsContract, explicitTests}, + std::tuple{dist::Role::Intermediate, base, explicitBase}, + std::tuple{dist::Role::SharedLibrary, sharedContract, explicitShared}}) { mi.role = role; mi.requested = requested; mi.explicitRequest = wasAsked; @@ -804,7 +874,7 @@ CompileFlags compute_flags(const BuildPlan& plan) { f.ldStdlibByRole[i] = r.unitFlags; f.contractByRole[i] = r.effective; if (r.streamInitShim) f.needsStreamInitShim = true; - if (!r.diagnostic.empty()) + if (!r.diagnostic.empty() && role_is_built(role)) f.diagnostics.push_back(std::format( "{} target: {}", dist::to_string(role), r.diagnostic)); } @@ -993,9 +1063,9 @@ CompileFlags compute_flags(const BuildPlan& plan) { // actually being present (see atomic_link_flag). std::string atomic_ld = atomic_link_flag(plan.toolchain.linkRuntimeDirs, !full_static.empty()); - f.ld = std::format("{}{}{}{}{}{}{}{}{}{}", full_static, + f.ld = std::format("{}{}{}{}{}{}{}{}{}", full_static, link_toolchain_flags, b_flag, runtime_dirs, - link_intent_ld, farm_ld, atomic_ld, payload_ld, + link_intent_ld, atomic_ld, payload_ld, user_ldflags, link_extra); } diff --git a/src/build/link_line.cppm b/src/build/link_line.cppm new file mode 100644 index 00000000..0a10ae9a --- /dev/null +++ b/src/build/link_line.cppm @@ -0,0 +1,115 @@ +// mcpp.build.link_line — the ORDER of one link command line, as a declaration. +// +// WHY THIS MODULE EXISTS +// +// DT_RPATH is a SEARCH ORDER, and on this ecosystem two directories routinely +// hold the same SONAME: mcpp compiles `compat.x11` from source into the +// artifact's own directory while xlings has `xim:libX11` in the SubOS library +// view. The loader takes the FIRST match, so the order does not decorate the +// artifact — it SELECTS which physical file the process runs against. +// +// Before this module that order was an accident of three `+=` in two files +// that did not know about each other: +// +// flags.cppm appended the SubOS farm to the GLOBAL ldflags, +// with a comment saying "so it is LAST" +// plan.cppm appended `$ORIGIN` to the PER-UNIT flags +// ninja_backend rendered `$cxx $in -o $out $ldflags $unit_ldflags` +// +// Each was locally right and the composition was wrong: the farm landed BEFORE +// `$ORIGIN`, so a GLFW/imgui application linked against the libX11 mcpp had +// just built and then LOADED the one xlings had installed. They are not +// interchangeable, and the program died before main with +// `undefined symbol: _ZNKSt13runtime_error4whatEv`. +// +// The fix is not "append it somewhere else" — that would leave the next caller +// free to make the same mistake. It is to give the line NAMED SLOTS whose +// relative order is written down once, asserted by a unit test, and impossible +// to bypass: a new producer must choose a slot, and choosing is where the +// question "before or after the artifact's own directory?" gets asked. +// +// FORMAT NEUTRALITY. Slots are named by ROLE, never by flag spelling, so no +// consumer needs a platform branch. PE leaves `runtimeFallback` and +// `loaderTag` empty because it has neither; Mach-O puts `@loader_path` in +// `dependencies` where ELF puts `$ORIGIN`. The spellings stay with the +// producers that already know the target format (`plan.cppm` for dependency +// rpath, `distribution.cppm` for the C++ runtime mechanism). +// +// Analysis: .agents/docs/2026-08-11-runtime-search-origin-precedence-analysis.md + +export module mcpp.build.link_line; + +import std; + +export namespace mcpp::build::link_line { + +// The per-unit tail of a link command, in emission order. +// +// Everything here follows the GLOBAL flags (toolchain payload `-L`/`-rpath`, +// `--sysroot`, `-B`, `-specs`) and, for a shared library, the soname flag. +// Those are properties of the toolchain and are the same for every unit; this +// struct is what differs per link unit, which is also why the C++ runtime +// mechanism lives here (two roles in one build may hold different contracts). +struct UnitTail { + // 1. What this unit links AGAINST: dependency `-L`/`-l`, plus the + // artifact-relative run-time search path that finds those dependencies + // again after the build directory moves (`$ORIGIN` on ELF, + // `@loader_path` on Mach-O, nothing on PE — DLLs resolve via the + // executable's directory and PATH). + // + // FIRST, and this is the load-bearing decision of the whole module: + // these directories hold the EXACT files this link resolved against. + // Anything that can supply the same SONAME must come after them, or the + // artifact runs against a different build than it was linked against. + std::string dependencies; + + // 2. The C++ runtime mechanism for this unit's role — `-static-libstdc++`, + // Mach-O's `-load_hidden` archives, MinGW's `-static`, and the + // `--exclude-libs` guard that keeps a statically embedded standard + // library out of a shared object's dynamic symbol table. + // + // After (1) so that a dependency's own definition is found first: an + // archive member is pulled only for symbols still undefined at the + // point the archive is processed, which is the ordering C++ drivers + // have always assumed. + std::string cxxRuntime; + + // 3. LAST-RESORT run-time search — today the SubOS library view ("the + // farm"), a flat symlink tree that `xlings install` rewrites. + // + // It may only supply what nothing else does. It must therefore follow + // BOTH the global payload directories AND slot (1): a mutable view that + // outranks either of them lets a later install silently change which + // library an ALREADY LINKED artifact loads. That is not hypothetical — + // it is the defect this module was created for. + std::string runtimeFallback; + + // 4. The loader tag (`--disable-new-dtags` / `--enable-new-dtags`). + // LITERALLY last, because ld honours the LAST one it sees and both gcc + // specs and clang config files supply the opposite of what mcpp wants. + // Nothing may be appended after this slot. + std::string loaderTag; + + // Slots are concatenated in declaration order. A non-empty slot that does + // not already begin with a separator gets one, so a producer cannot break + // the line by forgetting the leading space (every producer today supplies + // it, and the result is byte-identical to the hand-rolled concatenation + // this replaced). + std::string render() const { + std::string out; + for (const std::string* slot : + {&dependencies, &cxxRuntime, &runtimeFallback, &loaderTag}) { + if (slot->empty()) continue; + if (!slot->starts_with(' ')) out += ' '; + out += *slot; + } + return out; + } + + bool empty() const { + return dependencies.empty() && cxxRuntime.empty() + && runtimeFallback.empty() && loaderTag.empty(); + } +}; + +} // namespace mcpp::build::link_line diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index e668ca2e..067f2311 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -23,6 +23,7 @@ import mcpp.manifest; import mcpp.source_kind; import mcpp.build.distribution; import mcpp.build.graph_shape; +import mcpp.build.link_line; import mcpp.build.loader_contract; import mcpp.build.plan; import mcpp.build.flags; @@ -1404,16 +1405,23 @@ std::string emit_ninja_string(const BuildPlan& plan) { // the contract table's business, not this emitter's (#336 — // before, this switch WAS the policy, and `static_stdlib = false` // could not reach the test side of it). - std::string unit = join_flags(lu.linkFlags); - unit += flags.ldStdlibFor(role_of(lu.kind)); - // LAST, after every other linker argument, because the loader tag - // is decided by the last `--enable-new-dtags`/`--disable-new-dtags` - // ld sees — and both gcc specs and clang config files supply the - // former. `$unit_ldflags` is itself the final expansion in every - // link rule above, so "last here" is "last on the line". - if (!lu.loaderTagFlag.empty()) - unit += " " + lu.loaderTagFlag; - if (!unit.empty()) + // + // The slots and their ORDER belong to `link_line::UnitTail` — see + // that module for why each one is where it is. This emitter says + // WHAT goes in each slot and never WHERE it lands; deciding that + // here by hand is exactly how the SubOS farm ended up ahead of + // `$ORIGIN` in DT_RPATH and an artifact loaded a different build + // of libX11 than it was linked against. + mcpp::build::link_line::UnitTail tail; + tail.dependencies = join_flags(lu.linkFlags); + tail.cxxRuntime = flags.ldStdlibFor(role_of(lu.kind)); + // An archive has no run-time search path of its own: `ar` never + // reads `$unit_ldflags`, so rpath flags there would be dead bytes + // in every graph that builds a static library. + if (lu.kind != LinkUnit::StaticLibrary) + tail.runtimeFallback = flags.ldRuntimeFallback; + tail.loaderTag = lu.loaderTagFlag; + if (auto unit = tail.render(); !unit.empty()) out_line += " unit_ldflags =" + unit + "\n"; } append(std::move(out_line)); diff --git a/src/manifest/toml.cppm b/src/manifest/toml.cppm index c421f8cf..17c0bac9 100644 --- a/src/manifest/toml.cppm +++ b/src/manifest/toml.cppm @@ -931,8 +931,9 @@ std::expected parse_string(std::string_view content, // [build] — backend tunables if (auto v = doc->get_bool("build.static_stdlib")) m.buildConfig.staticStdlib = *v; // #336 — [build] cxx_runtime. Two spellings for one field, cargo-style: - // cxx_runtime = "host-coupled" (all roles) - // cxx_runtime = { default = "...", tests = "..." } (per role) + // cxx_runtime = "host-coupled" (all roles) + // cxx_runtime = { default = "...", tests = "...", shared = "..." } + // (per role) // Rejecting an unknown value here is what lets flags.cppm parse it later // with `value_or` and no second validation path. { @@ -953,18 +954,19 @@ std::expected parse_string(std::string_view content, m.buildConfig.cxxRuntime = s; } else if (val->is_table()) { for (auto& [key, v] : val->as_table()) { - if (key != "default" && key != "tests") + if (key != "default" && key != "tests" && key != "shared") return std::unexpected(error(origin, std::format( "[build].cxx_runtime has unsupported key '{}'; " - "expected 'default' or 'tests'", key))); + "expected 'default', 'tests' or 'shared'", key))); if (!v.is_string()) return std::unexpected(error(origin, std::format( "[build].cxx_runtime.{} must be a string", key))); auto s = v.as_string(); if (auto e = check(std::format("[build].cxx_runtime.{}", key), s)) return std::unexpected(*e); - (key == "tests" ? m.buildConfig.cxxRuntimeTests - : m.buildConfig.cxxRuntime) = s; + if (key == "tests") m.buildConfig.cxxRuntimeTests = s; + else if (key == "shared") m.buildConfig.cxxRuntimeShared = s; + else m.buildConfig.cxxRuntime = s; } } else { return std::unexpected(error(origin, @@ -1070,10 +1072,10 @@ std::expected parse_string(std::string_view content, // MUST stay in sync with the `doc->get_*("build.")` reads above. static constexpr std::string_view kKnownBuildKeys[] = { "allow_host_libs", "build_program_timeout", "c_standard", "cache", - "cflags", "cxxflags", "default-profile", "defines", "dialect_cxxflags", - "flags", "include_dirs", "include_dirs_after", "ldflags", - "macos_deployment_target", "module_extensions", "profile", "sources", - "static_stdlib", "target", + "cflags", "cxxflags", "cxx_runtime", "default-profile", "defines", + "dialect_cxxflags", "flags", "include_dirs", "include_dirs_after", + "ldflags", "macos_deployment_target", "module_extensions", "profile", + "sources", "static_stdlib", "target", }; if (auto* bt = doc->get_table("build")) { for (auto& [key, _] : *bt) { @@ -1085,8 +1087,8 @@ std::expected parse_string(std::string_view content, "sources, module_extensions, cflags, cxxflags, ldflags, " "defines, flags, include_dirs, include_dirs_after, " "dialect_cxxflags, c_standard, target, static_stdlib, " - "allow_host_libs, cache, profile, build_program_timeout, " - "macos_deployment_target.", key)); + "cxx_runtime, allow_host_libs, cache, profile, " + "build_program_timeout, macos_deployment_target.", key)); } } } diff --git a/src/manifest/types.cppm b/src/manifest/types.cppm index 2f7811a0..4cda3649 100644 --- a/src/manifest/types.cppm +++ b/src/manifest/types.cppm @@ -424,6 +424,13 @@ struct BuildConfig : BuildInputs { // leave the build machine, so "link the host's runtime" is a defensible // choice there and an indefensible one for a shipped artifact. std::string cxxRuntimeTests; + // Per-role override for shared libraries. Empty = the role default, which + // on ELF is toolchain-coupled (see `dist::default_contract`): a .so that + // embeds its own libstdc++ exports it into the process's single global + // symbol namespace and becomes the executable's C++ runtime by accident. + // Setting this to "self-contained" is supported and additionally emits + // `--exclude-libs` so the escape hatch cannot re-open that. + std::string cxxRuntimeShared; // "" (default = dynamic), "static", "dynamic" — chosen at resolve // time from --static / --target / [target.].linkage. Wired // through to ninja backend as the `-static` link flag. diff --git a/src/version.cppm b/src/version.cppm index 9662249c..cf19c7a8 100644 --- a/src/version.cppm +++ b/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.8.11.2"; +inline constexpr std::string_view MCPP_VERSION = "2026.8.11.3"; } // namespace mcpp diff --git a/tests/e2e/219_runtime_search_farm_is_last.sh b/tests/e2e/219_runtime_search_farm_is_last.sh index a6c1045d..31aaff72 100755 --- a/tests/e2e/219_runtime_search_farm_is_last.sh +++ b/tests/e2e/219_runtime_search_farm_is_last.sh @@ -13,14 +13,21 @@ # # THE INVARIANT # -# payload directories first, the SubOS farm LAST +# payload directories first, the artifact's own directory next, +# the SubOS farm LAST — literally last, `$ORIGIN` included # # and it is about mutability, not taste. `/lib` is a symlink view -# rewritten by every `xlings install`; a payload directory is written once. -# Payload-first keeps libc / libm / libstdc++ resolving from the pinned payload -# and leaves the farm to supply only what nothing else does. Farm-first would -# let a later install silently change which libc an ALREADY LINKED artifact -# loads — a failure that appears long after the build that caused it. +# rewritten by every `xlings install`; a payload directory is written once and +# the artifact's own directory holds the exact files this link resolved +# against. Payload-first keeps libc / libm / libstdc++ resolving from the +# pinned payload; `$ORIGIN` before the farm keeps the artifact running against +# what it was built with; the farm supplies only what nothing else does. +# +# Farm-first lets a later install silently change which library an ALREADY +# LINKED artifact loads — a failure that appears long after the build that +# caused it, and one that actually shipped: with the farm ahead of `$ORIGIN` a +# GLFW/imgui application linked against the libX11 mcpp had just built from +# `compat.x11` sources and then LOADED the `xim:libX11` xlings had installed. # # WHY THIS ASSERTS THE ARTIFACT AND THE RECORD # @@ -33,15 +40,50 @@ set -e TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT -mkdir -p "$TMP/proj/src" +# THE PROJECT CONSUMES A SHARED LIBRARY FROM A DEPENDENCY, AND THAT IS THE POINT. +# +# A bare `int main()` produces an executable with NO `$ORIGIN` in its DT_RPATH, +# so the ordering this test exists to check is not even present — the earlier +# version of this test passed on a binary that could not exhibit the bug. +# +# A shared TARGET in the same package is not enough either: mcpp links that +# package's module objects into the executable directly, so there is still no +# `-l` and no `$ORIGIN`. It takes a DEPENDENCY that ships a shared library — +# which is exactly the shape of the artifact that broke (an application whose +# `compat.x11` dependency builds `libX11.so` into the artifact directory). +mkdir -p "$TMP/greetdep/src" "$TMP/proj/src" +cat > "$TMP/greetdep/mcpp.toml" <<'EOF' +[package] +name = "greetdep" +version = "0.1.0" + +[targets.greetdep] +kind = "shared" +EOF +# Interface and implementation are split so the call is a real cross-library +# reference: an inline definition in the interface would be emitted into the +# consumer and the dependency edge would vanish. +cat > "$TMP/greetdep/src/greetdep.cppm" <<'EOF' +export module greetdep; +export int greet_value(); +EOF +cat > "$TMP/greetdep/src/greetdep.cpp" <<'EOF' +module greetdep; +int greet_value() { return 7; } +EOF + cd "$TMP/proj" cat > mcpp.toml <<'EOF' [package] name = "closure" version = "0.1.0" + +[dependencies.greetdep] +path = "../greetdep" EOF cat > src/main.cpp <<'EOF' -int main() { return 0; } +import greetdep; +int main() { return greet_value() == 7 ? 0 : 1; } EOF "$MCPP" build > build.log 2>&1 || { cat build.log; exit 1; } @@ -148,27 +190,54 @@ PY echo "DT_RPATH: $DT_RPATH" [[ -n "$DT_RPATH" ]] || { echo "FAIL: executable carries no DT_RPATH"; exit 1; } -# The farm must be the last ABSOLUTE entry — not literally the last entry. -# -# `$ORIGIN`-relative entries are a different kind: they address the artifact's -# own directory, not this machine, so they travel with it and their position -# says nothing about which machine-local directory wins. A project with a shared -# library dependency gets one appended after everything else, and an assertion -# of "literally last" would fail on every such project while the invariant it -# meant to check still held. (Measured on a real GLFW app, whose DT_RPATH ends -# `… : /lib : $ORIGIN`.) -RPATH_LAST_ABS="$(python3 -c " -p = [x for x in '''$DT_RPATH'''.split(':') if x.startswith('/')] -print(p[-1] if p else '') +# LITERALLY last, `$ORIGIN` included. +# +# An earlier version of this test filtered `$ORIGIN` out and asserted "the last +# ABSOLUTE entry", on the reasoning that an artifact-relative entry travels with +# the artifact and so "says nothing about which machine-local directory wins". +# That reasoning is backwards, and the filtered assertion is blind to the exact +# defect it was named after: with `… : : $ORIGIN` and with +# `… : $ORIGIN : ` the list of absolute entries is IDENTICAL, so it passed +# on both. It reported "farm is last" on a binary whose farm was not last. +# +# What decides the winner is a SONAME present in two directories, and that is +# routine here: mcpp builds `compat.x11` from source into the artifact's own +# directory while xlings has `xim:libX11` in the farm. Farm-first meant an +# application linked against one libX11 and loaded the other, dying before main +# with `undefined symbol: _ZNKSt13runtime_error4whatEv`. +RPATH_LAST="$(python3 -c " +print('''$DT_RPATH'''.split(':')[-1]) ")" -[[ "$RPATH_LAST_ABS" == "$FARM" ]] || { - echo "FAIL: the farm is not the last absolute entry of DT_RPATH" - echo " recorded farm: $FARM" - echo " last absolute entry: $RPATH_LAST_ABS" - echo " full: $DT_RPATH" +[[ "$RPATH_LAST" == "$FARM" ]] || { + echo "FAIL: the farm is not the last entry of DT_RPATH" + echo " recorded farm: $FARM" + echo " last entry: $RPATH_LAST" + echo " full: $DT_RPATH" exit 1 } +# ── invariant 2b: the artifact's own directory is ON the path, and ahead ───── +# +# Both halves are load-bearing. Without `$ORIGIN` the project under test cannot +# exhibit the bug and every other assertion here is vacuous; with `$ORIGIN` +# behind the farm, a mutable view outranks the exact files this link resolved +# against. +case ":$DT_RPATH:" in + *':$ORIGIN:'*) ;; + *) echo "FAIL: no \$ORIGIN in DT_RPATH — this project cannot exercise the" + echo " ordering it is meant to check. full: $DT_RPATH" + exit 1 ;; +esac +python3 - < p.index('''$FARM'''): + print("FAIL: the SubOS farm outranks \$ORIGIN, so this artifact can load a") + print(" different build of a library than it was linked against.") + print(" full: " + ':'.join(p)) + sys.exit(1) +PY + # ── invariant 3: libc still comes from the payload, not the farm ──────────── # # The point of the ordering. Both directories can hold a libc.so.6 (the farm's @@ -197,4 +266,46 @@ FIRST_POS=0 exit 1 } -echo "PASS: search closure is payload-first / farm-last, and the artifact agrees" +# ── invariant 4: the LOADER agrees, measured rather than inferred ─────────── +# +# Everything above reads a data structure. This runs the program and watches +# the dynamic linker walk the path, because the shape of DT_RPATH is a proxy +# and the behaviour is the thing: which physical file does the process open? +# +# THIS ASSERTION MUST NOT DEPEND ON A CRASH. The defect it guards produced a +# spectacular one (`undefined symbol: _ZNKSt13runtime_error4whatEv`), but that +# symptom exists only while shared libraries statically embed libstdc++. Once +# they stop, farm-first degrades from a crash to a SILENT version mismatch — +# the artifact quietly running a different build than it linked against — and +# an assertion written against the crash would go green for the wrong reason. +# So it asserts the search itself. +# +# `libgreetdep.so` lives in the artifact's directory and nowhere else, so if +# `$ORIGIN` is consulted first the farm is never tried for it at all. A farm +# path appearing in this trace means the farm was consulted FIRST. +LD_DEBUG=libs "$BIN" > "$TMP/run.log" 2> "$TMP/ld.log" || { + echo "FAIL: the built executable did not run"; tail -20 "$TMP/ld.log"; exit 1; +} +SONAME="$(ls "$(dirname "$BIN")" | grep -E '^libgreetdep\.so' | head -1)" +[[ -n "$SONAME" ]] || { + echo "FAIL: no shared library was produced, so invariant 4 is vacuous" + ls -la "$(dirname "$BIN")"; exit 1; +} +awk -v soname="$SONAME" -v farm="$FARM" ' + index($0, "find library=" soname) { inblock = 1; next } + inblock && /find library=/ { inblock = 0 } + inblock && index($0, "trying file=") { + if (index($0, farm "/")) { print; found = 1 } + } + END { exit found ? 1 : 0 } +' "$TMP/ld.log" || { + echo "FAIL: the loader consulted the SubOS farm for $SONAME before the" + echo " artifact's own directory. A mutable view is being searched" + echo " ahead of the exact files this artifact was linked against." + echo " farm: $FARM" + grep -A8 "find library=$SONAME" "$TMP/ld.log" | head -20 + exit 1 +} + +echo "PASS: search closure is payload-first / farm-last, the artifact agrees," +echo " and the loader resolves the artifact's own library from \$ORIGIN" diff --git a/tests/e2e/222_shared_library_cxx_runtime_contract.sh b/tests/e2e/222_shared_library_cxx_runtime_contract.sh new file mode 100755 index 00000000..24ffb499 --- /dev/null +++ b/tests/e2e/222_shared_library_cxx_runtime_contract.sh @@ -0,0 +1,305 @@ +#!/usr/bin/env bash +# requires: gcc elf python3 +# 222_shared_library_cxx_runtime_contract.sh — a shared library must not become +# the executable's C++ runtime. +# +# WHAT WENT WRONG +# +# `LinkUnit::SharedLibrary` shared the `Distributable` role with executables, so +# a .so got the same self-contained contract: `-static-libstdc++`. On ELF that +# is not a private copy. There is ONE global symbol namespace, a shared object +# exports every global it defines, and the embedded libstdc++ went into its +# dynamic symbol table UNVERSIONED — 777 GLOBAL std definitions (plus 2154 weak +# ones) out of a pure-C compat package. `libXau.so` was 9.5MB: 39KB of Xau and +# the rest libstdc++. +# +# The executable then linked `-lX11` BEFORE the driver's `-lstdc++`, so ld +# resolved its own `std::runtime_error::what()` against the .so and never +# pulled the archive member. Its `-static-libstdc++` became a no-op and its C++ +# runtime was, in fact, that .so. Swap the .so for another build of the same +# SONAME — which is exactly what a farm-first DT_RPATH did — and the program +# dies before main: +# +# undefined symbol: _ZNKSt13runtime_error4whatEv +# +# WHY THIS IS A SEPARATE TEST FROM 219 +# +# 219 asserts the SEARCH ORDER. This asserts that a wrong search order can no +# longer be fatal, and that the executable keeps the contract it was promised. +# They fail independently and neither subsumes the other. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +# ── ELF readers: no binutils ──────────────────────────────────────────────── +# A sandbox home does not reliably have binutils, and on at least one real +# machine its `readelf` shim pointed into a deleted directory. +cat > "$TMP/elf.py" <<'PY' +import struct, sys + +def load(path): + d = open(path, 'rb').read() + if d[:4] != b'\x7fELF' or d[4] != 2: + raise SystemExit("not an ELF64 file: " + path) + return d + +def sections(d): + # Elf64_Shdr: name(4) type(4) flags(8) addr(8) offset(8) size(8) link(4) … + shoff, = struct.unpack_from('> 4)) + if not found: + raise SystemExit( + "no .dynsym in " + path + " — this object was stripped of section " + "headers, so a symbol count here would be zero for the wrong reason") + return out + +def dynsym_defined(path): return dynsyms(path, True) +def dynsym_undefined(path): return dynsyms(path, False) + +def needed(path): + d = load(path) + phoff, = struct.unpack_from(' mcpp.toml <<'EOF' +[package] +name = "sharedrt" +version = "0.1.0" + +[targets.sharedrtlib] +kind = "shared" + +[targets.sharedrt] +kind = "bin" +main = "src/main.cpp" +EOF +# The library genuinely uses the standard library, and the executable +# genuinely throws — `std::runtime_error::what()` is the exact symbol whose +# disappearance produced the original crash. +cat > src/lib.cppm <<'EOF' +export module sharedrt.lib; +import std; +export std::string sharedrt_greet() { + try { throw std::runtime_error("hello"); } + catch (const std::exception& e) { return std::string(e.what()); } +} +EOF +cat > src/main.cpp <<'EOF' +import std; +import sharedrt.lib; +int main() { return sharedrt_greet() == "hello" ? 0 : 1; } +EOF + +"$MCPP" build > build.log 2>&1 || { cat build.log; exit 1; } + +BIN="$(ls target/*/*/bin/sharedrt 2>/dev/null | head -1)" +LIB="$(ls target/*/*/bin/libsharedrtlib.so 2>/dev/null | head -1)" +[[ -n "$BIN" ]] || { echo "FAIL: no executable"; ls -R target | head -40; exit 1; } +[[ -n "$LIB" ]] || { echo "FAIL: no shared library"; ls -R target | head -40; exit 1; } + +# ── invariant 1: the shared library exports no libstdc++ RUNTIME symbols ──── +# +# Zero GLOBAL std definitions, not "few". One is enough to satisfy an +# executable's reference and take over its runtime — that is not a matter of +# degree. (Weak/COMDAT template instantiations are excluded and must be: a +# library using std::string emits ~30 of them and unifying those across the +# process is the intended C++ ABI behaviour. See `std-exports` in elf.py.) +EXPORTED="$(elf std-exports "$LIB")" +[[ "$EXPORTED" == "0" ]] || { + echo "FAIL: the shared library exports $EXPORTED GLOBAL standard-library" + echo " symbols. Those come from libstdc++.a and nowhere else, so an" + echo " executable linking this library will bind ITS std references" + echo " here and silently lose its own C++ runtime contract." + exit 1 +} +[[ "$(elf defines "$LIB" "$CRASH_SYM")" == "no" ]] || { + echo "FAIL: the shared library exports $CRASH_SYM — the exact symbol whose" + echo " disappearance killed the artifact this test exists for." + exit 1 +} + +# ── invariant 2: …because it COUPLES to the runtime instead of embedding it ─ +# +# Asserted separately from invariant 1 on purpose. "Exports nothing" would also +# be true of a library that embedded a HIDDEN copy, and that is a different +# artifact with a different failure mode (two std runtimes in one process). +# What the default promises on ELF is one runtime, shared. +elf needed "$LIB" | grep -qx 'libstdc++.so.6' || { + echo "FAIL: the shared library declares no dependency on libstdc++.so.6, so" + echo " it is carrying a private C++ runtime after all. NEEDED was:" + elf needed "$LIB" + exit 1 +} + +# ── invariant 3: the EXECUTABLE keeps its own contract ────────────────────── +# +# This is the payoff, and the exact shape of the original crash: with the .so +# exporting std, the executable's `-static-libstdc++` silently became a no-op +# and it carried an UNDEFINED `_ZNKSt13runtime_error4whatEv` that only the .so +# could satisfy. +UNDEF="$(elf std-undefined "$BIN")" +[[ "$UNDEF" == "0" ]] || { + echo "FAIL: the executable has $UNDEF undefined standard-library symbols." + echo " Its self-contained C++ runtime is being supplied by something else." + exit 1 +} + +"$BIN" || { echo "FAIL: the executable did not run"; exit 1; } + +# ── invariant 4: the escape hatch works AND stays guarded ─────────────────── +# +# `cxx_runtime = { shared = "self-contained" }` is legitimate — a .so that +# ships alone needs it. What must not come back is the export. `--exclude-libs` +# is what keeps the hatch from re-opening the defect. +cat > mcpp.toml <<'EOF' +[package] +name = "sharedrt" +version = "0.1.0" + +[build] +cxx_runtime = { shared = "self-contained" } + +[targets.sharedrtlib] +kind = "shared" + +[targets.sharedrt] +kind = "bin" +main = "src/main.cpp" +EOF +rm -rf target +"$MCPP" build > build2.log 2>&1 || { cat build2.log; exit 1; } +LIB="$(ls target/*/*/bin/libsharedrtlib.so 2>/dev/null | head -1)" +[[ -n "$LIB" ]] || { echo "FAIL: no shared library on the second build"; exit 1; } + +elf needed "$LIB" | grep -qx 'libstdc++.so.6' && { + echo "FAIL: cxx_runtime = { shared = \"self-contained\" } was ignored —" + echo " the library still couples to libstdc++.so.6." + exit 1 +} +EXPORTED="$(elf std-exports "$LIB")" +[[ "$EXPORTED" == "0" ]] || { + echo "FAIL: an explicitly self-contained shared library exported $EXPORTED" + echo " GLOBAL standard-library symbols. --exclude-libs is missing, so" + echo " the escape hatch re-opens the defect it was allowed to work" + echo " around." + exit 1 +} +[[ "$(elf defines "$LIB" "$CRASH_SYM")" == "no" ]] || { + echo "FAIL: the embedded runtime is exported — $CRASH_SYM is visible from" + echo " an explicitly self-contained shared library." + exit 1 +} + +echo "PASS: a shared library couples to the C++ runtime instead of exporting it," +echo " the executable keeps its own contract, and the opt-out stays hidden" diff --git a/tests/unit/test_distribution.cpp b/tests/unit/test_distribution.cpp index 34641182..f9f08b6d 100644 --- a/tests/unit/test_distribution.cpp +++ b/tests/unit/test_distribution.cpp @@ -61,14 +61,16 @@ TEST(Distribution, HostCoupledReachesTestBinariesOnMacos) { // re-open #202 (system libc++ dylib against toolchain libc++ headers → // undefined __hash_memory on libc++ 22), so it is asserted, not assumed. TEST(Distribution, TestsDefaultToSelfContained) { - EXPECT_EQ(dist::default_contract(dist::Role::Test), - dist::Contract::SelfContained); - EXPECT_EQ(dist::default_contract(dist::Role::Distributable), - dist::Contract::SelfContained); + for (auto fmt : {dist::Format::Elf, dist::Format::MachO, dist::Format::Pe}) { + EXPECT_EQ(dist::default_contract(dist::Role::Test, fmt), + dist::Contract::SelfContained); + EXPECT_EQ(dist::default_contract(dist::Role::Distributable, fmt), + dist::Contract::SelfContained); + } auto in = macos_input(); in.role = dist::Role::Test; - in.requested = dist::default_contract(dist::Role::Test); + in.requested = dist::default_contract(dist::Role::Test, dist::Format::MachO); auto m = dist::resolve(in); EXPECT_NE(m.unitFlags.find("-load_hidden"), std::string::npos); EXPECT_TRUE(m.streamInitShim); @@ -332,3 +334,116 @@ TEST(Distribution, ShimIsNotGeneratedWhenTheSymbolIsAbsent) { EXPECT_EQ(m.effective, dist::Contract::SelfContained); EXPECT_NE(m.unitFlags.find("-load_hidden"), std::string::npos); } + +// --------------------------------------------------------------------------- +// Shared libraries. +// +// A .so is not a small executable: it is loaded INTO a process that already +// has a C++ runtime. On ELF that matters because there is ONE global symbol +// namespace and the first definition loaded wins — a .so that statically +// embedded libstdc++ exports ~3000 std symbols unversioned, the linker +// resolves the EXECUTABLE's std references against it (`-lfoo` precedes the +// driver's `-lstdc++`, so the archive member is never pulled), and the +// executable's own `-static-libstdc++` becomes a no-op. Swap that .so for +// another build of the same SONAME and `std::runtime_error::what()` is gone. +// +// The whole table is asserted cell by cell rather than "the ELF case", because +// the reason the other two formats keep the old answer is a real argument +// about each of them and a regression there would be silent. +TEST(Distribution, SharedLibraryDefaultIsFormatSpecific) { + EXPECT_EQ(dist::default_contract(dist::Role::SharedLibrary, dist::Format::Elf), + dist::Contract::ToolchainCoupled); + // Mach-O: self-contained ALREADY means hidden (-load_hidden), so dyld + // cannot unify the symbols; and toolchain-coupled is a documented dead end + // there (#202). PE: no global namespace at all, imports resolve per-DLL. + EXPECT_EQ(dist::default_contract(dist::Role::SharedLibrary, dist::Format::MachO), + dist::Contract::SelfContained); + EXPECT_EQ(dist::default_contract(dist::Role::SharedLibrary, dist::Format::Pe), + dist::Contract::SelfContained); +} + +// The ELF mechanism for the new default: nothing. The driver links +// libstdc++.so and the toolchain's lib directory is already an -L and an rpath +// entry on the line. "No flag" has to be asserted or a future edit that adds +// one back would look like an improvement. +TEST(Distribution, SharedLibraryOnElfEmbedsNothing) { + auto in = linux_gcc_input(); + in.role = dist::Role::SharedLibrary; + in.requested = dist::default_contract(dist::Role::SharedLibrary, dist::Format::Elf); + auto m = dist::resolve(in); + EXPECT_EQ(m.effective, dist::Contract::ToolchainCoupled); + EXPECT_EQ(m.unitFlags, ""); + EXPECT_FALSE(m.degraded); + EXPECT_TRUE(m.diagnostic.empty()); +} + +// The escape hatch stays usable — and is guarded. Asking for a self-contained +// .so is legitimate; letting it export the embedded runtime is not. +TEST(Distribution, ExplicitSelfContainedSharedLibraryHidesTheEmbeddedRuntime) { + auto in = linux_gcc_input(); + in.role = dist::Role::SharedLibrary; + in.requested = dist::Contract::SelfContained; + in.explicitRequest = true; + auto m = dist::resolve(in); + EXPECT_EQ(m.effective, dist::Contract::SelfContained); + EXPECT_NE(m.unitFlags.find("-static-libstdc++"), std::string::npos) << m.unitFlags; + EXPECT_NE(m.unitFlags.find("-Wl,--exclude-libs,libstdc++.a"), std::string::npos) + << m.unitFlags; + // No diagnostic: this is honored exactly as asked, not degraded. + EXPECT_FALSE(m.degraded); +} + +// The guard is for shared libraries only. An executable's static libstdc++ is +// already local (ld exports only what a loaded object references and mcpp +// passes no -rdynamic), so hiding it would be noise — and `--exclude-libs` on +// an executable link is a flag whose absence is part of the contract. +TEST(Distribution, ExecutablesDoNotGetTheExcludeLibsGuard) { + auto in = linux_gcc_input(); + in.role = dist::Role::Distributable; + in.requested = dist::Contract::SelfContained; + auto m = dist::resolve(in); + EXPECT_EQ(m.unitFlags, " -static-libstdc++"); +} + +// Same rule on the libc++/ELF path: the archives are linked by PATH, but +// --exclude-libs matches the archive BASENAME, so the names are spelled out. +TEST(Distribution, LibcxxSelfContainedSharedLibraryHidesItsArchives) { + dist::MechanismInput in; + in.format = dist::Format::Elf; + in.stdlibId = "libc++"; + in.role = dist::Role::SharedLibrary; + in.requested = dist::Contract::SelfContained; + in.explicitRequest = true; + in.libcxxArchive = "/tc/lib/libc++.a"; + in.libcxxAbiArchive = "/tc/lib/libc++abi.a"; + in.libunwindArchive = "/tc/lib/libunwind.a"; + auto m = dist::resolve(in); + for (auto needle : {"-Wl,--exclude-libs,libc++.a", + "-Wl,--exclude-libs,libc++abi.a", + "-Wl,--exclude-libs,libunwind.a"}) + EXPECT_NE(m.unitFlags.find(needle), std::string::npos) << needle + << " / " << m.unitFlags; +} + +// An archive embeds no runtime, so the role returns before any mechanism runs. +// Adding a fourth role must not have perturbed that early exit. +TEST(Distribution, IntermediateStillCarriesNoMechanism) { + auto in = linux_gcc_input(); + in.role = dist::Role::Intermediate; + in.requested = dist::Contract::SelfContained; + auto m = dist::resolve(in); + EXPECT_EQ(m.unitFlags, ""); + EXPECT_FALSE(m.degraded); +} + +// `ldStdlibByRole` indexes by the enum value. If a role is ever inserted +// rather than appended, every stored contract silently re-maps. +TEST(Distribution, RoleCountCoversEveryRole) { + EXPECT_EQ(static_cast(dist::Role::SharedLibrary) + 1, + dist::kRoleCount); + for (auto r : {dist::Role::Distributable, dist::Role::Test, + dist::Role::Intermediate, dist::Role::SharedLibrary}) { + EXPECT_LT(static_cast(r), dist::kRoleCount); + EXPECT_FALSE(dist::to_string(r).empty()); + } +} diff --git a/tests/unit/test_link_line.cpp b/tests/unit/test_link_line.cpp new file mode 100644 index 00000000..5a8d2def --- /dev/null +++ b/tests/unit/test_link_line.cpp @@ -0,0 +1,107 @@ +// The link-line slot order. +// +// This file exists because the order it asserts was, until now, an emergent +// property of three `+=` in two files — and the composition was wrong in a way +// every individual site looked right. A GLFW application linked against the +// libX11 mcpp had just built and then LOADED the one xlings had installed, +// because the SubOS farm reached DT_RPATH ahead of `$ORIGIN`. +// +// So these assertions are about the ORDER itself, not about "the flags look +// roughly right". Each one names the failure it prevents. + +#include + +import std; +import mcpp.build.link_line; + +namespace ll = mcpp::build::link_line; + +namespace { + +// Where does `needle` start in `haystack`? npos-safe ordering helper: gtest's +// output for a raw `EXPECT_LT(a.find(x), a.find(y))` on an absent needle is +// two huge numbers and no clue which one was missing. +std::size_t pos_of(std::string_view haystack, std::string_view needle) { + auto p = haystack.find(needle); + EXPECT_NE(p, std::string_view::npos) << "missing '" << needle << "' in: " << haystack; + return p; +} + +ll::UnitTail full_tail() { + ll::UnitTail t; + t.dependencies = " -Lbin -Wl,-rpath,'$ORIGIN' -lX11"; + t.cxxRuntime = " -static-libstdc++"; + t.runtimeFallback = " -Wl,-rpath,/home/u/.mcpp/registry/subos/default/lib"; + t.loaderTag = "-Wl,--disable-new-dtags"; + return t; +} + +} // namespace + +// THE invariant. The artifact's own directory holds the exact files this link +// resolved against; the farm is a mutable view that may hold another build of +// the same SONAME. Farm-first is how an already-linked artifact silently +// starts loading a different library. +TEST(LinkLine, ArtifactDirectoryPrecedesTheRuntimeFallback) { + auto line = full_tail().render(); + EXPECT_LT(pos_of(line, "$ORIGIN"), pos_of(line, "subos/default/lib")) << line; +} + +// ld honours the LAST `--enable/--disable-new-dtags` it sees, and both gcc +// specs and clang config files supply the opposite of what mcpp wants. A slot +// appended after this one silently changes RPATH into RUNPATH (or back), which +// changes whether the tag applies to transitive dependencies at all. +TEST(LinkLine, LoaderTagIsLiterallyLast) { + auto line = full_tail().render(); + EXPECT_TRUE(line.ends_with("-Wl,--disable-new-dtags")) + << "something was emitted after the loader tag: " << line; + EXPECT_GT(pos_of(line, "--disable-new-dtags"), + pos_of(line, "subos/default/lib")) << line; +} + +// The C++ runtime archive is processed after the dependencies it might +// otherwise pre-empt: an archive member is pulled only for symbols still +// undefined when the archive is reached. +TEST(LinkLine, CxxRuntimeFollowsDependenciesAndPrecedesTheFallback) { + auto line = full_tail().render(); + EXPECT_LT(pos_of(line, "-lX11"), pos_of(line, "-static-libstdc++")) << line; + EXPECT_LT(pos_of(line, "-static-libstdc++"), pos_of(line, "subos/default/lib")) << line; +} + +// PE has no rpath and no loader tag; Mach-O has no farm. A format that fills +// only some slots must not acquire stray separators — the rendered line is +// compared byte-for-byte against build.ninja by other tests. +TEST(LinkLine, EmptySlotsProduceNoStraySeparators) { + ll::UnitTail t; + t.cxxRuntime = " -static"; + EXPECT_EQ(t.render(), " -static"); + + ll::UnitTail only_deps; + only_deps.dependencies = " target/bin/foo.lib"; + EXPECT_EQ(only_deps.render(), " target/bin/foo.lib"); + + EXPECT_EQ(ll::UnitTail{}.render(), ""); + EXPECT_TRUE(ll::UnitTail{}.empty()); + EXPECT_FALSE(full_tail().empty()); +} + +// A producer that forgets the leading space must not weld two flags together. +// Every producer supplies it today; this keeps that from being load-bearing. +TEST(LinkLine, SlotWithoutLeadingSpaceGetsASeparator) { + ll::UnitTail t; + t.dependencies = " -lfoo"; + t.cxxRuntime = "-static-libstdc++"; // no leading space + EXPECT_EQ(t.render(), " -lfoo -static-libstdc++"); +} + +// Order is a property of the type, not of the caller's assignment order. +TEST(LinkLine, RenderOrderIsIndependentOfAssignmentOrder) { + ll::UnitTail t; + t.loaderTag = "-Wl,--disable-new-dtags"; + t.runtimeFallback = " -Wl,-rpath,/farm"; + t.cxxRuntime = " -static-libstdc++"; + t.dependencies = " -Wl,-rpath,'$ORIGIN'"; + EXPECT_EQ(t.render(), + " -Wl,-rpath,'$ORIGIN' -static-libstdc++ -Wl,-rpath,/farm" + " -Wl,--disable-new-dtags"); +} diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index d553c493..44eb679e 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -3423,3 +3423,56 @@ gtest = { version = "1.15.2", features = ["main"] } EXPECT_TRUE(got.contains("zlib")); EXPECT_TRUE(got.contains("gtest")); } + +// `cxx_runtime` gained a third role key. The table is validated by an explicit +// whitelist, so a new spelling that nobody added there is rejected with a +// message naming the key — which is the behaviour, not an accident. +TEST(Manifest, CxxRuntimeAcceptsThePerRoleTableIncludingShared) { + constexpr auto src = R"( +[package] +name = "app" +version = "0.1.0" +[build] +cxx_runtime = { default = "self-contained", tests = "host-coupled", shared = "toolchain-coupled" } +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + EXPECT_EQ(m->buildConfig.cxxRuntime, "self-contained"); + EXPECT_EQ(m->buildConfig.cxxRuntimeTests, "host-coupled"); + EXPECT_EQ(m->buildConfig.cxxRuntimeShared, "toolchain-coupled"); + // And it must not be reported as an unsupported [build] key. The whole + // feature is reachable only through this spelling, so a warning saying it + // is "ignored" would be both noisy and false. + EXPECT_TRUE(m->schemaWarnings.empty()) + << (m->schemaWarnings.empty() ? "" : m->schemaWarnings[0]); +} + +TEST(Manifest, CxxRuntimeRejectsAnUnknownRoleKey) { + constexpr auto src = R"( +[package] +name = "app" +version = "0.1.0" +[build] +cxx_runtime = { sharedd = "self-contained" } +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_FALSE(m.has_value()); + EXPECT_NE(m.error().format().find("sharedd"), std::string::npos) + << m.error().format(); +} + +// The scalar spelling still means "every role", including shared libraries. +TEST(Manifest, CxxRuntimeScalarStillAppliesToEveryRole) { + constexpr auto src = R"( +[package] +name = "app" +version = "0.1.0" +[build] +cxx_runtime = "host-coupled" +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + EXPECT_EQ(m->buildConfig.cxxRuntime, "host-coupled"); + EXPECT_TRUE(m->buildConfig.cxxRuntimeShared.empty()); + EXPECT_TRUE(m->schemaWarnings.empty()); +} diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index d388b6f8..7af42d59 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -11,6 +11,7 @@ import mcpp.manifest; import mcpp.toolchain.dialect; import mcpp.toolchain.model; import mcpp.platform; +import mcpp.platform.runtime_search; using namespace mcpp::build; @@ -557,6 +558,96 @@ TEST(NinjaBackend, PlainFlagsPassThroughUnquoted) { << ninja; } +// The SubOS farm must reach DT_RPATH AFTER the artifact's own directory. +// +// It did not, and the composition is why: the farm was appended to the GLOBAL +// `$ldflags` while `$ORIGIN` rides the PER-UNIT `$unit_ldflags`, and every link +// rule renders `$ldflags $unit_ldflags`. Each site read correctly on its own — +// flags.cppm even commented "so it is LAST" — and the artifact loaded a +// different build of libX11 than it had just been linked against. +// +// The assertion is on the EFFECTIVE line (global then unit, exactly as the +// rule expands it), not on either variable alone: checking one of them is what +// made the original mistake invisible. +TEST(NinjaBackend, SubosFarmRpathFollowsTheArtifactsOwnDirectory) { + if constexpr (!mcpp::platform::is_linux) + GTEST_SKIP() << "the SubOS farm rpath is emitted for ELF targets only"; + + const std::string farm = "/tmp/mcpp-ninja-test-farm/subos/default/lib"; + auto plan = minimal_plan(); + plan.runtimeSearch.push_back( + {farm, mcpp::platform::search::Origin::SubosFarm}); + plan.compileUnits.push_back({ + .source = "src/main.cpp", + .kind = mcpp::SourceKind::Cxx, + .object = "obj/main.o", + .packageName = "farm_order_test", + }); + plan.linkUnits.push_back({ + .targetName = "app", + .kind = mcpp::build::LinkUnit::Binary, + .objects = {"obj/main.o"}, + // What `shared_library_link_flags` produces for a shared-lib consumer. + .linkFlags = {"-Lbin", "-Wl,-rpath,'$$ORIGIN'", "-lgreet"}, + .output = "bin/app", + .entryMain = "src/main.cpp", + }); + + auto ninja = emit_ninja_string(plan); + + auto line_after = [&](std::string_view prefix) -> std::string { + auto at = ninja.find(prefix); + if (at == std::string::npos) return {}; + at += prefix.size(); + return ninja.substr(at, ninja.find('\n', at) - at); + }; + // `$cxx $in -o $out $ldflags $unit_ldflags` — reproduce that expansion. + const std::string effective = + line_after("\nldflags =") + " " + line_after("\n unit_ldflags ="); + + auto origin = effective.find("$$ORIGIN"); + auto fallback = effective.find(farm); + ASSERT_NE(origin, std::string::npos) << effective; + ASSERT_NE(fallback, std::string::npos) << effective; + EXPECT_LT(origin, fallback) + << "the SubOS farm outranks $ORIGIN, so the artifact can load a " + "different build of a library than it linked against:\n" << effective; + + // And it must have left the global channel entirely — leaving a copy there + // would restore the old order no matter what the unit tail says. + EXPECT_EQ(line_after("\nldflags =").find(farm), std::string::npos) << ninja; +} + +// An archive is produced by `ar`, whose rule never expands `$unit_ldflags`. +// Emitting run-time search flags there would be dead bytes in every graph. +TEST(NinjaBackend, StaticLibraryCarriesNoRuntimeFallback) { + if constexpr (!mcpp::platform::is_linux) + GTEST_SKIP() << "the SubOS farm rpath is emitted for ELF targets only"; + + const std::string farm = "/tmp/mcpp-ninja-test-farm/subos/default/lib"; + auto plan = minimal_plan(); + plan.runtimeSearch.push_back( + {farm, mcpp::platform::search::Origin::SubosFarm}); + plan.compileUnits.push_back({ + .source = "src/lib.cpp", + .kind = mcpp::SourceKind::Cxx, + .object = "obj/lib.o", + .packageName = "farm_order_test", + }); + plan.linkUnits.push_back({ + .targetName = "greet", + .kind = mcpp::build::LinkUnit::StaticLibrary, + .objects = {"obj/lib.o"}, + .output = "lib/libgreet.a", + }); + + auto ninja = emit_ninja_string(plan); + auto at = ninja.find("build lib/libgreet.a"); + ASSERT_NE(at, std::string::npos) << ninja; + auto stanza = ninja.substr(at, ninja.find("\nbuild ", at + 1) - at); + EXPECT_EQ(stanza.find(farm), std::string::npos) << stanza; +} + // Regression: mcpp-GENERATED per-unit LINK flags are already correctly // shell-quoted + ninja-escaped at construction — e.g. the shared-dep rpath // token `-Wl,-rpath,'$$ORIGIN'` (single quotes stop shell $-expansion, `$$`