fix(cache): a package's object layout must not depend on the consumer (#344)

A dependency's objects live in the global build cache under a key that
deliberately excludes the consuming project — that exclusion is what makes an
entry shareable across projects. Their LAYOUT inside the entry did not exclude
it: the address was the consumer's build-dir path with `obj/` stripped, and
build-dir object paths are decided by #233's basename disambiguation, whose
census spans the WHOLE build directory. So `compat.zlib`'s compress.o was

    obj/compress.o                          alone
    obj/compat_zlib/zlib-1.3.2/compress.o   beside compat.bzip2

under one key. Whichever project ran second asked the entry for a path the first
had never written, and ninja rejected the graph before running any command:

    ninja: error: '<cache>/…/obj/compress.o', needed by 'obj/compress.o',
           missing and no known rule to make it

one line after the CLI printed `Cached compat.zlib v1.3.2 (15 units)`.
32 of 47 mcpp-index workspace members across three platforms.

#233 (compile edges collided), #240 (link inputs didn't follow the rename) and
this are three products of one machine: a layout decided by a global census.
The fix is not a fourth place to keep in sync — it is to take dependencies out
of the census.

* plan.cppm now emits TWO addresses from ONE derivation: `object` (build dir)
  and `packageObjectRel` (inside a cache entry, a pure function of the owning
  package). Dependency objects go to `obj/<pkg-slug>/<mirrored relDir>/<name>.o`
  unconditionally — cross-package collisions are structurally impossible, so
  dependencies need no census at all. The root project, never cached, keeps its
  flat `obj/<name>.o` and now censuses only its own sources.
* prepare.cppm's `object_cache_path` — the second derivation — is deleted,
  along with its `filename()` fallback, which silently mapped distinct sources
  onto one entry address.
* `is_cached(key)` validated the entry's OWN file list while the caller went on
  to read addresses it computed itself; the two answers were never compared.
  It is now `probe_cached(key, requested)`. Any divergence is a MISS, never a
  build failure, and is reported — a systematic recurrence would otherwise show
  up only as "the cache never hits", with no signal at all.
* Cacheability is judged by where the payload is on disk, not by the
  `sourceKind` label. Multi-version mangling re-anchors a consumer package at
  `target/.mangled/…` and rewrites its sources while keeping that label; it
  stays out of the cache today only because axis F happens to differ.
  Containment is judged LEXICALLY (`path_is_under_any`): stores built out of
  symlinks are ordinary, and canonicalizing drops every such package out of the
  cache silently.
* All-or-nothing per package: one unaddressable unit takes the whole package
  out, rather than leaving it half staged.
* kCacheEpoch 1 → 2 (the artifact layout changed) and `cache verify` now
  reports obj addresses that escape their entry.

Tests: unit ObjectAddress pins "a dependency's addresses are immune to the rest
of the graph"; e2e 184 runs both orderings and asserts ZERO compile edges for
the reused dependency — asserting only "it builds" would be satisfied by a
build that quietly degraded every hit to a miss. e2e 123's assertion is
generalized from "the consumer's main must be renamed" (the shape of #240's
first fix) to "every link input is produced by some edge" (the property).

Author: Sunrisepeak

.agents/docs/2026-08-03-issue344-cache-object-address-design.md

@@ -3,6 +3,31 @@
 > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。
 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。
 
+## [2026.8.3.4] — 2026-08-03
+
+### 修复
+
+- **全局 build cache:同一个 key 下第二个消费者必挂 `missing and no known rule to make it`(#344)。** 一个依赖的 `.o` 存在 cache 条目里的**路径**,此前取自消费方 build dir 的相对路径。而 build dir 里的对象布局由 #233 的 basename 消歧决定,消歧的判据是一次**跨越整个 build dir**的普查 —— 也就是说,它取决于消费方还拉了哪些别的包:
+
+  - 同时拉了 `compat.zlib` 与 `compat.bzip2`(两者上游各有一个 `compress.c`)→ `obj/compat_zlib/zlib-1.3.2/compress.o`
+  - 只拉了 `compat.zlib` → `obj/compress.o`
+
+  cache key **刻意不含消费方**(这正是跨工程共享成立的前提),于是两种布局落进同一个条目,后跑的那个工程按自己的布局去取,必然缺一个文件。失败发生在 ninja 的 **graph 加载**阶段 —— 一条命令都还没跑,上一行却刚打印过 `Cached … (15 units)`。mcpp-index 全量 workspace 在三个平台上共 32 个成员因此失败。
+
+  条目内的对象地址现在是**包自身的纯函数**:镜像源文件相对它**自己**包根的路径,不含任何消费方信息。构建目录里,依赖包的对象一律落在 `obj/<pkg-slug>/…` 下 —— 跨包撞名结构性地不可能发生,依赖包因此完全退出消歧普查。根工程(永不入 cache)保持沿用至今的扁平 `obj/<name>.o`。
+
+  #233(编译边撞名)、#240(链接输入未跟改名)与本条是同一台机器的三个产物:**布局由一次全局普查决定**。所以修的不是再补一处同步,而是把依赖包从普查里彻底拿掉。
+
+### 改进
+
+- **cache 条目与本次构建的布局分歧,现在降级为 miss 并明确报告,而不是让 ninja 崩在图加载阶段。** `is_cached` 此前校验的是条目**自述的**文件表,而消费方随后按**自己算的**地址去取 —— 两处独立推导,从不比对。命中判据现在校验「本次实际要读的那批产物」,任何不匹配都只是一次重编。同时新增一行 warning:一个系统性的分歧否则会表现为「cache 永远不命中」而毫无信号,这正是 v2026.7.30.2 之前那个假 `Cached` 骗了三个月的失败模态。
+
+- **可缓存性改判磁盘出处,不再只认标签。** 规则一直写着「无法证明来自不可变的 xpkgs store 就不准入」,但代码实现的是更弱的代理(`sourceKind == "version"`)。多版本共存(mangling)会把消费方包的根重锚到 `<project>/target/.mangled/…` 并**改写其源码**,而标签仍是 `"version"` —— 它今天不出错只靠轴 F 侥幸。现在按包根的实际位置判定。同一批还加了「全有全无」:任何一个单元拿不到与机器无关的条目地址,整个包退出缓存,不留下半 staged 的包(那会表现为三条边之后的 BMI CRC mismatch)。
+
+- **`mcpp cache verify` 现在会报告逃出条目的对象地址。** 让「条目地址必须是包内相对路径」这条不变量可以离线审计,而不是只能通过复现一次双工程构建才看得见。
+
+- **`kCacheEpoch` 1 → 2。** 产物布局变了,旧条目描述的是本版本不会去要的布局。它们本来就会被判为 miss,但让两套布局共用一个目录会让 `cache gc` 的体积统计和 `cache verify` 的输出失去意义。用户侧表现为一次全量重建,无需任何手工步骤。
+
 ## [2026.8.3.3] — 2026-08-03
 
 ### 修复

CHANGELOG.md

@@ -1,6 +1,6 @@
 [package]
 name        = "mcpp"
-version     = "2026.8.3.3"
+version     = "2026.8.3.4"
 description = "Modern C++ build & package management tool"
 license     = "Apache-2.0"
 authors     = ["mcpp-community"]

mcpp.toml

@@ -5,7 +5,16 @@
 //   (the cache root is $MCPP_HOME/build-cache/v1 — see mcpp.home::cache_root)
 //     entry.json                       sentinel + self-description + file list
 //     bmi/<module>.{gcm,pcm}
-//     obj/<relative>.o
+//     obj/<package-internal path>.o
+//
+// The obj address is PACKAGE-INTERNAL: it mirrors the source's path relative to
+// its own package root and contains nothing about the consuming project
+// (mcpp#344). It used to be the consumer's build-dir path with `obj/` stripped,
+// which made the layout depend on which OTHER packages the consumer happened to
+// pull in — #233's basename disambiguation is triggered by a census over the
+// whole build dir — while the key deliberately excludes the consumer. Two
+// consumers then wrote and read incompatible layouts under one key and the
+// second one died in ninja's graph phase.
 //
 // <key16> comes from mcpp.build.cache_key: a per-package Merkle key over the
 // toolchain, the language/dialect settings, the resolved profile, the package's
@@ -75,16 +84,63 @@ struct CacheKey {
     std::filesystem::path objDir()    const { return dir() / "obj"; }
 };
 
-// File names (basenames for BMIs, output-relative paths for objects) belonging
-// to one package's cache entry.
+// One cached object file. The two addresses are deliberately separate
+// (mcpp#344):
+//
+//   cacheRel — where it lives INSIDE the entry (`<entry>/obj/<cacheRel>`).
+//              Must be a pure function of the package, because the key
+//              deliberately excludes the consumer. plan.cppm derives it.
+//   buildRel — where THIS build produces it (`<buildDir>/<buildRel>`).
+//              Consumer-side and therefore not recordable: two consumers of
+//              one entry may legitimately place the same object at different
+//              build-dir paths.
+//
+// Collapsing the two — recording the consumer's path as the entry's address —
+// is exactly what made the second consumer of an entry fail with ninja's
+// "missing and no known rule to make it". Only `cacheRel` ever reaches
+// entry.json; `buildRel` is populate-time input and is empty on read-back.
+struct ObjArtifact {
+    std::string           cacheRel;
+    std::filesystem::path buildRel;
+};
+
+// The artifacts belonging to one package's cache entry: BMI basenames plus the
+// objects above.
 struct DepArtifacts {
-    std::vector<std::string> bmiFiles;
-    std::vector<std::string> objFiles;
+    std::vector<std::string>  bmiFiles;
+    std::vector<ObjArtifact>  objFiles;
 };
 
-// True when entry.json exists, its schema matches, its recorded inputs equal
-// `key.inputs` field for field, and every listed file is present on disk.
-bool is_cached(const CacheKey& key);
+// Why an entry could not serve this build.
+struct CacheProbe {
+    bool ok = false;
+    // Non-empty ONLY when the entry itself validated (schema, key, inputs) but
+    // does not carry the artifacts THIS build asked for. After mcpp#344 that
+    // shape should be unreachable, which is precisely why it must be reported
+    // rather than silently folded into "miss": a systematic recurrence would
+    // otherwise present as "the cache simply never hits", with no signal at
+    // all — the same failure mode as the fake `Cached` that went unnoticed for
+    // three months.
+    std::vector<std::string> layoutMismatch;
+};
+
+// Validate an entry AGAINST WHAT THIS BUILD WILL ACTUALLY READ.
+//
+// A hit requires all of: entry.json exists, its schema matches, its recorded
+// key matches, its recorded inputs equal `key.inputs` field for field, and
+// every artifact in `requested` is both listed by the entry and present on
+// disk. Checking only the entry's OWN file list — which is what this used to
+// do — validates a different question than the one the caller goes on to ask,
+// and the two answers diverged the moment the object layout stopped being a
+// function of the package alone.
+//
+// Anything short of a full match is a MISS. This function must never be the
+// reason a build fails: an unusable entry costs a recompile, and the staging
+// edges that would read it are never emitted.
+CacheProbe probe_cached(const CacheKey& key, const DepArtifacts& requested);
+
+// probe_cached(...).ok
+bool is_cached(const CacheKey& key, const DepArtifacts& requested);
 
 // The artifact list of a validated entry. Does NOT copy anything: the ninja
 // backend stages cached files through its own `stage_file` edges, so that a
@@ -137,12 +193,15 @@ std::optional<nlohmann::json> read_entry(const std::filesystem::path& p) {
     return j;
 }
 
+// Read-back fills `cacheRel` only: `buildRel` is consumer-side and is not — and
+// must not be — recorded in the entry.
 DepArtifacts artifacts_from(const nlohmann::json& j) {
     DepArtifacts a;
     if (auto it = j.find("bmi"); it != j.end() && it->is_array())
         for (auto& v : *it) if (v.is_string()) a.bmiFiles.push_back(v.get<std::string>());
     if (auto it = j.find("obj"); it != j.end() && it->is_array())
-        for (auto& v : *it) if (v.is_string()) a.objFiles.push_back(v.get<std::string>());
+        for (auto& v : *it)
+            if (v.is_string()) a.objFiles.push_back({v.get<std::string>(), {}});
     return a;
 }
 
@@ -199,21 +258,39 @@ std::filesystem::path cached_obj_path(const CacheKey& key, std::string_view rel)
     return key.objDir() / std::filesystem::path(std::string(rel));
 }
 
-bool is_cached(const CacheKey& key) {
+CacheProbe probe_cached(const CacheKey& key, const DepArtifacts& requested) {
+    CacheProbe probe;
     auto j = read_entry(key.entryFile());
-    if (!j) return false;
-    if (j->value("schema", 0) != kEntrySchema) return false;
-    if (j->value("key", std::string{}) != key.keyHex) return false;
+    if (!j) return probe;
+    if (j->value("schema", 0) != kEntrySchema) return probe;
+    if (j->value("key", std::string{}) != key.keyHex) return probe;
     auto it = j->find("inputs");
-    if (it == j->end() || !inputs_match(*it, key.inputs)) return false;
+    if (it == j->end() || !inputs_match(*it, key.inputs)) return probe;
+
+    // The entry itself is valid. From here on, every remaining check is about
+    // whether it holds what THIS build is going to read.
+    auto recorded = artifacts_from(*j);
+    std::set<std::string> haveBmi(recorded.bmiFiles.begin(), recorded.bmiFiles.end());
+    std::set<std::string> haveObj;
+    for (auto& o : recorded.objFiles) haveObj.insert(o.cacheRel);
 
-    auto arts = artifacts_from(*j);
     std::error_code ec;
-    for (auto& g : arts.bmiFiles)
-        if (!std::filesystem::exists(cached_bmi_path(key, g), ec)) return false;
-    for (auto& o : arts.objFiles)
-        if (!std::filesystem::exists(cached_obj_path(key, o), ec)) return false;
-    return true;
+    for (auto& g : requested.bmiFiles) {
+        if (!haveBmi.contains(g)
+            || !std::filesystem::exists(cached_bmi_path(key, g), ec))
+            probe.layoutMismatch.push_back(g);
+    }
+    for (auto& o : requested.objFiles) {
+        if (!haveObj.contains(o.cacheRel)
+            || !std::filesystem::exists(cached_obj_path(key, o.cacheRel), ec))
+            probe.layoutMismatch.push_back(o.cacheRel);
+    }
+    probe.ok = probe.layoutMismatch.empty();
+    return probe;
+}
+
+bool is_cached(const CacheKey& key, const DepArtifacts& requested) {
+    return probe_cached(key, requested).ok;
 }
 
 std::expected<DepArtifacts, std::string> resolve_cached(const CacheKey& key) {
@@ -250,7 +327,6 @@ populate_from(const CacheKey& key,
     std::filesystem::create_directories(cacheObj, ec);
 
     auto projectBmi = projectTargetDir / key.bmiDirName;
-    auto projectObj = projectTargetDir / "obj";
 
     for (auto& g : arts.bmiFiles) {
         auto from = projectBmi / g;
@@ -263,15 +339,20 @@ populate_from(const CacheKey& key,
                 "populate bmi '{}': {}", g, ec.message()));
         }
     }
+    // Read from `buildRel`, write at `cacheRel`. These are NOT the same path in
+    // general (mcpp#344): the build-dir layout partitions objects by package,
+    // the entry's layout is package-internal, and a source that sits outside its
+    // package root is re-anchored for the entry. Deriving one from the other
+    // here is what this split exists to prevent.
     for (auto& o : arts.objFiles) {
-        auto from = projectObj / o;
-        if (!std::filesystem::exists(from)) {
+        auto from = projectTargetDir / o.buildRel;
+        if (o.buildRel.empty() || !std::filesystem::exists(from)) {
             return std::unexpected(std::format(
                 "expected build output missing: {}", from.string()));
         }
-        if (!copy_one(from, cached_obj_path(key, o), ec)) {
+        if (!copy_one(from, cached_obj_path(key, o.cacheRel), ec)) {
             return std::unexpected(std::format(
-                "populate obj '{}': {}", o, ec.message()));
+                "populate obj '{}': {}", o.cacheRel, ec.message()));
         }
     }
 
@@ -289,7 +370,13 @@ populate_from(const CacheKey& key,
     j["tag"]      = key.manifestTag;
     j["inputs"]   = key.inputs;
     j["bmi"]      = arts.bmiFiles;
-    j["obj"]      = arts.objFiles;
+    // Only the entry-internal addresses. Recording the consumer's build path
+    // here is mcpp#344 in one line.
+    {
+        auto objs = nlohmann::json::array();
+        for (auto& o : arts.objFiles) objs.push_back(o.cacheRel);
+        j["obj"] = std::move(objs);
+    }
     j["accessed"] = now_iso8601();
     return write_entry(key.entryFile(), j);
 }

src/bmi_cache.cppm

@@ -177,9 +177,24 @@ void check_pkg_files(Entry& e, const nlohmann::json& j) {
     if (auto it = j.find("obj"); it != j.end() && it->is_array()) {
         for (auto& v : *it) {
             if (!v.is_string()) continue;
-            if (missing(e.dir / "obj" / v.get<std::string>())) {
+            auto rel = v.get<std::string>();
+            // mcpp#344: an entry's obj addresses must be package-internal —
+            // downward, relative, no drive letter. An address that escapes the
+            // entry is one that was derived from some consumer's build tree,
+            // which is precisely the defect that made two consumers of one key
+            // disagree about the layout. Report it here so the invariant is
+            // auditable offline rather than only observable as a build that
+            // dies in ninja's graph phase.
+            if (rel.empty() || rel.starts_with("/") || rel.starts_with("..")
+                || rel.find(':') != std::string::npos) {
                 e.complete = false;
-                e.problem = std::format("missing obj/{}", v.get<std::string>());
+                e.problem = std::format(
+                    "obj address is not package-internal: '{}'", rel);
+                return;
+            }
+            if (missing(e.dir / "obj" / rel)) {
+                e.complete = false;
+                e.problem = std::format("missing obj/{}", rel);
                 return;
             }
         }

src/bmi_cache/maintenance.cppm

@@ -64,7 +64,13 @@ export namespace mcpp::build::cache_key {
 // Deliberately NOT the mcpp release number: folding the whole version in
 // orphaned every entry on every release, including plain C object files whose
 // validity has nothing to do with mcpp's version.
-inline constexpr int kCacheEpoch = 1;
+// 2 (mcpp#344): the artifact layout changed. An entry's obj addresses are now
+// package-internal instead of "the first consumer's build-dir path minus
+// `obj/`", so entries written by an older mcpp describe a layout this one does
+// not ask for. They would all miss anyway (probe_cached compares the REQUESTED
+// artifacts), but sharing a directory between two layouts makes `cache gc`'s
+// size accounting and `cache verify`'s output meaningless.
+inline constexpr int kCacheEpoch = 2;
 
 // Axes A/B/C — identical for every package in one build, computed once.
 struct BuildAxes {